2024-06-03T01:51:14.282 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-03T01:51:14.282 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/postgres}} --owner scheduled_teuthology@teuthology --name teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807 --verbose -- /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/orig.config.yaml 2024-06-03T01:51:14.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-03T01:51:14.424 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7738807' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2064 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 125/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK6MghrGqEca7wKIlzeYr3QhI3l9Nnld2DtW/jHXr9d2ok7v53nV8QmdjFw+seTM3SA3Eqkf++9mclb849gFnn8= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIJXJkDLvQZ35yL+8RZbUJGNjYkGbyf1A2DkrIG+4vyajHplLtkiEOh9eRtYITX1VxxwhAkJNINsy0XVssiVeHo= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxQycGGnNaYN/E88YePf5mEarkHEPb9dM5lvSGWoHRj+fun8sMrZiyo6KAQqntPUWykcVajW8QcjGAUUJtFkVQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-02_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-03T01:51:14.425 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa; will attempt to use it 2024-06-03T01:51:14.425 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks 2024-06-03T01:51:14.425 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-03T01:51:14.427 INFO:teuthology.task.internal:Checking packages... 2024-06-03T01:51:14.447 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-03T01:51:14.447 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-03T01:51:14.447 INFO:teuthology.packaging:ref: None 2024-06-03T01:51:14.447 INFO:teuthology.packaging:tag: None 2024-06-03T01:51:14.447 INFO:teuthology.packaging:branch: squid 2024-06-03T01:51:14.447 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T01:51:14.448 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-03T01:51:14.729 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2952.gdfcc4b3c 2024-06-03T01:51:14.731 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-03T01:51:14.737 INFO:teuthology.task.internal:no buildpackages task found 2024-06-03T01:51:14.737 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-03T01:51:14.770 INFO:teuthology.task.internal:Saving configuration 2024-06-03T01:51:14.790 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-03T01:51:14.796 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-03T01:51:14.820 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 01:46:14.565568', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK6MghrGqEca7wKIlzeYr3QhI3l9Nnld2DtW/jHXr9d2ok7v53nV8QmdjFw+seTM3SA3Eqkf++9mclb849gFnn8='} 2024-06-03T01:51:14.844 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 01:46:14.566919', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIJXJkDLvQZ35yL+8RZbUJGNjYkGbyf1A2DkrIG+4vyajHplLtkiEOh9eRtYITX1VxxwhAkJNINsy0XVssiVeHo='} 2024-06-03T01:51:14.866 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 01:46:14.563584', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxQycGGnNaYN/E88YePf5mEarkHEPb9dM5lvSGWoHRj+fun8sMrZiyo6KAQqntPUWykcVajW8QcjGAUUJtFkVQ='} 2024-06-03T01:51:14.866 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-03T01:51:14.872 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-03T01:51:14.872 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-03T01:51:14.872 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-03T01:51:14.872 INFO:teuthology.run_tasks:Running task console_log... 2024-06-03T01:51:14.959 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f699d937be0>, signals=[15]) 2024-06-03T01:51:14.959 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-03T01:51:14.965 INFO:teuthology.task.internal:Opening connections... 2024-06-03T01:51:14.965 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-06-03T01:51:14.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:51:15.040 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-06-03T01:51:15.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:51:15.116 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-06-03T01:51:15.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:51:15.189 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-03T01:51:15.196 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-06-03T01:51:15.214 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-06-03T01:51:15.214 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-06-03T01:51:15.271 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-06-03T01:51:15.271 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-06-03T01:51:15.271 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T01:51:15.272 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T01:51:15.273 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-06-03T01:51:15.293 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-06-03T01:51:15.311 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-06-03T01:51:15.311 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T01:51:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-06-03T01:51:15.369 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T01:51:15.369 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T01:51:15.369 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T01:51:15.369 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-06-03T01:51:15.392 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-06-03T01:51:15.410 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-06-03T01:51:15.410 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-06-03T01:51:15.467 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-06-03T01:51:15.467 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-06-03T01:51:15.467 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-06-03T01:51:15.467 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-06-03T01:51:15.467 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-06-03T01:51:15.467 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-06-03T01:51:15.467 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T01:51:15.468 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-06-03T01:51:15.468 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-06-03T01:51:15.468 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T01:51:15.468 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-06-03T01:51:15.468 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T01:51:15.468 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T01:51:15.468 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T01:51:15.468 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-06-03T01:51:15.494 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-03T01:51:15.504 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-03T01:51:15.509 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-03T01:51:15.509 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-06-03T01:51:15.511 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-06-03T01:51:15.514 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-06-03T01:51:15.529 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-03T01:51:15.535 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-03T01:51:15.535 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-06-03T01:51:15.569 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-06-03T01:51:15.571 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-06-03T01:51:15.630 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-03T01:51:15.701 INFO:teuthology.run_tasks:Running task kernel... 2024-06-03T01:51:15.715 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-03T01:51:15.715 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-06-03T01:51:15.715 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-06-03T01:51:15.715 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T01:51:15.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:51:15.733 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-06-03T01:51:15.895 DEBUG:teuthology.task.kernel:sha1 for client.0 is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-03T01:51:15.895 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-06-03T01:51:15.895 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-06-03T01:51:15.912 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-437.el9.x86_64 2024-06-03T01:51:15.912 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-03T01:51:15.913 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-06-03T01:51:15.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum upgrade -y linux-firmware 2024-06-03T01:51:18.101 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-06-03T01:51:20.010 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-06-03T01:51:24.344 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.2 MB 00:00 2024-06-03T01:51:25.711 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 17 kB 00:00 2024-06-03T01:51:27.256 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-06-03T01:51:32.487 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-03T01:51:34.601 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-06-03T01:51:34.602 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-06-03T01:51:34.602 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-06-03T01:51:34.719 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-06-03T01:51:34.719 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-03T01:51:34.864 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-06-03T01:51:34.864 DEBUG:teuthology.orchestra.run.smithi079:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-06-03T01:51:34.867 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi079.front.sepia.ceph.com to complete... 2024-06-03T01:51:47.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03 01:51:47 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-06-03T01:51:47.240 INFO:teuthology.orchestra.run.smithi079.stderr:FINISHED --2024-06-03 01:51:47-- 2024-06-03T01:51:47.240 INFO:teuthology.orchestra.run.smithi079.stderr:Total wall clock time: 12s 2024-06-03T01:51:47.240 INFO:teuthology.orchestra.run.smithi079.stderr:Downloaded: 1 files, 340M in 12s (28.4 MB/s) 2024-06-03T01:51:47.243 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-06-03T01:51:47.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-06-03T01:51:49.313 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying... ######################################## 2024-06-03T01:51:49.490 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing... ######################################## 2024-06-03T01:51:49.606 INFO:teuthology.orchestra.run.smithi079.stdout:Updating / installing... 2024-06-03T01:51:57.526 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-06-03T01:52:37.185 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi079.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-06-03T01:52:37.186 DEBUG:teuthology.orchestra.run.smithi079:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-06-03T01:52:37.314 INFO:teuthology.orchestra.run.smithi079.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-06-03T01:52:37.314 INFO:teuthology.orchestra.run.smithi079.stdout:/boot/config-6.9.0-gb9459a26b088 2024-06-03T01:52:37.314 INFO:teuthology.orchestra.run.smithi079.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-06-03T01:52:37.314 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-06-03T01:52:37.314 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-06-03T01:52:37.314 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-06-03T01:52:37.315 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-06-03T01:52:37.316 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-06-03T01:52:37.317 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-06-03T01:52:37.318 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-06-03T01:52:37.319 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-06-03T01:52:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-06-03T01:52:37.321 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-06-03T01:52:37.322 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-06-03T01:52:37.323 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-06-03T01:52:37.324 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-06-03T01:52:37.325 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-06-03T01:52:37.326 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-06-03T01:52:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-06-03T01:52:37.328 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-06-03T01:52:37.329 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-06-03T01:52:37.330 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-06-03T01:52:37.331 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-06-03T01:52:37.332 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-06-03T01:52:37.333 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-06-03T01:52:37.334 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-06-03T01:52:37.335 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-06-03T01:52:37.336 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-06-03T01:52:37.337 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-06-03T01:52:37.338 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-06-03T01:52:37.339 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-06-03T01:52:37.340 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-06-03T01:52:37.341 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-06-03T01:52:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-06-03T01:52:37.343 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-06-03T01:52:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-06-03T01:52:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-06-03T01:52:37.346 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-06-03T01:52:37.347 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-06-03T01:52:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-06-03T01:52:37.349 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-06-03T01:52:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-06-03T01:52:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-06-03T01:52:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-06-03T01:52:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-06-03T01:52:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-06-03T01:52:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-06-03T01:52:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-06-03T01:52:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-06-03T01:52:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-06-03T01:52:37.353 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-06-03T01:52:37.354 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-06-03T01:52:37.355 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-06-03T01:52:37.356 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-06-03T01:52:37.357 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-06-03T01:52:37.358 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-06-03T01:52:37.359 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-06-03T01:52:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-06-03T01:52:37.361 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-06-03T01:52:37.362 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-06-03T01:52:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-06-03T01:52:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-06-03T01:52:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-06-03T01:52:37.366 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-06-03T01:52:37.367 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-06-03T01:52:37.368 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-06-03T01:52:37.369 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-06-03T01:52:37.370 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-06-03T01:52:37.371 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-06-03T01:52:37.372 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-06-03T01:52:37.373 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-06-03T01:52:37.374 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-06-03T01:52:37.375 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-06-03T01:52:37.376 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-06-03T01:52:37.377 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-06-03T01:52:37.378 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-06-03T01:52:37.379 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-06-03T01:52:37.380 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-06-03T01:52:37.381 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-06-03T01:52:37.382 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-06-03T01:52:37.383 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-06-03T01:52:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-06-03T01:52:37.385 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-06-03T01:52:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-06-03T01:52:37.387 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-06-03T01:52:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-06-03T01:52:37.389 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-06-03T01:52:37.390 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-06-03T01:52:37.391 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-06-03T01:52:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-06-03T01:52:37.393 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-06-03T01:52:37.394 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-06-03T01:52:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-06-03T01:52:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-06-03T01:52:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-06-03T01:52:37.398 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-06-03T01:52:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-06-03T01:52:37.400 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-06-03T01:52:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-06-03T01:52:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-06-03T01:52:37.403 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-06-03T01:52:37.404 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-06-03T01:52:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-06-03T01:52:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-06-03T01:52:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-06-03T01:52:37.408 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-06-03T01:52:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-06-03T01:52:37.410 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-06-03T01:52:37.411 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-06-03T01:52:37.412 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-06-03T01:52:37.413 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-06-03T01:52:37.414 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-06-03T01:52:37.415 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-06-03T01:52:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-06-03T01:52:37.417 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-06-03T01:52:37.418 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-06-03T01:52:37.419 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-06-03T01:52:37.420 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-06-03T01:52:37.421 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-06-03T01:52:37.422 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-06-03T01:52:37.423 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-06-03T01:52:37.424 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-06-03T01:52:37.425 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-06-03T01:52:37.426 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-06-03T01:52:37.427 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-06-03T01:52:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-06-03T01:52:37.429 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-06-03T01:52:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-06-03T01:52:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-06-03T01:52:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-06-03T01:52:37.433 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-06-03T01:52:37.434 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-06-03T01:52:37.435 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-06-03T01:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-06-03T01:52:37.437 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-06-03T01:52:37.438 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-06-03T01:52:37.439 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-06-03T01:52:37.440 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-06-03T01:52:37.441 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-06-03T01:52:37.442 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-06-03T01:52:37.443 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-06-03T01:52:37.444 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-06-03T01:52:37.445 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-06-03T01:52:37.446 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-06-03T01:52:37.447 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-06-03T01:52:37.448 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-06-03T01:52:37.449 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-06-03T01:52:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-06-03T01:52:37.451 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-06-03T01:52:37.452 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-06-03T01:52:37.453 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-06-03T01:52:37.454 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-06-03T01:52:37.455 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-06-03T01:52:37.455 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-06-03T01:52:37.455 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-06-03T01:52:37.455 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-06-03T01:52:37.455 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-06-03T01:52:37.455 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-06-03T01:52:37.455 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-06-03T01:52:37.459 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-06-03T01:52:37.459 DEBUG:teuthology.orchestra.run.smithi079:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-06-03T01:52:37.480 INFO:teuthology.orchestra.run.smithi079.stdout:postinstall scriptlet (using /bin/sh): 2024-06-03T01:52:37.480 INFO:teuthology.orchestra.run.smithi079.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout:fi 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout:for file in vmlinuz System.map config; do 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: fi 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout:done 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout:fi 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-06-03T01:52:37.481 INFO:teuthology.orchestra.run.smithi079.stdout:fi 2024-06-03T01:52:37.482 INFO:teuthology.orchestra.run.smithi079.stdout:preuninstall scriptlet (using /bin/sh): 2024-06-03T01:52:37.482 INFO:teuthology.orchestra.run.smithi079.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-03T01:52:37.482 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-06-03T01:52:37.482 INFO:teuthology.orchestra.run.smithi079.stdout:fi 2024-06-03T01:52:37.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo rpm -qi grub2-tools 2024-06-03T01:52:37.563 INFO:teuthology.orchestra.run.smithi079.stdout:Name : grub2-tools 2024-06-03T01:52:37.563 INFO:teuthology.orchestra.run.smithi079.stdout:Epoch : 1 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Version : 2.06 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Release : 68.el9 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: x86_64 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Group : Unspecified 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Size : 8264254 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:License : GPLv3+ 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Packager : builder@centos.org 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Vendor : CentOS 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Summary : Support tools for GRUB. 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout:Description : 2024-06-03T01:52:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T01:52:37.565 INFO:teuthology.orchestra.run.smithi079.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-03T01:52:37.565 INFO:teuthology.orchestra.run.smithi079.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-03T01:52:37.565 INFO:teuthology.orchestra.run.smithi079.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-03T01:52:37.565 INFO:teuthology.orchestra.run.smithi079.stdout:hardware devices. 2024-06-03T01:52:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T01:52:37.565 INFO:teuthology.orchestra.run.smithi079.stdout:This subpackage provides tools for support of all platforms. 2024-06-03T01:52:37.566 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-03T01:52:37.566 INFO:teuthology.task.kernel:Updating grub on smithi079 to boot 6.9.0-gb9459a26b088 2024-06-03T01:52:37.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-03T01:52:38.174 INFO:teuthology.orchestra.run.smithi079.stderr:Generating grub configuration file ... 2024-06-03T01:52:39.479 INFO:teuthology.orchestra.run.smithi079.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-03T01:52:39.504 INFO:teuthology.orchestra.run.smithi079.stderr:done 2024-06-03T01:52:39.506 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-06-03T01:52:39.542 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.3MwQ8sIUcT 2024-06-03T01:52:39.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3MwQ8sIUcT 2024-06-03T01:52:39.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.3MwQ8sIUcT 2024-06-03T01:52:39.748 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.3MwQ8sIUcT is 6KB 2024-06-03T01:52:39.797 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.3MwQ8sIUcT 2024-06-03T01:52:39.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo /bin/ls /boot/loader/entries || true 2024-06-03T01:52:39.881 INFO:teuthology.orchestra.run.smithi079.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-03T01:52:39.881 INFO:teuthology.orchestra.run.smithi079.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-03T01:52:39.882 INFO:teuthology.orchestra.run.smithi079.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-03T01:52:39.882 INFO:teuthology.orchestra.run.smithi079.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-06-03T01:52:39.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-06-03T01:52:40.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo shutdown -r now 2024-06-03T01:53:10.055 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-03T01:53:10.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-06-03T01:53:10.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:53:28.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-06-03T01:53:37.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-06-03T01:53:37.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:53:40.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-06-03T01:53:52.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-06-03T01:53:52.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:54:52.674 DEBUG:teuthology.orchestra.remote:timed out 2024-06-03T01:55:07.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-06-03T01:55:07.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T01:55:08.064 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-06-03T01:55:08.683 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-06-03T01:55:08.684 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-gb9459a26b088"... 2024-06-03T01:55:08.684 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-06-03T01:55:08.733 INFO:teuthology.orchestra.run.smithi079.stdout:6.9.0-gb9459a26b088 2024-06-03T01:55:08.733 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-06-03T01:55:08.733 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-03T01:55:08.733 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-03T01:55:09.734 DEBUG:teuthology.parallel:result is None 2024-06-03T01:55:09.735 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-03T01:55:09.742 INFO:teuthology.task.internal:Creating test directory... 2024-06-03T01:55:09.742 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T01:55:09.749 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T01:55:09.752 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T01:55:09.796 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-03T01:55:09.802 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-03T01:55:09.811 INFO:teuthology.task.internal:Creating archive directory... 2024-06-03T01:55:09.811 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T01:55:09.816 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T01:55:09.818 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T01:55:09.930 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-03T01:55:09.936 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-03T01:55:09.936 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T01:55:09.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:55:09.953 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T01:55:09.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:55:09.969 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T01:55:09.977 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T01:55:09.995 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T01:55:10.027 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:55:10.046 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:55:10.053 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:55:10.093 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:55:10.104 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:55:10.156 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T01:55:10.161 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-03T01:55:10.167 INFO:teuthology.task.internal:Configuring sudo... 2024-06-03T01:55:10.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T01:55:10.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T01:55:10.177 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T01:55:10.254 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-03T01:55:10.262 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-03T01:55:10.263 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T01:55:10.270 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T01:55:10.273 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T01:55:10.317 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:55:10.399 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:55:10.502 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:55:10.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:55:10.675 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T01:55:10.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T01:55:10.757 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:55:10.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:55:10.856 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:55:10.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:55:10.980 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T01:55:10.980 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T01:55:11.046 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:55:11.090 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T01:55:11.144 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:55:11.221 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T01:55:11.270 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T01:55:11.270 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T01:55:11.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-06-03T01:55:11.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-06-03T01:55:11.347 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-06-03T01:55:11.381 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:55:11.419 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:55:11.453 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:55:11.927 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-03T01:55:11.935 INFO:teuthology.task.internal:Starting timer... 2024-06-03T01:55:11.935 INFO:teuthology.run_tasks:Running task pcp... 2024-06-03T01:55:11.944 INFO:teuthology.run_tasks:Running task selinux... 2024-06-03T01:55:11.952 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-03T01:55:11.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2024-06-03T01:55:12.050 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2024-06-03T01:55:12.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-06-03T01:55:12.124 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-06-03T01:55:12.126 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-06-03T01:55:12.204 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-06-03T01:55:12.206 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-03T01:55:12.206 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2024-06-03T01:55:12.273 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2024-06-03T01:55:12.273 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-06-03T01:55:12.297 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-06-03T01:55:12.298 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-06-03T01:55:12.326 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-06-03T01:55:12.326 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi079.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive'} 2024-06-03T01:55:12.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:55:12.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:55:12.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:55:12.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:55:12.431 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:55:12.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:55:12.459 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-03T01:55:12.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2024-06-03T01:55:12.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-06-03T01:55:12.574 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-06-03T01:55:12.618 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-03T01:55:12.627 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-03T01:55:12.630 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-03T01:55:12.747 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-03T01:55:12.758 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-03T01:55:12.759 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi079.front.sepia.ceph.com,smithi123.front.sepia.ceph.com,smithi135.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-03T02:01:32.489 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com')] 2024-06-03T02:01:32.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-06-03T02:01:32.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T02:01:32.597 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-06-03T02:01:32.747 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-06-03T02:01:32.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-06-03T02:01:32.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T02:01:32.822 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-06-03T02:01:32.903 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-06-03T02:01:32.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-06-03T02:01:32.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T02:01:32.978 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-06-03T02:01:33.058 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-06-03T02:01:33.059 INFO:teuthology.run_tasks:Running task clock... 2024-06-03T02:01:33.069 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-03T02:01:33.070 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T02:01:33.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T02:01:33.078 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T02:01:33.078 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T02:01:33.080 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T02:01:33.080 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T02:01:33.110 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T02:01:33.125 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T02:01:33.128 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T02:01:33.142 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T02:01:33.149 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-06-03T02:01:33.160 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-06-03T02:01:33.162 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T02:01:33.165 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-06-03T02:01:33.174 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T02:01:33.176 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-06-03T02:01:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T02:01:33.190 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T02:01:33.200 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T02:01:33.203 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T02:01:33.245 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-06-03T02:01:33.248 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T02:01:33.248 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-06-03T02:01:33.248 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.248 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.248 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.248 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.262 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-06-03T02:01:33.266 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T02:01:33.266 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-06-03T02:01:33.266 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.266 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.266 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.267 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.272 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2024-06-03T02:01:33.304 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2024-06-03T02:01:33.344 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T02:01:33.385 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T02:01:33.514 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-06-03T02:01:33.522 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T02:01:33.523 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-06-03T02:01:33.523 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.523 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.523 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.523 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:01:33.524 INFO:teuthology.run_tasks:Running task install... 2024-06-03T02:01:33.532 DEBUG:teuthology.task.install:project ceph 2024-06-03T02:01:33.532 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-03T02:01:33.533 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-03T02:01:33.533 INFO:teuthology.task.install:Using flavor: default 2024-06-03T02:01:33.538 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-03T02:01:33.538 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-03T02:01:33.539 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-03T02:01:33.539 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:01:33.541 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-03T02:01:33.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:01:33.542 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-03T02:01:33.542 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:01:33.686 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-03T02:01:33.686 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-03T02:01:33.760 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-03T02:01:33.760 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-03T02:01:33.774 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-03T02:01:33.775 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-03T02:01:33.809 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T02:01:33.809 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:01:33.809 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T02:01:33.898 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-03T02:01:33.898 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T02:01:33.929 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T02:01:33.929 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:01:33.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T02:01:33.943 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T02:01:33.944 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:01:33.944 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T02:01:33.957 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-03T02:01:33.957 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T02:01:33.970 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-03T02:01:33.971 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T02:01:34.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T02:01:34.023 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T02:01:34.037 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T02:01:34.098 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T02:01:34.111 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T02:01:34.128 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T02:01:34.161 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-06-03T02:01:34.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-06-03T02:01:34.174 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-06-03T02:01:34.176 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-06-03T02:01:34.202 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2024-06-03T02:01:34.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2024-06-03T02:01:34.387 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-06-03T02:01:34.398 INFO:teuthology.orchestra.run.smithi135.stdout:45 files removed 2024-06-03T02:01:34.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-03T02:01:34.424 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-03T02:01:34.519 INFO:teuthology.orchestra.run.smithi079.stdout:45 files removed 2024-06-03T02:01:34.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-03T02:01:35.116 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 194 kB/s | 83 kB 00:00 2024-06-03T02:01:35.143 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 176 kB/s | 83 kB 00:00 2024-06-03T02:01:35.537 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 152 kB/s | 83 kB 00:00 2024-06-03T02:01:35.541 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-06-03T02:01:35.561 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 31 kB/s | 11 kB 00:00 2024-06-03T02:01:35.967 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-06-03T02:01:35.986 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-06-03T02:01:36.046 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-06-03T02:01:36.514 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-06-03T02:01:36.601 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-06-03T02:01:36.628 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-06-03T02:01:37.136 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-06-03T02:01:38.311 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-06-03T02:01:38.396 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-06-03T02:01:39.046 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-06-03T02:01:42.312 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-06-03T02:01:42.805 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.2 MB 00:00 2024-06-03T02:01:43.396 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-06-03T02:01:43.822 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 17 kB 00:00 2024-06-03T02:01:44.631 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 17 kB 00:00 2024-06-03T02:01:44.882 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-06-03T02:01:44.939 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 17 kB 00:00 2024-06-03T02:01:46.068 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-06-03T02:01:46.140 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-06-03T02:01:49.958 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-06-03T02:01:51.317 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-06-03T02:01:52.198 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-03T02:01:52.242 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T02:01:52.243 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T02:01:52.254 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-03T02:01:52.255 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-03T02:01:52.256 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-03T02:01:52.257 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-03T02:01:52.258 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-03T02:01:52.260 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-03T02:01:52.261 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-03T02:01:52.325 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-06-03T02:01:52.348 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-06-03T02:01:52.348 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-06-03T02:01:52.348 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-06-03T02:01:52.348 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-06-03T02:01:52.348 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-03T02:01:52.348 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-03T02:01:52.348 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-03T02:01:52.349 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-03T02:01:52.350 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-03T02:01:52.351 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-03T02:01:52.352 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-03T02:01:52.353 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T02:01:52.354 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-03T02:01:52.355 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-03T02:01:52.356 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T02:01:52.356 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T02:01:52.356 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T02:01:52.356 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-03T02:01:52.356 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-03T02:01:52.357 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-03T02:01:52.357 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T02:01:52.357 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-03T02:01:52.357 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T02:01:52.357 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T02:01:52.358 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T02:01:52.358 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-03T02:01:52.358 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-03T02:01:52.358 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T02:01:52.358 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T02:01:52.359 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T02:01:52.359 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T02:01:52.359 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-03T02:01:52.359 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-03T02:01:52.359 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T02:01:52.360 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T02:01:52.360 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T02:01:52.360 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-03T02:01:52.360 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-03T02:01:52.360 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T02:01:52.361 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T02:01:52.361 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T02:01:52.361 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T02:01:52.361 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T02:01:52.361 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T02:01:52.361 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T02:01:52.362 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T02:01:52.362 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T02:01:52.362 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T02:01:52.362 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T02:01:52.362 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T02:01:52.363 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T02:01:52.363 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T02:01:52.363 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T02:01:52.363 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T02:01:52.363 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T02:01:52.364 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-03T02:01:52.364 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T02:01:52.364 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T02:01:52.364 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T02:01:52.364 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T02:01:52.365 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-03T02:01:52.365 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T02:01:52.365 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T02:01:52.365 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T02:01:52.365 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T02:01:52.365 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-03T02:01:52.366 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T02:01:52.366 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T02:01:52.366 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T02:01:52.366 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T02:01:52.366 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T02:01:52.367 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T02:01:52.367 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T02:01:52.367 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T02:01:52.367 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T02:01:52.367 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T02:01:52.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T02:01:52.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-03T02:01:52.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T02:01:52.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T02:01:52.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T02:01:52.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T02:01:52.369 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-03T02:01:52.369 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-03T02:01:52.369 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T02:01:52.369 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T02:01:52.369 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T02:01:52.370 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-03T02:01:52.370 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T02:01:52.370 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-03T02:01:52.370 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2024-06-03T02:01:52.370 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-03T02:01:52.370 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:01:52.370 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-06-03T02:01:52.371 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-06-03T02:01:52.371 INFO:teuthology.orchestra.run.smithi135.stdout:Install 148 Packages 2024-06-03T02:01:52.371 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 11 Packages 2024-06-03T02:01:52.371 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:01:52.371 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 225 M 2024-06-03T02:01:52.371 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-06-03T02:01:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T02:01:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T02:01:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-03T02:01:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-03T02:01:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-03T02:01:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-03T02:01:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-03T02:01:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-03T02:01:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-03T02:01:53.689 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-06-03T02:01:53.692 INFO:teuthology.orchestra.run.smithi135.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-03T02:01:53.705 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-03T02:01:53.706 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-03T02:01:53.707 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-03T02:01:53.708 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-03T02:01:53.709 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-03T02:01:53.710 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T02:01:53.710 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T02:01:53.710 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T02:01:53.710 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T02:01:53.710 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T02:01:53.710 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-03T02:01:53.710 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T02:01:53.710 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T02:01:53.710 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-03T02:01:53.711 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-03T02:01:53.712 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T02:01:53.713 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T02:01:53.714 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T02:01:53.715 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T02:01:53.716 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-06-03T02:01:53.717 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-06-03T02:01:53.718 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 11 Packages 2024-06-03T02:01:53.718 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:01:53.718 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 225 M 2024-06-03T02:01:53.718 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-06-03T02:01:54.009 INFO:teuthology.orchestra.run.smithi135.stdout:(2/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 8.4 MB/s | 5.4 MB 00:00 2024-06-03T02:01:54.060 INFO:teuthology.orchestra.run.smithi135.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 2.8 MB/s | 145 kB 00:00 2024-06-03T02:01:54.244 INFO:teuthology.orchestra.run.smithi135.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 13 MB/s | 2.4 MB 00:00 2024-06-03T02:01:54.277 INFO:teuthology.orchestra.run.smithi135.stdout:(5/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.5 MB/s | 890 kB 00:00 2024-06-03T02:01:54.361 INFO:teuthology.orchestra.run.smithi135.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 13 MB/s | 1.5 MB 00:00 2024-06-03T02:01:54.442 INFO:teuthology.orchestra.run.smithi123.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-06-03T02:01:54.797 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T02:01:54.798 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T02:01:54.809 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-03T02:01:54.810 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-03T02:01:54.811 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-03T02:01:54.812 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-03T02:01:54.813 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-03T02:01:54.816 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-03T02:01:54.816 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-03T02:01:54.881 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-06-03T02:01:54.898 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-06-03T02:01:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-06-03T02:01:54.898 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-06-03T02:01:54.898 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-06-03T02:01:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-03T02:01:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-03T02:01:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-03T02:01:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-03T02:01:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-03T02:01:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T02:01:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-03T02:01:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-03T02:01:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T02:01:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T02:01:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-03T02:01:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-03T02:01:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T02:01:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T02:01:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T02:01:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T02:01:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T02:01:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T02:01:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T02:01:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T02:01:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-03T02:01:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout:Install 148 Packages 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 11 Packages 2024-06-03T02:01:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:01:54.912 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 225 M 2024-06-03T02:01:54.912 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-06-03T02:01:55.442 INFO:teuthology.orchestra.run.smithi123.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 890 kB/s | 890 kB 00:00 2024-06-03T02:01:55.478 INFO:teuthology.orchestra.run.smithi135.stdout:(7/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x86 15 MB/s | 17 MB 00:01 2024-06-03T02:01:55.567 INFO:teuthology.orchestra.run.smithi123.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.1 MB/s | 145 kB 00:00 2024-06-03T02:01:56.241 INFO:teuthology.orchestra.run.smithi079.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-06-03T02:01:56.370 INFO:teuthology.orchestra.run.smithi135.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 11 MB/s | 10 MB 00:00 2024-06-03T02:01:56.470 INFO:teuthology.orchestra.run.smithi135.stdout:(9/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 2.1 MB/s | 4.7 MB 00:02 2024-06-03T02:01:56.496 INFO:teuthology.orchestra.run.smithi135.stdout:(10/159): ceph-selinux-19.0.0-2946.g3d89accf.el 201 kB/s | 25 kB 00:00 2024-06-03T02:01:56.579 INFO:teuthology.orchestra.run.smithi135.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.6 MB/s | 221 kB 00:00 2024-06-03T02:01:56.626 INFO:teuthology.orchestra.run.smithi123.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-06-03T02:01:56.655 INFO:teuthology.orchestra.run.smithi135.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 359 kB/s | 27 kB 00:00 2024-06-03T02:01:56.738 INFO:teuthology.orchestra.run.smithi135.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 381 kB/s | 32 kB 00:00 2024-06-03T02:01:56.769 INFO:teuthology.orchestra.run.smithi123.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 2.4 MB 00:01 2024-06-03T02:01:56.847 INFO:teuthology.orchestra.run.smithi135.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 6.7 MB/s | 747 kB 00:00 2024-06-03T02:01:56.931 INFO:teuthology.orchestra.run.smithi135.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.9 MB/s | 163 kB 00:00 2024-06-03T02:01:56.998 INFO:teuthology.orchestra.run.smithi135.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.8 MB/s | 127 kB 00:00 2024-06-03T02:01:57.082 INFO:teuthology.orchestra.run.smithi135.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 5.9 MB/s | 501 kB 00:00 2024-06-03T02:01:57.116 INFO:teuthology.orchestra.run.smithi079.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 890 kB 00:00 2024-06-03T02:01:57.501 INFO:teuthology.orchestra.run.smithi123.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-06-03T02:01:57.558 INFO:teuthology.orchestra.run.smithi079.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 329 kB/s | 145 kB 00:00 2024-06-03T02:01:57.632 INFO:teuthology.orchestra.run.smithi135.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 9.7 MB/s | 5.3 MB 00:00 2024-06-03T02:01:57.682 INFO:teuthology.orchestra.run.smithi135.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 907 kB/s | 45 kB 00:00 2024-06-03T02:01:57.733 INFO:teuthology.orchestra.run.smithi135.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 2.6 MB/s | 135 kB 00:00 2024-06-03T02:01:57.735 INFO:teuthology.orchestra.run.smithi123.stdout:(7/159): ceph-common-19.0.0-2946.g3d89accf.el9. 5.9 MB/s | 21 MB 00:03 2024-06-03T02:01:57.791 INFO:teuthology.orchestra.run.smithi135.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.7 MB/s | 162 kB 00:00 2024-06-03T02:01:57.875 INFO:teuthology.orchestra.run.smithi135.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.8 MB/s | 321 kB 00:00 2024-06-03T02:01:57.967 INFO:teuthology.orchestra.run.smithi135.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.2 MB/s | 300 kB 00:00 2024-06-03T02:01:58.034 INFO:teuthology.orchestra.run.smithi135.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 99 kB 00:00 2024-06-03T02:01:58.118 INFO:teuthology.orchestra.run.smithi135.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-03T02:01:58.502 INFO:teuthology.orchestra.run.smithi135.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 8.1 MB/s | 3.1 MB 00:00 2024-06-03T02:01:58.585 INFO:teuthology.orchestra.run.smithi135.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-03T02:01:58.661 INFO:teuthology.orchestra.run.smithi135.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 423 kB/s | 32 kB 00:00 2024-06-03T02:01:58.736 INFO:teuthology.orchestra.run.smithi135.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.8 MB/s | 142 kB 00:00 2024-06-03T02:01:59.125 INFO:teuthology.orchestra.run.smithi079.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-06-03T02:01:59.128 INFO:teuthology.orchestra.run.smithi135.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 9.0 MB/s | 3.5 MB 00:00 2024-06-03T02:01:59.143 INFO:teuthology.orchestra.run.smithi123.stdout:(8/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 4.7 MB 00:02 2024-06-03T02:01:59.260 INFO:teuthology.orchestra.run.smithi123.stdout:(9/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 6.7 MB/s | 10 MB 00:01 2024-06-03T02:01:59.277 INFO:teuthology.orchestra.run.smithi123.stdout:(10/159): ceph-selinux-19.0.0-2946.g3d89accf.el 189 kB/s | 25 kB 00:00 2024-06-03T02:01:59.452 INFO:teuthology.orchestra.run.smithi123.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.2 MB/s | 221 kB 00:00 2024-06-03T02:01:59.506 INFO:teuthology.orchestra.run.smithi135.stdout:(31/159): ceph-common-19.0.0-2946.g3d89accf.el9 3.4 MB/s | 21 MB 00:06 2024-06-03T02:01:59.544 INFO:teuthology.orchestra.run.smithi123.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 295 kB/s | 27 kB 00:00 2024-06-03T02:01:59.653 INFO:teuthology.orchestra.run.smithi123.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 293 kB/s | 32 kB 00:00 2024-06-03T02:01:59.696 INFO:teuthology.orchestra.run.smithi135.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.3 MB/s | 249 kB 00:00 2024-06-03T02:01:59.775 INFO:teuthology.orchestra.run.smithi079.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-06-03T02:01:59.796 INFO:teuthology.orchestra.run.smithi135.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 493 kB/s | 49 kB 00:00 2024-06-03T02:01:59.905 INFO:teuthology.orchestra.run.smithi135.stdout:(34/159): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-06-03T02:01:59.930 INFO:teuthology.orchestra.run.smithi135.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 123 kB/s | 16 kB 00:00 2024-06-03T02:01:59.961 INFO:teuthology.orchestra.run.smithi123.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.4 MB/s | 747 kB 00:00 2024-06-03T02:01:59.980 INFO:teuthology.orchestra.run.smithi135.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.4 MB/s | 264 kB 00:00 2024-06-03T02:02:00.053 INFO:teuthology.orchestra.run.smithi123.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.7 MB/s | 163 kB 00:00 2024-06-03T02:02:00.056 INFO:teuthology.orchestra.run.smithi135.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 242 kB/s | 18 kB 00:00 2024-06-03T02:02:00.075 INFO:teuthology.orchestra.run.smithi079.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-06-03T02:02:00.120 INFO:teuthology.orchestra.run.smithi123.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.9 MB/s | 127 kB 00:00 2024-06-03T02:02:00.181 INFO:teuthology.orchestra.run.smithi135.stdout:(38/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 3.0 MB/s | 763 kB 00:00 2024-06-03T02:02:00.207 INFO:teuthology.orchestra.run.smithi135.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-06-03T02:02:00.290 INFO:teuthology.orchestra.run.smithi135.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-06-03T02:02:00.312 INFO:teuthology.orchestra.run.smithi123.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 2.6 MB/s | 501 kB 00:00 2024-06-03T02:02:00.349 INFO:teuthology.orchestra.run.smithi135.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-06-03T02:02:00.449 INFO:teuthology.orchestra.run.smithi135.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-06-03T02:02:00.475 INFO:teuthology.orchestra.run.smithi135.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-06-03T02:02:00.508 INFO:teuthology.orchestra.run.smithi135.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-03T02:02:00.534 INFO:teuthology.orchestra.run.smithi135.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 135 kB 00:00 2024-06-03T02:02:00.567 INFO:teuthology.orchestra.run.smithi135.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-03T02:02:00.585 INFO:teuthology.orchestra.run.smithi135.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.1 MB/s | 215 kB 00:00 2024-06-03T02:02:00.710 INFO:teuthology.orchestra.run.smithi135.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-06-03T02:02:00.744 INFO:teuthology.orchestra.run.smithi135.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-06-03T02:02:00.769 INFO:teuthology.orchestra.run.smithi135.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-06-03T02:02:00.795 INFO:teuthology.orchestra.run.smithi135.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-06-03T02:02:00.820 INFO:teuthology.orchestra.run.smithi135.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-03T02:02:00.845 INFO:teuthology.orchestra.run.smithi135.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-06-03T02:02:00.887 INFO:teuthology.orchestra.run.smithi135.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 436 kB/s | 18 kB 00:00 2024-06-03T02:02:00.946 INFO:teuthology.orchestra.run.smithi135.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-06-03T02:02:00.971 INFO:teuthology.orchestra.run.smithi135.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-06-03T02:02:00.997 INFO:teuthology.orchestra.run.smithi135.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-03T02:02:01.022 INFO:teuthology.orchestra.run.smithi135.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-06-03T02:02:01.056 INFO:teuthology.orchestra.run.smithi135.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-06-03T02:02:01.089 INFO:teuthology.orchestra.run.smithi135.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-06-03T02:02:01.115 INFO:teuthology.orchestra.run.smithi135.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-06-03T02:02:01.140 INFO:teuthology.orchestra.run.smithi135.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-06-03T02:02:01.165 INFO:teuthology.orchestra.run.smithi135.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-03T02:02:01.191 INFO:teuthology.orchestra.run.smithi135.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-03T02:02:01.216 INFO:teuthology.orchestra.run.smithi135.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 699 kB/s | 35 kB 00:00 2024-06-03T02:02:01.317 INFO:teuthology.orchestra.run.smithi135.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-06-03T02:02:01.392 INFO:teuthology.orchestra.run.smithi135.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-06-03T02:02:01.434 INFO:teuthology.orchestra.run.smithi135.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-06-03T02:02:01.468 INFO:teuthology.orchestra.run.smithi135.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-06-03T02:02:01.493 INFO:teuthology.orchestra.run.smithi135.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-03T02:02:01.552 INFO:teuthology.orchestra.run.smithi135.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-06-03T02:02:01.636 INFO:teuthology.orchestra.run.smithi135.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-06-03T02:02:01.661 INFO:teuthology.orchestra.run.smithi135.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-06-03T02:02:01.695 INFO:teuthology.orchestra.run.smithi135.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-06-03T02:02:01.795 INFO:teuthology.orchestra.run.smithi135.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-06-03T02:02:01.821 INFO:teuthology.orchestra.run.smithi135.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-06-03T02:02:01.846 INFO:teuthology.orchestra.run.smithi135.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-03T02:02:01.872 INFO:teuthology.orchestra.run.smithi135.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-06-03T02:02:01.897 INFO:teuthology.orchestra.run.smithi135.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 379 kB/s | 19 kB 00:00 2024-06-03T02:02:01.931 INFO:teuthology.orchestra.run.smithi135.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-06-03T02:02:01.956 INFO:teuthology.orchestra.run.smithi135.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-06-03T02:02:01.998 INFO:teuthology.orchestra.run.smithi135.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-03T02:02:02.004 INFO:teuthology.orchestra.run.smithi123.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.1 MB/s | 5.3 MB 00:01 2024-06-03T02:02:02.073 INFO:teuthology.orchestra.run.smithi135.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-06-03T02:02:02.096 INFO:teuthology.orchestra.run.smithi123.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 494 kB/s | 45 kB 00:00 2024-06-03T02:02:02.099 INFO:teuthology.orchestra.run.smithi135.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-06-03T02:02:02.108 INFO:teuthology.orchestra.run.smithi079.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 4.7 MB 00:02 2024-06-03T02:02:02.124 INFO:teuthology.orchestra.run.smithi135.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-03T02:02:02.169 INFO:teuthology.orchestra.run.smithi135.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 600 kB/s | 42 kB 00:00 2024-06-03T02:02:02.188 INFO:teuthology.orchestra.run.smithi123.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.4 MB/s | 135 kB 00:00 2024-06-03T02:02:02.200 INFO:teuthology.orchestra.run.smithi135.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-06-03T02:02:02.225 INFO:teuthology.orchestra.run.smithi135.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.8 MB/s | 159 kB 00:00 2024-06-03T02:02:02.250 INFO:teuthology.orchestra.run.smithi135.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-03T02:02:02.272 INFO:teuthology.orchestra.run.smithi123.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.9 MB/s | 162 kB 00:00 2024-06-03T02:02:02.276 INFO:teuthology.orchestra.run.smithi135.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-03T02:02:02.301 INFO:teuthology.orchestra.run.smithi135.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-03T02:02:02.352 INFO:teuthology.orchestra.run.smithi135.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 826 kB/s | 42 kB 00:00 2024-06-03T02:02:02.394 INFO:teuthology.orchestra.run.smithi135.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-03T02:02:02.405 INFO:teuthology.orchestra.run.smithi123.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 2.4 MB/s | 321 kB 00:00 2024-06-03T02:02:02.417 INFO:teuthology.orchestra.run.smithi079.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 3.2 MB/s | 21 MB 00:06 2024-06-03T02:02:02.444 INFO:teuthology.orchestra.run.smithi135.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-03T02:02:02.476 INFO:teuthology.orchestra.run.smithi079.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 441 kB/s | 25 kB 00:00 2024-06-03T02:02:02.495 INFO:teuthology.orchestra.run.smithi135.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-06-03T02:02:02.539 INFO:teuthology.orchestra.run.smithi123.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.2 MB/s | 300 kB 00:00 2024-06-03T02:02:02.623 INFO:teuthology.orchestra.run.smithi123.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.2 MB/s | 99 kB 00:00 2024-06-03T02:02:02.678 INFO:teuthology.orchestra.run.smithi135.stdout:(96/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-06-03T02:02:02.698 INFO:teuthology.orchestra.run.smithi123.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-06-03T02:02:02.704 INFO:teuthology.orchestra.run.smithi135.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 227 kB/s | 47 kB 00:00 2024-06-03T02:02:02.738 INFO:teuthology.orchestra.run.smithi135.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-06-03T02:02:02.763 INFO:teuthology.orchestra.run.smithi135.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-06-03T02:02:02.788 INFO:teuthology.orchestra.run.smithi135.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-03T02:02:02.814 INFO:teuthology.orchestra.run.smithi135.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-03T02:02:02.839 INFO:teuthology.orchestra.run.smithi135.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-06-03T02:02:02.864 INFO:teuthology.orchestra.run.smithi135.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-06-03T02:02:02.898 INFO:teuthology.orchestra.run.smithi135.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-06-03T02:02:02.956 INFO:teuthology.orchestra.run.smithi123.stdout:(26/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 3.1 MB/s | 17 MB 00:05 2024-06-03T02:02:02.990 INFO:teuthology.orchestra.run.smithi135.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 4.3 MB/s | 548 kB 00:00 2024-06-03T02:02:03.015 INFO:teuthology.orchestra.run.smithi135.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-06-03T02:02:03.032 INFO:teuthology.orchestra.run.smithi123.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-06-03T02:02:03.041 INFO:teuthology.orchestra.run.smithi135.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-06-03T02:02:03.091 INFO:teuthology.orchestra.run.smithi135.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-06-03T02:02:03.107 INFO:teuthology.orchestra.run.smithi123.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 424 kB/s | 32 kB 00:00 2024-06-03T02:02:03.117 INFO:teuthology.orchestra.run.smithi135.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-03T02:02:03.150 INFO:teuthology.orchestra.run.smithi135.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-06-03T02:02:03.182 INFO:teuthology.orchestra.run.smithi123.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.9 MB/s | 142 kB 00:00 2024-06-03T02:02:03.192 INFO:teuthology.orchestra.run.smithi135.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-06-03T02:02:03.260 INFO:teuthology.orchestra.run.smithi135.stdout:(112/159): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-06-03T02:02:03.302 INFO:teuthology.orchestra.run.smithi135.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-06-03T02:02:03.352 INFO:teuthology.orchestra.run.smithi135.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-06-03T02:02:03.377 INFO:teuthology.orchestra.run.smithi135.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 577 kB/s | 43 kB 00:00 2024-06-03T02:02:03.403 INFO:teuthology.orchestra.run.smithi135.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 638 kB/s | 32 kB 00:00 2024-06-03T02:02:03.428 INFO:teuthology.orchestra.run.smithi135.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-06-03T02:02:03.454 INFO:teuthology.orchestra.run.smithi135.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-06-03T02:02:03.488 INFO:teuthology.orchestra.run.smithi135.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-06-03T02:02:03.512 INFO:teuthology.orchestra.run.smithi135.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-06-03T02:02:03.538 INFO:teuthology.orchestra.run.smithi135.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-06-03T02:02:03.613 INFO:teuthology.orchestra.run.smithi135.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-06-03T02:02:03.639 INFO:teuthology.orchestra.run.smithi135.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-06-03T02:02:03.664 INFO:teuthology.orchestra.run.smithi135.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-06-03T02:02:03.682 INFO:teuthology.orchestra.run.smithi123.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 3.1 MB/s | 3.1 MB 00:00 2024-06-03T02:02:03.689 INFO:teuthology.orchestra.run.smithi135.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-06-03T02:02:03.715 INFO:teuthology.orchestra.run.smithi135.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-06-03T02:02:03.740 INFO:teuthology.orchestra.run.smithi135.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-06-03T02:02:03.766 INFO:teuthology.orchestra.run.smithi135.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-06-03T02:02:03.807 INFO:teuthology.orchestra.run.smithi135.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-06-03T02:02:03.833 INFO:teuthology.orchestra.run.smithi135.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-06-03T02:02:03.858 INFO:teuthology.orchestra.run.smithi135.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-06-03T02:02:03.884 INFO:teuthology.orchestra.run.smithi135.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-06-03T02:02:03.909 INFO:teuthology.orchestra.run.smithi135.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-06-03T02:02:03.934 INFO:teuthology.orchestra.run.smithi135.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-06-03T02:02:03.968 INFO:teuthology.orchestra.run.smithi135.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-06-03T02:02:03.996 INFO:teuthology.orchestra.run.smithi135.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 463 kB/s | 28 kB 00:00 2024-06-03T02:02:04.021 INFO:teuthology.orchestra.run.smithi135.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 587 kB/s | 31 kB 00:00 2024-06-03T02:02:04.046 INFO:teuthology.orchestra.run.smithi135.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-06-03T02:02:04.071 INFO:teuthology.orchestra.run.smithi135.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-06-03T02:02:04.097 INFO:teuthology.orchestra.run.smithi135.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-06-03T02:02:04.122 INFO:teuthology.orchestra.run.smithi135.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-03T02:02:04.147 INFO:teuthology.orchestra.run.smithi135.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-06-03T02:02:04.164 INFO:teuthology.orchestra.run.smithi135.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-06-03T02:02:04.174 INFO:teuthology.orchestra.run.smithi123.stdout:(31/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 9.8 MB/s | 48 MB 00:04 2024-06-03T02:02:04.190 INFO:teuthology.orchestra.run.smithi135.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 10 MB/s | 427 kB 00:00 2024-06-03T02:02:04.215 INFO:teuthology.orchestra.run.smithi135.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-06-03T02:02:04.240 INFO:teuthology.orchestra.run.smithi135.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-03T02:02:04.241 INFO:teuthology.orchestra.run.smithi123.stdout:(32/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 3.3 MB/s | 3.5 MB 00:01 2024-06-03T02:02:04.267 INFO:teuthology.orchestra.run.smithi123.stdout:(33/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.7 MB/s | 249 kB 00:00 2024-06-03T02:02:04.282 INFO:teuthology.orchestra.run.smithi135.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-06-03T02:02:04.300 INFO:teuthology.orchestra.run.smithi123.stdout:(34/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 845 kB/s | 49 kB 00:00 2024-06-03T02:02:04.326 INFO:teuthology.orchestra.run.smithi123.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 281 kB/s | 16 kB 00:00 2024-06-03T02:02:04.409 INFO:teuthology.orchestra.run.smithi123.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.4 MB/s | 264 kB 00:00 2024-06-03T02:02:04.451 INFO:teuthology.orchestra.run.smithi123.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 6.0 MB/s | 763 kB 00:00 2024-06-03T02:02:04.476 INFO:teuthology.orchestra.run.smithi123.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 273 kB/s | 18 kB 00:00 2024-06-03T02:02:04.602 INFO:teuthology.orchestra.run.smithi123.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-06-03T02:02:04.685 INFO:teuthology.orchestra.run.smithi123.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-06-03T02:02:04.719 INFO:teuthology.orchestra.run.smithi123.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-06-03T02:02:04.744 INFO:teuthology.orchestra.run.smithi123.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-06-03T02:02:04.766 INFO:teuthology.orchestra.run.smithi135.stdout:(148/159): librados2-19.0.0-2946.g3d89accf.el9. 6.5 MB/s | 3.4 MB 00:00 2024-06-03T02:02:04.803 INFO:teuthology.orchestra.run.smithi123.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-03T02:02:04.836 INFO:teuthology.orchestra.run.smithi123.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-06-03T02:02:04.862 INFO:teuthology.orchestra.run.smithi123.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-03T02:02:04.895 INFO:teuthology.orchestra.run.smithi123.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-03T02:02:04.929 INFO:teuthology.orchestra.run.smithi123.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-06-03T02:02:05.046 INFO:teuthology.orchestra.run.smithi123.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-06-03T02:02:05.057 INFO:teuthology.orchestra.run.smithi135.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 7.9 MB/s | 2.3 MB 00:00 2024-06-03T02:02:05.088 INFO:teuthology.orchestra.run.smithi123.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-06-03T02:02:05.122 INFO:teuthology.orchestra.run.smithi123.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-06-03T02:02:05.147 INFO:teuthology.orchestra.run.smithi123.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-06-03T02:02:05.172 INFO:teuthology.orchestra.run.smithi123.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-03T02:02:05.174 INFO:teuthology.orchestra.run.smithi135.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 15 MB/s | 1.8 MB 00:00 2024-06-03T02:02:05.197 INFO:teuthology.orchestra.run.smithi123.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-06-03T02:02:05.231 INFO:teuthology.orchestra.run.smithi123.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-06-03T02:02:05.233 INFO:teuthology.orchestra.run.smithi135.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 711 kB/s | 41 kB 00:00 2024-06-03T02:02:05.281 INFO:teuthology.orchestra.run.smithi123.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 27 MB/s | 3.0 MB 00:00 2024-06-03T02:02:05.284 INFO:teuthology.orchestra.run.smithi079.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 3.2 MB/s | 17 MB 00:05 2024-06-03T02:02:05.291 INFO:teuthology.orchestra.run.smithi135.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-06-03T02:02:05.307 INFO:teuthology.orchestra.run.smithi123.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-06-03T02:02:05.332 INFO:teuthology.orchestra.run.smithi123.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-03T02:02:05.350 INFO:teuthology.orchestra.run.smithi135.stdout:(153/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-06-03T02:02:05.357 INFO:teuthology.orchestra.run.smithi123.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-06-03T02:02:05.367 INFO:teuthology.orchestra.run.smithi079.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.6 MB/s | 221 kB 00:00 2024-06-03T02:02:05.376 INFO:teuthology.orchestra.run.smithi135.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 439 kB/s | 37 kB 00:00 2024-06-03T02:02:05.399 INFO:teuthology.orchestra.run.smithi123.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-06-03T02:02:05.418 INFO:teuthology.orchestra.run.smithi079.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 552 kB/s | 27 kB 00:00 2024-06-03T02:02:05.434 INFO:teuthology.orchestra.run.smithi135.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 4.4 MB/s | 260 kB 00:00 2024-06-03T02:02:05.458 INFO:teuthology.orchestra.run.smithi123.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-06-03T02:02:05.468 INFO:teuthology.orchestra.run.smithi079.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 647 kB/s | 32 kB 00:00 2024-06-03T02:02:05.483 INFO:teuthology.orchestra.run.smithi123.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 306 kB/s | 25 kB 00:00 2024-06-03T02:02:05.509 INFO:teuthology.orchestra.run.smithi123.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-06-03T02:02:05.534 INFO:teuthology.orchestra.run.smithi123.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-03T02:02:05.534 INFO:teuthology.orchestra.run.smithi135.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 14 MB/s | 1.3 MB 00:00 2024-06-03T02:02:05.559 INFO:teuthology.orchestra.run.smithi123.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-03T02:02:05.560 INFO:teuthology.orchestra.run.smithi135.stdout:(157/159): elfutils-libelf-0.191-4.el9.x86_64.r 988 kB/s | 207 kB 00:00 2024-06-03T02:02:05.576 INFO:teuthology.orchestra.run.smithi123.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 841 kB/s | 35 kB 00:00 2024-06-03T02:02:05.635 INFO:teuthology.orchestra.run.smithi079.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.4 MB/s | 747 kB 00:00 2024-06-03T02:02:05.670 INFO:teuthology.orchestra.run.smithi135.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-06-03T02:02:05.670 INFO:teuthology.orchestra.run.smithi123.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 44 MB/s | 4.7 MB 00:00 2024-06-03T02:02:05.752 INFO:teuthology.orchestra.run.smithi123.stdout:(67/159): ceph-mgr-diskprediction-local-19.0.0- 3.6 MB/s | 7.4 MB 00:02 2024-06-03T02:02:05.794 INFO:teuthology.orchestra.run.smithi123.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-06-03T02:02:05.810 INFO:teuthology.orchestra.run.smithi079.stdout:(15/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 2.8 MB/s | 10 MB 00:03 2024-06-03T02:02:05.844 INFO:teuthology.orchestra.run.smithi079.stdout:(16/159): libcephsqlite-19.0.0-2946.g3d89accf.e 786 kB/s | 163 kB 00:00 2024-06-03T02:02:05.852 INFO:teuthology.orchestra.run.smithi123.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-06-03T02:02:05.877 INFO:teuthology.orchestra.run.smithi123.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-06-03T02:02:05.886 INFO:teuthology.orchestra.run.smithi079.stdout:(17/159): librados-devel-19.0.0-2946.g3d89accf. 1.7 MB/s | 127 kB 00:00 2024-06-03T02:02:05.928 INFO:teuthology.orchestra.run.smithi123.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-06-03T02:02:05.961 INFO:teuthology.orchestra.run.smithi123.stdout:(72/159): postgresql-contrib-13.14-1.el9.x86_64 4.3 MB/s | 911 kB 00:00 2024-06-03T02:02:05.969 INFO:teuthology.orchestra.run.smithi079.stdout:(18/159): libradosstriper1-19.0.0-2946.g3d89acc 3.9 MB/s | 501 kB 00:00 2024-06-03T02:02:05.986 INFO:teuthology.orchestra.run.smithi123.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-06-03T02:02:06.020 INFO:teuthology.orchestra.run.smithi079.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 909 kB/s | 45 kB 00:00 2024-06-03T02:02:06.054 INFO:teuthology.orchestra.run.smithi123.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-06-03T02:02:06.070 INFO:teuthology.orchestra.run.smithi123.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 732 kB/s | 61 kB 00:00 2024-06-03T02:02:06.078 INFO:teuthology.orchestra.run.smithi079.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 2.3 MB/s | 135 kB 00:00 2024-06-03T02:02:06.104 INFO:teuthology.orchestra.run.smithi123.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-06-03T02:02:06.130 INFO:teuthology.orchestra.run.smithi123.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-06-03T02:02:06.137 INFO:teuthology.orchestra.run.smithi079.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.8 MB/s | 162 kB 00:00 2024-06-03T02:02:06.154 INFO:teuthology.orchestra.run.smithi123.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-06-03T02:02:06.180 INFO:teuthology.orchestra.run.smithi123.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-06-03T02:02:06.205 INFO:teuthology.orchestra.run.smithi123.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-03T02:02:06.230 INFO:teuthology.orchestra.run.smithi123.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-06-03T02:02:06.246 INFO:teuthology.orchestra.run.smithi079.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 2.9 MB/s | 321 kB 00:00 2024-06-03T02:02:06.273 INFO:teuthology.orchestra.run.smithi123.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-03T02:02:06.321 INFO:teuthology.orchestra.run.smithi079.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 4.0 MB/s | 300 kB 00:00 2024-06-03T02:02:06.356 INFO:teuthology.orchestra.run.smithi123.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-06-03T02:02:06.380 INFO:teuthology.orchestra.run.smithi079.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.7 MB/s | 99 kB 00:00 2024-06-03T02:02:06.423 INFO:teuthology.orchestra.run.smithi123.stdout:(84/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-06-03T02:02:06.438 INFO:teuthology.orchestra.run.smithi079.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.5 MB/s | 85 kB 00:00 2024-06-03T02:02:06.448 INFO:teuthology.orchestra.run.smithi123.stdout:(85/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-06-03T02:02:06.498 INFO:teuthology.orchestra.run.smithi123.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 565 kB/s | 42 kB 00:00 2024-06-03T02:02:06.524 INFO:teuthology.orchestra.run.smithi123.stdout:(87/159): python3-packaging-20.9-5.el9.noarch.r 771 kB/s | 77 kB 00:00 2024-06-03T02:02:06.549 INFO:teuthology.orchestra.run.smithi123.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-03T02:02:06.574 INFO:teuthology.orchestra.run.smithi123.stdout:(89/159): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-06-03T02:02:06.599 INFO:teuthology.orchestra.run.smithi123.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-03T02:02:06.625 INFO:teuthology.orchestra.run.smithi123.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-03T02:02:06.650 INFO:teuthology.orchestra.run.smithi123.stdout:(92/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-06-03T02:02:06.675 INFO:teuthology.orchestra.run.smithi123.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-06-03T02:02:06.700 INFO:teuthology.orchestra.run.smithi123.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-06-03T02:02:06.726 INFO:teuthology.orchestra.run.smithi123.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-03T02:02:06.751 INFO:teuthology.orchestra.run.smithi123.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-06-03T02:02:06.776 INFO:teuthology.orchestra.run.smithi123.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-06-03T02:02:06.802 INFO:teuthology.orchestra.run.smithi123.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-06-03T02:02:06.827 INFO:teuthology.orchestra.run.smithi123.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-03T02:02:06.852 INFO:teuthology.orchestra.run.smithi123.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-03T02:02:06.878 INFO:teuthology.orchestra.run.smithi123.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-03T02:02:06.903 INFO:teuthology.orchestra.run.smithi123.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-06-03T02:02:06.939 INFO:teuthology.orchestra.run.smithi079.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 6.2 MB/s | 3.1 MB 00:00 2024-06-03T02:02:06.962 INFO:teuthology.orchestra.run.smithi123.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-06-03T02:02:06.997 INFO:teuthology.orchestra.run.smithi079.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.9 MB/s | 171 kB 00:00 2024-06-03T02:02:07.047 INFO:teuthology.orchestra.run.smithi079.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 646 kB/s | 32 kB 00:00 2024-06-03T02:02:07.106 INFO:teuthology.orchestra.run.smithi079.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.4 MB/s | 142 kB 00:00 2024-06-03T02:02:07.154 INFO:teuthology.orchestra.run.smithi123.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-06-03T02:02:07.220 INFO:teuthology.orchestra.run.smithi123.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-06-03T02:02:07.281 INFO:teuthology.orchestra.run.smithi079.stdout:(30/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.8 MB/s | 5.3 MB 00:01 2024-06-03T02:02:07.338 INFO:teuthology.orchestra.run.smithi123.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-06-03T02:02:07.438 INFO:teuthology.orchestra.run.smithi123.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-06-03T02:02:07.513 INFO:teuthology.orchestra.run.smithi123.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-06-03T02:02:07.555 INFO:teuthology.orchestra.run.smithi123.stdout:(109/159): abseil-cpp-20211102.0-3.el9.x86_64.r 924 kB/s | 548 kB 00:00 2024-06-03T02:02:07.580 INFO:teuthology.orchestra.run.smithi123.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 725 kB/s | 49 kB 00:00 2024-06-03T02:02:07.631 INFO:teuthology.orchestra.run.smithi123.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 896 kB/s | 67 kB 00:00 2024-06-03T02:02:07.656 INFO:teuthology.orchestra.run.smithi123.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-06-03T02:02:07.740 INFO:teuthology.orchestra.run.smithi123.stdout:(113/159): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-06-03T02:02:07.757 INFO:teuthology.orchestra.run.smithi079.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 5.4 MB/s | 3.5 MB 00:00 2024-06-03T02:02:07.773 INFO:teuthology.orchestra.run.smithi123.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-06-03T02:02:07.790 INFO:teuthology.orchestra.run.smithi123.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-06-03T02:02:07.815 INFO:teuthology.orchestra.run.smithi123.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 769 kB/s | 32 kB 00:00 2024-06-03T02:02:07.832 INFO:teuthology.orchestra.run.smithi123.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 358 kB/s | 15 kB 00:00 2024-06-03T02:02:07.848 INFO:teuthology.orchestra.run.smithi079.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.7 MB/s | 249 kB 00:00 2024-06-03T02:02:07.874 INFO:teuthology.orchestra.run.smithi123.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-06-03T02:02:07.907 INFO:teuthology.orchestra.run.smithi079.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 855 kB/s | 49 kB 00:00 2024-06-03T02:02:07.908 INFO:teuthology.orchestra.run.smithi123.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-06-03T02:02:07.933 INFO:teuthology.orchestra.run.smithi123.stdout:(120/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-06-03T02:02:07.958 INFO:teuthology.orchestra.run.smithi123.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-06-03T02:02:07.966 INFO:teuthology.orchestra.run.smithi079.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 286 kB/s | 16 kB 00:00 2024-06-03T02:02:07.983 INFO:teuthology.orchestra.run.smithi123.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-06-03T02:02:08.009 INFO:teuthology.orchestra.run.smithi123.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-06-03T02:02:08.034 INFO:teuthology.orchestra.run.smithi123.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-06-03T02:02:08.058 INFO:teuthology.orchestra.run.smithi079.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.8 MB/s | 264 kB 00:00 2024-06-03T02:02:08.060 INFO:teuthology.orchestra.run.smithi123.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 350 kB/s | 18 kB 00:00 2024-06-03T02:02:08.085 INFO:teuthology.orchestra.run.smithi123.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-06-03T02:02:08.110 INFO:teuthology.orchestra.run.smithi123.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-06-03T02:02:08.135 INFO:teuthology.orchestra.run.smithi123.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-06-03T02:02:08.186 INFO:teuthology.orchestra.run.smithi123.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 928 kB/s | 46 kB 00:00 2024-06-03T02:02:08.219 INFO:teuthology.orchestra.run.smithi123.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-06-03T02:02:08.233 INFO:teuthology.orchestra.run.smithi079.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 4.3 MB/s | 763 kB 00:00 2024-06-03T02:02:08.245 INFO:teuthology.orchestra.run.smithi123.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-06-03T02:02:08.278 INFO:teuthology.orchestra.run.smithi123.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 988 kB/s | 58 kB 00:00 2024-06-03T02:02:08.283 INFO:teuthology.orchestra.run.smithi079.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 372 kB/s | 18 kB 00:00 2024-06-03T02:02:08.304 INFO:teuthology.orchestra.run.smithi123.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-06-03T02:02:08.337 INFO:teuthology.orchestra.run.smithi123.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-06-03T02:02:08.362 INFO:teuthology.orchestra.run.smithi123.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-06-03T02:02:08.388 INFO:teuthology.orchestra.run.smithi123.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 558 kB/s | 28 kB 00:00 2024-06-03T02:02:08.421 INFO:teuthology.orchestra.run.smithi123.stdout:(137/159): python3-grpcio-1.46.7-10.el9.x86_64. 4.2 MB/s | 2.0 MB 00:00 2024-06-03T02:02:08.433 INFO:teuthology.orchestra.run.smithi079.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 539 kB/s | 80 kB 00:00 2024-06-03T02:02:08.447 INFO:teuthology.orchestra.run.smithi123.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 365 kB/s | 31 kB 00:00 2024-06-03T02:02:08.472 INFO:teuthology.orchestra.run.smithi123.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-06-03T02:02:08.497 INFO:teuthology.orchestra.run.smithi123.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-06-03T02:02:08.522 INFO:teuthology.orchestra.run.smithi123.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-06-03T02:02:08.548 INFO:teuthology.orchestra.run.smithi123.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-06-03T02:02:08.564 INFO:teuthology.orchestra.run.smithi123.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-06-03T02:02:08.598 INFO:teuthology.orchestra.run.smithi123.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-06-03T02:02:08.624 INFO:teuthology.orchestra.run.smithi123.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 340 kB/s | 20 kB 00:00 2024-06-03T02:02:08.649 INFO:teuthology.orchestra.run.smithi123.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-06-03T02:02:08.708 INFO:teuthology.orchestra.run.smithi123.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-06-03T02:02:08.741 INFO:teuthology.orchestra.run.smithi123.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-06-03T02:02:08.934 INFO:teuthology.orchestra.run.smithi079.stdout:(39/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 7.5 MB/s | 48 MB 00:06 2024-06-03T02:02:09.025 INFO:teuthology.orchestra.run.smithi123.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 8.2 MB/s | 2.3 MB 00:00 2024-06-03T02:02:09.118 INFO:teuthology.orchestra.run.smithi079.stdout:(40/159): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-06-03T02:02:09.133 INFO:teuthology.orchestra.run.smithi123.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-06-03T02:02:09.160 INFO:teuthology.orchestra.run.smithi079.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-06-03T02:02:09.184 INFO:teuthology.orchestra.run.smithi123.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 828 kB/s | 41 kB 00:00 2024-06-03T02:02:09.234 INFO:teuthology.orchestra.run.smithi079.stdout:(42/159): libquadmath-11.4.1-3.el9.x86_64.rpm 653 kB/s | 195 kB 00:00 2024-06-03T02:02:09.242 INFO:teuthology.orchestra.run.smithi123.stdout:(152/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-06-03T02:02:09.267 INFO:teuthology.orchestra.run.smithi123.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 2.9 MB/s | 248 kB 00:00 2024-06-03T02:02:09.293 INFO:teuthology.orchestra.run.smithi079.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-06-03T02:02:09.318 INFO:teuthology.orchestra.run.smithi079.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-06-03T02:02:09.326 INFO:teuthology.orchestra.run.smithi123.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-06-03T02:02:09.351 INFO:teuthology.orchestra.run.smithi123.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 337 kB/s | 37 kB 00:00 2024-06-03T02:02:09.352 INFO:teuthology.orchestra.run.smithi079.stdout:(45/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-06-03T02:02:09.385 INFO:teuthology.orchestra.run.smithi123.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 4.3 MB/s | 260 kB 00:00 2024-06-03T02:02:09.386 INFO:teuthology.orchestra.run.smithi079.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-06-03T02:02:09.411 INFO:teuthology.orchestra.run.smithi079.stdout:(47/159): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-06-03T02:02:09.453 INFO:teuthology.orchestra.run.smithi079.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-06-03T02:02:09.477 INFO:teuthology.orchestra.run.smithi123.stdout:(157/159): librados2-19.0.0-2946.g3d89accf.el9. 4.1 MB/s | 3.4 MB 00:00 2024-06-03T02:02:09.545 INFO:teuthology.orchestra.run.smithi079.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-06-03T02:02:09.554 INFO:teuthology.orchestra.run.smithi123.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-06-03T02:02:09.594 INFO:teuthology.orchestra.run.smithi123.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 1.3 MB 00:00 2024-06-03T02:02:09.596 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-06-03T02:02:09.597 INFO:teuthology.orchestra.run.smithi123.stdout:Total 14 MB/s | 225 MB 00:15 2024-06-03T02:02:09.612 INFO:teuthology.orchestra.run.smithi079.stdout:(50/159): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-06-03T02:02:09.637 INFO:teuthology.orchestra.run.smithi079.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 237 kB/s | 22 kB 00:00 2024-06-03T02:02:09.671 INFO:teuthology.orchestra.run.smithi079.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-06-03T02:02:09.696 INFO:teuthology.orchestra.run.smithi079.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 566 kB/s | 33 kB 00:00 2024-06-03T02:02:09.747 INFO:teuthology.orchestra.run.smithi079.stdout:(54/159): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-06-03T02:02:09.772 INFO:teuthology.orchestra.run.smithi079.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 244 kB/s | 18 kB 00:00 2024-06-03T02:02:09.797 INFO:teuthology.orchestra.run.smithi079.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 225 kB/s | 11 kB 00:00 2024-06-03T02:02:09.822 INFO:teuthology.orchestra.run.smithi079.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 160 kB 00:00 2024-06-03T02:02:09.847 INFO:teuthology.orchestra.run.smithi079.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 916 kB/s | 45 kB 00:00 2024-06-03T02:02:09.898 INFO:teuthology.orchestra.run.smithi079.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-06-03T02:02:09.940 INFO:teuthology.orchestra.run.smithi079.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 621 kB/s | 25 kB 00:00 2024-06-03T02:02:09.943 INFO:teuthology.orchestra.run.smithi135.stdout:(159/159): ceph-test-19.0.0-2946.g3d89accf.el9. 3.6 MB/s | 48 MB 00:13 2024-06-03T02:02:09.947 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-06-03T02:02:09.948 INFO:teuthology.orchestra.run.smithi135.stdout:Total 13 MB/s | 225 MB 00:17 2024-06-03T02:02:09.973 INFO:teuthology.orchestra.run.smithi079.stdout:(61/159): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 582 kB 00:00 2024-06-03T02:02:09.999 INFO:teuthology.orchestra.run.smithi079.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 820 kB/s | 47 kB 00:00 2024-06-03T02:02:10.056 INFO:teuthology.orchestra.run.smithi079.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.5 MB/s | 292 kB 00:00 2024-06-03T02:02:10.241 INFO:teuthology.orchestra.run.smithi079.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 781 kB/s | 188 kB 00:00 2024-06-03T02:02:10.267 INFO:teuthology.orchestra.run.smithi079.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 168 kB/s | 35 kB 00:00 2024-06-03T02:02:10.351 INFO:teuthology.orchestra.run.smithi079.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.4 MB/s | 3.0 MB 00:00 2024-06-03T02:02:10.450 INFO:teuthology.orchestra.run.smithi079.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-03T02:02:10.542 INFO:teuthology.orchestra.run.smithi079.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 9.8 MB/s | 911 kB 00:00 2024-06-03T02:02:10.634 INFO:teuthology.orchestra.run.smithi079.stdout:(69/159): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-06-03T02:02:10.718 INFO:teuthology.orchestra.run.smithi079.stdout:(70/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-06-03T02:02:10.743 INFO:teuthology.orchestra.run.smithi079.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 667 kB/s | 133 kB 00:00 2024-06-03T02:02:10.785 INFO:teuthology.orchestra.run.smithi079.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-06-03T02:02:10.785 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-06-03T02:02:10.835 INFO:teuthology.orchestra.run.smithi079.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-06-03T02:02:10.869 INFO:teuthology.orchestra.run.smithi079.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-06-03T02:02:10.919 INFO:teuthology.orchestra.run.smithi079.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-06-03T02:02:10.943 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-06-03T02:02:10.943 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-06-03T02:02:10.969 INFO:teuthology.orchestra.run.smithi079.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-06-03T02:02:11.037 INFO:teuthology.orchestra.run.smithi079.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 388 kB/s | 26 kB 00:00 2024-06-03T02:02:11.079 INFO:teuthology.orchestra.run.smithi079.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 469 kB/s | 19 kB 00:00 2024-06-03T02:02:11.121 INFO:teuthology.orchestra.run.smithi079.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.1 MB/s | 172 kB 00:00 2024-06-03T02:02:11.169 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-06-03T02:02:11.229 INFO:teuthology.orchestra.run.smithi079.stdout:(80/159): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-06-03T02:02:11.254 INFO:teuthology.orchestra.run.smithi079.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 262 kB/s | 35 kB 00:00 2024-06-03T02:02:11.305 INFO:teuthology.orchestra.run.smithi079.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-06-03T02:02:11.327 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-06-03T02:02:11.327 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-06-03T02:02:11.413 INFO:teuthology.orchestra.run.smithi079.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 9.5 MB/s | 6.0 MB 00:00 2024-06-03T02:02:11.447 INFO:teuthology.orchestra.run.smithi079.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-06-03T02:02:11.472 INFO:teuthology.orchestra.run.smithi079.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-06-03T02:02:11.531 INFO:teuthology.orchestra.run.smithi079.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 510 kB/s | 42 kB 00:00 2024-06-03T02:02:11.564 INFO:teuthology.orchestra.run.smithi079.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-06-03T02:02:11.598 INFO:teuthology.orchestra.run.smithi079.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-06-03T02:02:11.632 INFO:teuthology.orchestra.run.smithi079.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-06-03T02:02:11.657 INFO:teuthology.orchestra.run.smithi079.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 883 kB/s | 51 kB 00:00 2024-06-03T02:02:11.682 INFO:teuthology.orchestra.run.smithi079.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-06-03T02:02:11.724 INFO:teuthology.orchestra.run.smithi079.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-06-03T02:02:11.766 INFO:teuthology.orchestra.run.smithi079.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-06-03T02:02:11.817 INFO:teuthology.orchestra.run.smithi079.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-06-03T02:02:11.925 INFO:teuthology.orchestra.run.smithi079.stdout:(95/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.3 MB/s | 5.1 MB 00:00 2024-06-03T02:02:11.950 INFO:teuthology.orchestra.run.smithi079.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 270 kB/s | 36 kB 00:00 2024-06-03T02:02:11.992 INFO:teuthology.orchestra.run.smithi079.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 717 kB/s | 47 kB 00:00 2024-06-03T02:02:12.026 INFO:teuthology.orchestra.run.smithi079.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-06-03T02:02:12.051 INFO:teuthology.orchestra.run.smithi079.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-06-03T02:02:12.076 INFO:teuthology.orchestra.run.smithi079.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-06-03T02:02:12.102 INFO:teuthology.orchestra.run.smithi079.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-06-03T02:02:12.127 INFO:teuthology.orchestra.run.smithi079.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-06-03T02:02:12.211 INFO:teuthology.orchestra.run.smithi079.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-06-03T02:02:12.486 INFO:teuthology.orchestra.run.smithi079.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-06-03T02:02:12.545 INFO:teuthology.orchestra.run.smithi079.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-06-03T02:02:12.578 INFO:teuthology.orchestra.run.smithi079.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-06-03T02:02:12.629 INFO:teuthology.orchestra.run.smithi079.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 235 kB/s | 19 kB 00:00 2024-06-03T02:02:12.662 INFO:teuthology.orchestra.run.smithi079.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-06-03T02:02:12.688 INFO:teuthology.orchestra.run.smithi079.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 451 kB/s | 26 kB 00:00 2024-06-03T02:02:12.713 INFO:teuthology.orchestra.run.smithi079.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 983 kB/s | 49 kB 00:00 2024-06-03T02:02:12.732 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-06-03T02:02:12.732 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-06-03T02:02:12.738 INFO:teuthology.orchestra.run.smithi079.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-06-03T02:02:12.772 INFO:teuthology.orchestra.run.smithi079.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-06-03T02:02:12.839 INFO:teuthology.orchestra.run.smithi079.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-06-03T02:02:12.889 INFO:teuthology.orchestra.run.smithi079.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 839 kB 00:00 2024-06-03T02:02:12.906 INFO:teuthology.orchestra.run.smithi079.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 657 kB/s | 43 kB 00:00 2024-06-03T02:02:12.956 INFO:teuthology.orchestra.run.smithi079.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 487 kB/s | 32 kB 00:00 2024-06-03T02:02:12.973 INFO:teuthology.orchestra.run.smithi079.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 227 kB/s | 15 kB 00:00 2024-06-03T02:02:13.007 INFO:teuthology.orchestra.run.smithi079.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-06-03T02:02:13.040 INFO:teuthology.orchestra.run.smithi079.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-06-03T02:02:13.082 INFO:teuthology.orchestra.run.smithi079.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-06-03T02:02:13.116 INFO:teuthology.orchestra.run.smithi079.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-06-03T02:02:13.127 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-06-03T02:02:13.128 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-06-03T02:02:13.216 INFO:teuthology.orchestra.run.smithi079.stdout:(122/159): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-06-03T02:02:13.250 INFO:teuthology.orchestra.run.smithi079.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-06-03T02:02:13.267 INFO:teuthology.orchestra.run.smithi079.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 217 kB/s | 11 kB 00:00 2024-06-03T02:02:13.284 INFO:teuthology.orchestra.run.smithi079.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 541 kB/s | 18 kB 00:00 2024-06-03T02:02:13.301 INFO:teuthology.orchestra.run.smithi079.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 710 kB/s | 23 kB 00:00 2024-06-03T02:02:13.318 INFO:teuthology.orchestra.run.smithi079.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 594 kB/s | 19 kB 00:00 2024-06-03T02:02:13.334 INFO:teuthology.orchestra.run.smithi079.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 603 kB/s | 20 kB 00:00 2024-06-03T02:02:13.377 INFO:teuthology.orchestra.run.smithi079.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-06-03T02:02:13.444 INFO:teuthology.orchestra.run.smithi079.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-06-03T02:02:13.469 INFO:teuthology.orchestra.run.smithi079.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 866 kB/s | 79 kB 00:00 2024-06-03T02:02:13.536 INFO:teuthology.orchestra.run.smithi079.stdout:(132/159): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-06-03T02:02:13.563 INFO:teuthology.orchestra.run.smithi079.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 487 kB/s | 58 kB 00:00 2024-06-03T02:02:13.589 INFO:teuthology.orchestra.run.smithi079.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 319 kB/s | 16 kB 00:00 2024-06-03T02:02:13.614 INFO:teuthology.orchestra.run.smithi079.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-06-03T02:02:13.631 INFO:teuthology.orchestra.run.smithi079.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 687 kB/s | 28 kB 00:00 2024-06-03T02:02:13.656 INFO:teuthology.orchestra.run.smithi079.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 749 kB/s | 31 kB 00:00 2024-06-03T02:02:13.690 INFO:teuthology.orchestra.run.smithi079.stdout:(138/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-06-03T02:02:13.715 INFO:teuthology.orchestra.run.smithi079.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-06-03T02:02:13.740 INFO:teuthology.orchestra.run.smithi079.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 725 kB/s | 36 kB 00:00 2024-06-03T02:02:13.765 INFO:teuthology.orchestra.run.smithi079.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-06-03T02:02:13.783 INFO:teuthology.orchestra.run.smithi079.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 768 kB/s | 51 kB 00:00 2024-06-03T02:02:13.808 INFO:teuthology.orchestra.run.smithi079.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-06-03T02:02:13.841 INFO:teuthology.orchestra.run.smithi079.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-06-03T02:02:13.858 INFO:teuthology.orchestra.run.smithi079.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 406 kB/s | 20 kB 00:00 2024-06-03T02:02:13.909 INFO:teuthology.orchestra.run.smithi079.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-06-03T02:02:13.942 INFO:teuthology.orchestra.run.smithi079.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-06-03T02:02:14.017 INFO:teuthology.orchestra.run.smithi079.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-06-03T02:02:14.451 INFO:teuthology.orchestra.run.smithi079.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-06-03T02:02:14.560 INFO:teuthology.orchestra.run.smithi079.stdout:(150/159): NetworkManager-1.47.91-1.el9.x86_64. 4.9 MB/s | 2.3 MB 00:00 2024-06-03T02:02:14.610 INFO:teuthology.orchestra.run.smithi079.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 843 kB/s | 41 kB 00:00 2024-06-03T02:02:14.685 INFO:teuthology.orchestra.run.smithi079.stdout:(152/159): librados2-19.0.0-2946.g3d89accf.el9. 4.4 MB/s | 3.4 MB 00:00 2024-06-03T02:02:14.719 INFO:teuthology.orchestra.run.smithi079.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 2.2 MB/s | 248 kB 00:00 2024-06-03T02:02:14.811 INFO:teuthology.orchestra.run.smithi079.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 2.2 MB/s | 207 kB 00:00 2024-06-03T02:02:14.886 INFO:teuthology.orchestra.run.smithi079.stdout:(155/159): NetworkManager-libnm-1.47.91-1.el9.x 4.2 MB/s | 1.8 MB 00:00 2024-06-03T02:02:14.903 INFO:teuthology.orchestra.run.smithi079.stdout:(156/159): elfutils-debuginfod-client-0.191-4.e 169 kB/s | 37 kB 00:00 2024-06-03T02:02:14.937 INFO:teuthology.orchestra.run.smithi079.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.0 MB/s | 260 kB 00:00 2024-06-03T02:02:15.120 INFO:teuthology.orchestra.run.smithi079.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-06-03T02:02:15.770 INFO:teuthology.orchestra.run.smithi079.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 2.4 MB 00:00 2024-06-03T02:02:15.776 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-06-03T02:02:15.777 INFO:teuthology.orchestra.run.smithi079.stdout:Total 11 MB/s | 225 MB 00:20 2024-06-03T02:02:18.916 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-06-03T02:02:19.007 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-03T02:02:19.017 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-06-03T02:02:19.044 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-03T02:02:19.098 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-03T02:02:19.132 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-03T02:02:19.158 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-03T02:02:19.200 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-03T02:02:19.268 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-06-03T02:02:19.268 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-06-03T02:02:19.519 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-06-03T02:02:19.583 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-03T02:02:19.589 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-03T02:02:19.627 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-03T02:02:19.642 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T02:02:19.680 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-03T02:02:19.714 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-03T02:02:19.740 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-03T02:02:19.764 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T02:02:19.783 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-03T02:02:19.802 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T02:02:19.893 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T02:02:19.922 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T02:02:19.955 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-03T02:02:20.115 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-03T02:02:20.130 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-03T02:02:20.179 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-03T02:02:20.207 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T02:02:20.225 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T02:02:20.310 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T02:02:20.332 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T02:02:20.338 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T02:02:20.367 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T02:02:20.378 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T02:02:20.400 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T02:02:20.424 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T02:02:20.442 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T02:02:20.471 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T02:02:20.504 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-03T02:02:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T02:02:20.708 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-03T02:02:20.715 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T02:02:20.761 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-03T02:02:20.789 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T02:02:20.873 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T02:02:20.876 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T02:02:20.906 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T02:02:20.933 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T02:02:20.969 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T02:02:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T02:02:21.001 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T02:02:21.032 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T02:02:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T02:02:21.182 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T02:02:21.290 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T02:02:21.310 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T02:02:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-03T02:02:21.471 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T02:02:21.491 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-03T02:02:21.525 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-03T02:02:21.531 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T02:02:21.576 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-03T02:02:21.631 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T02:02:21.641 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-03T02:02:21.676 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-03T02:02:21.708 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T02:02:21.719 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-03T02:02:21.735 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-06-03T02:02:21.736 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-06-03T02:02:21.785 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-03T02:02:21.813 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-03T02:02:21.881 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-03T02:02:21.912 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T02:02:21.935 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-03T02:02:21.966 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-03T02:02:22.010 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-03T02:02:22.022 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-03T02:02:22.087 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-03T02:02:22.120 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-03T02:02:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-03T02:02:22.155 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-03T02:02:22.165 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-03T02:02:22.190 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-03T02:02:22.193 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-03T02:02:22.255 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-03T02:02:22.272 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-03T02:02:22.289 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-03T02:02:22.317 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-03T02:02:22.357 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-03T02:02:22.390 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-03T02:02:22.391 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-03T02:02:22.418 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-03T02:02:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-03T02:02:22.525 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-03T02:02:22.532 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-03T02:02:22.571 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-03T02:02:22.577 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-03T02:02:22.607 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-03T02:02:22.684 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-03T02:02:22.717 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-03T02:02:22.770 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-03T02:02:22.798 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-03T02:02:22.870 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-03T02:02:22.933 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-03T02:02:23.014 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-03T02:02:23.104 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-03T02:02:23.158 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-03T02:02:23.181 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-03T02:02:23.216 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-03T02:02:23.218 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-03T02:02:23.254 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-03T02:02:23.295 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-03T02:02:23.333 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-03T02:02:23.374 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-03T02:02:23.528 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-03T02:02:23.577 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-03T02:02:23.632 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-03T02:02:23.704 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-03T02:02:23.736 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-03T02:02:23.768 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-03T02:02:23.772 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-03T02:02:23.819 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-03T02:02:23.853 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-03T02:02:23.863 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-03T02:02:23.881 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-03T02:02:23.907 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-03T02:02:23.917 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-03T02:02:23.949 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-03T02:02:23.966 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-03T02:02:24.005 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-03T02:02:24.046 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-03T02:02:24.084 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-03T02:02:24.125 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-03T02:02:24.167 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-03T02:02:24.194 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-03T02:02:24.249 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-03T02:02:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-03T02:02:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-03T02:02:24.304 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-03T02:02:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-03T02:02:24.349 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-03T02:02:24.377 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-03T02:02:24.408 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-03T02:02:24.444 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-03T02:02:24.491 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-03T02:02:24.527 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-03T02:02:24.571 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-03T02:02:24.574 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-03T02:02:24.613 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-03T02:02:24.652 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-03T02:02:24.750 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-03T02:02:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-03T02:02:24.789 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-03T02:02:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-03T02:02:24.830 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-03T02:02:24.842 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-03T02:02:24.889 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-03T02:02:24.917 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-03T02:02:25.040 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-03T02:02:25.115 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-03T02:02:25.189 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-03T02:02:25.281 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-03T02:02:25.310 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-03T02:02:25.313 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-03T02:02:25.338 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-03T02:02:25.353 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-03T02:02:25.395 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-03T02:02:25.422 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-03T02:02:25.428 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-03T02:02:25.463 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-03T02:02:25.549 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-03T02:02:25.577 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-03T02:02:25.593 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-03T02:02:25.624 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-03T02:02:25.671 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-03T02:02:25.768 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-03T02:02:25.826 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-03T02:02:25.845 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-03T02:02:25.881 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-03T02:02:25.923 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-03T02:02:25.963 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-03T02:02:26.003 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-03T02:02:26.032 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-03T02:02:26.075 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-03T02:02:26.117 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-03T02:02:26.158 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-03T02:02:26.193 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-03T02:02:26.281 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-03T02:02:26.323 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-03T02:02:26.379 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-03T02:02:26.427 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-03T02:02:26.533 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-03T02:02:26.546 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-03T02:02:26.590 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-03T02:02:26.590 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-03T02:02:26.623 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-03T02:02:26.723 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-03T02:02:26.778 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-03T02:02:26.823 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-03T02:02:26.910 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-03T02:02:26.942 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-03T02:02:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-03T02:02:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-03T02:02:27.226 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-03T02:02:27.319 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-03T02:02:27.370 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-03T02:02:27.403 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-03T02:02:27.512 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-03T02:02:27.575 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-03T02:02:27.620 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-03T02:02:27.708 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-03T02:02:27.748 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-03T02:02:27.862 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-03T02:02:27.924 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-03T02:02:27.962 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-03T02:02:27.971 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-03T02:02:28.047 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-03T02:02:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-06-03T02:02:28.617 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-03T02:02:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-03T02:02:28.697 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-03T02:02:28.808 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-03T02:02:28.819 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-03T02:02:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-03T02:02:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-03T02:02:29.006 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-03T02:02:29.188 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-03T02:02:29.277 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-03T02:02:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-03T02:02:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-03T02:02:29.441 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-03T02:02:29.551 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T02:02:29.650 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-03T02:02:29.692 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T02:02:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T02:02:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T02:02:29.783 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T02:02:29.814 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-03T02:02:29.850 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-03T02:02:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T02:02:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T02:02:30.005 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-03T02:02:30.065 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-03T02:02:30.165 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-03T02:02:30.193 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-03T02:02:30.226 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-03T02:02:30.315 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-03T02:02:30.320 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-03T02:02:30.345 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T02:02:30.382 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-03T02:02:30.407 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T02:02:30.408 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T02:02:30.430 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-03T02:02:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-03T02:02:30.560 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T02:02:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T02:02:30.600 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T02:02:30.688 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T02:02:30.711 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T02:02:30.719 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-03T02:02:30.743 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T02:02:30.763 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-03T02:02:30.817 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T02:02:31.104 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T02:02:31.234 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-03T02:02:31.268 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T02:02:31.339 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T02:02:31.369 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-03T02:02:31.372 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T02:02:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T02:02:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T02:02:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T02:02:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T02:02:32.270 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T02:02:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-03T02:02:32.586 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-03T02:02:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-03T02:02:32.694 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T02:02:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-03T02:02:32.727 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T02:02:32.814 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-03T02:02:32.863 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-03T02:02:32.934 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-03T02:02:32.974 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-03T02:02:33.007 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-03T02:02:33.117 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-03T02:02:33.187 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-03T02:02:33.226 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-03T02:02:33.272 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T02:02:33.280 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-03T02:02:33.322 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T02:02:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-03T02:02:33.415 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T02:02:33.415 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T02:02:33.416 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T02:02:33.416 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:33.439 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-03T02:02:33.449 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T02:02:33.492 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-03T02:02:33.529 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-03T02:02:33.613 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T02:02:33.648 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T02:02:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-03T02:02:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-03T02:02:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-03T02:02:34.009 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-03T02:02:34.103 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-03T02:02:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T02:02:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T02:02:34.310 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T02:02:34.310 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T02:02:34.310 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T02:02:34.310 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:34.345 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T02:02:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-03T02:02:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-03T02:02:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-03T02:02:34.974 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-03T02:02:35.089 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-03T02:02:35.147 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-03T02:02:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-03T02:02:35.428 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-03T02:02:35.496 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-03T02:02:35.598 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-03T02:02:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-03T02:02:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-03T02:02:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-03T02:02:35.854 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-03T02:02:35.893 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-03T02:02:35.955 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-03T02:02:36.042 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-03T02:02:36.249 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-03T02:02:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-03T02:02:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-03T02:02:36.472 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-03T02:02:36.520 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-03T02:02:36.567 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-03T02:02:36.602 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-03T02:02:36.625 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-03T02:02:36.796 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-03T02:02:36.975 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-03T02:02:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-03T02:02:37.062 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-03T02:02:37.100 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-03T02:02:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-03T02:02:37.211 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-03T02:02:37.253 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-03T02:02:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-03T02:02:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-03T02:02:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-03T02:02:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-03T02:02:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-03T02:02:37.671 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-03T02:02:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-03T02:02:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-03T02:02:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-03T02:02:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-03T02:02:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-03T02:02:39.008 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-03T02:02:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-03T02:02:39.163 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-03T02:02:39.226 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-03T02:02:39.358 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-03T02:02:39.844 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-03T02:02:39.895 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-03T02:02:40.137 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-03T02:02:40.167 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-03T02:02:40.277 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-03T02:02:41.155 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-03T02:02:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-03T02:02:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-03T02:02:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-03T02:02:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-03T02:02:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-03T02:02:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-03T02:02:43.485 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T02:02:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T02:02:43.642 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-03T02:02:43.698 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-03T02:02:44.261 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-03T02:02:44.322 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T02:02:44.431 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T02:02:44.457 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-03T02:02:45.051 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T02:02:45.052 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-06-03T02:02:45.052 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-06-03T02:02:45.052 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-06-03T02:02:45.052 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-06-03T02:02:45.052 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-06-03T02:02:45.052 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-06-03T02:02:45.052 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-06-03T02:02:45.052 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:45.215 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T02:02:45.230 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T02:02:45.231 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:45.231 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-03T02:02:45.231 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:45.989 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T02:02:45.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-06-03T02:02:45.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-06-03T02:02:45.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-06-03T02:02:45.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-06-03T02:02:45.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-06-03T02:02:45.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-06-03T02:02:45.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-06-03T02:02:45.989 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:46.154 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T02:02:46.171 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T02:02:46.171 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:46.171 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-03T02:02:46.171 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:46.710 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T02:02:46.735 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T02:02:46.736 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:46.736 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T02:02:46.736 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T02:02:46.736 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T02:02:46.736 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:46.961 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T02:02:46.982 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T02:02:46.983 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:46.983 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T02:02:46.983 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T02:02:46.983 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T02:02:46.983 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:47.092 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T02:02:47.137 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T02:02:47.419 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T02:02:47.442 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T02:02:47.442 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:47.442 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T02:02:47.442 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T02:02:47.442 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T02:02:47.442 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-03T02:02:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-03T02:02:47.649 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T02:02:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-03T02:02:47.674 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T02:02:47.674 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:47.674 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T02:02:47.674 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T02:02:47.674 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T02:02:47.674 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:47.768 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T02:02:47.815 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T02:02:47.837 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-03T02:02:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-03T02:02:47.899 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T02:02:47.922 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T02:02:47.923 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:47.923 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T02:02:47.923 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T02:02:47.923 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T02:02:47.923 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:47.971 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-03T02:02:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T02:02:47.978 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T02:02:47.979 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T02:02:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:02:48.015 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T02:02:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-03T02:02:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-03T02:02:48.253 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-03T02:02:48.285 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-03T02:02:48.313 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-03T02:02:48.367 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T02:02:48.390 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T02:02:48.390 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:48.390 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T02:02:48.390 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T02:02:48.390 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T02:02:48.390 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:48.506 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-03T02:02:48.570 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-03T02:02:48.626 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-03T02:02:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T02:02:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T02:02:48.812 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-03T02:02:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-03T02:02:48.860 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-03T02:02:48.896 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-03T02:02:48.926 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T02:02:48.937 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-03T02:02:49.025 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-03T02:02:49.166 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-03T02:02:49.209 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-03T02:02:49.242 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-03T02:02:49.269 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-03T02:02:49.697 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T02:02:49.726 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T02:02:49.766 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-03T02:02:49.836 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-03T02:02:49.874 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T02:02:49.952 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T02:02:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T02:02:50.909 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T02:02:50.942 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T02:02:50.950 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T02:02:50.978 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T02:02:51.079 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T02:02:51.241 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-03T02:02:51.271 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T02:02:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T02:02:51.294 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:51.294 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T02:02:51.294 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T02:02:51.294 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T02:02:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T02:02:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T02:02:51.448 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-03T02:02:51.495 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-03T02:02:51.908 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T02:02:51.934 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T02:02:52.037 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T02:02:52.215 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-03T02:02:52.245 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T02:02:52.268 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T02:02:52.268 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:52.268 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T02:02:52.268 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T02:02:52.269 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T02:02:52.269 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:52.312 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T02:02:52.323 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T02:02:52.386 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-03T02:02:52.421 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-03T02:02:52.424 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T02:02:52.447 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T02:02:52.447 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:52.447 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T02:02:52.447 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T02:02:52.448 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T02:02:52.448 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:52.478 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-03T02:02:52.492 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T02:02:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T02:02:52.514 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:52.514 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T02:02:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:52.559 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T02:02:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T02:02:52.582 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:52.582 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-03T02:02:52.582 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T02:02:52.582 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T02:02:52.582 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:52.885 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T02:02:52.907 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T02:02:52.907 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:52.907 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T02:02:52.907 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T02:02:52.907 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T02:02:52.907 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:02:53.373 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-03T02:02:53.406 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T02:02:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T02:02:53.430 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:53.430 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T02:02:53.430 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T02:02:53.430 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T02:02:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:53.474 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T02:02:53.496 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T02:02:53.496 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:53.496 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T02:02:53.496 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:53.541 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T02:02:53.563 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T02:02:53.563 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:53.563 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-03T02:02:53.564 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T02:02:53.564 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T02:02:53.564 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:53.867 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T02:02:53.889 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T02:02:53.889 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:02:53.889 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T02:02:53.889 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T02:02:53.889 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T02:02:53.889 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:02:57.405 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-03T02:02:57.446 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-03T02:02:57.495 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-03T02:02:57.527 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-03T02:02:57.563 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-03T02:02:57.747 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-03T02:02:57.817 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-03T02:02:57.849 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-03T02:02:57.889 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-03T02:02:57.919 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-03T02:02:57.966 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T02:02:58.411 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-03T02:02:58.461 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-03T02:02:58.502 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-03T02:02:58.534 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-03T02:02:58.561 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-03T02:02:58.587 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T02:02:58.621 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T02:02:58.641 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T02:02:58.755 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-03T02:02:58.808 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-03T02:02:58.840 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-03T02:02:58.880 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-03T02:02:58.910 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-03T02:02:58.957 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T02:02:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-03T02:02:59.567 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-03T02:02:59.581 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T02:02:59.608 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-03T02:02:59.622 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T02:02:59.640 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T02:02:59.688 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-03T02:02:59.729 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-03T02:02:59.753 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T02:02:59.778 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-03T02:02:59.805 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-03T02:02:59.805 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T02:02:59.889 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T02:02:59.914 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-03T02:02:59.914 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T02:02:59.998 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T02:03:00.023 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-03T02:03:00.028 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T02:03:00.028 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T02:03:00.209 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T02:03:00.209 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T02:03:00.293 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T02:03:00.326 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-03T02:03:00.390 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T02:03:00.470 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-03T02:03:00.559 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-03T02:03:00.599 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-03T02:03:00.690 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-03T02:03:00.729 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-03T02:03:00.753 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T02:03:00.779 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-03T02:03:00.805 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-03T02:03:00.805 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T02:03:00.890 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T02:03:00.914 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-03T02:03:00.914 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T02:03:01.015 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T02:03:01.040 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-03T02:03:01.045 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T02:03:01.045 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T02:03:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T02:03:01.157 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2024-06-03T02:03:01.157 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2024-06-03T02:03:01.157 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2024-06-03T02:03:01.157 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2024-06-03T02:03:01.157 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2024-06-03T02:03:01.157 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /root 2024-06-03T02:03:01.157 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2024-06-03T02:03:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:01.225 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T02:03:01.225 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T02:03:01.309 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T02:03:01.334 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-03T02:03:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T02:03:01.401 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T02:03:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T02:03:01.431 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:03:01.431 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-03T02:03:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T02:03:03.133 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T02:03:03.133 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:03:03.133 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T02:03:03.133 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T02:03:03.133 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T02:03:03.133 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:03.376 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T02:03:03.445 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T02:03:03.445 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:03:03.445 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T02:03:03.445 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T02:03:03.445 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T02:03:03.445 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:03.935 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T02:03:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T02:03:04.008 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:03:04.008 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T02:03:04.008 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T02:03:04.008 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T02:03:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:04.143 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-03T02:03:04.228 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-03T02:03:04.309 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-03T02:03:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-03T02:03:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-03T02:03:04.691 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-03T02:03:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-03T02:03:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-03T02:03:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-03T02:03:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-03T02:03:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-03T02:03:05.688 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T02:03:05.732 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T02:03:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-03T02:03:05.898 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-03T02:03:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T02:03:05.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-03T02:03:05.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-03T02:03:05.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-03T02:03:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-03T02:03:05.941 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-03T02:03:05.942 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-03T02:03:05.943 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-03T02:03:05.944 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-03T02:03:05.945 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-03T02:03:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-03T02:03:05.947 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-03T02:03:05.948 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-03T02:03:05.949 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-03T02:03:07.232 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T02:03:07.278 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T02:03:07.296 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-03T02:03:07.297 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-03T02:03:07.298 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-03T02:03:07.299 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-03T02:03:07.299 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-03T02:03:07.299 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-03T02:03:07.311 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-03T02:03:07.311 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-03T02:03:07.311 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-03T02:03:07.311 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-03T02:03:07.313 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-03T02:03:07.314 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-03T02:03:07.315 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-03T02:03:07.316 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-03T02:03:07.317 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-03T02:03:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-03T02:03:07.319 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-03T02:03:07.320 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-03T02:03:07.321 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-03T02:03:07.322 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-03T02:03:07.323 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-03T02:03:07.323 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-03T02:03:07.323 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-03T02:03:07.323 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-03T02:03:08.375 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T02:03:08.411 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T02:03:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T02:03:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-03T02:03:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:03:08.609 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-06-03T02:03:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T02:03:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-06-03T02:03:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T02:03:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-06-03T02:03:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-03T02:03:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T02:03:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T02:03:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T02:03:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T02:03:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-03T02:03:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-03T02:03:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T02:03:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-03T02:03:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-03T02:03:08.622 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-06-03T02:03:08.745 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-03T02:03:08.772 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T02:03:08.850 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T02:03:08.850 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:03:08.850 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T02:03:08.850 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T02:03:08.850 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T02:03:08.850 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T02:03:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T02:03:09.051 DEBUG:teuthology.parallel:result is None 2024-06-03T02:03:09.103 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-03T02:03:09.175 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-03T02:03:10.163 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-03T02:03:10.174 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-03T02:03:10.174 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:03:10.174 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-03T02:03:10.175 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.2-8.el9.noarch 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.7.4-5.el9.x86_64 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.176 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.177 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-03T02:03:10.178 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.4-9.el9.x86_64 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T02:03:10.179 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-03T02:03:10.180 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T02:03:10.181 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-03T02:03:10.181 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-03T02:03:10.181 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T02:03:10.181 INFO:teuthology.orchestra.run.smithi135.stdout: lua-5.4.4-4.el9.x86_64 2024-06-03T02:03:10.181 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-03T02:03:10.181 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-03T02:03:10.181 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T02:03:10.181 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-13.14-1.el9.x86_64 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T02:03:10.182 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-03T02:03:10.183 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T02:03:10.184 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T02:03:10.185 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T02:03:10.185 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T02:03:10.185 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T02:03:10.185 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T02:03:10.185 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T02:03:10.185 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T02:03:10.185 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T02:03:10.185 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T02:03:10.185 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-03T02:03:10.186 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T02:03:10.186 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T02:03:10.186 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T02:03:10.186 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T02:03:10.186 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-03T02:03:10.186 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T02:03:10.186 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T02:03:10.186 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T02:03:10.186 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T02:03:10.187 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:10.188 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T02:03:10.189 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-03T02:03:10.189 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T02:03:10.189 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-03T02:03:10.189 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-03T02:03:10.189 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T02:03:10.189 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-03T02:03:10.189 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:03:10.189 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-06-03T02:03:10.192 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T02:03:10.262 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T02:03:10.262 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:03:10.262 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T02:03:10.262 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T02:03:10.262 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T02:03:10.262 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:10.326 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T02:03:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T02:03:10.396 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:03:10.396 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T02:03:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T02:03:10.553 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T02:03:10.553 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:03:10.553 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-03T02:03:10.553 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T02:03:10.553 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T02:03:10.553 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:10.634 DEBUG:teuthology.parallel:result is None 2024-06-03T02:03:10.928 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T02:03:11.009 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T02:03:11.009 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T02:03:11.009 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T02:03:11.009 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T02:03:11.009 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T02:03:11.009 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:15.908 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-03T02:03:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-03T02:03:16.007 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-03T02:03:16.040 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-03T02:03:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-03T02:03:16.336 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-03T02:03:16.397 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-03T02:03:16.431 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-03T02:03:16.469 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-03T02:03:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-03T02:03:16.572 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T02:03:17.669 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T02:03:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T02:03:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T02:03:19.126 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-03T02:03:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-03T02:03:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-03T02:03:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-03T02:03:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-03T02:03:19.530 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T02:03:19.564 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-03T02:03:19.605 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-03T02:03:19.605 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T02:03:19.739 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T02:03:19.793 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-03T02:03:19.793 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T02:03:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T02:03:19.969 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-03T02:03:20.003 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T02:03:20.004 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T02:03:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T02:03:20.324 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T02:03:20.461 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T02:03:20.522 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-03T02:03:20.645 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T02:03:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T02:03:20.905 INFO:teuthology.orchestra.run.smithi079.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-06-03T02:03:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-03T02:03:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-03T02:03:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-03T02:03:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-03T02:03:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-03T02:03:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-03T02:03:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-03T02:03:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T02:03:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-03T02:03:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-03T02:03:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-03T02:03:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-03T02:03:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-03T02:03:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-03T02:03:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-03T02:03:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-03T02:03:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-03T02:03:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-03T02:03:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-03T02:03:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-03T02:03:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-03T02:03:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-03T02:03:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-03T02:03:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-03T02:03:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-03T02:03:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-03T02:03:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-03T02:03:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-03T02:03:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-03T02:03:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-03T02:03:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-03T02:03:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-03T02:03:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.2-8.el9.noarch 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.7.4-5.el9.x86_64 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.4-9.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T02:03:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T02:03:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: lua-5.4.4-4.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T02:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-13.14-1.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T02:03:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T02:03:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T02:03:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T02:03:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T02:03:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T02:03:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-03T02:03:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T02:03:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-03T02:03:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-03T02:03:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T02:03:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-03T02:03:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:03:29.044 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-06-03T02:03:29.693 DEBUG:teuthology.parallel:result is None 2024-06-03T02:03:29.694 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:03:29.838 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T02:03:29.894 INFO:teuthology.orchestra.run.smithi079.stdout:19.0.0-2946.g3d89accf.el9 2024-06-03T02:03:29.894 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-03T02:03:29.895 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-03T02:03:29.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:03:30.113 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T02:03:30.139 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-2946.g3d89accf.el9 2024-06-03T02:03:30.139 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-03T02:03:30.139 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-03T02:03:30.141 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:03:30.361 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T02:03:30.387 INFO:teuthology.orchestra.run.smithi135.stdout:19.0.0-2946.g3d89accf.el9 2024-06-03T02:03:30.387 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-03T02:03:30.387 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-03T02:03:30.389 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-03T02:03:30.389 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:03:30.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T02:03:30.466 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:03:30.466 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T02:03:30.496 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:03:30.496 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T02:03:30.527 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-03T02:03:30.527 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:03:30.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T02:03:30.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T02:03:30.714 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:03:30.714 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T02:03:30.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T02:03:30.810 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:03:30.810 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T02:03:30.839 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T02:03:30.905 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-03T02:03:30.905 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:03:30.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T02:03:30.967 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T02:03:31.069 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:03:31.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T02:03:31.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T02:03:31.156 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:03:31.156 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T02:03:31.179 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T02:03:31.242 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-03T02:03:31.243 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:03:31.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T02:03:31.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T02:03:31.410 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:03:31.410 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T02:03:31.434 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T02:03:31.498 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:03:31.498 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T02:03:31.522 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T02:03:31.585 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-03T02:03:31.679 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-06-03T02:03:31.679 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:03:31.680 INFO:tasks.cephadm:Cluster fsid is 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:03:31.680 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-03T02:03:31.680 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.79', 'mon.b': '172.21.15.123', 'mon.c': '172.21.15.135'} 2024-06-03T02:03:31.680 INFO:tasks.cephadm:First mon is mon.a on smithi079 2024-06-03T02:03:31.680 INFO:tasks.cephadm:First mgr is x 2024-06-03T02:03:31.680 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-03T02:03:31.680 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname $(hostname -s) 2024-06-03T02:03:31.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-06-03T02:03:31.775 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname $(hostname -s) 2024-06-03T02:03:31.800 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-03T02:03:31.800 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:03:31.940 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-03T02:03:32.081 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:03:32.082 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-03T02:03:32.082 DEBUG:teuthology.orchestra.run.smithi079:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T02:03:33.229 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 3 02:03 /home/ubuntu/cephtest/cephadm 2024-06-03T02:03:33.229 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T02:03:34.409 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 3 02:03 /home/ubuntu/cephtest/cephadm 2024-06-03T02:03:34.410 DEBUG:teuthology.orchestra.run.smithi135:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T02:03:35.890 INFO:teuthology.orchestra.run.smithi135.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 3 02:03 /home/ubuntu/cephtest/cephadm 2024-06-03T02:03:35.890 DEBUG:teuthology.orchestra.run.smithi079:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T02:03:35.951 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T02:03:35.970 DEBUG:teuthology.orchestra.run.smithi135:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T02:03:36.004 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-03T02:03:36.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-03T02:03:36.011 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-03T02:03:36.013 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-03T02:03:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T02:03:36.828 INFO:teuthology.orchestra.run.smithi135.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T02:03:37.200 INFO:teuthology.orchestra.run.smithi079.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T02:04:12.802 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:04:12.803 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T02:04:12.803 INFO:teuthology.orchestra.run.smithi079.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-03T02:04:12.803 INFO:teuthology.orchestra.run.smithi079.stdout: "repo_digests": [ 2024-06-03T02:04:12.803 INFO:teuthology.orchestra.run.smithi079.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-03T02:04:12.803 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:04:12.803 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:04:18.786 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-03T02:04:18.787 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T02:04:18.787 INFO:teuthology.orchestra.run.smithi135.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-03T02:04:18.787 INFO:teuthology.orchestra.run.smithi135.stdout: "repo_digests": [ 2024-06-03T02:04:18.787 INFO:teuthology.orchestra.run.smithi135.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-03T02:04:18.787 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-06-03T02:04:18.787 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-03T02:04:30.608 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-03T02:04:30.608 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T02:04:30.608 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-03T02:04:30.608 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-06-03T02:04:30.609 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-03T02:04:30.609 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-06-03T02:04:30.609 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-03T02:04:30.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph 2024-06-03T02:04:30.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-06-03T02:04:30.760 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph 2024-06-03T02:04:30.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /etc/ceph 2024-06-03T02:04:30.878 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-06-03T02:04:30.903 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /etc/ceph 2024-06-03T02:04:30.930 INFO:tasks.cephadm:Writing seed config... 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-03T02:04:30.931 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-03T02:04:30.932 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-03T02:04:30.933 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-03T02:04:30.934 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-03T02:04:30.934 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-03T02:04:30.934 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-03T02:04:30.934 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-03T02:04:30.934 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-03T02:04:30.934 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-03T02:04:30.934 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-03T02:04:30.934 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-06-03T02:04:30.935 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:04:30.935 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-03T02:04:30.991 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7a1e9ad6-214d-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-03T02:04:30.992 DEBUG:teuthology.orchestra.run.smithi079:mon.a> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.a.service 2024-06-03T02:04:31.039 DEBUG:teuthology.orchestra.run.smithi079:mgr.x> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-03T02:04:31.086 INFO:tasks.cephadm:Bootstrapping... 2024-06-03T02:04:31.086 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.79 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-03T02:04:31.356 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-06-03T02:04:31.357 INFO:teuthology.orchestra.run.smithi079.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', '7a1e9ad6-214d-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.79', '--skip-admin-label'] 2024-06-03T02:04:31.357 INFO:teuthology.orchestra.run.smithi079.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-03T02:04:31.357 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying podman|docker is present... 2024-06-03T02:04:31.413 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stdout 5.0.2 2024-06-03T02:04:31.413 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying lvm2 is present... 2024-06-03T02:04:31.414 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying time synchronization is in place... 2024-06-03T02:04:31.428 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-03T02:04:31.428 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-03T02:04:31.441 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-03T02:04:31.442 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-06-03T02:04:31.458 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout enabled 2024-06-03T02:04:31.472 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout active 2024-06-03T02:04:31.472 INFO:teuthology.orchestra.run.smithi079.stdout:Unit chronyd.service is enabled and running 2024-06-03T02:04:31.473 INFO:teuthology.orchestra.run.smithi079.stdout:Repeating the final host check... 2024-06-03T02:04:31.523 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stdout 5.0.2 2024-06-03T02:04:31.523 INFO:teuthology.orchestra.run.smithi079.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-03T02:04:31.523 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl is present 2024-06-03T02:04:31.523 INFO:teuthology.orchestra.run.smithi079.stdout:lvcreate is present 2024-06-03T02:04:31.539 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-03T02:04:31.539 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-03T02:04:31.555 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-03T02:04:31.555 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-06-03T02:04:31.575 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout enabled 2024-06-03T02:04:31.593 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout active 2024-06-03T02:04:31.593 INFO:teuthology.orchestra.run.smithi079.stdout:Unit chronyd.service is enabled and running 2024-06-03T02:04:31.593 INFO:teuthology.orchestra.run.smithi079.stdout:Host looks OK 2024-06-03T02:04:31.593 INFO:teuthology.orchestra.run.smithi079.stdout:Cluster fsid: 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:31.594 INFO:teuthology.orchestra.run.smithi079.stdout:Acquiring lock 140110337717632 on /run/cephadm/7a1e9ad6-214d-11ef-bc9b-c7b262605968.lock 2024-06-03T02:04:31.594 INFO:teuthology.orchestra.run.smithi079.stdout:Lock 140110337717632 acquired on /run/cephadm/7a1e9ad6-214d-11ef-bc9b-c7b262605968.lock 2024-06-03T02:04:31.594 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying IP 172.21.15.79 port 3300 ... 2024-06-03T02:04:31.595 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying IP 172.21.15.79 port 6789 ... 2024-06-03T02:04:31.595 INFO:teuthology.orchestra.run.smithi079.stdout:Base mon IP(s) is [172.21.15.79:3300, 172.21.15.79:6789], mon addrv is [v2:172.21.15.79:3300,v1:172.21.15.79:6789] 2024-06-03T02:04:31.603 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.79 metric 100 2024-06-03T02:04:31.603 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.79 metric 100 2024-06-03T02:04:31.610 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-03T02:04:31.610 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-03T02:04:31.610 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1763sec hoplimit 64 pref medium 2024-06-03T02:04:31.617 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-03T02:04:31.617 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-03T02:04:31.617 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-03T02:04:31.617 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-03T02:04:31.617 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce87/64 scope link 2024-06-03T02:04:31.617 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-03T02:04:31.618 INFO:teuthology.orchestra.run.smithi079.stdout:Mon IP `172.21.15.79` is in CIDR network `172.21.0.0/20` 2024-06-03T02:04:31.618 INFO:teuthology.orchestra.run.smithi079.stdout:Mon IP `172.21.15.79` is in CIDR network `172.21.0.0/20` 2024-06-03T02:04:31.618 INFO:teuthology.orchestra.run.smithi079.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-03T02:04:31.619 INFO:teuthology.orchestra.run.smithi079.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-03T02:04:31.620 INFO:teuthology.orchestra.run.smithi079.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T02:04:32.251 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-03T02:04:32.252 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T02:04:32.252 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Getting image source signatures 2024-06-03T02:04:32.252 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-03T02:04:32.252 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-03T02:04:32.252 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-03T02:04:32.252 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-03T02:04:33.582 INFO:teuthology.orchestra.run.smithi079.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-03T02:04:33.583 INFO:teuthology.orchestra.run.smithi079.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-03T02:04:33.583 INFO:teuthology.orchestra.run.smithi079.stdout:Extracting ceph user uid/gid from container image... 2024-06-03T02:04:34.752 INFO:teuthology.orchestra.run.smithi079.stdout:stat: stdout 167 167 2024-06-03T02:04:34.752 INFO:teuthology.orchestra.run.smithi079.stdout:Creating initial keys... 2024-06-03T02:04:35.939 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph-authtool: stdout AQCzJF1m1BUSEBAAusLXelQLpKvt6Hb5LTtwoA== 2024-06-03T02:04:37.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph-authtool: stdout AQC0JF1m+LlyGRAA/fQvspWFZu5hcmqqMfoS4A== 2024-06-03T02:04:38.421 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph-authtool: stdout AQC1JF1m6FnGIRAAwH2GjNCxylFQRcIJmLGpKw== 2024-06-03T02:04:38.423 INFO:teuthology.orchestra.run.smithi079.stdout:Creating initial monmap... 2024-06-03T02:04:39.510 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-03T02:04:39.510 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-03T02:04:39.510 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:39.510 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-03T02:04:39.510 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool for a [v2:172.21.15.79:3300,v1:172.21.15.79:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-03T02:04:39.511 INFO:teuthology.orchestra.run.smithi079.stdout:setting min_mon_release = quincy 2024-06-03T02:04:39.511 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: set fsid to 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:39.511 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-03T02:04:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:04:39.511 INFO:teuthology.orchestra.run.smithi079.stdout:Creating mon... 2024-06-03T02:04:40.636 INFO:teuthology.orchestra.run.smithi079.stdout:create mon.a on 2024-06-03T02:04:41.012 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-03T02:04:41.255 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-03T02:04:41.552 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968.target. 2024-06-03T02:04:41.552 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968.target. 2024-06-03T02:04:41.946 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.a 2024-06-03T02:04:41.947 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-03T02:04:42.227 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968.target.wants/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@.service. 2024-06-03T02:04:43.006 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:42 smithi079 podman[31428]: 2024-06-03 02:04:42.612314705 +0000 UTC m=+0.133329975 container create 22398e90b3818e69731f23e1b90d2feeeadb0dccab9a941cdb41cf8a90e63b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-03T02:04:43.228 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T02:04:43.229 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout disabled 2024-06-03T02:04:43.246 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T02:04:43.246 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-06-03T02:04:43.246 INFO:teuthology.orchestra.run.smithi079.stdout:firewalld.service is not enabled 2024-06-03T02:04:43.246 INFO:teuthology.orchestra.run.smithi079.stdout:Not possible to enable service . firewalld.service is not available 2024-06-03T02:04:43.247 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mon to start... 2024-06-03T02:04:43.247 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mon... 2024-06-03T02:04:43.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:43 smithi079 podman[31428]: 2024-06-03 02:04:43.074567088 +0000 UTC m=+0.595582475 container init 22398e90b3818e69731f23e1b90d2feeeadb0dccab9a941cdb41cf8a90e63b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:04:43.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:43 smithi079 podman[31428]: 2024-06-03 02:04:43.085000166 +0000 UTC m=+0.606015483 container start 22398e90b3818e69731f23e1b90d2feeeadb0dccab9a941cdb41cf8a90e63b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-03T02:04:43.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:43 smithi079 ceph-mon[31446]: mkfs 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:43.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:43 smithi079 ceph-mon[31446]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-03T02:04:43.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:43 smithi079 bash[31428]: 22398e90b3818e69731f23e1b90d2feeeadb0dccab9a941cdb41cf8a90e63b38 2024-06-03T02:04:43.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:43 smithi079 systemd[1]: Started Ceph mon.a for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:04:44.313 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-03T02:04:44.313 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: monmap epoch 1 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: last_changed 2024-06-03T02:04:38.900256+0000 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: min_mon_release 19 (squid) 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: election_strategy: 1 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: fsmap 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: osdmap e1: 0 total, 0 up, 0 in 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: mgrmap e1: no daemons active 2024-06-03T02:04:44.314 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:44 smithi079 ceph-mon[31446]: from='client.? 172.21.15.79:0/971456378' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-03T02:04:44.653 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout cluster: 2024-06-03T02:04:44.654 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout id: 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:44.654 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-03T02:04:44.655 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:04:44.655 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout services: 2024-06-03T02:04:44.655 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.788832s) 2024-06-03T02:04:44.655 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-03T02:04:44.655 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-03T02:04:44.655 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:04:44.655 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout data: 2024-06-03T02:04:44.657 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-03T02:04:44.657 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-03T02:04:44.657 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-03T02:04:44.657 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout pgs: 2024-06-03T02:04:44.657 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:04:44.657 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.857+0000 7fe88e800700 1 Processor -- start 2024-06-03T02:04:44.657 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.857+0000 7fe88e800700 1 -- start start 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.858+0000 7fe88e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe8881083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.858+0000 7fe88e800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe888108910 con 0x7fe888107f90 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.858+0000 7fe887400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe8881083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.859+0000 7fe887400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe8881083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37064/0 (socket says 172.21.15.79:37064) 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.859+0000 7fe887400700 1 -- 172.21.15.79:0/2387553010 learned_addr learned my addr 172.21.15.79:0/2387553010 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.859+0000 7fe887400700 1 -- 172.21.15.79:0/2387553010 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8881090d0 con 0x7fe888107f90 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.859+0000 7fe887400700 1 --2- 172.21.15.79:0/2387553010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe8881083d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe8780098a0 tx=0x7fe878009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234ff0589d196a9a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.860+0000 7fe886a00700 1 -- 172.21.15.79:0/2387553010 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe878004160 con 0x7fe888107f90 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.860+0000 7fe886a00700 1 -- 172.21.15.79:0/2387553010 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878004300 con 0x7fe888107f90 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.860+0000 7fe886a00700 1 -- 172.21.15.79:0/2387553010 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8780036a0 con 0x7fe888107f90 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.860+0000 7fe88e800700 1 -- 172.21.15.79:0/2387553010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 msgr2=0x7fe8881083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:44.658 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.860+0000 7fe88e800700 1 --2- 172.21.15.79:0/2387553010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe8881083d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe8780098a0 tx=0x7fe878009bb0 comp rx=0 tx=0).stop 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.861+0000 7fe88e800700 1 -- 172.21.15.79:0/2387553010 shutdown_connections 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.861+0000 7fe88e800700 1 --2- 172.21.15.79:0/2387553010 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe8881083d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.861+0000 7fe88e800700 1 -- 172.21.15.79:0/2387553010 >> 172.21.15.79:0/2387553010 conn(0x7fe8881035e0 msgr2=0x7fe888105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.861+0000 7fe88e800700 1 -- 172.21.15.79:0/2387553010 shutdown_connections 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.861+0000 7fe88e800700 1 -- 172.21.15.79:0/2387553010 wait complete. 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.862+0000 7fe88e800700 1 Processor -- start 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.862+0000 7fe88e800700 1 -- start start 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.862+0000 7fe88e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe88819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.862+0000 7fe88e800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe888109480 con 0x7fe888107f90 2024-06-03T02:04:44.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.863+0000 7fe887400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe88819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.863+0000 7fe887400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe88819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37076/0 (socket says 172.21.15.79:37076) 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.863+0000 7fe887400700 1 -- 172.21.15.79:0/971456378 learned_addr learned my addr 172.21.15.79:0/971456378 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.863+0000 7fe887400700 1 -- 172.21.15.79:0/971456378 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88819dd10 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.863+0000 7fe887400700 1 --2- 172.21.15.79:0/971456378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe88819d7d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe878000c00 tx=0x7fe878004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.863+0000 7fe884c00700 1 -- 172.21.15.79:0/971456378 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe878003db0 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.863+0000 7fe884c00700 1 -- 172.21.15.79:0/971456378 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878011630 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.863+0000 7fe884c00700 1 -- 172.21.15.79:0/971456378 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe878011950 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.864+0000 7fe88e800700 1 -- 172.21.15.79:0/971456378 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88819dfa0 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.864+0000 7fe88e800700 1 -- 172.21.15.79:0/971456378 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8881a0c90 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.864+0000 7fe884c00700 1 -- 172.21.15.79:0/971456378 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe878011af0 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.864+0000 7fe884c00700 1 -- 172.21.15.79:0/971456378 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe87801c790 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.865+0000 7fe88e800700 1 -- 172.21.15.79:0/971456378 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888063af0 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.868+0000 7fe884c00700 1 -- 172.21.15.79:0/971456378 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe878032020 con 0x7fe888107f90 2024-06-03T02:04:44.663 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.947+0000 7fe88e800700 1 -- 172.21.15.79:0/971456378 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe8881a0e60 con 0x7fe888107f90 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.948+0000 7fe884c00700 1 -- 172.21.15.79:0/971456378 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe878004260 con 0x7fe888107f90 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.949+0000 7fe88e800700 1 -- 172.21.15.79:0/971456378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 msgr2=0x7fe88819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.949+0000 7fe88e800700 1 --2- 172.21.15.79:0/971456378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe88819d7d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe878000c00 tx=0x7fe878004750 comp rx=0 tx=0).stop 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.950+0000 7fe88e800700 1 -- 172.21.15.79:0/971456378 shutdown_connections 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.950+0000 7fe88e800700 1 --2- 172.21.15.79:0/971456378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe888107f90 0x7fe88819d7d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.950+0000 7fe88e800700 1 -- 172.21.15.79:0/971456378 >> 172.21.15.79:0/971456378 conn(0x7fe8881035e0 msgr2=0x7fe88807c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.950+0000 7fe88e800700 1 -- 172.21.15.79:0/971456378 shutdown_connections 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:43.950+0000 7fe88e800700 1 -- 172.21.15.79:0/971456378 wait complete. 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:mon is available 2024-06-03T02:04:44.664 INFO:teuthology.orchestra.run.smithi079.stdout:Assimilating anything we can from ceph.conf... 2024-06-03T02:04:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:45 smithi079 ceph-mon[31446]: from='client.? 172.21.15.79:0/1691658612' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-03T02:04:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:45 smithi079 ceph-mon[31446]: from='client.? 172.21.15.79:0/1691658612' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-03T02:04:46.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:04:46.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [global] 2024-06-03T02:04:46.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout fsid = 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:46.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.79:3300,v1:172.21.15.79:6789] 2024-06-03T02:04:46.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-03T02:04:46.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-03T02:04:46.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-03T02:04:46.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-03T02:04:46.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:04:46.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-03T02:04:46.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-03T02:04:46.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:04:46.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [osd] 2024-06-03T02:04:46.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.287+0000 7f5872800700 1 Processor -- start 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.287+0000 7f5872800700 1 -- start start 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.287+0000 7f5872800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c10ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.287+0000 7f5872800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f586c07c6f0 con 0x7f586c108990 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.288+0000 7f586b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c10ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:46.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.288+0000 7f586b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c10ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34440/0 (socket says 172.21.15.79:34440) 2024-06-03T02:04:46.095 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.288+0000 7f586b400700 1 -- 172.21.15.79:0/818320818 learned_addr learned my addr 172.21.15.79:0/818320818 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:46.095 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.289+0000 7f586b400700 1 -- 172.21.15.79:0/818320818 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c10b2c0 con 0x7f586c108990 2024-06-03T02:04:46.098 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.289+0000 7f586b400700 1 --2- 172.21.15.79:0/818320818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c10ad80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f585c0098a0 tx=0x7f585c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71141816f82477e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:46.098 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.289+0000 7f586a000700 1 -- 172.21.15.79:0/818320818 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f585c004160 con 0x7f586c108990 2024-06-03T02:04:46.098 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.289+0000 7f586a000700 1 -- 172.21.15.79:0/818320818 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c004300 con 0x7f586c108990 2024-06-03T02:04:46.098 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.289+0000 7f586a000700 1 -- 172.21.15.79:0/818320818 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f585c0036a0 con 0x7f586c108990 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.290+0000 7f5872800700 1 -- 172.21.15.79:0/818320818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 msgr2=0x7f586c10ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.290+0000 7f5872800700 1 --2- 172.21.15.79:0/818320818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c10ad80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f585c0098a0 tx=0x7f585c009bb0 comp rx=0 tx=0).stop 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.290+0000 7f5872800700 1 -- 172.21.15.79:0/818320818 shutdown_connections 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.290+0000 7f5872800700 1 --2- 172.21.15.79:0/818320818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c10ad80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.290+0000 7f5872800700 1 -- 172.21.15.79:0/818320818 >> 172.21.15.79:0/818320818 conn(0x7f586c102400 msgr2=0x7f586c104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.290+0000 7f5872800700 1 -- 172.21.15.79:0/818320818 shutdown_connections 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.290+0000 7f5872800700 1 -- 172.21.15.79:0/818320818 wait complete. 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.291+0000 7f5872800700 1 Processor -- start 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.292+0000 7f5872800700 1 -- start start 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.292+0000 7f5872800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.292+0000 7f5872800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f586c10b730 con 0x7f586c108990 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.293+0000 7f586b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:46.099 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.293+0000 7f586b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34452/0 (socket says 172.21.15.79:34452) 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.293+0000 7f586b400700 1 -- 172.21.15.79:0/1691658612 learned_addr learned my addr 172.21.15.79:0/1691658612 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.293+0000 7f586b400700 1 -- 172.21.15.79:0/1691658612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c19dd00 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.293+0000 7f586b400700 1 --2- 172.21.15.79:0/1691658612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c19d7c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f585c000c00 tx=0x7f585c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.293+0000 7f5863e00700 1 -- 172.21.15.79:0/1691658612 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f585c003db0 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.294+0000 7f5863e00700 1 -- 172.21.15.79:0/1691658612 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c011630 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.294+0000 7f5863e00700 1 -- 172.21.15.79:0/1691658612 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f585c011950 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.294+0000 7f5872800700 1 -- 172.21.15.79:0/1691658612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c19df90 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.294+0000 7f5872800700 1 -- 172.21.15.79:0/1691658612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c1a0c30 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.294+0000 7f5863e00700 1 -- 172.21.15.79:0/1691658612 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f585c011af0 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.294+0000 7f5863e00700 1 -- 172.21.15.79:0/1691658612 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f585c01c450 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.295+0000 7f5872800700 1 -- 172.21.15.79:0/1691658612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586c063af0 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.298+0000 7f5863e00700 1 -- 172.21.15.79:0/1691658612 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f585c032020 con 0x7f586c108990 2024-06-03T02:04:46.100 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.373+0000 7f5872800700 1 -- 172.21.15.79:0/1691658612 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f586c02de20 con 0x7f586c108990 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.377+0000 7f5863e00700 1 -- 172.21.15.79:0/1691658612 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f585c004260 con 0x7f586c108990 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.378+0000 7f5863e00700 1 -- 172.21.15.79:0/1691658612 <== mon.0 v2:172.21.15.79:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f585c01b450 con 0x7f586c108990 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.380+0000 7f5872800700 1 -- 172.21.15.79:0/1691658612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 msgr2=0x7f586c19d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.380+0000 7f5872800700 1 --2- 172.21.15.79:0/1691658612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c19d7c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f585c000c00 tx=0x7f585c004750 comp rx=0 tx=0).stop 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.380+0000 7f5872800700 1 -- 172.21.15.79:0/1691658612 shutdown_connections 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.380+0000 7f5872800700 1 --2- 172.21.15.79:0/1691658612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f586c108990 0x7f586c19d7c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.380+0000 7f5872800700 1 -- 172.21.15.79:0/1691658612 >> 172.21.15.79:0/1691658612 conn(0x7f586c102400 msgr2=0x7f586c103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.381+0000 7f5872800700 1 -- 172.21.15.79:0/1691658612 shutdown_connections 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:45.381+0000 7f5872800700 1 -- 172.21.15.79:0/1691658612 wait complete. 2024-06-03T02:04:46.101 INFO:teuthology.orchestra.run.smithi079.stdout:Generating new minimal ceph.conf... 2024-06-03T02:04:46.985 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:46 smithi079 ceph-mon[31446]: from='client.? 172.21.15.79:0/3229979883' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:04:47.394 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.653+0000 7f46ee400700 1 Processor -- start 2024-06-03T02:04:47.398 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.654+0000 7f46ee400700 1 -- start start 2024-06-03T02:04:47.398 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.654+0000 7f46ee400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e8107000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:47.398 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.654+0000 7f46ee400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81075d0 con 0x7f46e8106be0 2024-06-03T02:04:47.398 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.655+0000 7f46ed000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e8107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:47.398 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.655+0000 7f46ed000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e8107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34456/0 (socket says 172.21.15.79:34456) 2024-06-03T02:04:47.398 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.655+0000 7f46ed000700 1 -- 172.21.15.79:0/1855144437 learned_addr learned my addr 172.21.15.79:0/1855144437 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.655+0000 7f46ed000700 1 -- 172.21.15.79:0/1855144437 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e8107de0 con 0x7f46e8106be0 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.655+0000 7f46ed000700 1 --2- 172.21.15.79:0/1855144437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e8107000 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00bb40 tx=0x7f46dc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc3da4e7f550d63 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.656+0000 7f46e7400700 1 -- 172.21.15.79:0/1855144437 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46dc00dbe0 con 0x7f46e8106be0 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.656+0000 7f46e7400700 1 -- 172.21.15.79:0/1855144437 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46dc0044f0 con 0x7f46e8106be0 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.656+0000 7f46e7400700 1 -- 172.21.15.79:0/1855144437 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46dc010470 con 0x7f46e8106be0 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.656+0000 7f46ee400700 1 -- 172.21.15.79:0/1855144437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 msgr2=0x7f46e8107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.656+0000 7f46ee400700 1 --2- 172.21.15.79:0/1855144437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e8107000 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00bb40 tx=0x7f46dc00be50 comp rx=0 tx=0).stop 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.656+0000 7f46ee400700 1 -- 172.21.15.79:0/1855144437 shutdown_connections 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.656+0000 7f46ee400700 1 --2- 172.21.15.79:0/1855144437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e8107000 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.656+0000 7f46ee400700 1 -- 172.21.15.79:0/1855144437 >> 172.21.15.79:0/1855144437 conn(0x7f46e8102180 msgr2=0x7f46e81045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.657+0000 7f46ee400700 1 -- 172.21.15.79:0/1855144437 shutdown_connections 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.657+0000 7f46ee400700 1 -- 172.21.15.79:0/1855144437 wait complete. 2024-06-03T02:04:47.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.658+0000 7f46ee400700 1 Processor -- start 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.658+0000 7f46ee400700 1 -- start start 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.658+0000 7f46ee400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.658+0000 7f46ee400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46e807afc0 con 0x7f46e8106be0 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.658+0000 7f46ed000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.658+0000 7f46ed000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34468/0 (socket says 172.21.15.79:34468) 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.658+0000 7f46ed000700 1 -- 172.21.15.79:0/3229979883 learned_addr learned my addr 172.21.15.79:0/3229979883 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.659+0000 7f46ed000700 1 -- 172.21.15.79:0/3229979883 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e819e410 con 0x7f46e8106be0 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.659+0000 7f46ed000700 1 --2- 172.21.15.79:0/3229979883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e819ded0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00b340 tx=0x7f46dc00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.659+0000 7f46e5600700 1 -- 172.21.15.79:0/3229979883 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46dc010680 con 0x7f46e8106be0 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.659+0000 7f46e5600700 1 -- 172.21.15.79:0/3229979883 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46dc01d070 con 0x7f46e8106be0 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.659+0000 7f46e5600700 1 -- 172.21.15.79:0/3229979883 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46dc021b10 con 0x7f46e8106be0 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.659+0000 7f46ee400700 1 -- 172.21.15.79:0/3229979883 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e819e6a0 con 0x7f46e8106be0 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.659+0000 7f46ee400700 1 -- 172.21.15.79:0/3229979883 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e81a1390 con 0x7f46e8106be0 2024-06-03T02:04:47.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.660+0000 7f46e5600700 1 -- 172.21.15.79:0/3229979883 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f46dc01f030 con 0x7f46e8106be0 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.660+0000 7f46e5600700 1 -- 172.21.15.79:0/3229979883 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f46dc02acf0 con 0x7f46e8106be0 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.661+0000 7f46ee400700 1 -- 172.21.15.79:0/3229979883 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e8063af0 con 0x7f46e8106be0 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.663+0000 7f46e5600700 1 -- 172.21.15.79:0/3229979883 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f46dc015070 con 0x7f46e8106be0 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.736+0000 7f46ee400700 1 -- 172.21.15.79:0/3229979883 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f46e8193020 con 0x7f46e8106be0 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.737+0000 7f46e5600700 1 -- 172.21.15.79:0/3229979883 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f46e8193020 con 0x7f46e8106be0 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.738+0000 7f46ee400700 1 -- 172.21.15.79:0/3229979883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 msgr2=0x7f46e819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.738+0000 7f46ee400700 1 --2- 172.21.15.79:0/3229979883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e819ded0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00b340 tx=0x7f46dc00dd90 comp rx=0 tx=0).stop 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.738+0000 7f46ee400700 1 -- 172.21.15.79:0/3229979883 shutdown_connections 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.738+0000 7f46ee400700 1 --2- 172.21.15.79:0/3229979883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e8106be0 0x7f46e819ded0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.738+0000 7f46ee400700 1 -- 172.21.15.79:0/3229979883 >> 172.21.15.79:0/3229979883 conn(0x7f46e8102180 msgr2=0x7f46e81900e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.739+0000 7f46ee400700 1 -- 172.21.15.79:0/3229979883 shutdown_connections 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:46.739+0000 7f46ee400700 1 -- 172.21.15.79:0/3229979883 wait complete. 2024-06-03T02:04:47.401 INFO:teuthology.orchestra.run.smithi079.stdout:Restarting the monitor... 2024-06-03T02:04:47.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:47 smithi079 systemd[1]: Stopping Ceph mon.a for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:04:48.254 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:47 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a[31442]: 2024-06-03T02:04:47.884+0000 7fed02800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:04:48.254 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:47 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a[31442]: 2024-06-03T02:04:47.884+0000 7fed02800700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-03T02:04:48.254 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:47 smithi079 podman[31701]: 2024-06-03 02:04:47.968149718 +0000 UTC m=+0.196769807 container died 22398e90b3818e69731f23e1b90d2feeeadb0dccab9a941cdb41cf8a90e63b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:04:48.578 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:48 smithi079 podman[31701]: 2024-06-03 02:04:48.370398062 +0000 UTC m=+0.599018161 container cleanup 22398e90b3818e69731f23e1b90d2feeeadb0dccab9a941cdb41cf8a90e63b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-03T02:04:48.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:48 smithi079 bash[31701]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a 2024-06-03T02:04:48.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:48 smithi079 podman[31714]: 2024-06-03 02:04:48.579156145 +0000 UTC m=+0.591675029 container remove 22398e90b3818e69731f23e1b90d2feeeadb0dccab9a941cdb41cf8a90e63b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-06-03T02:04:49.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-03T02:04:49.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 systemd[1]: Stopped Ceph mon.a for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:04:49.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.a.service: Consumed 1.846s CPU time. 2024-06-03T02:04:49.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 systemd[1]: Starting Ceph mon.a for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:04:49.711 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 podman[31801]: 2024-06-03 02:04:49.359852244 +0000 UTC m=+0.030967555 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:04:49.711 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 podman[31801]: 2024-06-03 02:04:49.488415799 +0000 UTC m=+0.159531154 container create 5de779843158d430dc7548c8771faf16ee775b9749afa27296b350a1a23e87fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:04:49.711 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 podman[31801]: 2024-06-03 02:04:49.70015737 +0000 UTC m=+0.371272758 container init 5de779843158d430dc7548c8771faf16ee775b9749afa27296b350a1a23e87fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-03T02:04:49.711 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 podman[31801]: 2024-06-03 02:04:49.710553872 +0000 UTC m=+0.381669192 container start 5de779843158d430dc7548c8771faf16ee775b9749afa27296b350a1a23e87fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-03T02:04:49.783 INFO:teuthology.orchestra.run.smithi079.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-03T02:04:50.083 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: set uid:gid to 167:167 (ceph:ceph) 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: pidfile_write: ignore empty --pid-file 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: load: jerasure load: lrc 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: RocksDB version: 7.9.2 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Git sha 0 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: DB SUMMARY 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: DB Session ID: 6MLUXJM110B5PKRBO9RD 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: CURRENT file: CURRENT 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: IDENTITY file: IDENTITY 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-03T02:04:50.084 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104854 ; 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.error_if_exists: 0 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.create_if_missing: 0 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.paranoid_checks: 1 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.env: 0x56281b593c60 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.fs: PosixFileSystem 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.info_log: 0x56282b335b60 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_file_opening_threads: 16 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.statistics: (nil) 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.use_fsync: 0 2024-06-03T02:04:50.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_log_file_size: 0 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.keep_log_file_num: 1000 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.recycle_log_file_num: 0 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.allow_fallocate: 1 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.allow_mmap_reads: 0 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.allow_mmap_writes: 0 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.use_direct_reads: 0 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.create_missing_column_families: 0 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.db_log_dir: 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.wal_dir: 2024-06-03T02:04:50.086 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-03T02:04:50.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-03T02:04:50.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-03T02:04:50.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-03T02:04:50.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-03T02:04:50.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-03T02:04:50.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.advise_random_on_open: 1 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.db_write_buffer_size: 0 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.write_buffer_manager: 0x56282b3061e0 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.rate_limiter: (nil) 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.wal_recovery_mode: 2 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.enable_thread_tracking: 0 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.enable_pipelined_write: 0 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.unordered_write: 0 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-03T02:04:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.row_cache: None 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.wal_filter: None 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.allow_ingest_behind: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.two_write_queues: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.manual_wal_flush: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.wal_compression: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.atomic_flush: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.log_readahead_size: 0 2024-06-03T02:04:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.best_efforts_recovery: 0 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.allow_data_in_errors: 0 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.db_host_id: __hostname__ 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_background_jobs: 2 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_background_compactions: -1 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_subcompactions: 1 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-03T02:04:50.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_total_wal_size: 0 2024-06-03T02:04:50.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-03T02:04:50.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-03T02:04:50.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-03T02:04:50.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-03T02:04:50.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_open_files: -1 2024-06-03T02:04:50.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bytes_per_sync: 0 2024-06-03T02:04:50.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-03T02:04:50.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-03T02:04:50.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_readahead_size: 0 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_background_flushes: -1 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Compression algorithms supported: 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: kZSTD supported: 0 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: kXpressCompression supported: 0 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: kLZ4HCCompression supported: 1 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: kZlibCompression supported: 1 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: kSnappyCompression supported: 1 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: kLZ4Compression supported: 1 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: kBZip2Compression supported: 0 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-03T02:04:50.092 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-03T02:04:50.093 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-03T02:04:50.093 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.merge_operator: 2024-06-03T02:04:50.093 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_filter: None 2024-06-03T02:04:50.093 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_filter_factory: None 2024-06-03T02:04:50.093 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.sst_partitioner_factory: None 2024-06-03T02:04:50.093 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-03T02:04:50.093 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-03T02:04:50.093 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56282b335c80) 2024-06-03T02:04:50.094 INFO:journalctl@ceph.mon.a.smithi079.stdout: cache_index_and_filter_blocks: 1 2024-06-03T02:04:50.094 INFO:journalctl@ceph.mon.a.smithi079.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-03T02:04:50.094 INFO:journalctl@ceph.mon.a.smithi079.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-03T02:04:50.094 INFO:journalctl@ceph.mon.a.smithi079.stdout: pin_top_level_index_and_filter: 1 2024-06-03T02:04:50.094 INFO:journalctl@ceph.mon.a.smithi079.stdout: index_type: 0 2024-06-03T02:04:50.094 INFO:journalctl@ceph.mon.a.smithi079.stdout: data_block_index_type: 0 2024-06-03T02:04:50.095 INFO:journalctl@ceph.mon.a.smithi079.stdout: index_shortening: 1 2024-06-03T02:04:50.095 INFO:journalctl@ceph.mon.a.smithi079.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-03T02:04:50.095 INFO:journalctl@ceph.mon.a.smithi079.stdout: checksum: 4 2024-06-03T02:04:50.095 INFO:journalctl@ceph.mon.a.smithi079.stdout: no_block_cache: 0 2024-06-03T02:04:50.095 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_cache: 0x56282b337090 2024-06-03T02:04:50.095 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_cache_name: BinnedLRUCache 2024-06-03T02:04:50.096 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_cache_options: 2024-06-03T02:04:50.096 INFO:journalctl@ceph.mon.a.smithi079.stdout: capacity : 536870912 2024-06-03T02:04:50.096 INFO:journalctl@ceph.mon.a.smithi079.stdout: num_shard_bits : 4 2024-06-03T02:04:50.096 INFO:journalctl@ceph.mon.a.smithi079.stdout: strict_capacity_limit : 0 2024-06-03T02:04:50.096 INFO:journalctl@ceph.mon.a.smithi079.stdout: high_pri_pool_ratio: 0.000 2024-06-03T02:04:50.096 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_cache_compressed: (nil) 2024-06-03T02:04:50.097 INFO:journalctl@ceph.mon.a.smithi079.stdout: persistent_cache: (nil) 2024-06-03T02:04:50.097 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_size: 4096 2024-06-03T02:04:50.097 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_size_deviation: 10 2024-06-03T02:04:50.097 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_restart_interval: 16 2024-06-03T02:04:50.097 INFO:journalctl@ceph.mon.a.smithi079.stdout: index_block_restart_interval: 1 2024-06-03T02:04:50.097 INFO:journalctl@ceph.mon.a.smithi079.stdout: metadata_block_size: 4096 2024-06-03T02:04:50.097 INFO:journalctl@ceph.mon.a.smithi079.stdout: partition_filters: 0 2024-06-03T02:04:50.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: use_delta_encoding: 1 2024-06-03T02:04:50.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: filter_policy: bloomfilter 2024-06-03T02:04:50.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: whole_key_filtering: 1 2024-06-03T02:04:50.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: verify_compression: 0 2024-06-03T02:04:50.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: read_amp_bytes_per_bit: 0 2024-06-03T02:04:50.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: format_version: 5 2024-06-03T02:04:50.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: enable_index_compression: 1 2024-06-03T02:04:50.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_align: 0 2024-06-03T02:04:50.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: max_auto_readahead_size: 262144 2024-06-03T02:04:50.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: prepopulate_block_cache: 0 2024-06-03T02:04:50.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: initial_auto_readahead_size: 8192 2024-06-03T02:04:50.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: num_file_reads_for_auto_readahead: 2 2024-06-03T02:04:50.100 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.write_buffer_size: 33554432 2024-06-03T02:04:50.100 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_write_buffer_number: 2 2024-06-03T02:04:50.100 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression: NoCompression 2024-06-03T02:04:50.100 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression: Disabled 2024-06-03T02:04:50.100 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.prefix_extractor: nullptr 2024-06-03T02:04:50.100 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-03T02:04:50.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.num_levels: 7 2024-06-03T02:04:50.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-03T02:04:50.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-03T02:04:50.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-03T02:04:50.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-03T02:04:50.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-03T02:04:50.102 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-03T02:04:50.102 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-03T02:04:50.102 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-03T02:04:50.104 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-03T02:04:50.104 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-03T02:04:50.104 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-03T02:04:50.104 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-03T02:04:50.104 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-03T02:04:50.105 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression_opts.level: 32767 2024-06-03T02:04:50.105 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression_opts.strategy: 0 2024-06-03T02:04:50.105 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-03T02:04:50.105 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-03T02:04:50.105 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-03T02:04:50.105 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-03T02:04:50.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression_opts.enabled: false 2024-06-03T02:04:50.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-03T02:04:50.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-03T02:04:50.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-03T02:04:50.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-03T02:04:50.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.target_file_size_base: 67108864 2024-06-03T02:04:50.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-03T02:04:50.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-03T02:04:50.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-03T02:04:50.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-03T02:04:50.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-03T02:04:50.108 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-03T02:04:50.108 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-03T02:04:50.108 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-03T02:04:50.108 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-03T02:04:50.108 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-03T02:04:50.108 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-03T02:04:50.108 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-03T02:04:50.109 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-03T02:04:50.109 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 bash[31801]: 5de779843158d430dc7548c8771faf16ee775b9749afa27296b350a1a23e87fd 2024-06-03T02:04:50.109 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-03T02:04:50.109 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.arena_block_size: 1048576 2024-06-03T02:04:50.109 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-03T02:04:50.109 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-03T02:04:50.109 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.disable_auto_compactions: 0 2024-06-03T02:04:50.110 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-03T02:04:50.110 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-03T02:04:50.110 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-03T02:04:50.110 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-03T02:04:50.110 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-03T02:04:50.110 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-03T02:04:50.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-03T02:04:50.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-03T02:04:50.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-03T02:04:50.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-03T02:04:50.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.table_properties_collectors: 2024-06-03T02:04:50.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.inplace_update_support: 0 2024-06-03T02:04:50.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-03T02:04:50.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-03T02:04:50.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.bloom_locality: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.max_successive_merges: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.paranoid_file_checks: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.force_consistency_checks: 1 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.report_bg_io_stats: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.ttl: 2592000 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.enable_blob_files: false 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.min_blob_size: 0 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.blob_file_size: 268435456 2024-06-03T02:04:50.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.blob_file_starting_level: 0 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7931c5fa-66a2-468e-8e26-488c1eaae047 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717380289768660, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717380289771366, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99975, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97886, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14197, "raw_average_key_size": 49, "raw_value_size": 90162, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717380289, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7931c5fa-66a2-468e-8e26-488c1eaae047", "db_session_id": "6MLUXJM110B5PKRBO9RD", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717380289771654, "job": 1, "event": "recovery_finished"} 2024-06-03T02:04:50.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56282b426000 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: DB pointer 0x56282b410000 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout: ** DB Stats ** 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T02:04:50.114 INFO:journalctl@ceph.mon.a.smithi079.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: ** Compaction Stats [default] ** 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: L0 2/0 99.47 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: Sum 2/0 99.47 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: ** Compaction Stats [default] ** 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 63.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-06-03T02:04:50.115 INFO:journalctl@ceph.mon.a.smithi079.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: Cumulative compaction: 0.00 GB write, 9.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: Interval compaction: 0.00 GB write, 9.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: Block cache BinnedLRUCache@0x56282b337090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-06-03T02:04:50.116 INFO:journalctl@ceph.mon.a.smithi079.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: starting mon.a rank 0 at public addrs [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] at bind addrs [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mon.a@-1(???) e1 preinit fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mon.a@-1(???).mds e1 new map 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mon.a@-1(???).mds e1 print_map 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout: e1 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout: legacy client fscid: -1 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout: No filesystems configured 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T02:04:50.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 systemd[1]: Started Ceph mon.a for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: monmap epoch 1 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: last_changed 2024-06-03T02:04:38.900256+0000 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: min_mon_release 19 (squid) 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: election_strategy: 1 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: fsmap 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: osdmap e1: 0 total, 0 up, 0 in 2024-06-03T02:04:50.118 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:49 smithi079 ceph-mon[31819]: mgrmap e1: no daemons active 2024-06-03T02:04:51.116 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.369+0000 7fded5e00700 1 Processor -- start 2024-06-03T02:04:51.116 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.369+0000 7fded5e00700 1 -- start start 2024-06-03T02:04:51.116 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.370+0000 7fded5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded007cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:51.116 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.370+0000 7fded5e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fded007d080 con 0x7fded007e640 2024-06-03T02:04:51.116 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.370+0000 7fdecea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded007cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:51.116 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.370+0000 7fdecea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded007cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34480/0 (socket says 172.21.15.79:34480) 2024-06-03T02:04:51.116 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.370+0000 7fdecea00700 1 -- 172.21.15.79:0/1349115567 learned_addr learned my addr 172.21.15.79:0/1349115567 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:51.116 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.371+0000 7fdecea00700 1 -- 172.21.15.79:0/1349115567 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded007d200 con 0x7fded007e640 2024-06-03T02:04:51.120 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.371+0000 7fdecea00700 1 --2- 172.21.15.79:0/1349115567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded007cb40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdec00098a0 tx=0x7fdec0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8439ec11ddd854 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:51.120 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.371+0000 7fdecd600700 1 -- 172.21.15.79:0/1349115567 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec0004160 con 0x7fded007e640 2024-06-03T02:04:51.120 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.371+0000 7fdecd600700 1 -- 172.21.15.79:0/1349115567 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdec0004300 con 0x7fded007e640 2024-06-03T02:04:51.120 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.371+0000 7fdecd600700 1 -- 172.21.15.79:0/1349115567 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec0011b50 con 0x7fded007e640 2024-06-03T02:04:51.120 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.372+0000 7fded5e00700 1 -- 172.21.15.79:0/1349115567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 msgr2=0x7fded007cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:51.120 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.372+0000 7fded5e00700 1 --2- 172.21.15.79:0/1349115567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded007cb40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdec00098a0 tx=0x7fdec0009bb0 comp rx=0 tx=0).stop 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.372+0000 7fded5e00700 1 -- 172.21.15.79:0/1349115567 shutdown_connections 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.372+0000 7fded5e00700 1 --2- 172.21.15.79:0/1349115567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded007cb40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.372+0000 7fded5e00700 1 -- 172.21.15.79:0/1349115567 >> 172.21.15.79:0/1349115567 conn(0x7fded0103530 msgr2=0x7fded0105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.372+0000 7fded5e00700 1 -- 172.21.15.79:0/1349115567 shutdown_connections 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.372+0000 7fded5e00700 1 -- 172.21.15.79:0/1349115567 wait complete. 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.373+0000 7fded5e00700 1 Processor -- start 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.374+0000 7fded5e00700 1 -- start start 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.374+0000 7fded5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.374+0000 7fded5e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fded010e520 con 0x7fded007e640 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.374+0000 7fdecea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.374+0000 7fdecea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34490/0 (socket says 172.21.15.79:34490) 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.374+0000 7fdecea00700 1 -- 172.21.15.79:0/2512897960 learned_addr learned my addr 172.21.15.79:0/2512897960 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.375+0000 7fdecea00700 1 -- 172.21.15.79:0/2512897960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded019e5d0 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.375+0000 7fdecea00700 1 --2- 172.21.15.79:0/2512897960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded019e090 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009fb0 tx=0x7fdec0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.375+0000 7fded4a00700 1 -- 172.21.15.79:0/2512897960 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec00038b0 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.375+0000 7fded4a00700 1 -- 172.21.15.79:0/2512897960 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdec0011e50 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.375+0000 7fded5e00700 1 -- 172.21.15.79:0/2512897960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded019e860 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.375+0000 7fded4a00700 1 -- 172.21.15.79:0/2512897960 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec0019420 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.376+0000 7fded5e00700 1 -- 172.21.15.79:0/2512897960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded01a1550 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.376+0000 7fded4a00700 1 -- 172.21.15.79:0/2512897960 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdec0019a60 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.376+0000 7fded4a00700 1 -- 172.21.15.79:0/2512897960 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdec0024730 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.377+0000 7fded5e00700 1 -- 172.21.15.79:0/2512897960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded0063af0 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.380+0000 7fded4a00700 1 -- 172.21.15.79:0/2512897960 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdec0043dd0 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.456+0000 7fded5e00700 1 -- 172.21.15.79:0/2512897960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fded01a1720 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.461+0000 7fded4a00700 1 -- 172.21.15.79:0/2512897960 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fdec0032030 con 0x7fded007e640 2024-06-03T02:04:51.122 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.462+0000 7fded5e00700 1 -- 172.21.15.79:0/2512897960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 msgr2=0x7fded019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:51.123 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.462+0000 7fded5e00700 1 --2- 172.21.15.79:0/2512897960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded019e090 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009fb0 tx=0x7fdec0011bf0 comp rx=0 tx=0).stop 2024-06-03T02:04:51.123 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.462+0000 7fded5e00700 1 -- 172.21.15.79:0/2512897960 shutdown_connections 2024-06-03T02:04:51.123 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.462+0000 7fded5e00700 1 --2- 172.21.15.79:0/2512897960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fded007e640 0x7fded019e090 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:51.123 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.462+0000 7fded5e00700 1 -- 172.21.15.79:0/2512897960 >> 172.21.15.79:0/2512897960 conn(0x7fded0103530 msgr2=0x7fded0104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:51.123 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.463+0000 7fded5e00700 1 -- 172.21.15.79:0/2512897960 shutdown_connections 2024-06-03T02:04:51.123 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:50.463+0000 7fded5e00700 1 -- 172.21.15.79:0/2512897960 wait complete. 2024-06-03T02:04:51.123 INFO:teuthology.orchestra.run.smithi079.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-03T02:04:51.145 INFO:teuthology.orchestra.run.smithi079.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-03T02:04:51.145 INFO:teuthology.orchestra.run.smithi079.stdout:Creating mgr... 2024-06-03T02:04:51.146 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-03T02:04:51.147 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-03T02:04:51.589 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.x 2024-06-03T02:04:51.589 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-03T02:04:51.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:51 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2512897960' entity='client.admin' 2024-06-03T02:04:51.873 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968.target.wants/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@.service. 2024-06-03T02:04:52.279 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:51 smithi079 systemd[1]: Starting Ceph mgr.x for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:04:52.570 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:52 smithi079 podman[32028]: 2024-06-03 02:04:52.183865271 +0000 UTC m=+0.032815548 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:04:52.570 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:52 smithi079 podman[32028]: 2024-06-03 02:04:52.323702083 +0000 UTC m=+0.172652427 container create 35cbaa7fc85adb21c958d5413fac02c628da0a74594113f28434a943586778f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-03T02:04:52.570 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:52 smithi079 podman[32028]: 2024-06-03 02:04:52.568715793 +0000 UTC m=+0.417666124 container init 35cbaa7fc85adb21c958d5413fac02c628da0a74594113f28434a943586778f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-03T02:04:52.691 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T02:04:52.691 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout disabled 2024-06-03T02:04:52.708 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T02:04:52.708 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-06-03T02:04:52.708 INFO:teuthology.orchestra.run.smithi079.stdout:firewalld.service is not enabled 2024-06-03T02:04:52.708 INFO:teuthology.orchestra.run.smithi079.stdout:Not possible to enable service . firewalld.service is not available 2024-06-03T02:04:52.726 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T02:04:52.726 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout disabled 2024-06-03T02:04:52.741 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T02:04:52.741 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-06-03T02:04:52.741 INFO:teuthology.orchestra.run.smithi079.stdout:firewalld.service is not enabled 2024-06-03T02:04:52.741 INFO:teuthology.orchestra.run.smithi079.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-03T02:04:52.741 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mgr to start... 2024-06-03T02:04:52.741 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mgr... 2024-06-03T02:04:52.837 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:52 smithi079 podman[32028]: 2024-06-03 02:04:52.579735829 +0000 UTC m=+0.428686135 container start 35cbaa7fc85adb21c958d5413fac02c628da0a74594113f28434a943586778f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-03T02:04:52.837 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:52 smithi079 bash[32028]: 35cbaa7fc85adb21c958d5413fac02c628da0a74594113f28434a943586778f5 2024-06-03T02:04:52.837 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:52 smithi079 systemd[1]: Started Ceph mgr.x for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:04:53.088 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:52.836+0000 7f36d39dc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T02:04:53.352 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:53.098+0000 7f36d39dc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:04:53.353 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:53.231+0000 7f36d39dc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T02:04:54.006 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:53 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1876272339' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T02:04:54.007 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:53.651+0000 7f36d39dc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T02:04:54.007 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:53.772+0000 7f36d39dc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T02:04:54.469 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:04:54.470 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-06-03T02:04:54.470 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:04:54.470 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T02:04:54.470 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T02:04:54.470 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T02:04:54.470 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T02:04:54.471 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:54.471 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T02:04:54.471 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T02:04:54.471 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 0 2024-06-03T02:04:54.472 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:04:54.472 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T02:04:54.473 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T02:04:54.480 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T02:04:54.480 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T02:04:54.480 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T02:04:54.480 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:54.480 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T02:04:54.480 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T02:04:54.480 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T02:04:43.162607+0000", 2024-06-03T02:04:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.644+0000 7f514aa00700 1 Processor -- start 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.644+0000 7f514aa00700 1 -- start start 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.644+0000 7f514aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f51441081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.644+0000 7f514aa00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51441087c0 con 0x7f5144107dd0 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.645+0000 7f5149600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f51441081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.645+0000 7f5149600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f51441081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34514/0 (socket says 172.21.15.79:34514) 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.645+0000 7f5149600700 1 -- 172.21.15.79:0/2368970405 learned_addr learned my addr 172.21.15.79:0/2368970405 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.645+0000 7f5149600700 1 -- 172.21.15.79:0/2368970405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5144108f80 con 0x7f5144107dd0 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.646+0000 7f5149600700 1 --2- 172.21.15.79:0/2368970405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f51441081f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51380098a0 tx=0x7f5138009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d954c8cbb545de63 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:54.483 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.646+0000 7f5143e00700 1 -- 172.21.15.79:0/2368970405 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5138004160 con 0x7f5144107dd0 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.646+0000 7f5143e00700 1 -- 172.21.15.79:0/2368970405 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5138004300 con 0x7f5144107dd0 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.646+0000 7f5143e00700 1 -- 172.21.15.79:0/2368970405 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5138011b50 con 0x7f5144107dd0 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.647+0000 7f514aa00700 1 -- 172.21.15.79:0/2368970405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 msgr2=0x7f51441081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.647+0000 7f514aa00700 1 --2- 172.21.15.79:0/2368970405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f51441081f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51380098a0 tx=0x7f5138009bb0 comp rx=0 tx=0).stop 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.647+0000 7f514aa00700 1 -- 172.21.15.79:0/2368970405 shutdown_connections 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.647+0000 7f514aa00700 1 --2- 172.21.15.79:0/2368970405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f51441081f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.647+0000 7f514aa00700 1 -- 172.21.15.79:0/2368970405 >> 172.21.15.79:0/2368970405 conn(0x7f51441033b0 msgr2=0x7f51441057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.647+0000 7f514aa00700 1 -- 172.21.15.79:0/2368970405 shutdown_connections 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.647+0000 7f514aa00700 1 -- 172.21.15.79:0/2368970405 wait complete. 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.649+0000 7f514aa00700 1 Processor -- start 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.649+0000 7f514aa00700 1 -- start start 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.649+0000 7f514aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f514419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:54.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.649+0000 7f514aa00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f514410c330 con 0x7f5144107dd0 2024-06-03T02:04:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.650+0000 7f5149600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f514419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.650+0000 7f5149600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f514419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34516/0 (socket says 172.21.15.79:34516) 2024-06-03T02:04:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.650+0000 7f5149600700 1 -- 172.21.15.79:0/1876272339 learned_addr learned my addr 172.21.15.79:0/1876272339 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.650+0000 7f5149600700 1 -- 172.21.15.79:0/1876272339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514419e490 con 0x7f5144107dd0 2024-06-03T02:04:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.650+0000 7f5149600700 1 --2- 172.21.15.79:0/1876272339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f514419df50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5138011850 tx=0x7f5138011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.651+0000 7f5142000700 1 -- 172.21.15.79:0/1876272339 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5138003af0 con 0x7f5144107dd0 2024-06-03T02:04:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.651+0000 7f5142000700 1 -- 172.21.15.79:0/1876272339 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5138020070 con 0x7f5144107dd0 2024-06-03T02:04:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.651+0000 7f514aa00700 1 -- 172.21.15.79:0/1876272339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514419e720 con 0x7f5144107dd0 2024-06-03T02:04:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.651+0000 7f5142000700 1 -- 172.21.15.79:0/1876272339 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51380195a0 con 0x7f5144107dd0 2024-06-03T02:04:54.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.651+0000 7f514aa00700 1 -- 172.21.15.79:0/1876272339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51441a1410 con 0x7f5144107dd0 2024-06-03T02:04:54.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.651+0000 7f5142000700 1 -- 172.21.15.79:0/1876272339 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5138019c70 con 0x7f5144107dd0 2024-06-03T02:04:54.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.651+0000 7f5142000700 1 -- 172.21.15.79:0/1876272339 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5138011e00 con 0x7f5144107dd0 2024-06-03T02:04:54.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.652+0000 7f514aa00700 1 -- 172.21.15.79:0/1876272339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5144063af0 con 0x7f5144107dd0 2024-06-03T02:04:54.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.655+0000 7f5142000700 1 -- 172.21.15.79:0/1876272339 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5138003c90 con 0x7f5144107dd0 2024-06-03T02:04:54.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.733+0000 7f514aa00700 1 -- 172.21.15.79:0/1876272339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f51441a15e0 con 0x7f5144107dd0 2024-06-03T02:04:54.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.734+0000 7f5142000700 1 -- 172.21.15.79:0/1876272339 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5138025460 con 0x7f5144107dd0 2024-06-03T02:04:54.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.735+0000 7f514aa00700 1 -- 172.21.15.79:0/1876272339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 msgr2=0x7f514419df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:54.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.735+0000 7f514aa00700 1 --2- 172.21.15.79:0/1876272339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f514419df50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5138011850 tx=0x7f5138011c10 comp rx=0 tx=0).stop 2024-06-03T02:04:54.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.736+0000 7f514aa00700 1 -- 172.21.15.79:0/1876272339 shutdown_connections 2024-06-03T02:04:54.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.736+0000 7f514aa00700 1 --2- 172.21.15.79:0/1876272339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144107dd0 0x7f514419df50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:54.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.736+0000 7f514aa00700 1 -- 172.21.15.79:0/1876272339 >> 172.21.15.79:0/1876272339 conn(0x7f51441033b0 msgr2=0x7f5144103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:54.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.736+0000 7f514aa00700 1 -- 172.21.15.79:0/1876272339 shutdown_connections 2024-06-03T02:04:54.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:53.736+0000 7f514aa00700 1 -- 172.21.15.79:0/1876272339 wait complete. 2024-06-03T02:04:54.488 INFO:teuthology.orchestra.run.smithi079.stdout:mgr not available, waiting (1/15)... 2024-06-03T02:04:55.791 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:55.361+0000 7f36d39dc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T02:04:55.791 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:55.580+0000 7f36d39dc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T02:04:55.791 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:55.790+0000 7f36d39dc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T02:04:56.446 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:56.161+0000 7f36d39dc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T02:04:56.446 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:56.281+0000 7f36d39dc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T02:04:56.781 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:56.536+0000 7f36d39dc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T02:04:57.034 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:56.794+0000 7f36d39dc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:04:57.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:04:57 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/455218118' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T02:04:57.913 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:57 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:57.616+0000 7f36d39dc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T02:04:57.923 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:04:57.923 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-06-03T02:04:57.923 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:04:57.923 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T02:04:57.923 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T02:04:57.923 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 0 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T02:04:57.924 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:57.925 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T02:04:57.925 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:04:57.925 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T02:04:57.925 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T02:04:57.932 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T02:04:57.932 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T02:04:57.932 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T02:04:57.932 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T02:04:57.932 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:57.932 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:57.933 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T02:04:43.162607+0000", 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T02:04:57.934 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.085+0000 7f1e4fe00700 1 Processor -- start 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.086+0000 7f1e4fe00700 1 -- start start 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.086+0000 7f1e4fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.086+0000 7f1e4fe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4807c830 con 0x7f1e4807ddf0 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.086+0000 7f1e4d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.086+0000 7f1e4d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44310/0 (socket says 172.21.15.79:44310) 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.086+0000 7f1e4d400700 1 -- 172.21.15.79:0/4247025228 learned_addr learned my addr 172.21.15.79:0/4247025228 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.087+0000 7f1e4d400700 1 -- 172.21.15.79:0/4247025228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4807c9b0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.087+0000 7f1e4d400700 1 --2- 172.21.15.79:0/4247025228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4807c2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0098a0 tx=0x7f1e3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec56e4aec83aa7d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.087+0000 7f1e47e00700 1 -- 172.21.15.79:0/4247025228 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e3c004160 con 0x7f1e4807ddf0 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.088+0000 7f1e47e00700 1 -- 172.21.15.79:0/4247025228 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1e3c004300 con 0x7f1e4807ddf0 2024-06-03T02:04:57.935 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.088+0000 7f1e47e00700 1 -- 172.21.15.79:0/4247025228 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e3c011b50 con 0x7f1e4807ddf0 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.088+0000 7f1e4fe00700 1 -- 172.21.15.79:0/4247025228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 msgr2=0x7f1e4807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.088+0000 7f1e4fe00700 1 --2- 172.21.15.79:0/4247025228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4807c2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0098a0 tx=0x7f1e3c009bb0 comp rx=0 tx=0).stop 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.088+0000 7f1e4fe00700 1 -- 172.21.15.79:0/4247025228 shutdown_connections 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.088+0000 7f1e4fe00700 1 --2- 172.21.15.79:0/4247025228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4807c2f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.088+0000 7f1e4fe00700 1 -- 172.21.15.79:0/4247025228 >> 172.21.15.79:0/4247025228 conn(0x7f1e481035e0 msgr2=0x7f1e48105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.089+0000 7f1e4fe00700 1 -- 172.21.15.79:0/4247025228 shutdown_connections 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.089+0000 7f1e4fe00700 1 -- 172.21.15.79:0/4247025228 wait complete. 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.089+0000 7f1e4fe00700 1 Processor -- start 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.090+0000 7f1e4fe00700 1 -- start start 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.090+0000 7f1e4fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.090+0000 7f1e4fe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4810e4d0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.090+0000 7f1e4d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.090+0000 7f1e4d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4819e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44314/0 (socket says 172.21.15.79:44314) 2024-06-03T02:04:57.936 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.090+0000 7f1e4d400700 1 -- 172.21.15.79:0/455218118 learned_addr learned my addr 172.21.15.79:0/455218118 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.090+0000 7f1e4d400700 1 -- 172.21.15.79:0/455218118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4819e550 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.091+0000 7f1e4d400700 1 --2- 172.21.15.79:0/455218118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4819e010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c011670 tx=0x7f1e3c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.091+0000 7f1e46000700 1 -- 172.21.15.79:0/455218118 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e3c0039c0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.091+0000 7f1e46000700 1 -- 172.21.15.79:0/455218118 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1e3c011e50 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.091+0000 7f1e46000700 1 -- 172.21.15.79:0/455218118 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e3c01a5c0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.091+0000 7f1e4fe00700 1 -- 172.21.15.79:0/455218118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4819e7e0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.091+0000 7f1e4fe00700 1 -- 172.21.15.79:0/455218118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481a14d0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.092+0000 7f1e4fe00700 1 -- 172.21.15.79:0/455218118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e48063af0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.093+0000 7f1e46000700 1 -- 172.21.15.79:0/455218118 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e3c01a870 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.093+0000 7f1e46000700 1 -- 172.21.15.79:0/455218118 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1e3c02ddc0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.095+0000 7f1e46000700 1 -- 172.21.15.79:0/455218118 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1e3c003b60 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.170+0000 7f1e4fe00700 1 -- 172.21.15.79:0/455218118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1e481a16a0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.171+0000 7f1e46000700 1 -- 172.21.15.79:0/455218118 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1e3c02d3f0 con 0x7f1e4807ddf0 2024-06-03T02:04:57.937 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.172+0000 7f1e4fe00700 1 -- 172.21.15.79:0/455218118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 msgr2=0x7f1e4819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:04:57.938 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.172+0000 7f1e4fe00700 1 --2- 172.21.15.79:0/455218118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4819e010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c011670 tx=0x7f1e3c011b60 comp rx=0 tx=0).stop 2024-06-03T02:04:57.938 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.172+0000 7f1e4fe00700 1 -- 172.21.15.79:0/455218118 shutdown_connections 2024-06-03T02:04:57.938 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.172+0000 7f1e4fe00700 1 --2- 172.21.15.79:0/455218118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e4807ddf0 0x7f1e4819e010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:04:57.938 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.172+0000 7f1e4fe00700 1 -- 172.21.15.79:0/455218118 >> 172.21.15.79:0/455218118 conn(0x7f1e481035e0 msgr2=0x7f1e481040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:04:57.938 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.173+0000 7f1e4fe00700 1 -- 172.21.15.79:0/455218118 shutdown_connections 2024-06-03T02:04:57.938 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:04:57.173+0000 7f1e4fe00700 1 -- 172.21.15.79:0/455218118 wait complete. 2024-06-03T02:04:57.938 INFO:teuthology.orchestra.run.smithi079.stdout:mgr not available, waiting (2/15)... 2024-06-03T02:04:58.567 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:58.176+0000 7f36d39dc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T02:04:58.568 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:58.303+0000 7f36d39dc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T02:04:58.568 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:58.566+0000 7f36d39dc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T02:04:59.588 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:59 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:59.227+0000 7f36d39dc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T02:04:59.588 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:59 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:59.351+0000 7f36d39dc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T02:04:59.980 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:59 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:59.673+0000 7f36d39dc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T02:04:59.980 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:59 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:59.852+0000 7f36d39dc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T02:05:00.257 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:04:59 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:04:59.980+0000 7f36d39dc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T02:05:00.257 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:00.136+0000 7f36d39dc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T02:05:00.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:00 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1546731615' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T02:05:00.838 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:00.510+0000 7f36d39dc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T02:05:01.329 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:01.328+0000 7f36d39dc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T02:05:01.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:05:01.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-06-03T02:05:01.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:05:01.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T02:05:01.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T02:05:01.354 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T02:05:01.354 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T02:05:01.354 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:01.354 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T02:05:01.354 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T02:05:01.354 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 0 2024-06-03T02:05:01.354 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:05:01.354 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T02:05:01.354 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T02:05:01.355 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:05:01.355 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-06-03T02:05:01.355 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T02:05:01.355 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:05:01.355 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T02:05:01.355 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T02:05:01.355 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T02:05:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T02:04:43.162607+0000", 2024-06-03T02:05:01.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.561+0000 7f6993200700 1 Processor -- start 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.562+0000 7f6993200700 1 -- start start 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.562+0000 7f6993200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.562+0000 7f6993200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f699407c810 con 0x7f699407dd60 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.563+0000 7f6991e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.563+0000 7f6991e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44326/0 (socket says 172.21.15.79:44326) 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.563+0000 7f6991e00700 1 -- 172.21.15.79:0/2300890931 learned_addr learned my addr 172.21.15.79:0/2300890931 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.563+0000 7f6991e00700 1 -- 172.21.15.79:0/2300890931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699407c990 con 0x7f699407dd60 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.563+0000 7f6991e00700 1 --2- 172.21.15.79:0/2300890931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699407c2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f69840098a0 tx=0x7f6984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50076a79eda28df4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:01.366 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.564+0000 7f6990a00700 1 -- 172.21.15.79:0/2300890931 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6984004160 con 0x7f699407dd60 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.564+0000 7f6990a00700 1 -- 172.21.15.79:0/2300890931 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6984004300 con 0x7f699407dd60 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.564+0000 7f6990a00700 1 -- 172.21.15.79:0/2300890931 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6984011b50 con 0x7f699407dd60 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.564+0000 7f6993200700 1 -- 172.21.15.79:0/2300890931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 msgr2=0x7f699407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.564+0000 7f6993200700 1 --2- 172.21.15.79:0/2300890931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699407c2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f69840098a0 tx=0x7f6984009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.564+0000 7f6993200700 1 -- 172.21.15.79:0/2300890931 shutdown_connections 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.564+0000 7f6993200700 1 --2- 172.21.15.79:0/2300890931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699407c2d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.564+0000 7f6993200700 1 -- 172.21.15.79:0/2300890931 >> 172.21.15.79:0/2300890931 conn(0x7f69941032d0 msgr2=0x7f6994105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.565+0000 7f6993200700 1 -- 172.21.15.79:0/2300890931 shutdown_connections 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.565+0000 7f6993200700 1 -- 172.21.15.79:0/2300890931 wait complete. 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.566+0000 7f6993200700 1 Processor -- start 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.566+0000 7f6993200700 1 -- start start 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.566+0000 7f6993200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.566+0000 7f6993200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f699410e1a0 con 0x7f699407dd60 2024-06-03T02:05:01.367 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.566+0000 7f6991e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.566+0000 7f6991e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699419de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44338/0 (socket says 172.21.15.79:44338) 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.566+0000 7f6991e00700 1 -- 172.21.15.79:0/1546731615 learned_addr learned my addr 172.21.15.79:0/1546731615 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.567+0000 7f6991e00700 1 -- 172.21.15.79:0/1546731615 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699419e3c0 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.567+0000 7f6991e00700 1 --2- 172.21.15.79:0/1546731615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699419de80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6984011b60 tx=0x7f6984011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.567+0000 7f698aa00700 1 -- 172.21.15.79:0/1546731615 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6984003970 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.567+0000 7f698aa00700 1 -- 172.21.15.79:0/1546731615 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6984003b10 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.567+0000 7f698aa00700 1 -- 172.21.15.79:0/1546731615 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6984024610 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.567+0000 7f6993200700 1 -- 172.21.15.79:0/1546731615 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699419e650 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.568+0000 7f6993200700 1 -- 172.21.15.79:0/1546731615 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69941a1340 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.568+0000 7f698aa00700 1 -- 172.21.15.79:0/1546731615 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6984024ca0 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.568+0000 7f698aa00700 1 -- 172.21.15.79:0/1546731615 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6984011e00 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.569+0000 7f6993200700 1 -- 172.21.15.79:0/1546731615 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6994063af0 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.571+0000 7f698aa00700 1 -- 172.21.15.79:0/1546731615 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f698401a9f0 con 0x7f699407dd60 2024-06-03T02:05:01.368 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.664+0000 7f6993200700 1 -- 172.21.15.79:0/1546731615 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f69941a1510 con 0x7f699407dd60 2024-06-03T02:05:01.369 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.664+0000 7f698aa00700 1 -- 172.21.15.79:0/1546731615 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f698401a430 con 0x7f699407dd60 2024-06-03T02:05:01.369 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.666+0000 7f6993200700 1 -- 172.21.15.79:0/1546731615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 msgr2=0x7f699419de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:01.369 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.666+0000 7f6993200700 1 --2- 172.21.15.79:0/1546731615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699419de80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6984011b60 tx=0x7f6984011c40 comp rx=0 tx=0).stop 2024-06-03T02:05:01.369 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.666+0000 7f6993200700 1 -- 172.21.15.79:0/1546731615 shutdown_connections 2024-06-03T02:05:01.369 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.666+0000 7f6993200700 1 --2- 172.21.15.79:0/1546731615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699407dd60 0x7f699419de80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:01.369 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.666+0000 7f6993200700 1 -- 172.21.15.79:0/1546731615 >> 172.21.15.79:0/1546731615 conn(0x7f69941032d0 msgr2=0x7f6994103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:01.369 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.666+0000 7f6993200700 1 -- 172.21.15.79:0/1546731615 shutdown_connections 2024-06-03T02:05:01.369 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:00.666+0000 7f6993200700 1 -- 172.21.15.79:0/1546731615 wait complete. 2024-06-03T02:05:01.369 INFO:teuthology.orchestra.run.smithi079.stdout:mgr not available, waiting (3/15)... 2024-06-03T02:05:02.087 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:01.616+0000 7f36d39dc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: Activating manager daemon x 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: mgrmap e2: x(active, starting, since 0.00604533s) 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: Manager daemon x is now available 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T02:05:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' 2024-06-03T02:05:02.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T02:05:02.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' 2024-06-03T02:05:02.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:01 smithi079 ceph-mon[31819]: from='mgr.14100 172.21.15.79:0/3320231893' entity='mgr.x' 2024-06-03T02:05:03.822 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:03 smithi079 ceph-mon[31819]: mgrmap e3: x(active, since 1.02087s) 2024-06-03T02:05:04.899 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:04 smithi079 ceph-mon[31819]: mgrmap e4: x(active, since 2s) 2024-06-03T02:05:04.899 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:04 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4004968890' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T02:05:05.205 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 0 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T02:05:05.206 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T02:05:05.207 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T02:05:05.215 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T02:05:05.216 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T02:04:43.162607+0000", 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-06-03T02:05:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.985+0000 7f832c000700 1 Processor -- start 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.985+0000 7f832c000700 1 -- start start 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.986+0000 7f832c000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f8324106bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.986+0000 7f832c000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f832407c660 con 0x7f83241067d0 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.986+0000 7f8329600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f8324106bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.986+0000 7f8329600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f8324106bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44420/0 (socket says 172.21.15.79:44420) 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.986+0000 7f8329600700 1 -- 172.21.15.79:0/3824043593 learned_addr learned my addr 172.21.15.79:0/3824043593 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.987+0000 7f8329600700 1 -- 172.21.15.79:0/3824043593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8324107130 con 0x7f83241067d0 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.987+0000 7f8329600700 1 --2- 172.21.15.79:0/3824043593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f8324106bf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83180098a0 tx=0x7f8318009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460cafb5bd95f96a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.987+0000 7f8323e00700 1 -- 172.21.15.79:0/3824043593 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8318004160 con 0x7f83241067d0 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.987+0000 7f8323e00700 1 -- 172.21.15.79:0/3824043593 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8318004300 con 0x7f83241067d0 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.987+0000 7f8323e00700 1 -- 172.21.15.79:0/3824043593 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8318011b50 con 0x7f83241067d0 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.988+0000 7f832c000700 1 -- 172.21.15.79:0/3824043593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 msgr2=0x7f8324106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:05.218 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.988+0000 7f832c000700 1 --2- 172.21.15.79:0/3824043593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f8324106bf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83180098a0 tx=0x7f8318009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.988+0000 7f832c000700 1 -- 172.21.15.79:0/3824043593 shutdown_connections 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.988+0000 7f832c000700 1 --2- 172.21.15.79:0/3824043593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f8324106bf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.988+0000 7f832c000700 1 -- 172.21.15.79:0/3824043593 >> 172.21.15.79:0/3824043593 conn(0x7f83241023f0 msgr2=0x7f8324104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.988+0000 7f832c000700 1 -- 172.21.15.79:0/3824043593 shutdown_connections 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.988+0000 7f832c000700 1 -- 172.21.15.79:0/3824043593 wait complete. 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.989+0000 7f832c000700 1 Processor -- start 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.989+0000 7f832c000700 1 -- start start 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.990+0000 7f832c000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f832419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.990+0000 7f832c000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8324108040 con 0x7f83241067d0 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.990+0000 7f8329600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f832419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.990+0000 7f8329600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f832419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44426/0 (socket says 172.21.15.79:44426) 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.990+0000 7f8329600700 1 -- 172.21.15.79:0/4004968890 learned_addr learned my addr 172.21.15.79:0/4004968890 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.990+0000 7f8329600700 1 -- 172.21.15.79:0/4004968890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832419e520 con 0x7f83241067d0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.991+0000 7f8329600700 1 --2- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f832419dfe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8318011b60 tx=0x7f8318011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.991+0000 7f8322000700 1 -- 172.21.15.79:0/4004968890 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8318003b20 con 0x7f83241067d0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.991+0000 7f8322000700 1 -- 172.21.15.79:0/4004968890 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8318003cc0 con 0x7f83241067d0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.991+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832419e7b0 con 0x7f83241067d0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.991+0000 7f8322000700 1 -- 172.21.15.79:0/4004968890 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8318024840 con 0x7f83241067d0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.991+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241a14a0 con 0x7f83241067d0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.992+0000 7f8322000700 1 -- 172.21.15.79:0/4004968890 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8318022020 con 0x7f83241067d0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.992+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8324063af0 con 0x7f83241067d0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.993+0000 7f8322000700 1 --2- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f830c039050 0x7f830c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.993+0000 7f8322000700 1 -- 172.21.15.79:0/4004968890 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8318011e00 con 0x7f83241067d0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.993+0000 7f8328c00700 1 --2- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f830c039050 0x7f830c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.994+0000 7f8328c00700 1 --2- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f830c039050 0x7f830c03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8310006fd0 tx=0x7f8310006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:05.220 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:03.999+0000 7f8322000700 1 -- 172.21.15.79:0/4004968890 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f831804cb60 con 0x7f83241067d0 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.300+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f83240008d0 con 0x7f83241067d0 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.300+0000 7f8322000700 1 -- 172.21.15.79:0/4004968890 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f831801a7b0 con 0x7f83241067d0 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.303+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f830c039050 msgr2=0x7f830c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.303+0000 7f832c000700 1 --2- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f830c039050 0x7f830c03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8310006fd0 tx=0x7f8310006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.303+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 msgr2=0x7f832419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.303+0000 7f832c000700 1 --2- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f832419dfe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8318011b60 tx=0x7f8318011c40 comp rx=0 tx=0).stop 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.303+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 shutdown_connections 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.303+0000 7f832c000700 1 --2- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f830c039050 0x7f830c03b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.303+0000 7f832c000700 1 --2- 172.21.15.79:0/4004968890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83241067d0 0x7f832419dfe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.303+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 >> 172.21.15.79:0/4004968890 conn(0x7f83241023f0 msgr2=0x7f8324194c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.303+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 shutdown_connections 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:04.304+0000 7f832c000700 1 -- 172.21.15.79:0/4004968890 wait complete. 2024-06-03T02:05:05.221 INFO:teuthology.orchestra.run.smithi079.stdout:mgr is available 2024-06-03T02:05:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:06 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3676254954' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-03T02:05:06.750 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:05:06.750 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [global] 2024-06-03T02:05:06.750 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout fsid = 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:05:06.750 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-03T02:05:06.750 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-03T02:05:06.752 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-03T02:05:06.752 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-03T02:05:06.752 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:05:06.752 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-03T02:05:06.752 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-03T02:05:06.752 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-06-03T02:05:06.752 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [osd] 2024-06-03T02:05:06.752 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-03T02:05:06.753 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-03T02:05:06.753 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-03T02:05:06.758 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-03T02:05:06.758 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-03T02:05:06.758 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.797+0000 7fc8ff600700 1 Processor -- start 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.797+0000 7fc8ff600700 1 -- start start 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.797+0000 7fc8ff600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f80822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.797+0000 7fc8ff600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f80828c0 con 0x7fc8f8081ed0 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.798+0000 7fc8fcc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f80822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.798+0000 7fc8fcc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f80822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52304/0 (socket says 172.21.15.79:52304) 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.798+0000 7fc8fcc00700 1 -- 172.21.15.79:0/2920590593 learned_addr learned my addr 172.21.15.79:0/2920590593 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.798+0000 7fc8fcc00700 1 -- 172.21.15.79:0/2920590593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f810f6c0 con 0x7fc8f8081ed0 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.798+0000 7fc8fcc00700 1 --2- 172.21.15.79:0/2920590593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f80822f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0098a0 tx=0x7fc8ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cae8e4f9d7e8e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.799+0000 7fc8f7400700 1 -- 172.21.15.79:0/2920590593 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8ec004160 con 0x7fc8f8081ed0 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.799+0000 7fc8f7400700 1 -- 172.21.15.79:0/2920590593 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc8ec004300 con 0x7fc8f8081ed0 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.799+0000 7fc8f7400700 1 -- 172.21.15.79:0/2920590593 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8ec011b50 con 0x7fc8f8081ed0 2024-06-03T02:05:06.759 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.799+0000 7fc8ff600700 1 -- 172.21.15.79:0/2920590593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 msgr2=0x7fc8f80822f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.799+0000 7fc8ff600700 1 --2- 172.21.15.79:0/2920590593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f80822f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0098a0 tx=0x7fc8ec009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.800+0000 7fc8ff600700 1 -- 172.21.15.79:0/2920590593 shutdown_connections 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.800+0000 7fc8ff600700 1 --2- 172.21.15.79:0/2920590593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f80822f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.800+0000 7fc8ff600700 1 -- 172.21.15.79:0/2920590593 >> 172.21.15.79:0/2920590593 conn(0x7fc8f807d6b0 msgr2=0x7fc8f807fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.800+0000 7fc8ff600700 1 -- 172.21.15.79:0/2920590593 shutdown_connections 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.800+0000 7fc8ff600700 1 -- 172.21.15.79:0/2920590593 wait complete. 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.801+0000 7fc8ff600700 1 Processor -- start 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.801+0000 7fc8ff600700 1 -- start start 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.801+0000 7fc8ff600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f81a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.801+0000 7fc8ff600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f8112a70 con 0x7fc8f8081ed0 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.801+0000 7fc8fcc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f81a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.801+0000 7fc8fcc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f81a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52318/0 (socket says 172.21.15.79:52318) 2024-06-03T02:05:06.760 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.801+0000 7fc8fcc00700 1 -- 172.21.15.79:0/3676254954 learned_addr learned my addr 172.21.15.79:0/3676254954 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.802+0000 7fc8fcc00700 1 -- 172.21.15.79:0/3676254954 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f81a2950 con 0x7fc8f8081ed0 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.802+0000 7fc8fcc00700 1 --2- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f81a2410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec011b60 tx=0x7fc8ec011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.802+0000 7fc8f5600700 1 -- 172.21.15.79:0/3676254954 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8ec003b20 con 0x7fc8f8081ed0 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.802+0000 7fc8f5600700 1 -- 172.21.15.79:0/3676254954 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc8ec003cc0 con 0x7fc8f8081ed0 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.802+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f81a2be0 con 0x7fc8f8081ed0 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.802+0000 7fc8f5600700 1 -- 172.21.15.79:0/3676254954 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8ec024840 con 0x7fc8f8081ed0 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.802+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f81a58d0 con 0x7fc8f8081ed0 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.803+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f8063af0 con 0x7fc8f8081ed0 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.803+0000 7fc8f5600700 1 -- 172.21.15.79:0/3676254954 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc8ec022020 con 0x7fc8f8081ed0 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.804+0000 7fc8f5600700 1 --2- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc8e0039050 0x7fc8e003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:06.761 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.804+0000 7fc8f5600700 1 -- 172.21.15.79:0/3676254954 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc8ec011e00 con 0x7fc8f8081ed0 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.804+0000 7fc8f7e00700 1 --2- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc8e0039050 0x7fc8e003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.805+0000 7fc8f7e00700 1 --2- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc8e0039050 0x7fc8e003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4006fd0 tx=0x7fc8e4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:05.809+0000 7fc8f5600700 1 -- 172.21.15.79:0/3676254954 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc8ec04cb00 con 0x7fc8f8081ed0 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.028+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc8f81a5aa0 con 0x7fc8f8081ed0 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.029+0000 7fc8f5600700 1 -- 172.21.15.79:0/3676254954 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fc8f81a5aa0 con 0x7fc8f8081ed0 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc8e0039050 msgr2=0x7fc8e003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 --2- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc8e0039050 0x7fc8e003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4006fd0 tx=0x7fc8e4006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 msgr2=0x7fc8f81a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 --2- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f81a2410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec011b60 tx=0x7fc8ec011c40 comp rx=0 tx=0).stop 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 shutdown_connections 2024-06-03T02:05:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 --2- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc8e0039050 0x7fc8e003b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:06.763 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 --2- 172.21.15.79:0/3676254954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f8081ed0 0x7fc8f81a2410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:06.763 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 >> 172.21.15.79:0/3676254954 conn(0x7fc8f807d6b0 msgr2=0x7fc8f807f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:06.763 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 shutdown_connections 2024-06-03T02:05:06.763 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:06.032+0000 7fc8ff600700 1 -- 172.21.15.79:0/3676254954 wait complete. 2024-06-03T02:05:06.763 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling cephadm module... 2024-06-03T02:05:07.996 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:07 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3241662776' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-03T02:05:07.996 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:07 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: ignoring --setuser ceph since I am not root 2024-06-03T02:05:07.996 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:07 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: ignoring --setgroup ceph since I am not root 2024-06-03T02:05:08.338 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:07 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:07.995+0000 7f303e59e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T02:05:08.338 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:08.231+0000 7f303e59e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:05:08.480 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.360+0000 7f67bb000700 1 Processor -- start 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.360+0000 7f67bb000700 1 -- start start 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.361+0000 7f67bb000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b410a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.361+0000 7f67bb000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f67b410aae0 con 0x7f67b410a0f0 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.361+0000 7f67b9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b410a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.362+0000 7f67b9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b410a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52332/0 (socket says 172.21.15.79:52332) 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.362+0000 7f67b9c00700 1 -- 172.21.15.79:0/2605175325 learned_addr learned my addr 172.21.15.79:0/2605175325 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.362+0000 7f67b9c00700 1 -- 172.21.15.79:0/2605175325 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b410b060 con 0x7f67b410a0f0 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.363+0000 7f67b9c00700 1 --2- 172.21.15.79:0/2605175325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b410a510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67a80098a0 tx=0x7f67a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d697c32930c0131b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.363+0000 7f67b3e00700 1 -- 172.21.15.79:0/2605175325 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a8004160 con 0x7f67b410a0f0 2024-06-03T02:05:08.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.363+0000 7f67b3e00700 1 -- 172.21.15.79:0/2605175325 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f67a8004300 con 0x7f67b410a0f0 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.363+0000 7f67b3e00700 1 -- 172.21.15.79:0/2605175325 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a8011b50 con 0x7f67b410a0f0 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.364+0000 7f67bb000700 1 -- 172.21.15.79:0/2605175325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 msgr2=0x7f67b410a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.364+0000 7f67bb000700 1 --2- 172.21.15.79:0/2605175325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b410a510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67a80098a0 tx=0x7f67a8009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.364+0000 7f67bb000700 1 -- 172.21.15.79:0/2605175325 shutdown_connections 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.364+0000 7f67bb000700 1 --2- 172.21.15.79:0/2605175325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b410a510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.364+0000 7f67bb000700 1 -- 172.21.15.79:0/2605175325 >> 172.21.15.79:0/2605175325 conn(0x7f67b407d360 msgr2=0x7f67b407d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.364+0000 7f67bb000700 1 -- 172.21.15.79:0/2605175325 shutdown_connections 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.364+0000 7f67bb000700 1 -- 172.21.15.79:0/2605175325 wait complete. 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.365+0000 7f67bb000700 1 Processor -- start 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.365+0000 7f67bb000700 1 -- start start 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.365+0000 7f67bb000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.365+0000 7f67bb000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f67b410e320 con 0x7f67b410a0f0 2024-06-03T02:05:08.487 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.366+0000 7f67b9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.366+0000 7f67b9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b419dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52338/0 (socket says 172.21.15.79:52338) 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.366+0000 7f67b9c00700 1 -- 172.21.15.79:0/3241662776 learned_addr learned my addr 172.21.15.79:0/3241662776 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.366+0000 7f67b9c00700 1 -- 172.21.15.79:0/3241662776 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b419e3e0 con 0x7f67b410a0f0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.366+0000 7f67b9c00700 1 --2- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b419dea0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009fb0 tx=0x7f67a8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.367+0000 7f67b2000700 1 -- 172.21.15.79:0/3241662776 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a80038b0 con 0x7f67b410a0f0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.367+0000 7f67b2000700 1 -- 172.21.15.79:0/3241662776 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f67a8011e50 con 0x7f67b410a0f0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.367+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b419e670 con 0x7f67b410a0f0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.367+0000 7f67b2000700 1 -- 172.21.15.79:0/3241662776 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a8019420 con 0x7f67b410a0f0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.367+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b41a1360 con 0x7f67b410a0f0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.368+0000 7f67b2000700 1 -- 172.21.15.79:0/3241662776 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f67a8003a50 con 0x7f67b410a0f0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.368+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b410e4a0 con 0x7f67b410a0f0 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.369+0000 7f67b2000700 1 --2- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f679c039000 0x7f679c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:08.488 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.369+0000 7f67b2000700 1 -- 172.21.15.79:0/3241662776 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f67a804d210 con 0x7f67b410a0f0 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.369+0000 7f67b9200700 1 --2- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f679c039000 0x7f679c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.370+0000 7f67b9200700 1 --2- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f679c039000 0x7f679c03b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006fd0 tx=0x7f67a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.374+0000 7f67b2000700 1 -- 172.21.15.79:0/3241662776 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f67a804cc30 con 0x7f67b410a0f0 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.658+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f67b41a19b0 con 0x7f67b410a0f0 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.712+0000 7f67b2000700 1 -- 172.21.15.79:0/3241662776 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f67a800fda0 con 0x7f67b410a0f0 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.715+0000 7f67b2000700 1 -- 172.21.15.79:0/3241662776 <== mon.0 v2:172.21.15.79:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f67a8023390 con 0x7f67b410a0f0 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.716+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f679c039000 msgr2=0x7f679c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.716+0000 7f67bb000700 1 --2- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f679c039000 0x7f679c03b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006fd0 tx=0x7f67a0006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.716+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 msgr2=0x7f67b419dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.716+0000 7f67bb000700 1 --2- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b419dea0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009fb0 tx=0x7f67a8011bf0 comp rx=0 tx=0).stop 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.716+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 shutdown_connections 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.716+0000 7f67bb000700 1 --2- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7f679c039000 0x7f679c03b4c0 secure :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006fd0 tx=0x7f67a0006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:08.489 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.716+0000 7f67bb000700 1 --2- 172.21.15.79:0/3241662776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67b410a0f0 0x7f67b419dea0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:08.490 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.716+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 >> 172.21.15.79:0/3241662776 conn(0x7f67b407d360 msgr2=0x7f67b4107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:08.490 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.717+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 shutdown_connections 2024-06-03T02:05:08.490 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:07.717+0000 7f67bb000700 1 -- 172.21.15.79:0/3241662776 wait complete. 2024-06-03T02:05:08.629 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:08.363+0000 7f303e59e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T02:05:08.889 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:08.766+0000 7f303e59e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T02:05:08.889 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:08 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3241662776' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-03T02:05:08.890 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:08 smithi079 ceph-mon[31819]: mgrmap e5: x(active, since 6s) 2024-06-03T02:05:09.146 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:08.888+0000 7f303e59e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T02:05:09.831 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:09 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/104928662' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-03T02:05:10.191 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-06-03T02:05:10.191 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-03T02:05:10.191 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T02:05:10.191 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-03T02:05:10.191 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.146+0000 7fa9bf200700 1 Processor -- start 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.146+0000 7fa9bf200700 1 -- start start 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.147+0000 7fa9bf200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.147+0000 7fa9bf200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b807c810 con 0x7fa9b807ddd0 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.147+0000 7fa9bde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.147+0000 7fa9bde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52350/0 (socket says 172.21.15.79:52350) 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.147+0000 7fa9bde00700 1 -- 172.21.15.79:0/4016506867 learned_addr learned my addr 172.21.15.79:0/4016506867 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.147+0000 7fa9bde00700 1 -- 172.21.15.79:0/4016506867 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b807c990 con 0x7fa9b807ddd0 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.148+0000 7fa9bde00700 1 --2- 172.21.15.79:0/4016506867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b807c2d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098a0 tx=0x7fa9ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b0f70d5f94bf42 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.148+0000 7fa9bca00700 1 -- 172.21.15.79:0/4016506867 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac004160 con 0x7fa9b807ddd0 2024-06-03T02:05:10.192 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.148+0000 7fa9bca00700 1 -- 172.21.15.79:0/4016506867 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9ac004300 con 0x7fa9b807ddd0 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.148+0000 7fa9bca00700 1 -- 172.21.15.79:0/4016506867 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac011b50 con 0x7fa9b807ddd0 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.149+0000 7fa9bf200700 1 -- 172.21.15.79:0/4016506867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 msgr2=0x7fa9b807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.149+0000 7fa9bf200700 1 --2- 172.21.15.79:0/4016506867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b807c2d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098a0 tx=0x7fa9ac009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.149+0000 7fa9bf200700 1 -- 172.21.15.79:0/4016506867 shutdown_connections 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.149+0000 7fa9bf200700 1 --2- 172.21.15.79:0/4016506867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b807c2d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.149+0000 7fa9bf200700 1 -- 172.21.15.79:0/4016506867 >> 172.21.15.79:0/4016506867 conn(0x7fa9b81032d0 msgr2=0x7fa9b8105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.149+0000 7fa9bf200700 1 -- 172.21.15.79:0/4016506867 shutdown_connections 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.149+0000 7fa9bf200700 1 -- 172.21.15.79:0/4016506867 wait complete. 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.150+0000 7fa9bf200700 1 Processor -- start 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.150+0000 7fa9bf200700 1 -- start start 2024-06-03T02:05:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.150+0000 7fa9bf200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.150+0000 7fa9bf200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b810e2a0 con 0x7fa9b807ddd0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.151+0000 7fa9bde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.151+0000 7fa9bde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52366/0 (socket says 172.21.15.79:52366) 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.151+0000 7fa9bde00700 1 -- 172.21.15.79:0/104928662 learned_addr learned my addr 172.21.15.79:0/104928662 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.151+0000 7fa9bde00700 1 -- 172.21.15.79:0/104928662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b819e3f0 con 0x7fa9b807ddd0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.151+0000 7fa9bde00700 1 --2- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b819deb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009fb0 tx=0x7fa9ac011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.151+0000 7fa9b2a00700 1 -- 172.21.15.79:0/104928662 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac0038b0 con 0x7fa9b807ddd0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.152+0000 7fa9b2a00700 1 -- 172.21.15.79:0/104928662 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9ac011e50 con 0x7fa9b807ddd0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.152+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b819e680 con 0x7fa9b807ddd0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.152+0000 7fa9b2a00700 1 -- 172.21.15.79:0/104928662 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac019420 con 0x7fa9b807ddd0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.152+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b81a1370 con 0x7fa9b807ddd0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.153+0000 7fa9b2a00700 1 -- 172.21.15.79:0/104928662 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fa9ac003a50 con 0x7fa9b807ddd0 2024-06-03T02:05:10.194 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.153+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b8063af0 con 0x7fa9b807ddd0 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.153+0000 7fa9b2a00700 1 --2- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fa9a0039000 0x7fa9a003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.153+0000 7fa9b2a00700 1 -- 172.21.15.79:0/104928662 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa9ac04d1f0 con 0x7fa9b807ddd0 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.153+0000 7fa9bd400700 1 -- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fa9a0039000 msgr2=0x7fa9a003b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/1845575623 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.153+0000 7fa9bd400700 1 --2- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fa9a0039000 0x7fa9a003b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.159+0000 7fa9b2a00700 1 -- 172.21.15.79:0/104928662 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa9ac00fda0 con 0x7fa9b807ddd0 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.354+0000 7fa9bd400700 1 -- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fa9a0039000 msgr2=0x7fa9a003b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/1845575623 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.354+0000 7fa9bd400700 1 --2- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fa9a0039000 0x7fa9a003b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.475+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa9b81a1540 con 0x7fa9b807ddd0 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.476+0000 7fa9b2a00700 1 -- 172.21.15.79:0/104928662 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fa9b81a1540 con 0x7fa9b807ddd0 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fa9a0039000 msgr2=0x7fa9a003b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 --2- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fa9a0039000 0x7fa9a003b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 msgr2=0x7fa9b819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 --2- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b819deb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009fb0 tx=0x7fa9ac011bf0 comp rx=0 tx=0).stop 2024-06-03T02:05:10.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 shutdown_connections 2024-06-03T02:05:10.196 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 --2- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fa9a0039000 0x7fa9a003b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:10.196 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 --2- 172.21.15.79:0/104928662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9b807ddd0 0x7fa9b819deb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:10.196 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 >> 172.21.15.79:0/104928662 conn(0x7fa9b81032d0 msgr2=0x7fa9b8103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:10.196 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 shutdown_connections 2024-06-03T02:05:10.196 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:09.479+0000 7fa9bf200700 1 -- 172.21.15.79:0/104928662 wait complete. 2024-06-03T02:05:10.196 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for the mgr to restart... 2024-06-03T02:05:10.196 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mgr epoch 5... 2024-06-03T02:05:10.737 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:10 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:10.444+0000 7f303e59e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T02:05:10.737 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:10 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:10.669+0000 7f303e59e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T02:05:11.031 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:10 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:10.888+0000 7f303e59e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T02:05:11.338 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:11 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:11.277+0000 7f303e59e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T02:05:11.668 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:11 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:11.407+0000 7f303e59e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T02:05:11.928 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:11 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:11.668+0000 7f303e59e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T02:05:12.233 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:11 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:11.927+0000 7f303e59e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:05:13.088 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:12 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:12.735+0000 7f303e59e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T02:05:13.588 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:13 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:13.278+0000 7f303e59e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T02:05:13.588 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:13 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:13.404+0000 7f303e59e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T02:05:14.088 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:13 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:13.667+0000 7f303e59e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T02:05:14.588 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:14.304+0000 7f303e59e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T02:05:14.588 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:14.424+0000 7f303e59e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T02:05:15.043 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:14.740+0000 7f303e59e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T02:05:15.043 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:14.917+0000 7f303e59e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T02:05:15.043 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:15 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:15.042+0000 7f303e59e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T02:05:15.338 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:15 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:15.192+0000 7f303e59e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T02:05:15.838 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:15 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:15.556+0000 7f303e59e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T02:05:16.588 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:16.303+0000 7f303e59e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T02:05:17.088 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:16.587+0000 7f303e59e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T02:05:17.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: Active manager daemon x restarted 2024-06-03T02:05:17.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: Activating manager daemon x 2024-06-03T02:05:17.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: osdmap e2: 0 total, 0 up, 0 in 2024-06-03T02:05:17.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: mgrmap e6: x(active, starting, since 0.0092056s) 2024-06-03T02:05:17.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:05:17.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T02:05:17.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T02:05:17.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T02:05:17.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T02:05:17.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: Manager daemon x is now available 2024-06-03T02:05:17.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:16 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:18.069 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: Found migration_current of "None". Setting to last migration. 2024-06-03T02:05:18.069 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:18.069 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:05:18.069 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:05:18.069 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T02:05:18.070 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T02:05:18.070 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:18.070 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:18.070 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:05:18.070 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:17 smithi079 ceph-mon[31819]: mgrmap e7: x(active, since 1.01628s) 2024-06-03T02:05:18.397 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-06-03T02:05:18.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-03T02:05:18.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-03T02:05:18.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-06-03T02:05:18.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.822+0000 7fc2b2000700 1 Processor -- start 2024-06-03T02:05:18.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.822+0000 7fc2b2000700 1 -- start start 2024-06-03T02:05:18.399 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.822+0000 7fc2b2000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac07c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:18.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.822+0000 7fc2b2000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac07c850 con 0x7fc2ac07de10 2024-06-03T02:05:18.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.823+0000 7fc2ab400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac07c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:18.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.823+0000 7fc2ab400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac07c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52380/0 (socket says 172.21.15.79:52380) 2024-06-03T02:05:18.400 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.823+0000 7fc2ab400700 1 -- 172.21.15.79:0/1295504296 learned_addr learned my addr 172.21.15.79:0/1295504296 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:18.405 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.823+0000 7fc2ab400700 1 -- 172.21.15.79:0/1295504296 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac07c9d0 con 0x7fc2ac07de10 2024-06-03T02:05:18.405 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.823+0000 7fc2ab400700 1 --2- 172.21.15.79:0/1295504296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac07c310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0098a0 tx=0x7fc29c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedab4de35c946d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:18.405 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.824+0000 7fc2aa000700 1 -- 172.21.15.79:0/1295504296 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc29c004160 con 0x7fc2ac07de10 2024-06-03T02:05:18.405 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.824+0000 7fc2aa000700 1 -- 172.21.15.79:0/1295504296 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc29c004300 con 0x7fc2ac07de10 2024-06-03T02:05:18.405 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.824+0000 7fc2aa000700 1 -- 172.21.15.79:0/1295504296 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc29c011b50 con 0x7fc2ac07de10 2024-06-03T02:05:18.405 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.824+0000 7fc2b2000700 1 -- 172.21.15.79:0/1295504296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 msgr2=0x7fc2ac07c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:18.405 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.824+0000 7fc2b2000700 1 --2- 172.21.15.79:0/1295504296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac07c310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0098a0 tx=0x7fc29c009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:18.405 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.824+0000 7fc2b2000700 1 -- 172.21.15.79:0/1295504296 shutdown_connections 2024-06-03T02:05:18.405 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.824+0000 7fc2b2000700 1 --2- 172.21.15.79:0/1295504296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac07c310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:18.406 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.824+0000 7fc2b2000700 1 -- 172.21.15.79:0/1295504296 >> 172.21.15.79:0/1295504296 conn(0x7fc2ac1035e0 msgr2=0x7fc2ac105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:18.406 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.825+0000 7fc2b2000700 1 -- 172.21.15.79:0/1295504296 shutdown_connections 2024-06-03T02:05:18.406 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.825+0000 7fc2b2000700 1 -- 172.21.15.79:0/1295504296 wait complete. 2024-06-03T02:05:18.406 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.826+0000 7fc2b2000700 1 Processor -- start 2024-06-03T02:05:18.406 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.826+0000 7fc2b2000700 1 -- start start 2024-06-03T02:05:18.406 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.827+0000 7fc2b2000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:18.406 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.827+0000 7fc2b2000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac10e4b0 con 0x7fc2ac07de10 2024-06-03T02:05:18.406 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.827+0000 7fc2ab400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:18.407 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.827+0000 7fc2ab400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac19e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52394/0 (socket says 172.21.15.79:52394) 2024-06-03T02:05:18.407 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.827+0000 7fc2ab400700 1 -- 172.21.15.79:0/3543455101 learned_addr learned my addr 172.21.15.79:0/3543455101 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:18.407 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.827+0000 7fc2ab400700 1 -- 172.21.15.79:0/3543455101 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac19e550 con 0x7fc2ac07de10 2024-06-03T02:05:18.407 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.828+0000 7fc2ab400700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac19e010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc29c011670 tx=0x7fc29c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:18.407 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.828+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc29c0039c0 con 0x7fc2ac07de10 2024-06-03T02:05:18.407 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.828+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc29c011e50 con 0x7fc2ac07de10 2024-06-03T02:05:18.407 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.828+0000 7fc2b2000700 1 -- 172.21.15.79:0/3543455101 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac19e7e0 con 0x7fc2ac07de10 2024-06-03T02:05:18.407 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.828+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc29c01a5c0 con 0x7fc2ac07de10 2024-06-03T02:05:18.407 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.828+0000 7fc2b2000700 1 -- 172.21.15.79:0/3543455101 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1a14d0 con 0x7fc2ac07de10 2024-06-03T02:05:18.408 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.829+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fc29c024430 con 0x7fc2ac07de10 2024-06-03T02:05:18.408 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.830+0000 7fc2a3e00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 0x7fc29003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:18.408 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.830+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 --> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc29003bc20 con 0x7fc290039050 2024-06-03T02:05:18.408 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.830+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc29c04d410 con 0x7fc2ac07de10 2024-06-03T02:05:18.408 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.830+0000 7fc2aaa00700 1 -- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 msgr2=0x7fc29003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/1845575623 2024-06-03T02:05:18.408 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:10.830+0000 7fc2aaa00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 0x7fc29003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T02:05:18.408 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:11.031+0000 7fc2aaa00700 1 -- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 msgr2=0x7fc29003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/1845575623 2024-06-03T02:05:18.408 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:11.031+0000 7fc2aaa00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 0x7fc29003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:11.432+0000 7fc2aaa00700 1 -- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 msgr2=0x7fc29003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/1845575623 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:11.432+0000 7fc2aaa00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 0x7fc29003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:12.233+0000 7fc2aaa00700 1 -- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 msgr2=0x7fc29003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/1845575623 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:12.233+0000 7fc2aaa00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 0x7fc29003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:13.835+0000 7fc2aaa00700 1 -- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 msgr2=0x7fc29003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/1845575623 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:13.835+0000 7fc2aaa00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 0x7fc29003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:16.596+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fc29c024770 con 0x7fc2ac07de10 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:16.596+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 msgr2=0x7fc29003b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:16.596+0000 7fc2a3e00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 0x7fc29003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:18.409 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.600+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fc29c04ef20 con 0x7fc2ac07de10 2024-06-03T02:05:18.411 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.600+0000 7fc2a3e00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fc29003caa0 0x7fc29003ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:18.411 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.600+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc29c025e70 con 0x7fc29003caa0 2024-06-03T02:05:18.411 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.602+0000 7fc2aaa00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fc29003caa0 0x7fc29003ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.607+0000 7fc2aaa00700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fc29003caa0 0x7fc29003ee90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc298003a10 tx=0x7fc2980092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.609+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 <== mgr.14120 v2:172.21.15.79:6800/54480245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fc29c025e70 con 0x7fc29003caa0 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.622+0000 7fc2b2000700 1 -- 172.21.15.79:0/3543455101 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc2ac063af0 con 0x7fc29003caa0 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.622+0000 7fc2a3e00700 1 -- 172.21.15.79:0/3543455101 <== mgr.14120 v2:172.21.15.79:6800/54480245 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fc2ac063af0 con 0x7fc29003caa0 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.623+0000 7fc2b2000700 1 -- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fc29003caa0 msgr2=0x7fc29003ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.623+0000 7fc2b2000700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fc29003caa0 0x7fc29003ee90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc298003a10 tx=0x7fc2980092b0 comp rx=0 tx=0).stop 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.623+0000 7fc2b2000700 1 -- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 msgr2=0x7fc2ac19e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.623+0000 7fc2b2000700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac19e010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc29c011670 tx=0x7fc29c011b60 comp rx=0 tx=0).stop 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.623+0000 7fc2b2000700 1 -- 172.21.15.79:0/3543455101 shutdown_connections 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.623+0000 7fc2b2000700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fc29003caa0 0x7fc29003ee90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.623+0000 7fc2b2000700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2ac07de10 0x7fc2ac19e010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.623+0000 7fc2b2000700 1 --2- 172.21.15.79:0/3543455101 >> [v2:172.21.15.79:6800/1845575623,v1:172.21.15.79:6801/1845575623] conn(0x7fc290039050 0x7fc29003b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.623+0000 7fc2b2000700 1 -- 172.21.15.79:0/3543455101 >> 172.21.15.79:0/3543455101 conn(0x7fc2ac1035e0 msgr2=0x7fc2ac1040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:18.412 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.624+0000 7fc2b2000700 1 -- 172.21.15.79:0/3543455101 shutdown_connections 2024-06-03T02:05:18.413 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:17.624+0000 7fc2b2000700 1 -- 172.21.15.79:0/3543455101 wait complete. 2024-06-03T02:05:18.413 INFO:teuthology.orchestra.run.smithi079.stdout:mgr epoch 5 is available 2024-06-03T02:05:18.413 INFO:teuthology.orchestra.run.smithi079.stdout:Setting orchestrator backend to cephadm... 2024-06-03T02:05:18.960 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:18 smithi079 ceph-mon[31819]: [03/Jun/2024:02:05:17] ENGINE Bus STARTING 2024-06-03T02:05:18.960 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:18 smithi079 ceph-mon[31819]: [03/Jun/2024:02:05:17] ENGINE Serving on http://172.21.15.79:8765 2024-06-03T02:05:18.960 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:18 smithi079 ceph-mon[31819]: [03/Jun/2024:02:05:17] ENGINE Serving on https://172.21.15.79:7150 2024-06-03T02:05:18.960 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:18 smithi079 ceph-mon[31819]: [03/Jun/2024:02:05:17] ENGINE Bus STARTED 2024-06-03T02:05:18.960 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:18 smithi079 ceph-mon[31819]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-03T02:05:18.960 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:18 smithi079 ceph-mon[31819]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-03T02:05:19.980 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:19 smithi079 ceph-mon[31819]: mgrmap e8: x(active, since 2s) 2024-06-03T02:05:19.980 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:19 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:19.980 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:19 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:05:19.989 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.023+0000 7f1eace00700 1 Processor -- start 2024-06-03T02:05:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.023+0000 7f1eace00700 1 -- start start 2024-06-03T02:05:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.024+0000 7f1eace00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea81071f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.024+0000 7f1eace00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81077c0 con 0x7f1ea8106dd0 2024-06-03T02:05:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.024+0000 7f1ea5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea81071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.024+0000 7f1ea5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea81071f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50784/0 (socket says 172.21.15.79:50784) 2024-06-03T02:05:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.024+0000 7f1ea5e00700 1 -- 172.21.15.79:0/583375775 learned_addr learned my addr 172.21.15.79:0/583375775 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.025+0000 7f1ea5e00700 1 -- 172.21.15.79:0/583375775 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea8107fd0 con 0x7f1ea8106dd0 2024-06-03T02:05:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.025+0000 7f1ea5e00700 1 --2- 172.21.15.79:0/583375775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea81071f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1e980098a0 tx=0x7f1e98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3495ee79ec601681 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.026+0000 7f1ea4a00700 1 -- 172.21.15.79:0/583375775 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e98004160 con 0x7f1ea8106dd0 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.026+0000 7f1ea4a00700 1 -- 172.21.15.79:0/583375775 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1e98004300 con 0x7f1ea8106dd0 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.026+0000 7f1ea4a00700 1 -- 172.21.15.79:0/583375775 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e98011b50 con 0x7f1ea8106dd0 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.026+0000 7f1eace00700 1 -- 172.21.15.79:0/583375775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 msgr2=0x7f1ea81071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.026+0000 7f1eace00700 1 --2- 172.21.15.79:0/583375775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea81071f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1e980098a0 tx=0x7f1e98009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.026+0000 7f1eace00700 1 -- 172.21.15.79:0/583375775 shutdown_connections 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.026+0000 7f1eace00700 1 --2- 172.21.15.79:0/583375775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea81071f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.026+0000 7f1eace00700 1 -- 172.21.15.79:0/583375775 >> 172.21.15.79:0/583375775 conn(0x7f1ea81023f0 msgr2=0x7f1ea8104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.027+0000 7f1eace00700 1 -- 172.21.15.79:0/583375775 shutdown_connections 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.027+0000 7f1eace00700 1 -- 172.21.15.79:0/583375775 wait complete. 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.028+0000 7f1eace00700 1 Processor -- start 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.028+0000 7f1eace00700 1 -- start start 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.028+0000 7f1eace00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.028+0000 7f1eace00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea810b380 con 0x7f1ea8106dd0 2024-06-03T02:05:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.028+0000 7f1ea5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.028+0000 7f1ea5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea819e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50788/0 (socket says 172.21.15.79:50788) 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.028+0000 7f1ea5e00700 1 -- 172.21.15.79:0/2049555694 learned_addr learned my addr 172.21.15.79:0/2049555694 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.029+0000 7f1ea5e00700 1 -- 172.21.15.79:0/2049555694 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea819e610 con 0x7f1ea8106dd0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.029+0000 7f1ea5e00700 1 --2- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea819e0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1e98000c00 tx=0x7f1e98011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.029+0000 7f1e9ea00700 1 -- 172.21.15.79:0/2049555694 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e98003b20 con 0x7f1ea8106dd0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.029+0000 7f1e9ea00700 1 -- 172.21.15.79:0/2049555694 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1e98003cc0 con 0x7f1ea8106dd0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.029+0000 7f1e9ea00700 1 -- 172.21.15.79:0/2049555694 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e98024910 con 0x7f1ea8106dd0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.029+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea819e8a0 con 0x7f1ea8106dd0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.029+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea81a1590 con 0x7f1ea8106dd0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.031+0000 7f1e9ea00700 1 -- 172.21.15.79:0/2049555694 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1e98022020 con 0x7f1ea8106dd0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.031+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea8063af0 con 0x7f1ea8106dd0 2024-06-03T02:05:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.031+0000 7f1e9ea00700 1 --2- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1e8c0390a0 0x7f1e8c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.031+0000 7f1e9ea00700 1 -- 172.21.15.79:0/2049555694 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1e98011e00 con 0x7f1ea8106dd0 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.031+0000 7f1ea5400700 1 --2- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1e8c0390a0 0x7f1e8c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.032+0000 7f1ea5400700 1 --2- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1e8c0390a0 0x7f1e8c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e94006fd0 tx=0x7f1e94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.037+0000 7f1e9ea00700 1 -- 172.21.15.79:0/2049555694 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1e9801ad80 con 0x7f1ea8106dd0 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.272+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f1ea80008d0 con 0x7f1e8c0390a0 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.281+0000 7f1e9ea00700 1 -- 172.21.15.79:0/2049555694 <== mgr.14120 v2:172.21.15.79:6800/54480245 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1ea80008d0 con 0x7f1e8c0390a0 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.284+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1e8c0390a0 msgr2=0x7f1e8c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.284+0000 7f1eace00700 1 --2- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1e8c0390a0 0x7f1e8c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e94006fd0 tx=0x7f1e94006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.284+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 msgr2=0x7f1ea819e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.284+0000 7f1eace00700 1 --2- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea819e0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1e98000c00 tx=0x7f1e98011c10 comp rx=0 tx=0).stop 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.285+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 shutdown_connections 2024-06-03T02:05:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.285+0000 7f1eace00700 1 --2- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1e8c0390a0 0x7f1e8c03b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:20.000 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.285+0000 7f1eace00700 1 --2- 172.21.15.79:0/2049555694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ea8106dd0 0x7f1ea819e0d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:20.000 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.285+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 >> 172.21.15.79:0/2049555694 conn(0x7f1ea81023f0 msgr2=0x7f1ea8103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:20.000 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.285+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 shutdown_connections 2024-06-03T02:05:20.000 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:19.285+0000 7f1eace00700 1 -- 172.21.15.79:0/2049555694 wait complete. 2024-06-03T02:05:20.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:20 smithi079 ceph-mon[31819]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:05:21.572 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-03T02:05:21.572 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.612+0000 7f58a2400700 1 Processor -- start 2024-06-03T02:05:21.572 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.612+0000 7f58a2400700 1 -- start start 2024-06-03T02:05:21.572 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.612+0000 7f58a2400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c10a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:21.572 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.612+0000 7f58a2400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f589c10aaa0 con 0x7f589c10a0b0 2024-06-03T02:05:21.575 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.613+0000 7f589b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c10a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:21.575 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.613+0000 7f589b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c10a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50798/0 (socket says 172.21.15.79:50798) 2024-06-03T02:05:21.575 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.613+0000 7f589b400700 1 -- 172.21.15.79:0/1123862525 learned_addr learned my addr 172.21.15.79:0/1123862525 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:21.575 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.613+0000 7f589b400700 1 -- 172.21.15.79:0/1123862525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c10b2c0 con 0x7f589c10a0b0 2024-06-03T02:05:21.575 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.614+0000 7f589b400700 1 --2- 172.21.15.79:0/1123862525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c10a4d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f588c0098a0 tx=0x7f588c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dedab91458280c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:21.575 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.614+0000 7f589a000700 1 -- 172.21.15.79:0/1123862525 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f588c004160 con 0x7f589c10a0b0 2024-06-03T02:05:21.575 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.614+0000 7f589a000700 1 -- 172.21.15.79:0/1123862525 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f588c004300 con 0x7f589c10a0b0 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.614+0000 7f589a000700 1 -- 172.21.15.79:0/1123862525 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f588c011b50 con 0x7f589c10a0b0 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.615+0000 7f58a2400700 1 -- 172.21.15.79:0/1123862525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 msgr2=0x7f589c10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.615+0000 7f58a2400700 1 --2- 172.21.15.79:0/1123862525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c10a4d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f588c0098a0 tx=0x7f588c009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.615+0000 7f58a2400700 1 -- 172.21.15.79:0/1123862525 shutdown_connections 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.615+0000 7f58a2400700 1 --2- 172.21.15.79:0/1123862525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c10a4d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.615+0000 7f58a2400700 1 -- 172.21.15.79:0/1123862525 >> 172.21.15.79:0/1123862525 conn(0x7f589c07d6b0 msgr2=0x7f589c07dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.615+0000 7f58a2400700 1 -- 172.21.15.79:0/1123862525 shutdown_connections 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.615+0000 7f58a2400700 1 -- 172.21.15.79:0/1123862525 wait complete. 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.616+0000 7f58a2400700 1 Processor -- start 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.616+0000 7f58a2400700 1 -- start start 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.617+0000 7f58a2400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c19e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.617+0000 7f58a2400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f589c10e670 con 0x7f589c10a0b0 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.617+0000 7f589b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c19e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:21.576 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.617+0000 7f589b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c19e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50814/0 (socket says 172.21.15.79:50814) 2024-06-03T02:05:21.577 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.617+0000 7f589b400700 1 -- 172.21.15.79:0/746556786 learned_addr learned my addr 172.21.15.79:0/746556786 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:21.577 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.617+0000 7f589b400700 1 -- 172.21.15.79:0/746556786 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c19e590 con 0x7f589c10a0b0 2024-06-03T02:05:21.579 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.618+0000 7f589b400700 1 --2- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c19e050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f588c000c00 tx=0x7f588c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:21.579 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.618+0000 7f5893e00700 1 -- 172.21.15.79:0/746556786 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f588c003b20 con 0x7f589c10a0b0 2024-06-03T02:05:21.579 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.618+0000 7f5893e00700 1 -- 172.21.15.79:0/746556786 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f588c003cc0 con 0x7f589c10a0b0 2024-06-03T02:05:21.579 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.618+0000 7f5893e00700 1 -- 172.21.15.79:0/746556786 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f588c024910 con 0x7f589c10a0b0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.618+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c19e820 con 0x7f589c10a0b0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.618+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c1a1510 con 0x7f589c10a0b0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.619+0000 7f5893e00700 1 -- 172.21.15.79:0/746556786 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f588c022020 con 0x7f589c10a0b0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.619+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c063af0 con 0x7f589c10a0b0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.619+0000 7f5893e00700 1 --2- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f58800390a0 0x7f588003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.620+0000 7f5893e00700 1 -- 172.21.15.79:0/746556786 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f588c011e00 con 0x7f589c10a0b0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.620+0000 7f589aa00700 1 --2- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f58800390a0 0x7f588003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.620+0000 7f589aa00700 1 --2- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f58800390a0 0x7f588003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5888006fd0 tx=0x7f5888006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.625+0000 7f5893e00700 1 -- 172.21.15.79:0/746556786 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f588c01ad80 con 0x7f589c10a0b0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.850+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f589c108460 con 0x7f58800390a0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.852+0000 7f5893e00700 1 -- 172.21.15.79:0/746556786 <== mgr.14120 v2:172.21.15.79:6800/54480245 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f589c108460 con 0x7f58800390a0 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f58800390a0 msgr2=0x7f588003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:21.580 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 --2- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f58800390a0 0x7f588003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5888006fd0 tx=0x7f5888006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:21.581 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 msgr2=0x7f589c19e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:21.581 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 --2- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c19e050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f588c000c00 tx=0x7f588c011c10 comp rx=0 tx=0).stop 2024-06-03T02:05:21.581 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 shutdown_connections 2024-06-03T02:05:21.581 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 --2- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f58800390a0 0x7f588003b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:21.581 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 --2- 172.21.15.79:0/746556786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f589c10a0b0 0x7f589c19e050 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:21.581 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 >> 172.21.15.79:0/746556786 conn(0x7f589c07d6b0 msgr2=0x7f589c107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:21.581 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 shutdown_connections 2024-06-03T02:05:21.581 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:20.855+0000 7f58a2400700 1 -- 172.21.15.79:0/746556786 wait complete. 2024-06-03T02:05:21.581 INFO:teuthology.orchestra.run.smithi079.stdout:Generating ssh key... 2024-06-03T02:05:22.188 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:21 smithi079 ceph-mon[31819]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:05:23.297 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-mon[31819]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:05:23.297 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-mon[31819]: Generating ssh key... 2024-06-03T02:05:23.297 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: Generating public/private rsa key pair. 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: Your identification has been saved in /tmp/tmpgcmyvt5i/key. 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: Your public key has been saved in /tmp/tmpgcmyvt5i/key.pub. 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: The key fingerprint is: 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: SHA256:wkFUZbOH/k02JK0Qwb/GRFgWoZ6Zk7zLpyqkQy3b52I ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: The key's randomart image is: 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: +---[RSA 3072]----+ 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: | .o..o*+=o | 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: | . .oB.. | 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: | . =oo o | 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: | . . + Bo+ | 2024-06-03T02:05:23.298 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: | .o S Xo..+ | 2024-06-03T02:05:23.299 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: | o o. +++ . | 2024-06-03T02:05:23.299 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: | . * ... . | 2024-06-03T02:05:23.299 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: | + E .. .. | 2024-06-03T02:05:23.299 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: | o =o.+o | 2024-06-03T02:05:23.299 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: +----[SHA256]-----+ 2024-06-03T02:05:23.614 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.206+0000 7f4d02200700 1 Processor -- start 2024-06-03T02:05:23.619 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.206+0000 7f4d02200700 1 -- start start 2024-06-03T02:05:23.619 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.207+0000 7f4d02200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc10a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:23.619 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.207+0000 7f4d02200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc10aaf0 con 0x7f4cfc10a100 2024-06-03T02:05:23.619 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.207+0000 7f4cfb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc10a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:23.619 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.207+0000 7f4cfb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc10a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50824/0 (socket says 172.21.15.79:50824) 2024-06-03T02:05:23.619 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.207+0000 7f4cfb400700 1 -- 172.21.15.79:0/1532923174 learned_addr learned my addr 172.21.15.79:0/1532923174 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.208+0000 7f4cfb400700 1 -- 172.21.15.79:0/1532923174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc10b2b0 con 0x7f4cfc10a100 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.208+0000 7f4cfb400700 1 --2- 172.21.15.79:0/1532923174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc10a520 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0098a0 tx=0x7f4cec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76f545d7e5d04e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.208+0000 7f4cfa000700 1 -- 172.21.15.79:0/1532923174 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec004160 con 0x7f4cfc10a100 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.208+0000 7f4cfa000700 1 -- 172.21.15.79:0/1532923174 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4cec004300 con 0x7f4cfc10a100 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.208+0000 7f4cfa000700 1 -- 172.21.15.79:0/1532923174 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec011b50 con 0x7f4cfc10a100 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.209+0000 7f4d02200700 1 -- 172.21.15.79:0/1532923174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 msgr2=0x7f4cfc10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.209+0000 7f4d02200700 1 --2- 172.21.15.79:0/1532923174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc10a520 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0098a0 tx=0x7f4cec009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.209+0000 7f4d02200700 1 -- 172.21.15.79:0/1532923174 shutdown_connections 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.209+0000 7f4d02200700 1 --2- 172.21.15.79:0/1532923174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc10a520 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.209+0000 7f4d02200700 1 -- 172.21.15.79:0/1532923174 >> 172.21.15.79:0/1532923174 conn(0x7f4cfc07d6b0 msgr2=0x7f4cfc07dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.209+0000 7f4d02200700 1 -- 172.21.15.79:0/1532923174 shutdown_connections 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.209+0000 7f4d02200700 1 -- 172.21.15.79:0/1532923174 wait complete. 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.210+0000 7f4d02200700 1 Processor -- start 2024-06-03T02:05:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.210+0000 7f4d02200700 1 -- start start 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.211+0000 7f4d02200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc19e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.211+0000 7f4d02200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc10e660 con 0x7f4cfc10a100 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.211+0000 7f4cfb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc19e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.211+0000 7f4cfb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc19e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50836/0 (socket says 172.21.15.79:50836) 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.211+0000 7f4cfb400700 1 -- 172.21.15.79:0/1124609303 learned_addr learned my addr 172.21.15.79:0/1124609303 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.211+0000 7f4cfb400700 1 -- 172.21.15.79:0/1124609303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc19e580 con 0x7f4cfc10a100 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.212+0000 7f4cfb400700 1 --2- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc19e040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4cec011670 tx=0x7f4cec011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.212+0000 7f4cf3e00700 1 -- 172.21.15.79:0/1124609303 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec0039c0 con 0x7f4cfc10a100 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.212+0000 7f4cf3e00700 1 -- 172.21.15.79:0/1124609303 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4cec011e50 con 0x7f4cfc10a100 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.212+0000 7f4cf3e00700 1 -- 172.21.15.79:0/1124609303 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec01a5c0 con 0x7f4cfc10a100 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.212+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cfc19e810 con 0x7f4cfc10a100 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.212+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cfc1a1500 con 0x7f4cfc10a100 2024-06-03T02:05:23.621 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.213+0000 7f4cf3e00700 1 -- 172.21.15.79:0/1124609303 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4cec024430 con 0x7f4cfc10a100 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.213+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cfc063af0 con 0x7f4cfc10a100 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.214+0000 7f4cf3e00700 1 --2- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f4ce0039050 0x7f4ce003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.214+0000 7f4cf3e00700 1 -- 172.21.15.79:0/1124609303 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4cec04d4f0 con 0x7f4cfc10a100 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.214+0000 7f4cfaa00700 1 --2- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f4ce0039050 0x7f4ce003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.215+0000 7f4cfaa00700 1 --2- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f4ce0039050 0x7f4ce003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8006fd0 tx=0x7f4ce8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.219+0000 7f4cf3e00700 1 -- 172.21.15.79:0/1124609303 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4cec04ce40 con 0x7f4cfc10a100 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.441+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f4cfc107b80 con 0x7f4ce0039050 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.948+0000 7f4cf3e00700 1 -- 172.21.15.79:0/1124609303 <== mgr.14120 v2:172.21.15.79:6800/54480245 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4cfc107b80 con 0x7f4ce0039050 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f4ce0039050 msgr2=0x7f4ce003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 --2- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f4ce0039050 0x7f4ce003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8006fd0 tx=0x7f4ce8006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 msgr2=0x7f4cfc19e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 --2- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc19e040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4cec011670 tx=0x7f4cec011b60 comp rx=0 tx=0).stop 2024-06-03T02:05:23.622 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 shutdown_connections 2024-06-03T02:05:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 --2- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f4ce0039050 0x7f4ce003b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 --2- 172.21.15.79:0/1124609303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc10a100 0x7f4cfc19e040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 >> 172.21.15.79:0/1124609303 conn(0x7f4cfc07d6b0 msgr2=0x7f4cfc107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 shutdown_connections 2024-06-03T02:05:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:22.951+0000 7f4d02200700 1 -- 172.21.15.79:0/1124609303 wait complete. 2024-06-03T02:05:24.203 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:23 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:24.203 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:23 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:25.224 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:24 smithi079 ceph-mon[31819]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:05:25.242 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCiYUCumZdKhmuFl5RG+pbaJjAW2m8cnJsx9UL1U56NOf1kSPvE9gNyuxG4HSQkw5NX44ocwfy4v4OvIUS/Ol7WQsFRXv/DVnIQaEsclJIFzNFBK8E1dcsFEr8JJ8xUediSe+82L0YDP8Ibj8uPLHL7g1cNopCgJJfdcC0784ztuGZdmb6ZyvW0xCM2Q6NZUePVh4uy4WUx0Y8zePSmVfmW9S5yTZ4NBpn+exOF8lq+VMer6iCzFOPXmS+Ej+ltvZU4kwtCtjUeOHu6Jx42fWASisSABXp8IszVND3qI3H8A5xjU+J3zMSdEkogvYwN1ZOUnZUKTyUlNmMRx39zt79ErwvNHV2bKkcF7UlpNR4szRshbgp87JJw1u/xqUORiEGSjgUW5kiNDd7DAKiibIUgeaWj5obhEdzgmyikkt8iVJPlvgdx8tu5AWnbC2HuBp4ZhpMqWBRwMUSBHxpmw/8PD7D9RcFp9yw4LXJtsebwDHgoWEIJ7kqpjMUvOhq9MbE= ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:05:25.243 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.274+0000 7f41aac00700 1 Processor -- start 2024-06-03T02:05:25.243 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.274+0000 7f41aac00700 1 -- start start 2024-06-03T02:05:25.243 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.275+0000 7f41aac00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a407c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:25.243 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.275+0000 7f41aac00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f41a407c850 con 0x7f41a407de10 2024-06-03T02:05:25.243 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.275+0000 7f41a3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a407c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:25.243 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.275+0000 7f41a3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a407c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50848/0 (socket says 172.21.15.79:50848) 2024-06-03T02:05:25.243 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.275+0000 7f41a3e00700 1 -- 172.21.15.79:0/107281270 learned_addr learned my addr 172.21.15.79:0/107281270 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:25.246 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.276+0000 7f41a3e00700 1 -- 172.21.15.79:0/107281270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a407c9d0 con 0x7f41a407de10 2024-06-03T02:05:25.246 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.276+0000 7f41a3e00700 1 --2- 172.21.15.79:0/107281270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a407c310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f41940098a0 tx=0x7f4194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158b3cda7a63be69 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:25.246 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.276+0000 7f41a3400700 1 -- 172.21.15.79:0/107281270 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4194004160 con 0x7f41a407de10 2024-06-03T02:05:25.246 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.276+0000 7f41a3400700 1 -- 172.21.15.79:0/107281270 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4194004300 con 0x7f41a407de10 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.277+0000 7f41a3400700 1 -- 172.21.15.79:0/107281270 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4194011b50 con 0x7f41a407de10 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.277+0000 7f41aac00700 1 -- 172.21.15.79:0/107281270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 msgr2=0x7f41a407c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.277+0000 7f41aac00700 1 --2- 172.21.15.79:0/107281270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a407c310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f41940098a0 tx=0x7f4194009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.277+0000 7f41aac00700 1 -- 172.21.15.79:0/107281270 shutdown_connections 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.277+0000 7f41aac00700 1 --2- 172.21.15.79:0/107281270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a407c310 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.277+0000 7f41aac00700 1 -- 172.21.15.79:0/107281270 >> 172.21.15.79:0/107281270 conn(0x7f41a41035e0 msgr2=0x7f41a4105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.277+0000 7f41aac00700 1 -- 172.21.15.79:0/107281270 shutdown_connections 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.277+0000 7f41aac00700 1 -- 172.21.15.79:0/107281270 wait complete. 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.278+0000 7f41aac00700 1 Processor -- start 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.279+0000 7f41aac00700 1 -- start start 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.279+0000 7f41aac00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a410b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.279+0000 7f41aac00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f41a410e4b0 con 0x7f41a407de10 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.279+0000 7f41a3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a410b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:25.247 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.279+0000 7f41a3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a410b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50856/0 (socket says 172.21.15.79:50856) 2024-06-03T02:05:25.248 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.279+0000 7f41a3e00700 1 -- 172.21.15.79:0/3829810430 learned_addr learned my addr 172.21.15.79:0/3829810430 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:25.248 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.280+0000 7f41a3e00700 1 -- 172.21.15.79:0/3829810430 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a410b790 con 0x7f41a407de10 2024-06-03T02:05:25.248 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.280+0000 7f41a3e00700 1 --2- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a410b250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4194011b60 tx=0x7f4194011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:25.248 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.280+0000 7f41a1600700 1 -- 172.21.15.79:0/3829810430 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4194003b20 con 0x7f41a407de10 2024-06-03T02:05:25.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.280+0000 7f41a1600700 1 -- 172.21.15.79:0/3829810430 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4194003cc0 con 0x7f41a407de10 2024-06-03T02:05:25.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.280+0000 7f41a1600700 1 -- 172.21.15.79:0/3829810430 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4194024840 con 0x7f41a407de10 2024-06-03T02:05:25.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.280+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a410ba20 con 0x7f41a407de10 2024-06-03T02:05:25.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.280+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a4107fe0 con 0x7f41a407de10 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.281+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a410bbf0 con 0x7f41a407de10 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.281+0000 7f41a1600700 1 -- 172.21.15.79:0/3829810430 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4194022020 con 0x7f41a407de10 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.282+0000 7f41a1600700 1 --2- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f41880390a0 0x7f418803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.282+0000 7f41a1600700 1 -- 172.21.15.79:0/3829810430 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4194011e00 con 0x7f41a407de10 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.282+0000 7f419b400700 1 --2- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f41880390a0 0x7f418803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.283+0000 7f419b400700 1 --2- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f41880390a0 0x7f418803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f418c006fd0 tx=0x7f418c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.287+0000 7f41a1600700 1 -- 172.21.15.79:0/3829810430 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f41a410bbf0 con 0x7f41a407de10 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.513+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f41a4108780 con 0x7f41880390a0 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.515+0000 7f41a1600700 1 -- 172.21.15.79:0/3829810430 <== mgr.14120 v2:172.21.15.79:6800/54480245 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f41a4108780 con 0x7f41880390a0 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.518+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f41880390a0 msgr2=0x7f418803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.518+0000 7f41aac00700 1 --2- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f41880390a0 0x7f418803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f418c006fd0 tx=0x7f418c006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.518+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 msgr2=0x7f41a410b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.518+0000 7f41aac00700 1 --2- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a410b250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4194011b60 tx=0x7f4194011c40 comp rx=0 tx=0).stop 2024-06-03T02:05:25.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.518+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 shutdown_connections 2024-06-03T02:05:25.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.518+0000 7f41aac00700 1 --2- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f41880390a0 0x7f418803b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:25.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.518+0000 7f41aac00700 1 --2- 172.21.15.79:0/3829810430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41a407de10 0x7f41a410b250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:25.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.518+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 >> 172.21.15.79:0/3829810430 conn(0x7f41a41035e0 msgr2=0x7f41a41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:25.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.518+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 shutdown_connections 2024-06-03T02:05:25.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:24.519+0000 7f41aac00700 1 -- 172.21.15.79:0/3829810430 wait complete. 2024-06-03T02:05:25.252 INFO:teuthology.orchestra.run.smithi079.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-03T02:05:25.252 INFO:teuthology.orchestra.run.smithi079.stdout:Adding key to root@localhost authorized_keys... 2024-06-03T02:05:25.254 INFO:teuthology.orchestra.run.smithi079.stdout:Adding host smithi079... 2024-06-03T02:05:27.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:26 smithi079 ceph-mon[31819]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi079", "addr": "172.21.15.79", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:05:27.941 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:27 smithi079 ceph-mon[31819]: Deploying cephadm binary to smithi079 2024-06-03T02:05:32.866 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout Added host 'smithi079' with addr '172.21.15.79' 2024-06-03T02:05:32.866 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.884+0000 7f9998a00700 1 Processor -- start 2024-06-03T02:05:32.867 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.884+0000 7f9998a00700 1 -- start start 2024-06-03T02:05:32.867 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.885+0000 7f9998a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999407cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:32.867 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.885+0000 7f9998a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f999407d0a0 con 0x7f999407e660 2024-06-03T02:05:32.867 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.885+0000 7f9993400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999407cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:32.867 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.885+0000 7f9993400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999407cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50840/0 (socket says 172.21.15.79:50840) 2024-06-03T02:05:32.867 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.885+0000 7f9993400700 1 -- 172.21.15.79:0/3595929460 learned_addr learned my addr 172.21.15.79:0/3595929460 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:32.868 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.886+0000 7f9993400700 1 -- 172.21.15.79:0/3595929460 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999407d220 con 0x7f999407e660 2024-06-03T02:05:32.868 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.886+0000 7f9993400700 1 --2- 172.21.15.79:0/3595929460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999407cb60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99840098a0 tx=0x7f9984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f9318f3266854a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:32.868 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.886+0000 7f9992000700 1 -- 172.21.15.79:0/3595929460 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9984004160 con 0x7f999407e660 2024-06-03T02:05:32.868 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.886+0000 7f9992000700 1 -- 172.21.15.79:0/3595929460 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9984004300 con 0x7f999407e660 2024-06-03T02:05:32.868 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.886+0000 7f9992000700 1 -- 172.21.15.79:0/3595929460 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9984011b50 con 0x7f999407e660 2024-06-03T02:05:32.872 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.887+0000 7f9998a00700 1 -- 172.21.15.79:0/3595929460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 msgr2=0x7f999407cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:32.872 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.887+0000 7f9998a00700 1 --2- 172.21.15.79:0/3595929460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999407cb60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99840098a0 tx=0x7f9984009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.887+0000 7f9998a00700 1 -- 172.21.15.79:0/3595929460 shutdown_connections 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.887+0000 7f9998a00700 1 --2- 172.21.15.79:0/3595929460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999407cb60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.887+0000 7f9998a00700 1 -- 172.21.15.79:0/3595929460 >> 172.21.15.79:0/3595929460 conn(0x7f9994103330 msgr2=0x7f9994105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.887+0000 7f9998a00700 1 -- 172.21.15.79:0/3595929460 shutdown_connections 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.887+0000 7f9998a00700 1 -- 172.21.15.79:0/3595929460 wait complete. 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.888+0000 7f9998a00700 1 Processor -- start 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.889+0000 7f9998a00700 1 -- start start 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.889+0000 7f9998a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.889+0000 7f9998a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f999410e2e0 con 0x7f999407e660 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.889+0000 7f9993400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.889+0000 7f9993400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999419dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50858/0 (socket says 172.21.15.79:50858) 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.889+0000 7f9993400700 1 -- 172.21.15.79:0/1578253919 learned_addr learned my addr 172.21.15.79:0/1578253919 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:32.873 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.890+0000 7f9993400700 1 -- 172.21.15.79:0/1578253919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999419e400 con 0x7f999407e660 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.890+0000 7f9993400700 1 --2- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999419dec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9984009fb0 tx=0x7f9984011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.890+0000 7f998be00700 1 -- 172.21.15.79:0/1578253919 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99840038b0 con 0x7f999407e660 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.890+0000 7f998be00700 1 -- 172.21.15.79:0/1578253919 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9984011e50 con 0x7f999407e660 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.890+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999419e690 con 0x7f999407e660 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.890+0000 7f998be00700 1 -- 172.21.15.79:0/1578253919 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9984019420 con 0x7f999407e660 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.890+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99941a1380 con 0x7f999407e660 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.891+0000 7f998be00700 1 -- 172.21.15.79:0/1578253919 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f998401a430 con 0x7f999407e660 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.891+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9994063af0 con 0x7f999407e660 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.892+0000 7f998be00700 1 --2- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f9978039050 0x7f997803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.892+0000 7f998be00700 1 -- 172.21.15.79:0/1578253919 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f998404d380 con 0x7f999407e660 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.892+0000 7f9992a00700 1 --2- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f9978039050 0x7f997803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.893+0000 7f9992a00700 1 --2- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f9978039050 0x7f997803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f997c006fd0 tx=0x7f997c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:32.874 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:25.897+0000 7f998be00700 1 -- 172.21.15.79:0/1578253919 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9984023e50 con 0x7f999407e660 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:26.124+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi079", "addr": "172.21.15.79", "target": ["mon-mgr", ""]}) -- 0x7f99941045b0 con 0x7f9978039050 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.137+0000 7f998be00700 1 -- 172.21.15.79:0/1578253919 <== mgr.14120 v2:172.21.15.79:6800/54480245 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f99941045b0 con 0x7f9978039050 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.141+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f9978039050 msgr2=0x7f997803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.141+0000 7f9998a00700 1 --2- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f9978039050 0x7f997803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f997c006fd0 tx=0x7f997c006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.141+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 msgr2=0x7f999419dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.141+0000 7f9998a00700 1 --2- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999419dec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9984009fb0 tx=0x7f9984011bf0 comp rx=0 tx=0).stop 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.141+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 shutdown_connections 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.141+0000 7f9998a00700 1 --2- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f9978039050 0x7f997803b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.141+0000 7f9998a00700 1 --2- 172.21.15.79:0/1578253919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f999407e660 0x7f999419dec0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.141+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 >> 172.21.15.79:0/1578253919 conn(0x7f9994103330 msgr2=0x7f9994103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.142+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 shutdown_connections 2024-06-03T02:05:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:32.142+0000 7f9998a00700 1 -- 172.21.15.79:0/1578253919 wait complete. 2024-06-03T02:05:32.876 INFO:teuthology.orchestra.run.smithi079.stdout:Deploying unmanaged mon service... 2024-06-03T02:05:33.527 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:33 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:33.527 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:33 smithi079 ceph-mon[31819]: Added host smithi079 2024-06-03T02:05:33.527 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:33 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:05:34.955 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-03T02:05:34.955 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.697+0000 7fb091600700 1 Processor -- start 2024-06-03T02:05:34.957 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.697+0000 7fb091600700 1 -- start start 2024-06-03T02:05:34.957 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.697+0000 7fb091600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c0822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:34.957 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.697+0000 7fb091600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c0828c0 con 0x7fb08c081ed0 2024-06-03T02:05:34.957 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.698+0000 7fb08aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c0822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.698+0000 7fb08aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c0822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50860/0 (socket says 172.21.15.79:50860) 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.698+0000 7fb08aa00700 1 -- 172.21.15.79:0/1692448909 learned_addr learned my addr 172.21.15.79:0/1692448909 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.699+0000 7fb08aa00700 1 -- 172.21.15.79:0/1692448909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c10f6c0 con 0x7fb08c081ed0 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.699+0000 7fb08aa00700 1 --2- 172.21.15.79:0/1692448909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c0822f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0098a0 tx=0x7fb07c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0e8637eae3cf65 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.699+0000 7fb089600700 1 -- 172.21.15.79:0/1692448909 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb07c004160 con 0x7fb08c081ed0 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.699+0000 7fb089600700 1 -- 172.21.15.79:0/1692448909 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb07c004300 con 0x7fb08c081ed0 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.699+0000 7fb089600700 1 -- 172.21.15.79:0/1692448909 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb07c011b50 con 0x7fb08c081ed0 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.700+0000 7fb091600700 1 -- 172.21.15.79:0/1692448909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 msgr2=0x7fb08c0822f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.700+0000 7fb091600700 1 --2- 172.21.15.79:0/1692448909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c0822f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0098a0 tx=0x7fb07c009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.700+0000 7fb091600700 1 -- 172.21.15.79:0/1692448909 shutdown_connections 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.700+0000 7fb091600700 1 --2- 172.21.15.79:0/1692448909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c0822f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:34.958 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.700+0000 7fb091600700 1 -- 172.21.15.79:0/1692448909 >> 172.21.15.79:0/1692448909 conn(0x7fb08c07d6b0 msgr2=0x7fb08c07fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:34.962 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.700+0000 7fb091600700 1 -- 172.21.15.79:0/1692448909 shutdown_connections 2024-06-03T02:05:34.962 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.700+0000 7fb091600700 1 -- 172.21.15.79:0/1692448909 wait complete. 2024-06-03T02:05:34.962 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.701+0000 7fb091600700 1 Processor -- start 2024-06-03T02:05:34.962 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.701+0000 7fb091600700 1 -- start start 2024-06-03T02:05:34.962 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.702+0000 7fb091600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c1a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:34.962 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.702+0000 7fb091600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c112a70 con 0x7fb08c081ed0 2024-06-03T02:05:34.962 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.702+0000 7fb08aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c1a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.702+0000 7fb08aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c1a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50866/0 (socket says 172.21.15.79:50866) 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.702+0000 7fb08aa00700 1 -- 172.21.15.79:0/1920581001 learned_addr learned my addr 172.21.15.79:0/1920581001 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.702+0000 7fb08aa00700 1 -- 172.21.15.79:0/1920581001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c1a2950 con 0x7fb08c081ed0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.703+0000 7fb08aa00700 1 --2- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c1a2410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb07c011b60 tx=0x7fb07c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.703+0000 7fb083400700 1 -- 172.21.15.79:0/1920581001 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb07c003b20 con 0x7fb08c081ed0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.703+0000 7fb083400700 1 -- 172.21.15.79:0/1920581001 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb07c003cc0 con 0x7fb08c081ed0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.703+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c1a2be0 con 0x7fb08c081ed0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.703+0000 7fb083400700 1 -- 172.21.15.79:0/1920581001 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb07c024840 con 0x7fb08c081ed0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.703+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c1a58d0 con 0x7fb08c081ed0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.704+0000 7fb083400700 1 -- 172.21.15.79:0/1920581001 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb07c022020 con 0x7fb08c081ed0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.704+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08c063af0 con 0x7fb08c081ed0 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.704+0000 7fb083400700 1 --2- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fb0700390a0 0x7fb07003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:34.963 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.705+0000 7fb083400700 1 -- 172.21.15.79:0/1920581001 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb07c011e00 con 0x7fb08c081ed0 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.705+0000 7fb08a000700 1 --2- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fb0700390a0 0x7fb07003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.706+0000 7fb08a000700 1 --2- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fb0700390a0 0x7fb07003b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb078006fd0 tx=0x7fb078006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.710+0000 7fb083400700 1 -- 172.21.15.79:0/1920581001 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb07c04cc80 con 0x7fb08c081ed0 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.957+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb08c07f8f0 con 0x7fb0700390a0 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.964+0000 7fb083400700 1 -- 172.21.15.79:0/1920581001 <== mgr.14120 v2:172.21.15.79:6800/54480245 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb08c07f8f0 con 0x7fb0700390a0 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.967+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fb0700390a0 msgr2=0x7fb07003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.967+0000 7fb091600700 1 --2- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fb0700390a0 0x7fb07003b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb078006fd0 tx=0x7fb078006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.967+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 msgr2=0x7fb08c1a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.967+0000 7fb091600700 1 --2- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c1a2410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb07c011b60 tx=0x7fb07c011c40 comp rx=0 tx=0).stop 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.967+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 shutdown_connections 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.967+0000 7fb091600700 1 --2- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fb0700390a0 0x7fb07003b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:34.964 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.967+0000 7fb091600700 1 --2- 172.21.15.79:0/1920581001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb08c081ed0 0x7fb08c1a2410 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:34.965 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.967+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 >> 172.21.15.79:0/1920581001 conn(0x7fb08c07d6b0 msgr2=0x7fb08c07f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:34.965 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.967+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 shutdown_connections 2024-06-03T02:05:34.965 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:33.968+0000 7fb091600700 1 -- 172.21.15.79:0/1920581001 wait complete. 2024-06-03T02:05:34.965 INFO:teuthology.orchestra.run.smithi079.stdout:Deploying unmanaged mgr service... 2024-06-03T02:05:35.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:34 smithi079 ceph-mon[31819]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:05:35.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:34 smithi079 ceph-mon[31819]: Saving service mon spec with placement count:5 2024-06-03T02:05:35.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:34 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:37.287 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:36 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:37.287 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:36 smithi079 ceph-mon[31819]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:05:37.287 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:36 smithi079 ceph-mon[31819]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:05:37.288 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:36 smithi079 ceph-mon[31819]: Saving service mgr spec with placement count:2 2024-06-03T02:05:37.288 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:36 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:37.288 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:36 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:37.298 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-03T02:05:37.298 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.344+0000 7faa55e00700 1 Processor -- start 2024-06-03T02:05:37.298 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.344+0000 7faa55e00700 1 -- start start 2024-06-03T02:05:37.298 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.344+0000 7faa55e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa50108150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:37.298 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.344+0000 7faa55e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faa50108720 con 0x7faa50107d30 2024-06-03T02:05:37.298 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.345+0000 7faa54a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa50108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.345+0000 7faa54a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa50108150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38732/0 (socket says 172.21.15.79:38732) 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.345+0000 7faa54a00700 1 -- 172.21.15.79:0/146033148 learned_addr learned my addr 172.21.15.79:0/146033148 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.345+0000 7faa54a00700 1 -- 172.21.15.79:0/146033148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa50108e90 con 0x7faa50107d30 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.346+0000 7faa54a00700 1 --2- 172.21.15.79:0/146033148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa50108150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa440098a0 tx=0x7faa44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9a80ed52a69510 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.346+0000 7faa4f400700 1 -- 172.21.15.79:0/146033148 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa44004160 con 0x7faa50107d30 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.346+0000 7faa4f400700 1 -- 172.21.15.79:0/146033148 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa44004300 con 0x7faa50107d30 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.346+0000 7faa4f400700 1 -- 172.21.15.79:0/146033148 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa44011b20 con 0x7faa50107d30 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.347+0000 7faa55e00700 1 -- 172.21.15.79:0/146033148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 msgr2=0x7faa50108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.347+0000 7faa55e00700 1 --2- 172.21.15.79:0/146033148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa50108150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa440098a0 tx=0x7faa44009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.347+0000 7faa55e00700 1 -- 172.21.15.79:0/146033148 shutdown_connections 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.347+0000 7faa55e00700 1 --2- 172.21.15.79:0/146033148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa50108150 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:37.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.347+0000 7faa55e00700 1 -- 172.21.15.79:0/146033148 >> 172.21.15.79:0/146033148 conn(0x7faa501032d0 msgr2=0x7faa50105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.347+0000 7faa55e00700 1 -- 172.21.15.79:0/146033148 shutdown_connections 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.347+0000 7faa55e00700 1 -- 172.21.15.79:0/146033148 wait complete. 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.348+0000 7faa55e00700 1 Processor -- start 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.349+0000 7faa55e00700 1 -- start start 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.349+0000 7faa55e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa5019de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.349+0000 7faa55e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faa5010c240 con 0x7faa50107d30 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.349+0000 7faa54a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa5019de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.349+0000 7faa54a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa5019de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38740/0 (socket says 172.21.15.79:38740) 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.349+0000 7faa54a00700 1 -- 172.21.15.79:0/2196501655 learned_addr learned my addr 172.21.15.79:0/2196501655 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.350+0000 7faa54a00700 1 -- 172.21.15.79:0/2196501655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5019e3c0 con 0x7faa50107d30 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.350+0000 7faa54a00700 1 --2- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa5019de80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa44000c00 tx=0x7faa44011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.350+0000 7faa4d600700 1 -- 172.21.15.79:0/2196501655 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa44003b60 con 0x7faa50107d30 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.350+0000 7faa4d600700 1 -- 172.21.15.79:0/2196501655 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa4401a910 con 0x7faa50107d30 2024-06-03T02:05:37.300 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.350+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5019e650 con 0x7faa50107d30 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.350+0000 7faa4d600700 1 -- 172.21.15.79:0/2196501655 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa44024930 con 0x7faa50107d30 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.350+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa5019eab0 con 0x7faa50107d30 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.352+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa50063af0 con 0x7faa50107d30 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.352+0000 7faa4d600700 1 -- 172.21.15.79:0/2196501655 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7faa44022020 con 0x7faa50107d30 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.352+0000 7faa4d600700 1 --2- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7faa38039050 0x7faa3803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.352+0000 7faa4d600700 1 -- 172.21.15.79:0/2196501655 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faa4404d2a0 con 0x7faa50107d30 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.353+0000 7faa4fe00700 1 --2- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7faa38039050 0x7faa3803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.353+0000 7faa4fe00700 1 --2- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7faa38039050 0x7faa3803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006fd0 tx=0x7faa3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.363+0000 7faa4d600700 1 -- 172.21.15.79:0/2196501655 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faa4400f2e0 con 0x7faa50107d30 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.605+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7faa501044c0 con 0x7faa38039050 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.617+0000 7faa4d600700 1 -- 172.21.15.79:0/2196501655 <== mgr.14120 v2:172.21.15.79:6800/54480245 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faa501044c0 con 0x7faa38039050 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.620+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7faa38039050 msgr2=0x7faa3803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:37.301 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.620+0000 7faa55e00700 1 --2- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7faa38039050 0x7faa3803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006fd0 tx=0x7faa3c006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:37.302 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.620+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 msgr2=0x7faa5019de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:37.302 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.620+0000 7faa55e00700 1 --2- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa5019de80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa44000c00 tx=0x7faa44011c10 comp rx=0 tx=0).stop 2024-06-03T02:05:37.302 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.621+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 shutdown_connections 2024-06-03T02:05:37.302 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.621+0000 7faa55e00700 1 --2- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7faa38039050 0x7faa3803b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:37.302 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.621+0000 7faa55e00700 1 --2- 172.21.15.79:0/2196501655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa50107d30 0x7faa5019de80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:37.302 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.621+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 >> 172.21.15.79:0/2196501655 conn(0x7faa501032d0 msgr2=0x7faa50103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:37.302 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.621+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 shutdown_connections 2024-06-03T02:05:37.302 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:36.621+0000 7faa55e00700 1 -- 172.21.15.79:0/2196501655 wait complete. 2024-06-03T02:05:39.076 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.865+0000 7f2a6c400700 1 Processor -- start 2024-06-03T02:05:39.076 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.865+0000 7f2a6c400700 1 -- start start 2024-06-03T02:05:39.076 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.865+0000 7f2a6c400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a641083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:39.076 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.865+0000 7f2a6c400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64108910 con 0x7f2a64107f90 2024-06-03T02:05:39.076 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.866+0000 7f2a69a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a641083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:39.076 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.866+0000 7f2a69a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a641083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38752/0 (socket says 172.21.15.79:38752) 2024-06-03T02:05:39.076 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.866+0000 7f2a69a00700 1 -- 172.21.15.79:0/44069282 learned_addr learned my addr 172.21.15.79:0/44069282 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.866+0000 7f2a69a00700 1 -- 172.21.15.79:0/44069282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a641090d0 con 0x7f2a64107f90 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.866+0000 7f2a69a00700 1 --2- 172.21.15.79:0/44069282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a641083d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a580098a0 tx=0x7f2a58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23d25812fe1947d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.867+0000 7f2a63e00700 1 -- 172.21.15.79:0/44069282 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a58004160 con 0x7f2a64107f90 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.867+0000 7f2a63e00700 1 -- 172.21.15.79:0/44069282 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a58004300 con 0x7f2a64107f90 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.867+0000 7f2a63e00700 1 -- 172.21.15.79:0/44069282 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a58011b20 con 0x7f2a64107f90 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.868+0000 7f2a6c400700 1 -- 172.21.15.79:0/44069282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 msgr2=0x7f2a641083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.868+0000 7f2a6c400700 1 --2- 172.21.15.79:0/44069282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a641083d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a580098a0 tx=0x7f2a58009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.868+0000 7f2a6c400700 1 -- 172.21.15.79:0/44069282 shutdown_connections 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.868+0000 7f2a6c400700 1 --2- 172.21.15.79:0/44069282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a641083d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.868+0000 7f2a6c400700 1 -- 172.21.15.79:0/44069282 >> 172.21.15.79:0/44069282 conn(0x7f2a641035e0 msgr2=0x7f2a64105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:39.077 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.868+0000 7f2a6c400700 1 -- 172.21.15.79:0/44069282 shutdown_connections 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.868+0000 7f2a6c400700 1 -- 172.21.15.79:0/44069282 wait complete. 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.869+0000 7f2a6c400700 1 Processor -- start 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.869+0000 7f2a6c400700 1 -- start start 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.870+0000 7f2a6c400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a6419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.870+0000 7f2a6c400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410c390 con 0x7f2a64107f90 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.870+0000 7f2a69a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a6419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.870+0000 7f2a69a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a6419e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38760/0 (socket says 172.21.15.79:38760) 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.870+0000 7f2a69a00700 1 -- 172.21.15.79:0/957862035 learned_addr learned my addr 172.21.15.79:0/957862035 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.870+0000 7f2a69a00700 1 -- 172.21.15.79:0/957862035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6419e570 con 0x7f2a64107f90 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.871+0000 7f2a69a00700 1 --2- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a6419e030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a580117e0 tx=0x7f2a58011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.871+0000 7f2a62000700 1 -- 172.21.15.79:0/957862035 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a58003bf0 con 0x7f2a64107f90 2024-06-03T02:05:39.078 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.871+0000 7f2a62000700 1 -- 172.21.15.79:0/957862035 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5801a910 con 0x7f2a64107f90 2024-06-03T02:05:39.079 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.871+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6419e800 con 0x7f2a64107f90 2024-06-03T02:05:39.079 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.871+0000 7f2a62000700 1 -- 172.21.15.79:0/957862035 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a58024970 con 0x7f2a64107f90 2024-06-03T02:05:39.079 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.871+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6419ec60 con 0x7f2a64107f90 2024-06-03T02:05:39.079 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.872+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a64063af0 con 0x7f2a64107f90 2024-06-03T02:05:39.079 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.872+0000 7f2a62000700 1 -- 172.21.15.79:0/957862035 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2a58022020 con 0x7f2a64107f90 2024-06-03T02:05:39.079 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.873+0000 7f2a62000700 1 --2- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f2a4c0390a0 0x7f2a4c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:39.079 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.873+0000 7f2a62000700 1 -- 172.21.15.79:0/957862035 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2a5804d8a0 con 0x7f2a64107f90 2024-06-03T02:05:39.079 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.873+0000 7f2a69000700 1 --2- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f2a4c0390a0 0x7f2a4c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:39.079 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.874+0000 7f2a69000700 1 --2- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f2a4c0390a0 0x7f2a4c03b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2a50006fd0 tx=0x7f2a50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:37.883+0000 7f2a62000700 1 -- 172.21.15.79:0/957862035 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a5801ab70 con 0x7f2a64107f90 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.124+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f2a641a14f0 con 0x7f2a64107f90 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.128+0000 7f2a62000700 1 -- 172.21.15.79:0/957862035 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f2a5802d3f0 con 0x7f2a64107f90 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.131+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f2a4c0390a0 msgr2=0x7f2a4c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.132+0000 7f2a6c400700 1 --2- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f2a4c0390a0 0x7f2a4c03b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2a50006fd0 tx=0x7f2a50006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.132+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 msgr2=0x7f2a6419e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.132+0000 7f2a6c400700 1 --2- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a6419e030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a580117e0 tx=0x7f2a58011ba0 comp rx=0 tx=0).stop 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.132+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 shutdown_connections 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.132+0000 7f2a6c400700 1 --2- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f2a4c0390a0 0x7f2a4c03b560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.132+0000 7f2a6c400700 1 --2- 172.21.15.79:0/957862035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a64107f90 0x7f2a6419e030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.132+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 >> 172.21.15.79:0/957862035 conn(0x7f2a641035e0 msgr2=0x7f2a64103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:39.081 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.132+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 shutdown_connections 2024-06-03T02:05:39.081 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:38.132+0000 7f2a6c400700 1 -- 172.21.15.79:0/957862035 wait complete. 2024-06-03T02:05:39.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:39 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/957862035' entity='client.admin' 2024-06-03T02:05:39.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:39 smithi079 ceph-mon[31819]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:05:41.061 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:40 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:41.061 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:40 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/668470082' entity='client.admin' 2024-06-03T02:05:41.087 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.002+0000 7fe50e000700 1 Processor -- start 2024-06-03T02:05:41.087 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.003+0000 7fe50e000700 1 -- start start 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.003+0000 7fe50e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe50810ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.003+0000 7fe50e000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe50807be60 con 0x7fe508108980 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.004+0000 7fe507400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe50810ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.004+0000 7fe507400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe50810ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38764/0 (socket says 172.21.15.79:38764) 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.004+0000 7fe507400700 1 -- 172.21.15.79:0/2498579183 learned_addr learned my addr 172.21.15.79:0/2498579183 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.004+0000 7fe507400700 1 -- 172.21.15.79:0/2498579183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50810b2b0 con 0x7fe508108980 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.004+0000 7fe507400700 1 --2- 172.21.15.79:0/2498579183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe50810ad70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80098a0 tx=0x7fe4f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906c93c3cf026258 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.005+0000 7fe506a00700 1 -- 172.21.15.79:0/2498579183 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4f8004160 con 0x7fe508108980 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.005+0000 7fe506a00700 1 -- 172.21.15.79:0/2498579183 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f8004300 con 0x7fe508108980 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.005+0000 7fe506a00700 1 -- 172.21.15.79:0/2498579183 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4f8011b20 con 0x7fe508108980 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.005+0000 7fe50e000700 1 -- 172.21.15.79:0/2498579183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 msgr2=0x7fe50810ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.005+0000 7fe50e000700 1 --2- 172.21.15.79:0/2498579183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe50810ad70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80098a0 tx=0x7fe4f8009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.006+0000 7fe50e000700 1 -- 172.21.15.79:0/2498579183 shutdown_connections 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.006+0000 7fe50e000700 1 --2- 172.21.15.79:0/2498579183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe50810ad70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.006+0000 7fe50e000700 1 -- 172.21.15.79:0/2498579183 >> 172.21.15.79:0/2498579183 conn(0x7fe5081023f0 msgr2=0x7fe508104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.006+0000 7fe50e000700 1 -- 172.21.15.79:0/2498579183 shutdown_connections 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.006+0000 7fe50e000700 1 -- 172.21.15.79:0/2498579183 wait complete. 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.007+0000 7fe50e000700 1 Processor -- start 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.007+0000 7fe50e000700 1 -- start start 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.007+0000 7fe50e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe508108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.007+0000 7fe50e000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe50810c310 con 0x7fe508108980 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.008+0000 7fe507400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe508108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.008+0000 7fe507400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe508108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38780/0 (socket says 172.21.15.79:38780) 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.008+0000 7fe507400700 1 -- 172.21.15.79:0/668470082 learned_addr learned my addr 172.21.15.79:0/668470082 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.008+0000 7fe507400700 1 -- 172.21.15.79:0/668470082 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe508108740 con 0x7fe508108980 2024-06-03T02:05:41.089 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.008+0000 7fe507400700 1 --2- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe508108200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80117e0 tx=0x7fe4f8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.009+0000 7fe504c00700 1 -- 172.21.15.79:0/668470082 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4f8003ba0 con 0x7fe508108980 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.009+0000 7fe504c00700 1 -- 172.21.15.79:0/668470082 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f801a910 con 0x7fe508108980 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.009+0000 7fe504c00700 1 -- 172.21.15.79:0/668470082 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4f80248f0 con 0x7fe508108980 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.009+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe508106930 con 0x7fe508108980 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.009+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe508106df0 con 0x7fe508108980 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.010+0000 7fe504c00700 1 -- 172.21.15.79:0/668470082 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe4f8022020 con 0x7fe508108980 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.010+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe508063af0 con 0x7fe508108980 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.011+0000 7fe504c00700 1 --2- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fe4ec039100 0x7fe4ec03b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.011+0000 7fe504c00700 1 -- 172.21.15.79:0/668470082 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe4f804d560 con 0x7fe508108980 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.011+0000 7fe4ffe00700 1 --2- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fe4ec039100 0x7fe4ec03b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.012+0000 7fe4ffe00700 1 --2- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fe4ec039100 0x7fe4ec03b5c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006fd0 tx=0x7fe4f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.017+0000 7fe504c00700 1 -- 172.21.15.79:0/668470082 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe4f801ab70 con 0x7fe508108980 2024-06-03T02:05:41.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.242+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fe508107070 con 0x7fe508108980 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.248+0000 7fe504c00700 1 -- 172.21.15.79:0/668470082 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fe4f80212c0 con 0x7fe508108980 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.251+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fe4ec039100 msgr2=0x7fe4ec03b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.251+0000 7fe50e000700 1 --2- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fe4ec039100 0x7fe4ec03b5c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006fd0 tx=0x7fe4f0006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.251+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 msgr2=0x7fe508108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.251+0000 7fe50e000700 1 --2- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe508108200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80117e0 tx=0x7fe4f8011ba0 comp rx=0 tx=0).stop 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.252+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 shutdown_connections 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.252+0000 7fe50e000700 1 --2- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7fe4ec039100 0x7fe4ec03b5c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.252+0000 7fe50e000700 1 --2- 172.21.15.79:0/668470082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe508108980 0x7fe508108200 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.252+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 >> 172.21.15.79:0/668470082 conn(0x7fe5081023f0 msgr2=0x7fe508102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.252+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 shutdown_connections 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:40.253+0000 7fe50e000700 1 -- 172.21.15.79:0/668470082 wait complete. 2024-06-03T02:05:41.091 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling the dashboard module... 2024-06-03T02:05:41.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:41 smithi079 ceph-mon[31819]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:05:43.233 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:42 smithi079 ceph-mon[31819]: from='mgr.14120 172.21.15.79:0/1157530258' entity='mgr.x' 2024-06-03T02:05:43.234 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:42 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/488999330' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-03T02:05:43.234 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:42 smithi079 ceph-mon[31819]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:05:43.234 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: ignoring --setuser ceph since I am not root 2024-06-03T02:05:43.234 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: ignoring --setgroup ceph since I am not root 2024-06-03T02:05:43.554 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:43.304+0000 7f1c37112200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T02:05:43.554 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:43.553+0000 7f1c37112200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:05:43.838 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:43.699+0000 7f1c37112200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T02:05:44.132 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:43 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/488999330' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-03T02:05:44.132 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:43 smithi079 ceph-mon[31819]: mgrmap e9: x(active, since 26s) 2024-06-03T02:05:44.132 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:44 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:44.095+0000 7f1c37112200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T02:05:44.587 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:44 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:44.217+0000 7f1c37112200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T02:05:44.646 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.659+0000 7f8458c00700 1 Processor -- start 2024-06-03T02:05:44.646 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.659+0000 7f8458c00700 1 -- start start 2024-06-03T02:05:44.646 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.659+0000 7f8458c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f8454108310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:44.646 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.659+0000 7f8458c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8454108850 con 0x7f8454107ef0 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.660+0000 7f8451e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f8454108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.660+0000 7f8451e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f8454108310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38786/0 (socket says 172.21.15.79:38786) 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.660+0000 7f8451e00700 1 -- 172.21.15.79:0/2574746334 learned_addr learned my addr 172.21.15.79:0/2574746334 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.661+0000 7f8451e00700 1 -- 172.21.15.79:0/2574746334 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8454109040 con 0x7f8454107ef0 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.661+0000 7f8451e00700 1 --2- 172.21.15.79:0/2574746334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f8454108310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84440098a0 tx=0x7f8444009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9e62638823c66e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.661+0000 7f8450a00700 1 -- 172.21.15.79:0/2574746334 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8444004160 con 0x7f8454107ef0 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.662+0000 7f8450a00700 1 -- 172.21.15.79:0/2574746334 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8444004300 con 0x7f8454107ef0 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.662+0000 7f8450a00700 1 -- 172.21.15.79:0/2574746334 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8444011b20 con 0x7f8454107ef0 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.662+0000 7f8458c00700 1 -- 172.21.15.79:0/2574746334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 msgr2=0x7f8454108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.662+0000 7f8458c00700 1 --2- 172.21.15.79:0/2574746334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f8454108310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84440098a0 tx=0x7f8444009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.663+0000 7f8458c00700 1 -- 172.21.15.79:0/2574746334 shutdown_connections 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.663+0000 7f8458c00700 1 --2- 172.21.15.79:0/2574746334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f8454108310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.663+0000 7f8458c00700 1 -- 172.21.15.79:0/2574746334 >> 172.21.15.79:0/2574746334 conn(0x7f8454103540 msgr2=0x7f8454105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.663+0000 7f8458c00700 1 -- 172.21.15.79:0/2574746334 shutdown_connections 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.663+0000 7f8458c00700 1 -- 172.21.15.79:0/2574746334 wait complete. 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.664+0000 7f8458c00700 1 Processor -- start 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.664+0000 7f8458c00700 1 -- start start 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.664+0000 7f8458c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f845419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.664+0000 7f8458c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f845410c300 con 0x7f8454107ef0 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.664+0000 7f8451e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f845419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.665+0000 7f8451e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f845419e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38794/0 (socket says 172.21.15.79:38794) 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.665+0000 7f8451e00700 1 -- 172.21.15.79:0/488999330 learned_addr learned my addr 172.21.15.79:0/488999330 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.665+0000 7f8451e00700 1 -- 172.21.15.79:0/488999330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845419e5c0 con 0x7f8454107ef0 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.665+0000 7f8451e00700 1 --2- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f845419e080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84440117e0 tx=0x7f8444011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.665+0000 7f844aa00700 1 -- 172.21.15.79:0/488999330 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8444003bf0 con 0x7f8454107ef0 2024-06-03T02:05:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.666+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845419e850 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.666+0000 7f844aa00700 1 -- 172.21.15.79:0/488999330 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f844401a910 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.666+0000 7f844aa00700 1 -- 172.21.15.79:0/488999330 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8444024970 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.666+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845419ecb0 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.667+0000 7f844aa00700 1 -- 172.21.15.79:0/488999330 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8444022020 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.667+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8454063af0 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.667+0000 7f844aa00700 1 --2- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f84380390a0 0x7f843803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.667+0000 7f844aa00700 1 -- 172.21.15.79:0/488999330 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f844404d410 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.668+0000 7f8451400700 1 --2- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f84380390a0 0x7f843803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.668+0000 7f8451400700 1 --2- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f84380390a0 0x7f843803b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8440006fd0 tx=0x7f8440006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.673+0000 7f844aa00700 1 -- 172.21.15.79:0/488999330 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f844400b3e0 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:41.962+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f84541a1a20 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.959+0000 7f844aa00700 1 -- 172.21.15.79:0/488999330 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8444020c10 con 0x7f8454107ef0 2024-06-03T02:05:44.649 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.962+0000 7f844aa00700 1 -- 172.21.15.79:0/488999330 <== mon.0 v2:172.21.15.79:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f844401aac0 con 0x7f8454107ef0 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.964+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f84380390a0 msgr2=0x7f843803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.964+0000 7f8458c00700 1 --2- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f84380390a0 0x7f843803b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8440006fd0 tx=0x7f8440006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.964+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 msgr2=0x7f845419e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.964+0000 7f8458c00700 1 --2- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f845419e080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84440117e0 tx=0x7f8444011ba0 comp rx=0 tx=0).stop 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.964+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 shutdown_connections 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.964+0000 7f8458c00700 1 --2- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f84380390a0 0x7f843803b560 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8440006fd0 tx=0x7f8440006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.964+0000 7f8458c00700 1 --2- 172.21.15.79:0/488999330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8454107ef0 0x7f845419e080 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.964+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 >> 172.21.15.79:0/488999330 conn(0x7f8454103540 msgr2=0x7f8454103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.965+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 shutdown_connections 2024-06-03T02:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:42.965+0000 7f8458c00700 1 -- 172.21.15.79:0/488999330 wait complete. 2024-06-03T02:05:45.837 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:45 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:45.788+0000 7f1c37112200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T02:05:45.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:45 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/339205144' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-03T02:05:46.230 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:46.020+0000 7f1c37112200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T02:05:46.240 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-06-03T02:05:46.248 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-03T02:05:46.248 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T02:05:46.248 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-03T02:05:46.248 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.258+0000 7f1bf4400700 1 Processor -- start 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.258+0000 7f1bf4400700 1 -- start start 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.259+0000 7f1bf4400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec108340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.259+0000 7f1bf4400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec108880 con 0x7f1bec107f20 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.259+0000 7f1bf1a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.259+0000 7f1bf1a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec108340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51290/0 (socket says 172.21.15.79:51290) 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.259+0000 7f1bf1a00700 1 -- 172.21.15.79:0/1379473455 learned_addr learned my addr 172.21.15.79:0/1379473455 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.260+0000 7f1bf1a00700 1 -- 172.21.15.79:0/1379473455 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec108fe0 con 0x7f1bec107f20 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.260+0000 7f1bf1a00700 1 --2- 172.21.15.79:0/1379473455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec108340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098a0 tx=0x7f1be0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2dac4cb85fa647 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.260+0000 7f1bebe00700 1 -- 172.21.15.79:0/1379473455 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0004160 con 0x7f1bec107f20 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.261+0000 7f1bebe00700 1 -- 172.21.15.79:0/1379473455 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be0004300 con 0x7f1bec107f20 2024-06-03T02:05:46.249 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.261+0000 7f1bebe00700 1 -- 172.21.15.79:0/1379473455 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0011b20 con 0x7f1bec107f20 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.261+0000 7f1bf4400700 1 -- 172.21.15.79:0/1379473455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 msgr2=0x7f1bec108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.261+0000 7f1bf4400700 1 --2- 172.21.15.79:0/1379473455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec108340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098a0 tx=0x7f1be0009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.261+0000 7f1bf4400700 1 -- 172.21.15.79:0/1379473455 shutdown_connections 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.261+0000 7f1bf4400700 1 --2- 172.21.15.79:0/1379473455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec108340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.261+0000 7f1bf4400700 1 -- 172.21.15.79:0/1379473455 >> 172.21.15.79:0/1379473455 conn(0x7f1bec103520 msgr2=0x7f1bec105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.262+0000 7f1bf4400700 1 -- 172.21.15.79:0/1379473455 shutdown_connections 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.262+0000 7f1bf4400700 1 -- 172.21.15.79:0/1379473455 wait complete. 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.263+0000 7f1bf4400700 1 Processor -- start 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.263+0000 7f1bf4400700 1 -- start start 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.263+0000 7f1bf4400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:46.250 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.263+0000 7f1bf4400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec07cae0 con 0x7f1bec107f20 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.263+0000 7f1bf1a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.263+0000 7f1bf1a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51300/0 (socket says 172.21.15.79:51300) 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.263+0000 7f1bf1a00700 1 -- 172.21.15.79:0/339205144 learned_addr learned my addr 172.21.15.79:0/339205144 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.264+0000 7f1bf1a00700 1 -- 172.21.15.79:0/339205144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec19a120 con 0x7f1bec107f20 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.264+0000 7f1bf1a00700 1 --2- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec199be0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1be00117e0 tx=0x7f1be0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.264+0000 7f1bea000700 1 -- 172.21.15.79:0/339205144 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0003ba0 con 0x7f1bec107f20 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.264+0000 7f1bea000700 1 -- 172.21.15.79:0/339205144 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be001a910 con 0x7f1bec107f20 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.264+0000 7f1bf4400700 1 -- 172.21.15.79:0/339205144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bec19a3b0 con 0x7f1bec107f20 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.265+0000 7f1bea000700 1 -- 172.21.15.79:0/339205144 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be00248f0 con 0x7f1bec107f20 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.265+0000 7f1bf4400700 1 -- 172.21.15.79:0/339205144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bec19a810 con 0x7f1bec107f20 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.266+0000 7f1bdf400700 1 -- 172.21.15.79:0/339205144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bec063af0 con 0x7f1bec107f20 2024-06-03T02:05:46.251 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.267+0000 7f1bea000700 1 -- 172.21.15.79:0/339205144 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1be0022020 con 0x7f1bec107f20 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.267+0000 7f1bea000700 1 --2- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1bd403d550 0x7f1bd403fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.267+0000 7f1bea000700 1 -- 172.21.15.79:0/339205144 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1be004da30 con 0x7f1bec107f20 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.267+0000 7f1bf1000700 1 -- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1bd403d550 msgr2=0x7f1bd403fa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/54480245 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.267+0000 7f1bf1000700 1 --2- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1bd403d550 0x7f1bd403fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.272+0000 7f1bea000700 1 -- 172.21.15.79:0/339205144 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1be004c420 con 0x7f1bec107f20 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.468+0000 7f1bf1000700 1 -- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1bd403d550 msgr2=0x7f1bd403fa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/54480245 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.468+0000 7f1bf1000700 1 --2- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1bd403d550 0x7f1bd403fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.559+0000 7f1bdf400700 1 -- 172.21.15.79:0/339205144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1bec0631c0 con 0x7f1bec107f20 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.559+0000 7f1bea000700 1 -- 172.21.15.79:0/339205144 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f1be004cb20 con 0x7f1bec107f20 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.563+0000 7f1bdf400700 1 -- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1bd403d550 msgr2=0x7f1bd403fa10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.563+0000 7f1bdf400700 1 --2- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1bd403d550 0x7f1bd403fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:46.252 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.563+0000 7f1bdf400700 1 -- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 msgr2=0x7f1bec199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:46.253 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.563+0000 7f1bdf400700 1 --2- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec199be0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1be00117e0 tx=0x7f1be0011ba0 comp rx=0 tx=0).stop 2024-06-03T02:05:46.253 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.564+0000 7f1bdf400700 1 -- 172.21.15.79:0/339205144 shutdown_connections 2024-06-03T02:05:46.253 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.564+0000 7f1bdf400700 1 --2- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f1bd403d550 0x7f1bd403fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:46.253 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.564+0000 7f1bdf400700 1 --2- 172.21.15.79:0/339205144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1bec107f20 0x7f1bec199be0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:46.253 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.564+0000 7f1bdf400700 1 -- 172.21.15.79:0/339205144 >> 172.21.15.79:0/339205144 conn(0x7f1bec103520 msgr2=0x7f1bec105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:46.253 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.564+0000 7f1bdf400700 1 -- 172.21.15.79:0/339205144 shutdown_connections 2024-06-03T02:05:46.253 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:45.564+0000 7f1bdf400700 1 -- 172.21.15.79:0/339205144 wait complete. 2024-06-03T02:05:46.253 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for the mgr to restart... 2024-06-03T02:05:46.253 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mgr epoch 9... 2024-06-03T02:05:46.488 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:46.237+0000 7f1c37112200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T02:05:46.741 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:46.628+0000 7f1c37112200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T02:05:47.008 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:46.757+0000 7f1c37112200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T02:05:47.272 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:47 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:47.018+0000 7f1c37112200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T02:05:47.272 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:47 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:47.273+0000 7f1c37112200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:05:48.337 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:48 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:48.071+0000 7f1c37112200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T02:05:49.025 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:48 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:48.626+0000 7f1c37112200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T02:05:49.025 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:48 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:48.757+0000 7f1c37112200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T02:05:49.026 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:49.025+0000 7f1c37112200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T02:05:50.088 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:49.670+0000 7f1c37112200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T02:05:50.088 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:49.795+0000 7f1c37112200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T02:05:50.422 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:50.114+0000 7f1c37112200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T02:05:50.422 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:50.293+0000 7f1c37112200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T02:05:50.422 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:50.421+0000 7f1c37112200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T02:05:50.838 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:50.573+0000 7f1c37112200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T02:05:51.338 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:50.962+0000 7f1c37112200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T02:05:52.006 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:51 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:51.715+0000 7f1c37112200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T02:05:52.337 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:05:52.004+0000 7f1c37112200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: Active manager daemon x restarted 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: Activating manager daemon x 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: osdmap e3: 0 total, 0 up, 0 in 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: mgrmap e10: x(active, starting, since 0.00506904s) 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T02:05:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:52 smithi079 ceph-mon[31819]: Manager daemon x is now available 2024-06-03T02:05:53.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:05:53.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T02:05:53.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T02:05:53.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:53 smithi079 ceph-mon[31819]: mgrmap e11: x(active, since 1.0089s) 2024-06-03T02:05:53.895 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-06-03T02:05:53.895 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-03T02:05:53.895 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-03T02:05:53.895 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-06-03T02:05:53.895 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.799+0000 7f661c800700 1 Processor -- start 2024-06-03T02:05:53.895 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.800+0000 7f661c800700 1 -- start start 2024-06-03T02:05:53.895 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.800+0000 7f661c800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:53.895 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.800+0000 7f661c800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f661407c830 con 0x7f661407ddf0 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.800+0000 7f6619e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.800+0000 7f6619e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51312/0 (socket says 172.21.15.79:51312) 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.801+0000 7f6619e00700 1 -- 172.21.15.79:0/2329932923 learned_addr learned my addr 172.21.15.79:0/2329932923 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.801+0000 7f6619e00700 1 -- 172.21.15.79:0/2329932923 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661407c9b0 con 0x7f661407ddf0 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.801+0000 7f6619e00700 1 --2- 172.21.15.79:0/2329932923 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661407c2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66080098a0 tx=0x7f6608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39061a469b3669ab server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.801+0000 7f6618a00700 1 -- 172.21.15.79:0/2329932923 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6608004160 con 0x7f661407ddf0 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.802+0000 7f6618a00700 1 -- 172.21.15.79:0/2329932923 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6608004300 con 0x7f661407ddf0 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.802+0000 7f6618a00700 1 -- 172.21.15.79:0/2329932923 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6608011b20 con 0x7f661407ddf0 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.802+0000 7f661c800700 1 -- 172.21.15.79:0/2329932923 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 msgr2=0x7f661407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.802+0000 7f661c800700 1 --2- 172.21.15.79:0/2329932923 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661407c2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66080098a0 tx=0x7f6608009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.802+0000 7f661c800700 1 -- 172.21.15.79:0/2329932923 shutdown_connections 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.802+0000 7f661c800700 1 --2- 172.21.15.79:0/2329932923 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661407c2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:53.896 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.802+0000 7f661c800700 1 -- 172.21.15.79:0/2329932923 >> 172.21.15.79:0/2329932923 conn(0x7f6614103540 msgr2=0x7f6614105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.803+0000 7f661c800700 1 -- 172.21.15.79:0/2329932923 shutdown_connections 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.803+0000 7f661c800700 1 -- 172.21.15.79:0/2329932923 wait complete. 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.804+0000 7f661c800700 1 Processor -- start 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.804+0000 7f661c800700 1 -- start start 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.804+0000 7f661c800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.804+0000 7f661c800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f661410e4e0 con 0x7f661407ddf0 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.804+0000 7f6619e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.804+0000 7f6619e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51324/0 (socket says 172.21.15.79:51324) 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.804+0000 7f6619e00700 1 -- 172.21.15.79:0/3097956934 learned_addr learned my addr 172.21.15.79:0/3097956934 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.805+0000 7f6619e00700 1 -- 172.21.15.79:0/3097956934 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661419e4d0 con 0x7f661407ddf0 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.805+0000 7f6619e00700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661419df90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66080117e0 tx=0x7f6608011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.805+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6608003ba0 con 0x7f661407ddf0 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.805+0000 7f661c800700 1 -- 172.21.15.79:0/3097956934 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661419e760 con 0x7f661407ddf0 2024-06-03T02:05:53.897 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.806+0000 7f661c800700 1 -- 172.21.15.79:0/3097956934 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661419ebc0 con 0x7f661407ddf0 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.806+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f660801a910 con 0x7f661407ddf0 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.806+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66080248f0 con 0x7f661407ddf0 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.807+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f6608022020 con 0x7f661407ddf0 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.807+0000 7f660ea00700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 0x7f65fc03b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.807+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 --> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f65fc03bcd0 con 0x7f65fc039100 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.807+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f660804d950 con 0x7f661407ddf0 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.807+0000 7f6619400700 1 -- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 msgr2=0x7f65fc03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/54480245 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:46.807+0000 7f6619400700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 0x7f65fc03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:47.008+0000 7f6619400700 1 -- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 msgr2=0x7f65fc03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/54480245 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:47.008+0000 7f6619400700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 0x7f65fc03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:47.410+0000 7f6619400700 1 -- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 msgr2=0x7f65fc03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/54480245 2024-06-03T02:05:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:47.410+0000 7f6619400700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 0x7f65fc03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:48.211+0000 7f6619400700 1 -- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 msgr2=0x7f65fc03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/54480245 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:48.211+0000 7f6619400700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 0x7f65fc03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:49.813+0000 7f6619400700 1 -- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 msgr2=0x7f65fc03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6800/54480245 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:49.813+0000 7f6619400700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 0x7f65fc03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:52.013+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f660802d410 con 0x7f661407ddf0 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:52.014+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 msgr2=0x7f65fc03b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:52.014+0000 7f660ea00700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 0x7f65fc03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.014+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f660804fad0 con 0x7f661407ddf0 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.014+0000 7f660ea00700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65fc03cbc0 0x7f65fc03efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.014+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f65fc03bcd0 con 0x7f65fc03cbc0 2024-06-03T02:05:53.899 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.015+0000 7f6619400700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65fc03cbc0 0x7f65fc03efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.016+0000 7f6619400700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65fc03cbc0 0x7f65fc03efb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6604003a10 tx=0x7f6604009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.017+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f65fc03bcd0 con 0x7f65fc03cbc0 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.024+0000 7f661c800700 1 -- 172.21.15.79:0/3097956934 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f65f8002630 con 0x7f65fc03cbc0 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.024+0000 7f660ea00700 1 -- 172.21.15.79:0/3097956934 <== mgr.14152 v2:172.21.15.79:6800/2278164 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f65f8002630 con 0x7f65fc03cbc0 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 -- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65fc03cbc0 msgr2=0x7f65fc03efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65fc03cbc0 0x7f65fc03efb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6604003a10 tx=0x7f6604009270 comp rx=0 tx=0).stop 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 -- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 msgr2=0x7f661419df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661419df90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66080117e0 tx=0x7f6608011ba0 comp rx=0 tx=0).stop 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 -- 172.21.15.79:0/3097956934 shutdown_connections 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65fc03cbc0 0x7f65fc03efb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f661407ddf0 0x7f661419df90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 --2- 172.21.15.79:0/3097956934 >> [v2:172.21.15.79:6800/54480245,v1:172.21.15.79:6801/54480245] conn(0x7f65fc039100 0x7f65fc03b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:53.900 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 -- 172.21.15.79:0/3097956934 >> 172.21.15.79:0/3097956934 conn(0x7f6614103540 msgr2=0x7f6614103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:53.901 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 -- 172.21.15.79:0/3097956934 shutdown_connections 2024-06-03T02:05:53.901 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:53.025+0000 7f661c800700 1 -- 172.21.15.79:0/3097956934 wait complete. 2024-06-03T02:05:53.901 INFO:teuthology.orchestra.run.smithi079.stdout:mgr epoch 9 is available 2024-06-03T02:05:53.901 INFO:teuthology.orchestra.run.smithi079.stdout:Generating a dashboard self-signed certificate... 2024-06-03T02:05:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:54 smithi079 ceph-mon[31819]: [03/Jun/2024:02:05:52] ENGINE Bus STARTING 2024-06-03T02:05:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:54 smithi079 ceph-mon[31819]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-03T02:05:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:54 smithi079 ceph-mon[31819]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-03T02:05:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:54 smithi079 ceph-mon[31819]: [03/Jun/2024:02:05:53] ENGINE Serving on https://172.21.15.79:7150 2024-06-03T02:05:55.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:55 smithi079 ceph-mon[31819]: [03/Jun/2024:02:05:53] ENGINE Serving on http://172.21.15.79:8765 2024-06-03T02:05:55.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:55 smithi079 ceph-mon[31819]: [03/Jun/2024:02:05:53] ENGINE Bus STARTED 2024-06-03T02:05:55.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:55 smithi079 ceph-mon[31819]: mgrmap e12: x(active, since 2s) 2024-06-03T02:05:55.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:55 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:05:56.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-03T02:05:56.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.037+0000 7f0949c00700 1 Processor -- start 2024-06-03T02:05:56.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.037+0000 7f0949c00700 1 -- start start 2024-06-03T02:05:56.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.038+0000 7f0949c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094410adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.038+0000 7f0949c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f094407bde0 con 0x7f0944108a00 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.038+0000 7f0942a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094410adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.038+0000 7f0942a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094410adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44814/0 (socket says 172.21.15.79:44814) 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.038+0000 7f0942a00700 1 -- 172.21.15.79:0/855750625 learned_addr learned my addr 172.21.15.79:0/855750625 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.039+0000 7f0942a00700 1 -- 172.21.15.79:0/855750625 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094410b330 con 0x7f0944108a00 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.039+0000 7f0942a00700 1 --2- 172.21.15.79:0/855750625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094410adf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f09340098a0 tx=0x7f0934009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1897fa10e120105 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.039+0000 7f0942000700 1 -- 172.21.15.79:0/855750625 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0934004160 con 0x7f0944108a00 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.039+0000 7f0942000700 1 -- 172.21.15.79:0/855750625 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0934004300 con 0x7f0944108a00 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.039+0000 7f0942000700 1 -- 172.21.15.79:0/855750625 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0934011b20 con 0x7f0944108a00 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.040+0000 7f0949c00700 1 -- 172.21.15.79:0/855750625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 msgr2=0x7f094410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.040+0000 7f0949c00700 1 --2- 172.21.15.79:0/855750625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094410adf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f09340098a0 tx=0x7f0934009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.040+0000 7f0949c00700 1 -- 172.21.15.79:0/855750625 shutdown_connections 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.040+0000 7f0949c00700 1 --2- 172.21.15.79:0/855750625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094410adf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.040+0000 7f0949c00700 1 -- 172.21.15.79:0/855750625 >> 172.21.15.79:0/855750625 conn(0x7f09441023f0 msgr2=0x7f0944104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.040+0000 7f0949c00700 1 -- 172.21.15.79:0/855750625 shutdown_connections 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.040+0000 7f0949c00700 1 -- 172.21.15.79:0/855750625 wait complete. 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.041+0000 7f0949c00700 1 Processor -- start 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.041+0000 7f0949c00700 1 -- start start 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.042+0000 7f0949c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094419bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.042+0000 7f0949c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f094410c2e0 con 0x7f0944108a00 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.042+0000 7f0942a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094419bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.042+0000 7f0942a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094419bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44828/0 (socket says 172.21.15.79:44828) 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.042+0000 7f0942a00700 1 -- 172.21.15.79:0/4077764078 learned_addr learned my addr 172.21.15.79:0/4077764078 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.042+0000 7f0942a00700 1 -- 172.21.15.79:0/4077764078 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094419c310 con 0x7f0944108a00 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.043+0000 7f0942a00700 1 --2- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094419bdd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f09340117e0 tx=0x7f0934011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:56.158 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.043+0000 7f093b400700 1 -- 172.21.15.79:0/4077764078 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0934003ba0 con 0x7f0944108a00 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.043+0000 7f093b400700 1 -- 172.21.15.79:0/4077764078 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f093401a910 con 0x7f0944108a00 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.043+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094419c5a0 con 0x7f0944108a00 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.043+0000 7f093b400700 1 -- 172.21.15.79:0/4077764078 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09340248f0 con 0x7f0944108a00 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.043+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094419ca00 con 0x7f0944108a00 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.045+0000 7f093b400700 1 -- 172.21.15.79:0/4077764078 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f0934022020 con 0x7f0944108a00 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.045+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0944063af0 con 0x7f0944108a00 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.045+0000 7f093b400700 1 --2- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0928039100 0x7f092803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.045+0000 7f093b400700 1 -- 172.21.15.79:0/4077764078 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f093404d660 con 0x7f0944108a00 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.046+0000 7f093be00700 1 --2- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0928039100 0x7f092803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.046+0000 7f093be00700 1 --2- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0928039100 0x7f092803b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f092c006fd0 tx=0x7f092c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.051+0000 7f093b400700 1 -- 172.21.15.79:0/4077764078 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f093400b3e0 con 0x7f0944108a00 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.298+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0944103500 con 0x7f0928039100 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.458+0000 7f093b400700 1 -- 172.21.15.79:0/4077764078 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0944103500 con 0x7f0928039100 2024-06-03T02:05:56.159 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.461+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0928039100 msgr2=0x7f092803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.461+0000 7f0949c00700 1 --2- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0928039100 0x7f092803b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f092c006fd0 tx=0x7f092c006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.461+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 msgr2=0x7f094419bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.461+0000 7f0949c00700 1 --2- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094419bdd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f09340117e0 tx=0x7f0934011ba0 comp rx=0 tx=0).stop 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.462+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 shutdown_connections 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.462+0000 7f0949c00700 1 --2- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0928039100 0x7f092803b5c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.462+0000 7f0949c00700 1 --2- 172.21.15.79:0/4077764078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0944108a00 0x7f094419bdd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.462+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 >> 172.21.15.79:0/4077764078 conn(0x7f09441023f0 msgr2=0x7f0944102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.462+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 shutdown_connections 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:55.462+0000 7f0949c00700 1 -- 172.21.15.79:0/4077764078 wait complete. 2024-06-03T02:05:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:Creating initial admin user... 2024-06-03T02:05:56.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:56 smithi079 ceph-mon[31819]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:05:56.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:56 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:05:56.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:56 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:05:58.036 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:57 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:05:58.037 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:57 smithi079 ceph-mon[31819]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:05:58.037 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:57 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:05:58.046 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$mipQ5VnV4roXMyFvWMPbg.fEHE9Rg68wBpNUjaAF0tuCOxNjUi3hi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717380357, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-03T02:05:58.046 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.778+0000 7f25b3400700 1 Processor -- start 2024-06-03T02:05:58.047 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.778+0000 7f25b3400700 1 -- start start 2024-06-03T02:05:58.047 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.778+0000 7f25b3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:58.047 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.778+0000 7f25b3400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac07c810 con 0x7f25ac07ddd0 2024-06-03T02:05:58.047 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.779+0000 7f25b0a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:58.052 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.779+0000 7f25b0a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44842/0 (socket says 172.21.15.79:44842) 2024-06-03T02:05:58.052 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.779+0000 7f25b0a00700 1 -- 172.21.15.79:0/1806076818 learned_addr learned my addr 172.21.15.79:0/1806076818 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:58.052 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.780+0000 7f25b0a00700 1 -- 172.21.15.79:0/1806076818 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac07c990 con 0x7f25ac07ddd0 2024-06-03T02:05:58.052 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.780+0000 7f25b0a00700 1 --2- 172.21.15.79:0/1806076818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac07c2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25a00098a0 tx=0x7f25a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa0a9219b001b95 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:58.052 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.780+0000 7f25ab400700 1 -- 172.21.15.79:0/1806076818 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a0004160 con 0x7f25ac07ddd0 2024-06-03T02:05:58.052 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.780+0000 7f25ab400700 1 -- 172.21.15.79:0/1806076818 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25a0004300 con 0x7f25ac07ddd0 2024-06-03T02:05:58.052 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.780+0000 7f25ab400700 1 -- 172.21.15.79:0/1806076818 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a0011b20 con 0x7f25ac07ddd0 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.781+0000 7f25b3400700 1 -- 172.21.15.79:0/1806076818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 msgr2=0x7f25ac07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.781+0000 7f25b3400700 1 --2- 172.21.15.79:0/1806076818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac07c2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25a00098a0 tx=0x7f25a0009bb0 comp rx=0 tx=0).stop 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.781+0000 7f25b3400700 1 -- 172.21.15.79:0/1806076818 shutdown_connections 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.781+0000 7f25b3400700 1 --2- 172.21.15.79:0/1806076818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac07c2d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.781+0000 7f25b3400700 1 -- 172.21.15.79:0/1806076818 >> 172.21.15.79:0/1806076818 conn(0x7f25ac1035e0 msgr2=0x7f25ac105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.782+0000 7f25b3400700 1 -- 172.21.15.79:0/1806076818 shutdown_connections 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.782+0000 7f25b3400700 1 -- 172.21.15.79:0/1806076818 wait complete. 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.783+0000 7f25b3400700 1 Processor -- start 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.783+0000 7f25b3400700 1 -- start start 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.783+0000 7f25b3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.783+0000 7f25b3400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac10e540 con 0x7f25ac07ddd0 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.783+0000 7f25b0a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:58.053 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.784+0000 7f25b0a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac19dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44856/0 (socket says 172.21.15.79:44856) 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.784+0000 7f25b0a00700 1 -- 172.21.15.79:0/849106183 learned_addr learned my addr 172.21.15.79:0/849106183 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.784+0000 7f25b0a00700 1 -- 172.21.15.79:0/849106183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac19e500 con 0x7f25ac07ddd0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.784+0000 7f25b0a00700 1 --2- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac19dfc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25a00117e0 tx=0x7f25a0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.784+0000 7f25a9600700 1 -- 172.21.15.79:0/849106183 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a0003ba0 con 0x7f25ac07ddd0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.785+0000 7f25a9600700 1 -- 172.21.15.79:0/849106183 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25a001a910 con 0x7f25ac07ddd0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.785+0000 7f25a9600700 1 -- 172.21.15.79:0/849106183 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a00248f0 con 0x7f25ac07ddd0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.785+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac19e790 con 0x7f25ac07ddd0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.785+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac19ebf0 con 0x7f25ac07ddd0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.786+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ac063af0 con 0x7f25ac07ddd0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.786+0000 7f25a9600700 1 -- 172.21.15.79:0/849106183 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f25a0022020 con 0x7f25ac07ddd0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.787+0000 7f25a9600700 1 --2- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25940390a0 0x7f259403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.787+0000 7f25a9600700 1 -- 172.21.15.79:0/849106183 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f25a004d670 con 0x7f25ac07ddd0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.787+0000 7f25abe00700 1 --2- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25940390a0 0x7f259403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:05:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.788+0000 7f25abe00700 1 --2- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25940390a0 0x7f259403b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2598006fd0 tx=0x7f2598006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:56.792+0000 7f25a9600700 1 -- 172.21.15.79:0/849106183 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f25a001ab70 con 0x7f25ac07ddd0 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.045+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f25ac1a15a0 con 0x7f25940390a0 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.286+0000 7f25a9600700 1 -- 172.21.15.79:0/849106183 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f25ac1a15a0 con 0x7f25940390a0 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25940390a0 msgr2=0x7f259403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 --2- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25940390a0 0x7f259403b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2598006fd0 tx=0x7f2598006e40 comp rx=0 tx=0).stop 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 msgr2=0x7f25ac19dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 --2- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac19dfc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25a00117e0 tx=0x7f25a0011ba0 comp rx=0 tx=0).stop 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 shutdown_connections 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 --2- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25940390a0 0x7f259403b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 --2- 172.21.15.79:0/849106183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25ac07ddd0 0x7f25ac19dfc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 >> 172.21.15.79:0/849106183 conn(0x7f25ac1035e0 msgr2=0x7f25ac104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 shutdown_connections 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:57.289+0000 7f25b3400700 1 -- 172.21.15.79:0/849106183 wait complete. 2024-06-03T02:05:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:Fetching dashboard port number... 2024-06-03T02:06:00.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:59 smithi079 ceph-mon[31819]: mgrmap e13: x(active, since 6s) 2024-06-03T02:06:00.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:05:59 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2755306228' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-03T02:06:00.496 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 8443 2024-06-03T02:06:00.496 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.666+0000 7f829a800700 1 Processor -- start 2024-06-03T02:06:00.496 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.666+0000 7f829a800700 1 -- start start 2024-06-03T02:06:00.496 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.666+0000 7f829a800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f82941080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:00.496 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.666+0000 7f829a800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8294108610 con 0x7f8294107cb0 2024-06-03T02:06:00.496 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.667+0000 7f8299400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f82941080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.667+0000 7f8299400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f82941080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44872/0 (socket says 172.21.15.79:44872) 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.667+0000 7f8299400700 1 -- 172.21.15.79:0/1649099535 learned_addr learned my addr 172.21.15.79:0/1649099535 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.667+0000 7f8299400700 1 -- 172.21.15.79:0/1649099535 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8294108790 con 0x7f8294107cb0 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.668+0000 7f8299400700 1 --2- 172.21.15.79:0/1649099535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f82941080d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82880098a0 tx=0x7f8288009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfa5d66882d84d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.668+0000 7f8293e00700 1 -- 172.21.15.79:0/1649099535 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8288004160 con 0x7f8294107cb0 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.668+0000 7f8293e00700 1 -- 172.21.15.79:0/1649099535 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8288004300 con 0x7f8294107cb0 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.668+0000 7f8293e00700 1 -- 172.21.15.79:0/1649099535 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8288011b20 con 0x7f8294107cb0 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.669+0000 7f829a800700 1 -- 172.21.15.79:0/1649099535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 msgr2=0x7f82941080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.669+0000 7f829a800700 1 --2- 172.21.15.79:0/1649099535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f82941080d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82880098a0 tx=0x7f8288009bb0 comp rx=0 tx=0).stop 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.669+0000 7f829a800700 1 -- 172.21.15.79:0/1649099535 shutdown_connections 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.669+0000 7f829a800700 1 --2- 172.21.15.79:0/1649099535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f82941080d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.669+0000 7f829a800700 1 -- 172.21.15.79:0/1649099535 >> 172.21.15.79:0/1649099535 conn(0x7f82941032d0 msgr2=0x7f8294105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.669+0000 7f829a800700 1 -- 172.21.15.79:0/1649099535 shutdown_connections 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.669+0000 7f829a800700 1 -- 172.21.15.79:0/1649099535 wait complete. 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.670+0000 7f829a800700 1 Processor -- start 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.670+0000 7f829a800700 1 -- start start 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.671+0000 7f829a800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f829419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.671+0000 7f829a800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f829410c120 con 0x7f8294107cb0 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.671+0000 7f8299400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f829419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.671+0000 7f8299400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f829419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44886/0 (socket says 172.21.15.79:44886) 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.671+0000 7f8299400700 1 -- 172.21.15.79:0/2755306228 learned_addr learned my addr 172.21.15.79:0/2755306228 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.671+0000 7f8299400700 1 -- 172.21.15.79:0/2755306228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829419e350 con 0x7f8294107cb0 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.672+0000 7f8299400700 1 --2- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f829419de10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8288011b60 tx=0x7f8288011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.672+0000 7f8292000700 1 -- 172.21.15.79:0/2755306228 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8288003b60 con 0x7f8294107cb0 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.672+0000 7f8292000700 1 -- 172.21.15.79:0/2755306228 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f828801a910 con 0x7f8294107cb0 2024-06-03T02:06:00.498 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.672+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829419e5e0 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.672+0000 7f8292000700 1 -- 172.21.15.79:0/2755306228 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8288024930 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.672+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829419ea40 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.673+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8294063af0 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.674+0000 7f8292000700 1 -- 172.21.15.79:0/2755306228 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f8288022020 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.674+0000 7f8292000700 1 --2- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f827c0390a0 0x7f827c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.674+0000 7f8292000700 1 -- 172.21.15.79:0/2755306228 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f828804d3e0 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.675+0000 7f8298a00700 1 --2- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f827c0390a0 0x7f827c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.675+0000 7f8298a00700 1 --2- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f827c0390a0 0x7f827c03b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8280006fd0 tx=0x7f8280006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.680+0000 7f8292000700 1 -- 172.21.15.79:0/2755306228 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f828800b3e0 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.848+0000 7f8292000700 1 -- 172.21.15.79:0/2755306228 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f828801a430 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.911+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f82941a14d0 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.912+0000 7f8292000700 1 -- 172.21.15.79:0/2755306228 <== mon.0 v2:172.21.15.79:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f828802ddb0 con 0x7f8294107cb0 2024-06-03T02:06:00.499 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.915+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f827c0390a0 msgr2=0x7f827c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:00.500 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.915+0000 7f829a800700 1 --2- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f827c0390a0 0x7f827c03b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8280006fd0 tx=0x7f8280006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:00.500 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.915+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 msgr2=0x7f829419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:00.500 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.915+0000 7f829a800700 1 --2- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f829419de10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8288011b60 tx=0x7f8288011c40 comp rx=0 tx=0).stop 2024-06-03T02:06:00.500 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.916+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 shutdown_connections 2024-06-03T02:06:00.500 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.916+0000 7f829a800700 1 --2- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f827c0390a0 0x7f827c03b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:00.500 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.916+0000 7f829a800700 1 --2- 172.21.15.79:0/2755306228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8294107cb0 0x7f829419de10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:00.500 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.916+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 >> 172.21.15.79:0/2755306228 conn(0x7f82941032d0 msgr2=0x7f8294103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:00.500 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.916+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 shutdown_connections 2024-06-03T02:06:00.500 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:05:58.916+0000 7f829a800700 1 -- 172.21.15.79:0/2755306228 wait complete. 2024-06-03T02:06:00.525 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T02:06:00.525 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout disabled 2024-06-03T02:06:00.552 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T02:06:00.552 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-06-03T02:06:00.552 INFO:teuthology.orchestra.run.smithi079.stdout:firewalld.service is not enabled 2024-06-03T02:06:00.552 INFO:teuthology.orchestra.run.smithi079.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-03T02:06:00.558 INFO:teuthology.orchestra.run.smithi079.stdout:Ceph Dashboard is now available at: 2024-06-03T02:06:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: URL: https://smithi079.front.sepia.ceph.com:8443/ 2024-06-03T02:06:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: User: admin 2024-06-03T02:06:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: Password: 41syfhsh2r 2024-06-03T02:06:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:00.560 INFO:teuthology.orchestra.run.smithi079.stdout:Saving cluster configuration to /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config directory 2024-06-03T02:06:00.562 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling autotune for osd_memory_target 2024-06-03T02:06:02.147 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.221+0000 7ff4acc00700 1 Processor -- start 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.222+0000 7ff4acc00700 1 -- start start 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.222+0000 7ff4acc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a80822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.222+0000 7ff4acc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a80828c0 con 0x7ff4a8081ed0 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.222+0000 7ff4a5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a80822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.223+0000 7ff4a5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a80822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44892/0 (socket says 172.21.15.79:44892) 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.223+0000 7ff4a5e00700 1 -- 172.21.15.79:0/2220087315 learned_addr learned my addr 172.21.15.79:0/2220087315 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.223+0000 7ff4a5e00700 1 -- 172.21.15.79:0/2220087315 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a810f6c0 con 0x7ff4a8081ed0 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.223+0000 7ff4a5e00700 1 --2- 172.21.15.79:0/2220087315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a80822f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4980098a0 tx=0x7ff498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cf62bc957ae61 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.223+0000 7ff4a4a00700 1 -- 172.21.15.79:0/2220087315 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498004160 con 0x7ff4a8081ed0 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.224+0000 7ff4a4a00700 1 -- 172.21.15.79:0/2220087315 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff498004300 con 0x7ff4a8081ed0 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.224+0000 7ff4a4a00700 1 -- 172.21.15.79:0/2220087315 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498011b20 con 0x7ff4a8081ed0 2024-06-03T02:06:02.149 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.224+0000 7ff4acc00700 1 -- 172.21.15.79:0/2220087315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 msgr2=0x7ff4a80822f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.224+0000 7ff4acc00700 1 --2- 172.21.15.79:0/2220087315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a80822f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4980098a0 tx=0x7ff498009bb0 comp rx=0 tx=0).stop 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.224+0000 7ff4acc00700 1 -- 172.21.15.79:0/2220087315 shutdown_connections 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.224+0000 7ff4acc00700 1 --2- 172.21.15.79:0/2220087315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a80822f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.224+0000 7ff4acc00700 1 -- 172.21.15.79:0/2220087315 >> 172.21.15.79:0/2220087315 conn(0x7ff4a807d6b0 msgr2=0x7ff4a807fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.224+0000 7ff4acc00700 1 -- 172.21.15.79:0/2220087315 shutdown_connections 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.224+0000 7ff4acc00700 1 -- 172.21.15.79:0/2220087315 wait complete. 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.226+0000 7ff4acc00700 1 Processor -- start 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.226+0000 7ff4acc00700 1 -- start start 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.226+0000 7ff4acc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a81a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.226+0000 7ff4acc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8112a70 con 0x7ff4a8081ed0 2024-06-03T02:06:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.226+0000 7ff4a5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a81a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.226+0000 7ff4a5e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a81a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44896/0 (socket says 172.21.15.79:44896) 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.226+0000 7ff4a5e00700 1 -- 172.21.15.79:0/3473772541 learned_addr learned my addr 172.21.15.79:0/3473772541 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.227+0000 7ff4a5e00700 1 -- 172.21.15.79:0/3473772541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a81a28b0 con 0x7ff4a8081ed0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.227+0000 7ff4a5e00700 1 --2- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a81a2370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4980117e0 tx=0x7ff498011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.227+0000 7ff49ea00700 1 -- 172.21.15.79:0/3473772541 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498003ba0 con 0x7ff4a8081ed0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.227+0000 7ff49ea00700 1 -- 172.21.15.79:0/3473772541 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff49801a910 con 0x7ff4a8081ed0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.227+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a81a2b40 con 0x7ff4a8081ed0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.227+0000 7ff49ea00700 1 -- 172.21.15.79:0/3473772541 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4980248f0 con 0x7ff4a8081ed0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.227+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81a2fa0 con 0x7ff4a8081ed0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.229+0000 7ff49ea00700 1 -- 172.21.15.79:0/3473772541 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff498022020 con 0x7ff4a8081ed0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.229+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a8063af0 con 0x7ff4a8081ed0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.229+0000 7ff49ea00700 1 --2- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff48c0391a0 0x7ff48c03b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.229+0000 7ff49ea00700 1 -- 172.21.15.79:0/3473772541 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff49804d680 con 0x7ff4a8081ed0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.229+0000 7ff4a5400700 1 --2- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff48c0391a0 0x7ff48c03b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:02.155 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.230+0000 7ff4a5400700 1 --2- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff48c0391a0 0x7ff48c03b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff494006fd0 tx=0x7ff494006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:02.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.235+0000 7ff49ea00700 1 -- 172.21.15.79:0/3473772541 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff49801ab70 con 0x7ff4a8081ed0 2024-06-03T02:06:02.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.454+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff4a81a5970 con 0x7ff4a8081ed0 2024-06-03T02:06:02.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.455+0000 7ff49ea00700 1 -- 172.21.15.79:0/3473772541 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff498010380 con 0x7ff4a8081ed0 2024-06-03T02:06:02.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.458+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff48c0391a0 msgr2=0x7ff48c03b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:02.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.458+0000 7ff4acc00700 1 --2- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff48c0391a0 0x7ff48c03b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff494006fd0 tx=0x7ff494006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:02.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.458+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 msgr2=0x7ff4a81a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:02.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.458+0000 7ff4acc00700 1 --2- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a81a2370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4980117e0 tx=0x7ff498011ba0 comp rx=0 tx=0).stop 2024-06-03T02:06:02.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.458+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 shutdown_connections 2024-06-03T02:06:02.156 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.458+0000 7ff4acc00700 1 --2- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff48c0391a0 0x7ff48c03b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:02.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.458+0000 7ff4acc00700 1 --2- 172.21.15.79:0/3473772541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4a8081ed0 0x7ff4a81a2370 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:02.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.458+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 >> 172.21.15.79:0/3473772541 conn(0x7ff4a807d6b0 msgr2=0x7ff4a807f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:02.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.459+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 shutdown_connections 2024-06-03T02:06:02.157 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:01.459+0000 7ff4acc00700 1 -- 172.21.15.79:0/3473772541 wait complete. 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.774+0000 7fc684e00700 1 Processor -- start 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.774+0000 7fc684e00700 1 -- start start 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.774+0000 7fc684e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc6801083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.774+0000 7fc684e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc680108910 con 0x7fc680107f90 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.775+0000 7fc67de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc6801083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.775+0000 7fc67de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc6801083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44904/0 (socket says 172.21.15.79:44904) 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.775+0000 7fc67de00700 1 -- 172.21.15.79:0/950544515 learned_addr learned my addr 172.21.15.79:0/950544515 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.776+0000 7fc67de00700 1 -- 172.21.15.79:0/950544515 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6801090d0 con 0x7fc680107f90 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.776+0000 7fc67de00700 1 --2- 172.21.15.79:0/950544515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc6801083d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6700098a0 tx=0x7fc670009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f7086e6cbb516d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.776+0000 7fc67d400700 1 -- 172.21.15.79:0/950544515 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc670004160 con 0x7fc680107f90 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.777+0000 7fc67d400700 1 -- 172.21.15.79:0/950544515 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc670004300 con 0x7fc680107f90 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.777+0000 7fc67d400700 1 -- 172.21.15.79:0/950544515 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc670011b20 con 0x7fc680107f90 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.777+0000 7fc684e00700 1 -- 172.21.15.79:0/950544515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 msgr2=0x7fc6801083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.777+0000 7fc684e00700 1 --2- 172.21.15.79:0/950544515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc6801083d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6700098a0 tx=0x7fc670009bb0 comp rx=0 tx=0).stop 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.777+0000 7fc684e00700 1 -- 172.21.15.79:0/950544515 shutdown_connections 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.777+0000 7fc684e00700 1 --2- 172.21.15.79:0/950544515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc6801083d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.777+0000 7fc684e00700 1 -- 172.21.15.79:0/950544515 >> 172.21.15.79:0/950544515 conn(0x7fc6801035e0 msgr2=0x7fc680105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.778+0000 7fc684e00700 1 -- 172.21.15.79:0/950544515 shutdown_connections 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.778+0000 7fc684e00700 1 -- 172.21.15.79:0/950544515 wait complete. 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.779+0000 7fc684e00700 1 Processor -- start 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.779+0000 7fc684e00700 1 -- start start 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.779+0000 7fc684e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc68019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.779+0000 7fc684e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc68010c390 con 0x7fc680107f90 2024-06-03T02:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.779+0000 7fc67de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc68019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.779+0000 7fc67de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc68019e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44912/0 (socket says 172.21.15.79:44912) 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.779+0000 7fc67de00700 1 -- 172.21.15.79:0/3686904511 learned_addr learned my addr 172.21.15.79:0/3686904511 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.780+0000 7fc67de00700 1 -- 172.21.15.79:0/3686904511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68019e540 con 0x7fc680107f90 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.780+0000 7fc67de00700 1 --2- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc68019e000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6700117e0 tx=0x7fc670011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.780+0000 7fc677400700 1 -- 172.21.15.79:0/3686904511 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc670003bf0 con 0x7fc680107f90 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.780+0000 7fc677400700 1 -- 172.21.15.79:0/3686904511 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67001a910 con 0x7fc680107f90 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.780+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68019e7d0 con 0x7fc680107f90 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.780+0000 7fc677400700 1 -- 172.21.15.79:0/3686904511 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc670024970 con 0x7fc680107f90 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.781+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68019ec30 con 0x7fc680107f90 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.782+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc680063af0 con 0x7fc680107f90 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.782+0000 7fc677400700 1 -- 172.21.15.79:0/3686904511 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc670022020 con 0x7fc680107f90 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.783+0000 7fc677400700 1 --2- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc664039140 0x7fc66403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.783+0000 7fc677400700 1 -- 172.21.15.79:0/3686904511 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc67004da70 con 0x7fc680107f90 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.783+0000 7fc675400700 1 --2- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc664039140 0x7fc66403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.784+0000 7fc675400700 1 --2- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc664039140 0x7fc66403b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc668006fd0 tx=0x7fc668006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:02.788+0000 7fc677400700 1 -- 172.21.15.79:0/3686904511 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc670010380 con 0x7fc680107f90 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.123+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc6801a1600 con 0x7fc680107f90 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.128+0000 7fc677400700 1 -- 172.21.15.79:0/3686904511 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fc6700212c0 con 0x7fc680107f90 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc664039140 msgr2=0x7fc66403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 --2- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc664039140 0x7fc66403b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc668006fd0 tx=0x7fc668006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 msgr2=0x7fc68019e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 --2- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc68019e000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6700117e0 tx=0x7fc670011ba0 comp rx=0 tx=0).stop 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 shutdown_connections 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 --2- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc664039140 0x7fc66403b600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 --2- 172.21.15.79:0/3686904511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc680107f90 0x7fc68019e000 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 >> 172.21.15.79:0/3686904511 conn(0x7fc6801035e0 msgr2=0x7fc680103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 shutdown_connections 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr 2024-06-03T02:06:03.131+0000 7fc684e00700 1 -- 172.21.15.79:0/3686904511 wait complete. 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout:Or, if you are only running a single cluster on this host: 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-03T02:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph telemetry on 2024-06-03T02:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout:For more information see: 2024-06-03T02:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-03T02:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout:Bootstrap complete. 2024-06-03T02:06:03.897 INFO:tasks.cephadm:Fetching config... 2024-06-03T02:06:03.897 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:06:03.897 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-03T02:06:03.950 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-03T02:06:03.950 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:06:03.951 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-03T02:06:03.990 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-03T02:06:03.990 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:06:03.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-03T02:06:04.098 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-03T02:06:04.099 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:06:04.099 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-03T02:06:04.192 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-03T02:06:04.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCiYUCumZdKhmuFl5RG+pbaJjAW2m8cnJsx9UL1U56NOf1kSPvE9gNyuxG4HSQkw5NX44ocwfy4v4OvIUS/Ol7WQsFRXv/DVnIQaEsclJIFzNFBK8E1dcsFEr8JJ8xUediSe+82L0YDP8Ibj8uPLHL7g1cNopCgJJfdcC0784ztuGZdmb6ZyvW0xCM2Q6NZUePVh4uy4WUx0Y8zePSmVfmW9S5yTZ4NBpn+exOF8lq+VMer6iCzFOPXmS+Ej+ltvZU4kwtCtjUeOHu6Jx42fWASisSABXp8IszVND3qI3H8A5xjU+J3zMSdEkogvYwN1ZOUnZUKTyUlNmMRx39zt79ErwvNHV2bKkcF7UlpNR4szRshbgp87JJw1u/xqUORiEGSjgUW5kiNDd7DAKiibIUgeaWj5obhEdzgmyikkt8iVJPlvgdx8tu5AWnbC2HuBp4ZhpMqWBRwMUSBHxpmw/8PD7D9RcFp9yw4LXJtsebwDHgoWEIJ7kqpjMUvOhq9MbE= ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T02:06:04.305 INFO:teuthology.orchestra.run.smithi079.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCiYUCumZdKhmuFl5RG+pbaJjAW2m8cnJsx9UL1U56NOf1kSPvE9gNyuxG4HSQkw5NX44ocwfy4v4OvIUS/Ol7WQsFRXv/DVnIQaEsclJIFzNFBK8E1dcsFEr8JJ8xUediSe+82L0YDP8Ibj8uPLHL7g1cNopCgJJfdcC0784ztuGZdmb6ZyvW0xCM2Q6NZUePVh4uy4WUx0Y8zePSmVfmW9S5yTZ4NBpn+exOF8lq+VMer6iCzFOPXmS+Ej+ltvZU4kwtCtjUeOHu6Jx42fWASisSABXp8IszVND3qI3H8A5xjU+J3zMSdEkogvYwN1ZOUnZUKTyUlNmMRx39zt79ErwvNHV2bKkcF7UlpNR4szRshbgp87JJw1u/xqUORiEGSjgUW5kiNDd7DAKiibIUgeaWj5obhEdzgmyikkt8iVJPlvgdx8tu5AWnbC2HuBp4ZhpMqWBRwMUSBHxpmw/8PD7D9RcFp9yw4LXJtsebwDHgoWEIJ7kqpjMUvOhq9MbE= ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:06:04.335 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:04 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3686904511' entity='client.admin' 2024-06-03T02:06:04.345 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCiYUCumZdKhmuFl5RG+pbaJjAW2m8cnJsx9UL1U56NOf1kSPvE9gNyuxG4HSQkw5NX44ocwfy4v4OvIUS/Ol7WQsFRXv/DVnIQaEsclJIFzNFBK8E1dcsFEr8JJ8xUediSe+82L0YDP8Ibj8uPLHL7g1cNopCgJJfdcC0784ztuGZdmb6ZyvW0xCM2Q6NZUePVh4uy4WUx0Y8zePSmVfmW9S5yTZ4NBpn+exOF8lq+VMer6iCzFOPXmS+Ej+ltvZU4kwtCtjUeOHu6Jx42fWASisSABXp8IszVND3qI3H8A5xjU+J3zMSdEkogvYwN1ZOUnZUKTyUlNmMRx39zt79ErwvNHV2bKkcF7UlpNR4szRshbgp87JJw1u/xqUORiEGSjgUW5kiNDd7DAKiibIUgeaWj5obhEdzgmyikkt8iVJPlvgdx8tu5AWnbC2HuBp4ZhpMqWBRwMUSBHxpmw/8PD7D9RcFp9yw4LXJtsebwDHgoWEIJ7kqpjMUvOhq9MbE= ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T02:06:04.386 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCiYUCumZdKhmuFl5RG+pbaJjAW2m8cnJsx9UL1U56NOf1kSPvE9gNyuxG4HSQkw5NX44ocwfy4v4OvIUS/Ol7WQsFRXv/DVnIQaEsclJIFzNFBK8E1dcsFEr8JJ8xUediSe+82L0YDP8Ibj8uPLHL7g1cNopCgJJfdcC0784ztuGZdmb6ZyvW0xCM2Q6NZUePVh4uy4WUx0Y8zePSmVfmW9S5yTZ4NBpn+exOF8lq+VMer6iCzFOPXmS+Ej+ltvZU4kwtCtjUeOHu6Jx42fWASisSABXp8IszVND3qI3H8A5xjU+J3zMSdEkogvYwN1ZOUnZUKTyUlNmMRx39zt79ErwvNHV2bKkcF7UlpNR4szRshbgp87JJw1u/xqUORiEGSjgUW5kiNDd7DAKiibIUgeaWj5obhEdzgmyikkt8iVJPlvgdx8tu5AWnbC2HuBp4ZhpMqWBRwMUSBHxpmw/8PD7D9RcFp9yw4LXJtsebwDHgoWEIJ7kqpjMUvOhq9MbE= ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:06:04.397 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCiYUCumZdKhmuFl5RG+pbaJjAW2m8cnJsx9UL1U56NOf1kSPvE9gNyuxG4HSQkw5NX44ocwfy4v4OvIUS/Ol7WQsFRXv/DVnIQaEsclJIFzNFBK8E1dcsFEr8JJ8xUediSe+82L0YDP8Ibj8uPLHL7g1cNopCgJJfdcC0784ztuGZdmb6ZyvW0xCM2Q6NZUePVh4uy4WUx0Y8zePSmVfmW9S5yTZ4NBpn+exOF8lq+VMer6iCzFOPXmS+Ej+ltvZU4kwtCtjUeOHu6Jx42fWASisSABXp8IszVND3qI3H8A5xjU+J3zMSdEkogvYwN1ZOUnZUKTyUlNmMRx39zt79ErwvNHV2bKkcF7UlpNR4szRshbgp87JJw1u/xqUORiEGSjgUW5kiNDd7DAKiibIUgeaWj5obhEdzgmyikkt8iVJPlvgdx8tu5AWnbC2HuBp4ZhpMqWBRwMUSBHxpmw/8PD7D9RcFp9yw4LXJtsebwDHgoWEIJ7kqpjMUvOhq9MbE= ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T02:06:04.439 INFO:teuthology.orchestra.run.smithi135.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCiYUCumZdKhmuFl5RG+pbaJjAW2m8cnJsx9UL1U56NOf1kSPvE9gNyuxG4HSQkw5NX44ocwfy4v4OvIUS/Ol7WQsFRXv/DVnIQaEsclJIFzNFBK8E1dcsFEr8JJ8xUediSe+82L0YDP8Ibj8uPLHL7g1cNopCgJJfdcC0784ztuGZdmb6ZyvW0xCM2Q6NZUePVh4uy4WUx0Y8zePSmVfmW9S5yTZ4NBpn+exOF8lq+VMer6iCzFOPXmS+Ej+ltvZU4kwtCtjUeOHu6Jx42fWASisSABXp8IszVND3qI3H8A5xjU+J3zMSdEkogvYwN1ZOUnZUKTyUlNmMRx39zt79ErwvNHV2bKkcF7UlpNR4szRshbgp87JJw1u/xqUORiEGSjgUW5kiNDd7DAKiibIUgeaWj5obhEdzgmyikkt8iVJPlvgdx8tu5AWnbC2HuBp4ZhpMqWBRwMUSBHxpmw/8PD7D9RcFp9yw4LXJtsebwDHgoWEIJ7kqpjMUvOhq9MbE= ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:06:04.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-03T02:06:04.864 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:06:07.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.691+0000 7f5bd4c00700 1 -- 172.21.15.79:0/2105982890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 msgr2=0x7f5bd0074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:07.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.691+0000 7f5bd4c00700 1 --2- 172.21.15.79:0/2105982890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 0x7f5bd0074d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009910 tx=0x7f5bc0009c20 comp rx=0 tx=0).stop 2024-06-03T02:06:07.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.692+0000 7f5bd4c00700 1 -- 172.21.15.79:0/2105982890 shutdown_connections 2024-06-03T02:06:07.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.692+0000 7f5bd4c00700 1 --2- 172.21.15.79:0/2105982890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 0x7f5bd0074d40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:07.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.692+0000 7f5bd4c00700 1 -- 172.21.15.79:0/2105982890 >> 172.21.15.79:0/2105982890 conn(0x7f5bd00fd4e0 msgr2=0x7f5bd00ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:07.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.692+0000 7f5bd4c00700 1 -- 172.21.15.79:0/2105982890 shutdown_connections 2024-06-03T02:06:07.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.692+0000 7f5bd4c00700 1 -- 172.21.15.79:0/2105982890 wait complete. 2024-06-03T02:06:07.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.693+0000 7f5bd4c00700 1 Processor -- start 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.694+0000 7f5bd4c00700 1 -- start start 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.694+0000 7f5bd4c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 0x7f5bd01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.694+0000 7f5bd4c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd010a2b0 con 0x7f5bd0074920 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.694+0000 7f5bcde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 0x7f5bd01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.694+0000 7f5bcde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 0x7f5bd01998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49298/0 (socket says 172.21.15.79:49298) 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.694+0000 7f5bcde00700 1 -- 172.21.15.79:0/1865865335 learned_addr learned my addr 172.21.15.79:0/1865865335 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.695+0000 7f5bcde00700 1 -- 172.21.15.79:0/1865865335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd0199e30 con 0x7f5bd0074920 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.695+0000 7f5bcde00700 1 --2- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 0x7f5bd01998f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0004fa0 tx=0x7f5bc0005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.695+0000 7f5bc6a00700 1 -- 172.21.15.79:0/1865865335 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bc0027070 con 0x7f5bd0074920 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.695+0000 7f5bc6a00700 1 -- 172.21.15.79:0/1865865335 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc000be40 con 0x7f5bd0074920 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.695+0000 7f5bc6a00700 1 -- 172.21.15.79:0/1865865335 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bc000f460 con 0x7f5bd0074920 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.695+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd019a0c0 con 0x7f5bd0074920 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.696+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd019cdb0 con 0x7f5bd0074920 2024-06-03T02:06:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.697+0000 7f5bc6a00700 1 -- 172.21.15.79:0/1865865335 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5bc002a070 con 0x7f5bd0074920 2024-06-03T02:06:07.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.697+0000 7f5bc6a00700 1 --2- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bb40390f0 0x7f5bb403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:07.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.697+0000 7f5bc6a00700 1 -- 172.21.15.79:0/1865865335 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5bc001e030 con 0x7f5bd0074920 2024-06-03T02:06:07.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.697+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd010aa30 con 0x7f5bd0074920 2024-06-03T02:06:07.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.697+0000 7f5bcd400700 1 --2- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bb40390f0 0x7f5bb403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:07.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.698+0000 7f5bcd400700 1 --2- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bb40390f0 0x7f5bb403b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc006fd0 tx=0x7f5bbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:07.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.703+0000 7f5bc6a00700 1 -- 172.21.15.79:0/1865865335 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5bc0013070 con 0x7f5bd0074920 2024-06-03T02:06:07.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.941+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f5bd002dca0 con 0x7f5bd0074920 2024-06-03T02:06:07.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.947+0000 7f5bc6a00700 1 -- 172.21.15.79:0/1865865335 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5bc0022830 con 0x7f5bd0074920 2024-06-03T02:06:07.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.950+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bb40390f0 msgr2=0x7f5bb403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:07.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.950+0000 7f5bd4c00700 1 --2- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bb40390f0 0x7f5bb403b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc006fd0 tx=0x7f5bbc006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:07.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.950+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 msgr2=0x7f5bd01998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:07.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.950+0000 7f5bd4c00700 1 --2- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 0x7f5bd01998f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0004fa0 tx=0x7f5bc0005080 comp rx=0 tx=0).stop 2024-06-03T02:06:07.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.950+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 shutdown_connections 2024-06-03T02:06:07.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.950+0000 7f5bd4c00700 1 --2- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bb40390f0 0x7f5bb403b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:07.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.950+0000 7f5bd4c00700 1 --2- 172.21.15.79:0/1865865335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bd0074920 0x7f5bd01998f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:07.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.950+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 >> 172.21.15.79:0/1865865335 conn(0x7f5bd00fd4e0 msgr2=0x7f5bd00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:07.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.951+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 shutdown_connections 2024-06-03T02:06:07.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:07.951+0000 7f5bd4c00700 1 -- 172.21.15.79:0/1865865335 wait complete. 2024-06-03T02:06:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:06:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:06:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:08 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1865865335' entity='client.admin' 2024-06-03T02:06:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:06:08.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:08.781 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-03T02:06:08.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-03T02:06:09.382 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:06:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.863+0000 7ff42be00700 1 -- 172.21.15.79:0/1310016310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 msgr2=0x7ff424104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.863+0000 7ff42be00700 1 --2- 172.21.15.79:0/1310016310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 0x7ff424104d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff418009910 tx=0x7ff418009c20 comp rx=0 tx=0).stop 2024-06-03T02:06:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.863+0000 7ff42be00700 1 -- 172.21.15.79:0/1310016310 shutdown_connections 2024-06-03T02:06:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.863+0000 7ff42be00700 1 --2- 172.21.15.79:0/1310016310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 0x7ff424104d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:11.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.863+0000 7ff42be00700 1 -- 172.21.15.79:0/1310016310 >> 172.21.15.79:0/1310016310 conn(0x7ff4240fc3d0 msgr2=0x7ff4240fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:11.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.864+0000 7ff42be00700 1 -- 172.21.15.79:0/1310016310 shutdown_connections 2024-06-03T02:06:11.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.864+0000 7ff42be00700 1 -- 172.21.15.79:0/1310016310 wait complete. 2024-06-03T02:06:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.865+0000 7ff42be00700 1 Processor -- start 2024-06-03T02:06:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.865+0000 7ff42be00700 1 -- start start 2024-06-03T02:06:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.865+0000 7ff42be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 0x7ff424197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.865+0000 7ff42be00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff424107f40 con 0x7ff4241029a0 2024-06-03T02:06:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.865+0000 7ff429400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 0x7ff424197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.865+0000 7ff429400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 0x7ff424197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49304/0 (socket says 172.21.15.79:49304) 2024-06-03T02:06:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.865+0000 7ff429400700 1 -- 172.21.15.79:0/1046013276 learned_addr learned my addr 172.21.15.79:0/1046013276 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.866+0000 7ff429400700 1 -- 172.21.15.79:0/1046013276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff424197ac0 con 0x7ff4241029a0 2024-06-03T02:06:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.866+0000 7ff429400700 1 --2- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 0x7ff424197580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff418005950 tx=0x7ff418005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.866+0000 7ff422000700 1 -- 172.21.15.79:0/1046013276 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff418026070 con 0x7ff4241029a0 2024-06-03T02:06:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.866+0000 7ff422000700 1 -- 172.21.15.79:0/1046013276 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff41800bbd0 con 0x7ff4241029a0 2024-06-03T02:06:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.866+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff424197db0 con 0x7ff4241029a0 2024-06-03T02:06:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.866+0000 7ff422000700 1 -- 172.21.15.79:0/1046013276 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff418003ad0 con 0x7ff4241029a0 2024-06-03T02:06:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.867+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42419aa40 con 0x7ff4241029a0 2024-06-03T02:06:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.868+0000 7ff422000700 1 -- 172.21.15.79:0/1046013276 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff418029070 con 0x7ff4241029a0 2024-06-03T02:06:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.868+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4241086c0 con 0x7ff4241029a0 2024-06-03T02:06:11.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.868+0000 7ff422000700 1 --2- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff40c039150 0x7ff40c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:11.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.868+0000 7ff422000700 1 -- 172.21.15.79:0/1046013276 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff41801e070 con 0x7ff4241029a0 2024-06-03T02:06:11.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.868+0000 7ff428a00700 1 --2- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff40c039150 0x7ff40c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:11.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.869+0000 7ff428a00700 1 --2- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff40c039150 0x7ff40c03b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff410006fd0 tx=0x7ff410006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:11.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:11.874+0000 7ff422000700 1 -- 172.21.15.79:0/1046013276 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff418013070 con 0x7ff4241029a0 2024-06-03T02:06:12.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.100+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff42402def0 con 0x7ff40c039150 2024-06-03T02:06:12.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.108+0000 7ff422000700 1 -- 172.21.15.79:0/1046013276 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff42402def0 con 0x7ff40c039150 2024-06-03T02:06:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.110+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff40c039150 msgr2=0x7ff40c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.110+0000 7ff42be00700 1 --2- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff40c039150 0x7ff40c03b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff410006fd0 tx=0x7ff410006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.110+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 msgr2=0x7ff424197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.110+0000 7ff42be00700 1 --2- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 0x7ff424197580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff418005950 tx=0x7ff418005030 comp rx=0 tx=0).stop 2024-06-03T02:06:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.110+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 shutdown_connections 2024-06-03T02:06:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.110+0000 7ff42be00700 1 --2- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff40c039150 0x7ff40c03b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.110+0000 7ff42be00700 1 --2- 172.21.15.79:0/1046013276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff4241029a0 0x7ff424197580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.110+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 >> 172.21.15.79:0/1046013276 conn(0x7ff4240fc3d0 msgr2=0x7ff4240fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.112+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 shutdown_connections 2024-06-03T02:06:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:12.112+0000 7ff42be00700 1 -- 172.21.15.79:0/1046013276 wait complete. 2024-06-03T02:06:12.928 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2024-06-03T02:06:12.929 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:06:12.929 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2024-06-03T02:06:12.947 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:06:12.947 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:06:13.005 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2024-06-03T02:06:13.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch host add smithi123 2024-06-03T02:06:13.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:13 smithi079 ceph-mon[31819]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:13.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:13 smithi079 ceph-mon[31819]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:06:13.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:13.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:06:13.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:13.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:06:13.405 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:06:14.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:14 smithi079 ceph-mon[31819]: Updating smithi079:/etc/ceph/ceph.conf 2024-06-03T02:06:14.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:14 smithi079 ceph-mon[31819]: Updating smithi079:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:06:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:15 smithi079 ceph-mon[31819]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:06:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:15 smithi079 ceph-mon[31819]: Updating smithi079:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T02:06:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:15 smithi079 ceph-mon[31819]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:15.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:15.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:15.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:15.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.544+0000 7f48e2600700 1 -- 172.21.15.79:0/3579369263 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 msgr2=0x7f48dc104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:15.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.544+0000 7f48e2600700 1 --2- 172.21.15.79:0/3579369263 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 0x7f48dc104700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009910 tx=0x7f48cc009c20 comp rx=0 tx=0).stop 2024-06-03T02:06:15.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.544+0000 7f48e2600700 1 -- 172.21.15.79:0/3579369263 shutdown_connections 2024-06-03T02:06:15.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.544+0000 7f48e2600700 1 --2- 172.21.15.79:0/3579369263 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 0x7f48dc104700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:15.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.544+0000 7f48e2600700 1 -- 172.21.15.79:0/3579369263 >> 172.21.15.79:0/3579369263 conn(0x7f48dc0ffbb0 msgr2=0x7f48dc101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:15.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.544+0000 7f48e2600700 1 -- 172.21.15.79:0/3579369263 shutdown_connections 2024-06-03T02:06:15.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.544+0000 7f48e2600700 1 -- 172.21.15.79:0/3579369263 wait complete. 2024-06-03T02:06:15.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.546+0000 7f48e2600700 1 Processor -- start 2024-06-03T02:06:15.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.546+0000 7f48e2600700 1 -- start start 2024-06-03T02:06:15.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.547+0000 7f48e2600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 0x7f48dc19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:15.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.547+0000 7f48e2600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc10e6f0 con 0x7f48dc104320 2024-06-03T02:06:15.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.547+0000 7f48db400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 0x7f48dc19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:15.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.547+0000 7f48db400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 0x7f48dc19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46042/0 (socket says 172.21.15.79:46042) 2024-06-03T02:06:15.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.547+0000 7f48db400700 1 -- 172.21.15.79:0/2580614655 learned_addr learned my addr 172.21.15.79:0/2580614655 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:15.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.548+0000 7f48db400700 1 -- 172.21.15.79:0/2580614655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc19fd60 con 0x7f48dc104320 2024-06-03T02:06:15.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.548+0000 7f48db400700 1 --2- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 0x7f48dc19f820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48cc005850 tx=0x7f48cc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:15.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.548+0000 7f48d3e00700 1 -- 172.21.15.79:0/2580614655 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48cc026070 con 0x7f48dc104320 2024-06-03T02:06:15.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.548+0000 7f48d3e00700 1 -- 172.21.15.79:0/2580614655 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc00bbd0 con 0x7f48dc104320 2024-06-03T02:06:15.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.548+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48dc1a0050 con 0x7f48dc104320 2024-06-03T02:06:15.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.548+0000 7f48d3e00700 1 -- 172.21.15.79:0/2580614655 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48cc003970 con 0x7f48dc104320 2024-06-03T02:06:15.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.549+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48dc1999e0 con 0x7f48dc104320 2024-06-03T02:06:15.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.550+0000 7f48d3e00700 1 -- 172.21.15.79:0/2580614655 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f48cc029070 con 0x7f48dc104320 2024-06-03T02:06:15.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.550+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48dc105800 con 0x7f48dc104320 2024-06-03T02:06:15.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.550+0000 7f48d3e00700 1 --2- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48c00390f0 0x7f48c003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:15.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.550+0000 7f48d3e00700 1 -- 172.21.15.79:0/2580614655 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48cc01e070 con 0x7f48dc104320 2024-06-03T02:06:15.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.551+0000 7f48daa00700 1 --2- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48c00390f0 0x7f48c003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:15.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.551+0000 7f48daa00700 1 --2- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48c00390f0 0x7f48c003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48c8006fd0 tx=0x7f48c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:15.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.556+0000 7f48d3e00700 1 -- 172.21.15.79:0/2580614655 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f48cc013070 con 0x7f48dc104320 2024-06-03T02:06:15.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:15.783+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}) -- 0x7f48dc19af80 con 0x7f48c00390f0 2024-06-03T02:06:17.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:17 smithi079 ceph-mon[31819]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:06:17.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:17 smithi079 ceph-mon[31819]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:18.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:18 smithi079 ceph-mon[31819]: Deploying cephadm binary to smithi123 2024-06-03T02:06:19.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:19 smithi079 ceph-mon[31819]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:20.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.576+0000 7f48d3e00700 1 -- 172.21.15.79:0/2580614655 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f48dc19af80 con 0x7f48c00390f0 2024-06-03T02:06:20.577 INFO:teuthology.orchestra.run.smithi079.stdout:Added host 'smithi123' with addr '172.21.15.123' 2024-06-03T02:06:20.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48c00390f0 msgr2=0x7f48c003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:20.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 --2- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48c00390f0 0x7f48c003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48c8006fd0 tx=0x7f48c8006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:20.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 msgr2=0x7f48dc19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:20.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 --2- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 0x7f48dc19f820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48cc005850 tx=0x7f48cc005030 comp rx=0 tx=0).stop 2024-06-03T02:06:20.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 shutdown_connections 2024-06-03T02:06:20.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 --2- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48c00390f0 0x7f48c003b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:20.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 --2- 172.21.15.79:0/2580614655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48dc104320 0x7f48dc19f820 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:20.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 >> 172.21.15.79:0/2580614655 conn(0x7f48dc0ffbb0 msgr2=0x7f48dc105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:20.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 shutdown_connections 2024-06-03T02:06:20.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:20.580+0000 7f48e2600700 1 -- 172.21.15.79:0/2580614655 wait complete. 2024-06-03T02:06:21.261 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:21 smithi079 ceph-mon[31819]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:21.261 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:21.261 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:06:21.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-03T02:06:21.772 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:06:22.258 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:22 smithi079 ceph-mon[31819]: Added host smithi123 2024-06-03T02:06:22.258 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:22 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:22.259 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:22 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:23.325 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:23 smithi079 ceph-mon[31819]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.140+0000 7fa0b3a00700 1 -- 172.21.15.79:0/1478788445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 msgr2=0x7fa0ac104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.140+0000 7fa0b3a00700 1 --2- 172.21.15.79:0/1478788445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 0x7fa0ac104700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009910 tx=0x7fa0a0009c20 comp rx=0 tx=0).stop 2024-06-03T02:06:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.140+0000 7fa0b3a00700 1 -- 172.21.15.79:0/1478788445 shutdown_connections 2024-06-03T02:06:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.140+0000 7fa0b3a00700 1 --2- 172.21.15.79:0/1478788445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 0x7fa0ac104700 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.140+0000 7fa0b3a00700 1 -- 172.21.15.79:0/1478788445 >> 172.21.15.79:0/1478788445 conn(0x7fa0ac0ffbb0 msgr2=0x7fa0ac101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.140+0000 7fa0b3a00700 1 -- 172.21.15.79:0/1478788445 shutdown_connections 2024-06-03T02:06:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.140+0000 7fa0b3a00700 1 -- 172.21.15.79:0/1478788445 wait complete. 2024-06-03T02:06:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.141+0000 7fa0b3a00700 1 Processor -- start 2024-06-03T02:06:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.141+0000 7fa0b3a00700 1 -- start start 2024-06-03T02:06:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.142+0000 7fa0b3a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 0x7fa0ac19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.142+0000 7fa0b3a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac10e6f0 con 0x7fa0ac104320 2024-06-03T02:06:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.142+0000 7fa0b1000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 0x7fa0ac19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.142+0000 7fa0b1000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 0x7fa0ac19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46058/0 (socket says 172.21.15.79:46058) 2024-06-03T02:06:24.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.142+0000 7fa0b1000700 1 -- 172.21.15.79:0/4147449199 learned_addr learned my addr 172.21.15.79:0/4147449199 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:24.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.142+0000 7fa0b1000700 1 -- 172.21.15.79:0/4147449199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac19fd60 con 0x7fa0ac104320 2024-06-03T02:06:24.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.143+0000 7fa0b1000700 1 --2- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 0x7fa0ac19f820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0006010 tx=0x7fa0a0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:24.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.143+0000 7fa0a9600700 1 -- 172.21.15.79:0/4147449199 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0026070 con 0x7fa0ac104320 2024-06-03T02:06:24.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.143+0000 7fa0a9600700 1 -- 172.21.15.79:0/4147449199 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a000bbd0 con 0x7fa0ac104320 2024-06-03T02:06:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.143+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ac1a0050 con 0x7fa0ac104320 2024-06-03T02:06:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.143+0000 7fa0a9600700 1 -- 172.21.15.79:0/4147449199 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0003910 con 0x7fa0ac104320 2024-06-03T02:06:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.144+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ac1999e0 con 0x7fa0ac104320 2024-06-03T02:06:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.145+0000 7fa0a9600700 1 -- 172.21.15.79:0/4147449199 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa0a0029020 con 0x7fa0ac104320 2024-06-03T02:06:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.145+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ac105800 con 0x7fa0ac104320 2024-06-03T02:06:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.145+0000 7fa0a9600700 1 --2- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa094039150 0x7fa09403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.145+0000 7fa0a9600700 1 -- 172.21.15.79:0/4147449199 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa0a001e070 con 0x7fa0ac104320 2024-06-03T02:06:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.146+0000 7fa0abe00700 1 --2- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa094039150 0x7fa09403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.146+0000 7fa0abe00700 1 --2- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa094039150 0x7fa09403b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa098006fd0 tx=0x7fa098006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:24.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.151+0000 7fa0a9600700 1 -- 172.21.15.79:0/4147449199 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa0a0013070 con 0x7fa0ac104320 2024-06-03T02:06:24.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.377+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0ac19af80 con 0x7fa094039150 2024-06-03T02:06:24.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.380+0000 7fa0a9600700 1 -- 172.21.15.79:0/4147449199 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fa0ac19af80 con 0x7fa094039150 2024-06-03T02:06:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:24.380 INFO:teuthology.orchestra.run.smithi079.stdout:[{"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.382+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa094039150 msgr2=0x7fa09403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.382+0000 7fa0b3a00700 1 --2- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa094039150 0x7fa09403b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa098006fd0 tx=0x7fa098006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.382+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 msgr2=0x7fa0ac19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.382+0000 7fa0b3a00700 1 --2- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 0x7fa0ac19f820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0006010 tx=0x7fa0a0005030 comp rx=0 tx=0).stop 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.382+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 shutdown_connections 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.383+0000 7fa0b3a00700 1 --2- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa094039150 0x7fa09403b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.383+0000 7fa0b3a00700 1 --2- 172.21.15.79:0/4147449199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0ac104320 0x7fa0ac19f820 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.383+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 >> 172.21.15.79:0/4147449199 conn(0x7fa0ac0ffbb0 msgr2=0x7fa0ac105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.383+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 shutdown_connections 2024-06-03T02:06:24.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:24.383+0000 7fa0b3a00700 1 -- 172.21.15.79:0/4147449199 wait complete. 2024-06-03T02:06:24.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:24 smithi079 ceph-mon[31819]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:25.287 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi135 2024-06-03T02:06:25.287 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:06:25.288 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.conf 2024-06-03T02:06:25.305 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:06:25.306 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:06:25.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:25 smithi079 ceph-mon[31819]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:06:25.363 INFO:tasks.cephadm:Adding host smithi135 to orchestrator... 2024-06-03T02:06:25.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch host add smithi135 2024-06-03T02:06:25.767 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:06:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:26 smithi079 ceph-mon[31819]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:27.717 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:27.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:27.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:27.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:27.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:06:27.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:27.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:06:27.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:27 smithi079 ceph-mon[31819]: Updating smithi123:/etc/ceph/ceph.conf 2024-06-03T02:06:27.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:27 smithi079 ceph-mon[31819]: Updating smithi123:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:06:27.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.901+0000 7f4890600700 1 -- 172.21.15.79:0/1132121792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 msgr2=0x7f488810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:27.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.901+0000 7f4890600700 1 --2- 172.21.15.79:0/1132121792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 0x7f488810d8b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f487c009910 tx=0x7f487c009c20 comp rx=0 tx=0).stop 2024-06-03T02:06:27.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.901+0000 7f4890600700 1 -- 172.21.15.79:0/1132121792 shutdown_connections 2024-06-03T02:06:27.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.901+0000 7f4890600700 1 --2- 172.21.15.79:0/1132121792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 0x7f488810d8b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:27.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.901+0000 7f4890600700 1 -- 172.21.15.79:0/1132121792 >> 172.21.15.79:0/1132121792 conn(0x7f4888077c30 msgr2=0x7f488807a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:27.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.902+0000 7f4890600700 1 -- 172.21.15.79:0/1132121792 shutdown_connections 2024-06-03T02:06:27.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.902+0000 7f4890600700 1 -- 172.21.15.79:0/1132121792 wait complete. 2024-06-03T02:06:27.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.903+0000 7f4890600700 1 Processor -- start 2024-06-03T02:06:27.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.903+0000 7f4890600700 1 -- start start 2024-06-03T02:06:27.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.903+0000 7f4890600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 0x7f488819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:27.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.903+0000 7f4890600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4888112b40 con 0x7f488810d4d0 2024-06-03T02:06:27.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.904+0000 7f488dc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 0x7f488819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:27.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.904+0000 7f488dc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 0x7f488819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:59636/0 (socket says 172.21.15.79:59636) 2024-06-03T02:06:27.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.904+0000 7f488dc00700 1 -- 172.21.15.79:0/1401390003 learned_addr learned my addr 172.21.15.79:0/1401390003 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:27.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.904+0000 7f488dc00700 1 -- 172.21.15.79:0/1401390003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488819e5b0 con 0x7f488810d4d0 2024-06-03T02:06:27.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.904+0000 7f488dc00700 1 --2- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 0x7f488819e070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f487c0092e0 tx=0x7f487c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:27.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.904+0000 7f4886000700 1 -- 172.21.15.79:0/1401390003 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c026070 con 0x7f488810d4d0 2024-06-03T02:06:27.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.905+0000 7f4886000700 1 -- 172.21.15.79:0/1401390003 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487c00b750 con 0x7f488810d4d0 2024-06-03T02:06:27.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.905+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488819e840 con 0x7f488810d4d0 2024-06-03T02:06:27.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.905+0000 7f4886000700 1 -- 172.21.15.79:0/1401390003 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c003970 con 0x7f488810d4d0 2024-06-03T02:06:27.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.905+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488819eca0 con 0x7f488810d4d0 2024-06-03T02:06:27.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.906+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4888109bd0 con 0x7f488810d4d0 2024-06-03T02:06:27.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.906+0000 7f4886000700 1 -- 172.21.15.79:0/1401390003 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f487c029070 con 0x7f488810d4d0 2024-06-03T02:06:27.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.906+0000 7f4886000700 1 --2- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4870039150 0x7f487003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:27.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.907+0000 7f4886000700 1 -- 172.21.15.79:0/1401390003 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f487c01e070 con 0x7f488810d4d0 2024-06-03T02:06:27.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.907+0000 7f488d200700 1 --2- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4870039150 0x7f487003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:27.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.907+0000 7f488d200700 1 --2- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4870039150 0x7f487003b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4874006fd0 tx=0x7f4874006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:27.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:27.912+0000 7f4886000700 1 -- 172.21.15.79:0/1401390003 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f487c013070 con 0x7f488810d4d0 2024-06-03T02:06:28.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:28.140+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}) -- 0x7f488819f440 con 0x7f4870039150 2024-06-03T02:06:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:28 smithi079 ceph-mon[31819]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:06:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:28 smithi079 ceph-mon[31819]: Updating smithi123:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T02:06:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:28 smithi079 ceph-mon[31819]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:30.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:29 smithi079 ceph-mon[31819]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:06:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:29 smithi079 ceph-mon[31819]: Deploying cephadm binary to smithi135 2024-06-03T02:06:31.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:30 smithi079 ceph-mon[31819]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:32.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.705+0000 7f4886000700 1 -- 172.21.15.79:0/1401390003 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f488819f440 con 0x7f4870039150 2024-06-03T02:06:32.705 INFO:teuthology.orchestra.run.smithi079.stdout:Added host 'smithi135' with addr '172.21.15.135' 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.709+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4870039150 msgr2=0x7f487003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.709+0000 7f4890600700 1 --2- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4870039150 0x7f487003b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4874006fd0 tx=0x7f4874006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.709+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 msgr2=0x7f488819e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.709+0000 7f4890600700 1 --2- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 0x7f488819e070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f487c0092e0 tx=0x7f487c005030 comp rx=0 tx=0).stop 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.709+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 shutdown_connections 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.709+0000 7f4890600700 1 --2- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4870039150 0x7f487003b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.709+0000 7f4890600700 1 --2- 172.21.15.79:0/1401390003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f488810d4d0 0x7f488819e070 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.709+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 >> 172.21.15.79:0/1401390003 conn(0x7f4888077c30 msgr2=0x7f488810fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.710+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 shutdown_connections 2024-06-03T02:06:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:32.710+0000 7f4890600700 1 -- 172.21.15.79:0/1401390003 wait complete. 2024-06-03T02:06:33.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:33 smithi079 ceph-mon[31819]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:33.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:33.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:06:33.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-03T02:06:34.051 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:06:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:34 smithi079 ceph-mon[31819]: Added host smithi135 2024-06-03T02:06:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:34 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:34.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:34 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:35.233 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:35 smithi079 ceph-mon[31819]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:36.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.010+0000 7fe82d400700 1 -- 172.21.15.79:0/1666206648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 msgr2=0x7fe828074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:36.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.010+0000 7fe82d400700 1 --2- 172.21.15.79:0/1666206648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 0x7fe828074c10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe818009910 tx=0x7fe818009c20 comp rx=0 tx=0).stop 2024-06-03T02:06:36.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.010+0000 7fe82d400700 1 -- 172.21.15.79:0/1666206648 shutdown_connections 2024-06-03T02:06:36.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.010+0000 7fe82d400700 1 --2- 172.21.15.79:0/1666206648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 0x7fe828074c10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:36.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.010+0000 7fe82d400700 1 -- 172.21.15.79:0/1666206648 >> 172.21.15.79:0/1666206648 conn(0x7fe8280fd9d0 msgr2=0x7fe8280ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:36.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.010+0000 7fe82d400700 1 -- 172.21.15.79:0/1666206648 shutdown_connections 2024-06-03T02:06:36.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.010+0000 7fe82d400700 1 -- 172.21.15.79:0/1666206648 wait complete. 2024-06-03T02:06:36.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.011+0000 7fe82d400700 1 Processor -- start 2024-06-03T02:06:36.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.012+0000 7fe82d400700 1 -- start start 2024-06-03T02:06:36.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.012+0000 7fe82d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 0x7fe82819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:36.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.012+0000 7fe82d400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe828112b40 con 0x7fe828074830 2024-06-03T02:06:36.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.012+0000 7fe826800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 0x7fe82819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:36.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.012+0000 7fe826800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 0x7fe82819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47958/0 (socket says 172.21.15.79:47958) 2024-06-03T02:06:36.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.012+0000 7fe826800700 1 -- 172.21.15.79:0/2907712947 learned_addr learned my addr 172.21.15.79:0/2907712947 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:36.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.013+0000 7fe826800700 1 -- 172.21.15.79:0/2907712947 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82819e5b0 con 0x7fe828074830 2024-06-03T02:06:36.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.013+0000 7fe826800700 1 --2- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 0x7fe82819e070 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe818006010 tx=0x7fe818005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:36.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.013+0000 7fe81f400700 1 -- 172.21.15.79:0/2907712947 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe818026070 con 0x7fe828074830 2024-06-03T02:06:36.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.013+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82819e840 con 0x7fe828074830 2024-06-03T02:06:36.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.013+0000 7fe81f400700 1 -- 172.21.15.79:0/2907712947 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe81800bc30 con 0x7fe828074830 2024-06-03T02:06:36.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.013+0000 7fe81f400700 1 -- 172.21.15.79:0/2907712947 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe818003ab0 con 0x7fe828074830 2024-06-03T02:06:36.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.014+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82819eca0 con 0x7fe828074830 2024-06-03T02:06:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.014+0000 7fe81f400700 1 -- 172.21.15.79:0/2907712947 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe818029070 con 0x7fe828074830 2024-06-03T02:06:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.015+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe828075f70 con 0x7fe828074830 2024-06-03T02:06:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.015+0000 7fe81f400700 1 --2- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe80c0390f0 0x7fe80c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.015+0000 7fe81f400700 1 -- 172.21.15.79:0/2907712947 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe81801e070 con 0x7fe828074830 2024-06-03T02:06:36.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.015+0000 7fe81fe00700 1 --2- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe80c0390f0 0x7fe80c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:36.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.016+0000 7fe81fe00700 1 --2- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe80c0390f0 0x7fe80c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe810006fd0 tx=0x7fe810006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:36.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.021+0000 7fe81f400700 1 -- 172.21.15.79:0/2907712947 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe818013070 con 0x7fe828074830 2024-06-03T02:06:36.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.253+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe82819f3d0 con 0x7fe80c0390f0 2024-06-03T02:06:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.256+0000 7fe81f400700 1 -- 172.21.15.79:0/2907712947 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fe82819f3d0 con 0x7fe80c0390f0 2024-06-03T02:06:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:06:36.256 INFO:teuthology.orchestra.run.smithi079.stdout:[{"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}] 2024-06-03T02:06:36.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe80c0390f0 msgr2=0x7fe80c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:36.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 --2- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe80c0390f0 0x7fe80c03b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe810006fd0 tx=0x7fe810006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:36.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 msgr2=0x7fe82819e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:36.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 --2- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 0x7fe82819e070 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe818006010 tx=0x7fe818005050 comp rx=0 tx=0).stop 2024-06-03T02:06:36.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 shutdown_connections 2024-06-03T02:06:36.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 --2- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe80c0390f0 0x7fe80c03b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:36.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 --2- 172.21.15.79:0/2907712947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe828074830 0x7fe82819e070 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:36.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 >> 172.21.15.79:0/2907712947 conn(0x7fe8280fd9d0 msgr2=0x7fe8280ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:36.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 shutdown_connections 2024-06-03T02:06:36.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:36.259+0000 7fe82d400700 1 -- 172.21.15.79:0/2907712947 wait complete. 2024-06-03T02:06:36.675 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:36 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:36.675 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:36 smithi079 ceph-mon[31819]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:37.050 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-03T02:06:37.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-03T02:06:37.496 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:06:37.769 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:37 smithi079 ceph-mon[31819]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:06:38.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:38 smithi079 ceph-mon[31819]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:38.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:38 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:38.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:38 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:38.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:38 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:39.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.676+0000 7f57da400700 1 -- 172.21.15.79:0/2134732133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 msgr2=0x7f57d4069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:39.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.676+0000 7f57da400700 1 --2- 172.21.15.79:0/2134732133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 0x7f57d4069ef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009910 tx=0x7f57c8009c20 comp rx=0 tx=0).stop 2024-06-03T02:06:39.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.676+0000 7f57da400700 1 -- 172.21.15.79:0/2134732133 shutdown_connections 2024-06-03T02:06:39.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.676+0000 7f57da400700 1 --2- 172.21.15.79:0/2134732133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 0x7f57d4069ef0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:39.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.676+0000 7f57da400700 1 -- 172.21.15.79:0/2134732133 >> 172.21.15.79:0/2134732133 conn(0x7f57d4076700 msgr2=0x7f57d4076b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:39.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.676+0000 7f57da400700 1 -- 172.21.15.79:0/2134732133 shutdown_connections 2024-06-03T02:06:39.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.676+0000 7f57da400700 1 -- 172.21.15.79:0/2134732133 wait complete. 2024-06-03T02:06:39.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.677+0000 7f57da400700 1 Processor -- start 2024-06-03T02:06:39.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.677+0000 7f57da400700 1 -- start start 2024-06-03T02:06:39.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.678+0000 7f57da400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 0x7f57d419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:39.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.678+0000 7f57da400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f57d410c2d0 con 0x7f57d4069b10 2024-06-03T02:06:39.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.678+0000 7f57d9000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 0x7f57d419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:39.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.678+0000 7f57d9000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 0x7f57d419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47978/0 (socket says 172.21.15.79:47978) 2024-06-03T02:06:39.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.678+0000 7f57d9000700 1 -- 172.21.15.79:0/4007887968 learned_addr learned my addr 172.21.15.79:0/4007887968 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:06:39.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.679+0000 7f57d9000700 1 -- 172.21.15.79:0/4007887968 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d419fc40 con 0x7f57d4069b10 2024-06-03T02:06:39.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.679+0000 7f57d9000700 1 --2- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 0x7f57d419f700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f57c8005850 tx=0x7f57c8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:39.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.679+0000 7f57d1600700 1 -- 172.21.15.79:0/4007887968 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c8026070 con 0x7f57d4069b10 2024-06-03T02:06:39.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.679+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d419fed0 con 0x7f57d4069b10 2024-06-03T02:06:39.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.679+0000 7f57d1600700 1 -- 172.21.15.79:0/4007887968 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c800bbd0 con 0x7f57d4069b10 2024-06-03T02:06:39.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.679+0000 7f57d1600700 1 -- 172.21.15.79:0/4007887968 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c8003970 con 0x7f57d4069b10 2024-06-03T02:06:39.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.680+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d4199860 con 0x7f57d4069b10 2024-06-03T02:06:39.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.681+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d4103400 con 0x7f57d4069b10 2024-06-03T02:06:39.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.681+0000 7f57d1600700 1 -- 172.21.15.79:0/4007887968 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f57c8029070 con 0x7f57d4069b10 2024-06-03T02:06:39.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.682+0000 7f57d1600700 1 --2- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57bc039150 0x7f57bc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:39.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.682+0000 7f57d1600700 1 -- 172.21.15.79:0/4007887968 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f57c801e070 con 0x7f57d4069b10 2024-06-03T02:06:39.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.682+0000 7f57d3e00700 1 --2- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57bc039150 0x7f57bc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:39.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.683+0000 7f57d3e00700 1 --2- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57bc039150 0x7f57bc03b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f57c0006fd0 tx=0x7f57c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:39.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.687+0000 7f57d1600700 1 -- 172.21.15.79:0/4007887968 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f57c8051070 con 0x7f57d4069b10 2024-06-03T02:06:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:06:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:06:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:39 smithi079 ceph-mon[31819]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-03T02:06:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:39 smithi079 ceph-mon[31819]: Updating smithi135:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:06:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:39 smithi079 ceph-mon[31819]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:06:39.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:39.921+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f57d4063af0 con 0x7f57d4069b10 2024-06-03T02:06:40.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.687+0000 7f57d1600700 1 -- 172.21.15.79:0/4007887968 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f57c8030090 con 0x7f57d4069b10 2024-06-03T02:06:40.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.691+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57bc039150 msgr2=0x7f57bc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:40.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.691+0000 7f57da400700 1 --2- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57bc039150 0x7f57bc03b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f57c0006fd0 tx=0x7f57c0006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:40.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.691+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 msgr2=0x7f57d419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:40.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.691+0000 7f57da400700 1 --2- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 0x7f57d419f700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f57c8005850 tx=0x7f57c8005030 comp rx=0 tx=0).stop 2024-06-03T02:06:40.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.692+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 shutdown_connections 2024-06-03T02:06:40.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.692+0000 7f57da400700 1 --2- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57bc039150 0x7f57bc03b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:40.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.692+0000 7f57da400700 1 --2- 172.21.15.79:0/4007887968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57d4069b10 0x7f57d419f700 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:40.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.692+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 >> 172.21.15.79:0/4007887968 conn(0x7f57d4076700 msgr2=0x7f57d4105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:40.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.692+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 shutdown_connections 2024-06-03T02:06:40.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:06:40.692+0000 7f57da400700 1 -- 172.21.15.79:0/4007887968 wait complete. 2024-06-03T02:06:40.693 INFO:teuthology.orchestra.run.smithi079.stderr:adjusted tunables profile to default 2024-06-03T02:06:41.065 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:40 smithi079 ceph-mon[31819]: Updating smithi135:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T02:06:41.065 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:41.065 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:41.065 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:41.065 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:40 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4007887968' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-03T02:06:41.065 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:40 smithi079 ceph-mon[31819]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:41.466 INFO:tasks.cephadm:Adding mon.a on smithi079 2024-06-03T02:06:41.467 INFO:tasks.cephadm:Adding mon.b on smithi123 2024-06-03T02:06:41.467 INFO:tasks.cephadm:Adding mon.c on smithi135 2024-06-03T02:06:41.467 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi079:172.21.15.79=a;smithi123:172.21.15.123=b;smithi135:172.21.15.135=c' 2024-06-03T02:06:41.704 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:06:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:41 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4007887968' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-03T02:06:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:41 smithi079 ceph-mon[31819]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:06:42.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:42 smithi079 ceph-mon[31819]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:43.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.138+0000 7efbffd33700 1 -- 172.21.15.135:0/4088806907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 msgr2=0x7efbf81034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.138+0000 7efbffd33700 1 --2- 172.21.15.135:0/4088806907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 0x7efbf81034e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009910 tx=0x7efbf4009c20 comp rx=0 tx=0).stop 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.138+0000 7efbffd33700 1 -- 172.21.15.135:0/4088806907 shutdown_connections 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.138+0000 7efbffd33700 1 --2- 172.21.15.135:0/4088806907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 0x7efbf81034e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.138+0000 7efbffd33700 1 -- 172.21.15.135:0/4088806907 >> 172.21.15.135:0/4088806907 conn(0x7efbf80fc8f0 msgr2=0x7efbf80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.138+0000 7efbffd33700 1 -- 172.21.15.135:0/4088806907 shutdown_connections 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.138+0000 7efbffd33700 1 -- 172.21.15.135:0/4088806907 wait complete. 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.139+0000 7efbffd33700 1 Processor -- start 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.139+0000 7efbffd33700 1 -- start start 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.139+0000 7efbffd33700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 0x7efbf819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:43.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.139+0000 7efbffd33700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efbf810c580 con 0x7efbf8103100 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.139+0000 7efbfdacf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 0x7efbf819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.139+0000 7efbfdacf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 0x7efbf819d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.135:41408/0 (socket says 172.21.15.135:41408) 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.139+0000 7efbfdacf700 1 -- 172.21.15.135:0/49644178 learned_addr learned my addr 172.21.15.135:0/49644178 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.140+0000 7efbfdacf700 1 -- 172.21.15.135:0/49644178 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf819dc30 con 0x7efbf8103100 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.140+0000 7efbfdacf700 1 --2- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 0x7efbf819d6f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006010 tx=0x7efbf4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.140+0000 7efbeeffd700 1 -- 172.21.15.135:0/49644178 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf4026070 con 0x7efbf8103100 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.141+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf819dec0 con 0x7efbf8103100 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.141+0000 7efbeeffd700 1 -- 172.21.15.135:0/49644178 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf400bc30 con 0x7efbf8103100 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.141+0000 7efbeeffd700 1 -- 172.21.15.135:0/49644178 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf4003ab0 con 0x7efbf8103100 2024-06-03T02:06:43.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.141+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf81978b0 con 0x7efbf8103100 2024-06-03T02:06:43.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.141+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf806ab80 con 0x7efbf8103100 2024-06-03T02:06:43.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.142+0000 7efbeeffd700 1 -- 172.21.15.135:0/49644178 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efbf4029070 con 0x7efbf8103100 2024-06-03T02:06:43.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.142+0000 7efbeeffd700 1 --2- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efbe40390f0 0x7efbe403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:43.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.142+0000 7efbeeffd700 1 -- 172.21.15.135:0/49644178 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efbf401e070 con 0x7efbf8103100 2024-06-03T02:06:43.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.143+0000 7efbfd2ce700 1 --2- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efbe40390f0 0x7efbe403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:43.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.143+0000 7efbfd2ce700 1 --2- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efbe40390f0 0x7efbe403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efbe8006fd0 tx=0x7efbe8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:43.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.150+0000 7efbeeffd700 1 -- 172.21.15.135:0/49644178 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efbf4013070 con 0x7efbf8103100 2024-06-03T02:06:43.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.356+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi079:172.21.15.79=a;smithi123:172.21.15.123=b;smithi135:172.21.15.135=c", "target": ["mon-mgr", ""]}) -- 0x7efbf8198de0 con 0x7efbe40390f0 2024-06-03T02:06:43.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.370+0000 7efbeeffd700 1 -- 172.21.15.135:0/49644178 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efbf8198de0 con 0x7efbe40390f0 2024-06-03T02:06:43.370 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mon update... 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efbe40390f0 msgr2=0x7efbe403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 --2- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efbe40390f0 0x7efbe403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efbe8006fd0 tx=0x7efbe8006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 msgr2=0x7efbf819d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 --2- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 0x7efbf819d6f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006010 tx=0x7efbf4005050 comp rx=0 tx=0).stop 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 shutdown_connections 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 --2- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efbe40390f0 0x7efbe403b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 --2- 172.21.15.135:0/49644178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efbf8103100 0x7efbf819d6f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 >> 172.21.15.135:0/49644178 conn(0x7efbf80fc8f0 msgr2=0x7efbf8103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 shutdown_connections 2024-06-03T02:06:43.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:43.371+0000 7efbffd33700 1 -- 172.21.15.135:0/49644178 wait complete. 2024-06-03T02:06:43.992 DEBUG:teuthology.orchestra.run.smithi123:mon.b> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.b.service 2024-06-03T02:06:43.995 DEBUG:teuthology.orchestra.run.smithi135:mon.c> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.c.service 2024-06-03T02:06:43.997 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T02:06:43.997 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T02:06:44.615 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:06:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi079:172.21.15.79=a;smithi123:172.21.15.123=b;smithi135:172.21.15.135=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:06:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: Saving service mon spec with placement smithi079:172.21.15.79=a;smithi123:172.21.15.123=b;smithi135:172.21.15.135=c;count:3 2024-06-03T02:06:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:06:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:06:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:06:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: Deploying daemon mon.c on smithi135 2024-06-03T02:06:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:44 smithi079 ceph-mon[31819]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:47.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:47 smithi079 ceph-mon[31819]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:47.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.508+0000 7ff0f407a700 1 -- 172.21.15.135:0/2042512800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 msgr2=0x7ff0ec1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:47.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.508+0000 7ff0f407a700 1 --2- 172.21.15.135:0/2042512800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 0x7ff0ec1024e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009910 tx=0x7ff0e4009c20 comp rx=0 tx=0).stop 2024-06-03T02:06:47.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.508+0000 7ff0f407a700 1 -- 172.21.15.135:0/2042512800 shutdown_connections 2024-06-03T02:06:47.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.508+0000 7ff0f407a700 1 --2- 172.21.15.135:0/2042512800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 0x7ff0ec1024e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:47.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.508+0000 7ff0f407a700 1 -- 172.21.15.135:0/2042512800 >> 172.21.15.135:0/2042512800 conn(0x7ff0ec0fd9b0 msgr2=0x7ff0ec0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:47.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.508+0000 7ff0f407a700 1 -- 172.21.15.135:0/2042512800 shutdown_connections 2024-06-03T02:06:47.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.508+0000 7ff0f407a700 1 -- 172.21.15.135:0/2042512800 wait complete. 2024-06-03T02:06:47.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.509+0000 7ff0f407a700 1 Processor -- start 2024-06-03T02:06:47.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.509+0000 7ff0f407a700 1 -- start start 2024-06-03T02:06:47.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.509+0000 7ff0f407a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 0x7ff0ec19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:47.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.509+0000 7ff0f407a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec10c4d0 con 0x7ff0ec102100 2024-06-03T02:06:47.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.509+0000 7ff0f1e16700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 0x7ff0ec19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:47.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.509+0000 7ff0f1e16700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 0x7ff0ec19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.135:41420/0 (socket says 172.21.15.135:41420) 2024-06-03T02:06:47.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.509+0000 7ff0f1e16700 1 -- 172.21.15.135:0/1665215183 learned_addr learned my addr 172.21.15.135:0/1665215183 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:06:47.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.510+0000 7ff0f1e16700 1 -- 172.21.15.135:0/1665215183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec19dbc0 con 0x7ff0ec102100 2024-06-03T02:06:47.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.510+0000 7ff0f1e16700 1 --2- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 0x7ff0ec19d680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4006010 tx=0x7ff0e4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:47.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.511+0000 7ff0eaffd700 1 -- 172.21.15.135:0/1665215183 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e4026070 con 0x7ff0ec102100 2024-06-03T02:06:47.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.511+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec19deb0 con 0x7ff0ec102100 2024-06-03T02:06:47.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.511+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec197800 con 0x7ff0ec102100 2024-06-03T02:06:47.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.511+0000 7ff0eaffd700 1 -- 172.21.15.135:0/1665215183 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e400bbd0 con 0x7ff0ec102100 2024-06-03T02:06:47.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.511+0000 7ff0eaffd700 1 -- 172.21.15.135:0/1665215183 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e4030060 con 0x7ff0ec102100 2024-06-03T02:06:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.512+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d8005330 con 0x7ff0ec102100 2024-06-03T02:06:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.512+0000 7ff0eaffd700 1 -- 172.21.15.135:0/1665215183 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff0e4029070 con 0x7ff0ec102100 2024-06-03T02:06:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.512+0000 7ff0eaffd700 1 --2- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0d40390f0 0x7ff0d403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.513+0000 7ff0eaffd700 1 -- 172.21.15.135:0/1665215183 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff0e401e070 con 0x7ff0ec102100 2024-06-03T02:06:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.513+0000 7ff0f1615700 1 --2- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0d40390f0 0x7ff0d403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:47.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.514+0000 7ff0f1615700 1 --2- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0d40390f0 0x7ff0d403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0006fd0 tx=0x7ff0e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:47.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.529+0000 7ff0eaffd700 1 -- 172.21.15.135:0/1665215183 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff0e4013070 con 0x7ff0ec102100 2024-06-03T02:06:47.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.841+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff0d80058f0 con 0x7ff0ec102100 2024-06-03T02:06:47.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.842+0000 7ff0eaffd700 1 -- 172.21.15.135:0/1665215183 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7ff0e400e840 con 0x7ff0ec102100 2024-06-03T02:06:47.842 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:06:47.842 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":1,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","modified":"2024-06-03T02:04:38.900256Z","created":"2024-06-03T02:04:38.900256Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T02:06:47.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.844+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0d40390f0 msgr2=0x7ff0d403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.844+0000 7ff0f407a700 1 --2- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0d40390f0 0x7ff0d403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0006fd0 tx=0x7ff0e0006e40 comp rx=0 tx=0).stop 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.844+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 msgr2=0x7ff0ec19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.844+0000 7ff0f407a700 1 --2- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 0x7ff0ec19d680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4006010 tx=0x7ff0e4005030 comp rx=0 tx=0).stop 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.845+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 shutdown_connections 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.845+0000 7ff0f407a700 1 --2- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0d40390f0 0x7ff0d403b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.845+0000 7ff0f407a700 1 --2- 172.21.15.135:0/1665215183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0ec102100 0x7ff0ec19d680 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.845+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 >> 172.21.15.135:0/1665215183 conn(0x7ff0ec0fd9b0 msgr2=0x7ff0ec103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.845+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 shutdown_connections 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:47.845+0000 7ff0f407a700 1 -- 172.21.15.135:0/1665215183 wait complete. 2024-06-03T02:06:47.845 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 1 2024-06-03T02:06:48.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:48 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/1665215183' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T02:06:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 podman[36634]: 2024-06-03 02:06:48.541933548 +0000 UTC m=+0.123124787 container create abe230c9a7e3aba64925ecf47c45931d0c2767466f84fea2fdb8d5523c8f5c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-c, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-03T02:06:49.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 podman[36634]: 2024-06-03 02:06:48.72797186 +0000 UTC m=+0.309163108 container init abe230c9a7e3aba64925ecf47c45931d0c2767466f84fea2fdb8d5523c8f5c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-06-03T02:06:49.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 podman[36634]: 2024-06-03 02:06:48.73113763 +0000 UTC m=+0.312328872 container start abe230c9a7e3aba64925ecf47c45931d0c2767466f84fea2fdb8d5523c8f5c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-03T02:06:49.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: set uid:gid to 167:167 (ceph:ceph) 2024-06-03T02:06:49.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-03T02:06:49.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pidfile_write: ignore empty --pid-file 2024-06-03T02:06:49.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: load: jerasure load: lrc 2024-06-03T02:06:49.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: RocksDB version: 7.9.2 2024-06-03T02:06:49.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Git sha 0 2024-06-03T02:06:49.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-03T02:06:49.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: DB SUMMARY 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: DB Session ID: G1TBANN241ZM0Z0D5KBT 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: CURRENT file: CURRENT 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: IDENTITY file: IDENTITY 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.error_if_exists: 0 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.create_if_missing: 0 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.paranoid_checks: 1 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.env: 0x55c255793c60 2024-06-03T02:06:49.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.fs: PosixFileSystem 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.info_log: 0x55c257333b20 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_file_opening_threads: 16 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.statistics: (nil) 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.use_fsync: 0 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_log_file_size: 0 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.keep_log_file_num: 1000 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.recycle_log_file_num: 0 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.allow_fallocate: 1 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.allow_mmap_reads: 0 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.allow_mmap_writes: 0 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.use_direct_reads: 0 2024-06-03T02:06:49.098 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.create_missing_column_families: 0 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.db_log_dir: 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.wal_dir: 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.advise_random_on_open: 1 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.db_write_buffer_size: 0 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.write_buffer_manager: 0x55c2573041e0 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-03T02:06:49.099 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.rate_limiter: (nil) 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.wal_recovery_mode: 2 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.enable_thread_tracking: 0 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.enable_pipelined_write: 0 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.unordered_write: 0 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.row_cache: None 2024-06-03T02:06:49.100 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.wal_filter: None 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.allow_ingest_behind: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.two_write_queues: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.manual_wal_flush: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.wal_compression: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.atomic_flush: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.log_readahead_size: 0 2024-06-03T02:06:49.102 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.best_efforts_recovery: 0 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.allow_data_in_errors: 0 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.db_host_id: __hostname__ 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_background_jobs: 2 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_background_compactions: -1 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_subcompactions: 1 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_total_wal_size: 0 2024-06-03T02:06:49.103 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_open_files: -1 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bytes_per_sync: 0 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_readahead_size: 0 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_background_flushes: -1 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Compression algorithms supported: 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: kZSTD supported: 0 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: kXpressCompression supported: 0 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: kLZ4HCCompression supported: 1 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: kZlibCompression supported: 1 2024-06-03T02:06:49.104 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: kSnappyCompression supported: 1 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: kLZ4Compression supported: 1 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: kBZip2Compression supported: 0 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.merge_operator: 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_filter: None 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_filter_factory: None 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.sst_partitioner_factory: None 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c257333c40) 2024-06-03T02:06:49.105 INFO:journalctl@ceph.mon.c.smithi135.stdout: cache_index_and_filter_blocks: 1 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: pin_top_level_index_and_filter: 1 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: index_type: 0 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: data_block_index_type: 0 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: index_shortening: 1 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: checksum: 4 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: no_block_cache: 0 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: block_cache: 0x55c257335090 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: block_cache_name: BinnedLRUCache 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: block_cache_options: 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: capacity : 536870912 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: num_shard_bits : 4 2024-06-03T02:06:49.106 INFO:journalctl@ceph.mon.c.smithi135.stdout: strict_capacity_limit : 0 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: high_pri_pool_ratio: 0.000 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: block_cache_compressed: (nil) 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: persistent_cache: (nil) 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: block_size: 4096 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: block_size_deviation: 10 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: block_restart_interval: 16 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: index_block_restart_interval: 1 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: metadata_block_size: 4096 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: partition_filters: 0 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: use_delta_encoding: 1 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: filter_policy: bloomfilter 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: whole_key_filtering: 1 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: verify_compression: 0 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: read_amp_bytes_per_bit: 0 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: format_version: 5 2024-06-03T02:06:49.107 INFO:journalctl@ceph.mon.c.smithi135.stdout: enable_index_compression: 1 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout: block_align: 0 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout: max_auto_readahead_size: 262144 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout: prepopulate_block_cache: 0 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout: initial_auto_readahead_size: 8192 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout: num_file_reads_for_auto_readahead: 2 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.write_buffer_size: 33554432 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_write_buffer_number: 2 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression: NoCompression 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression: Disabled 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.prefix_extractor: nullptr 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.num_levels: 7 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-03T02:06:49.108 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression_opts.level: 32767 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression_opts.strategy: 0 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-03T02:06:49.109 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression_opts.enabled: false 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.target_file_size_base: 67108864 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-03T02:06:49.110 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.arena_block_size: 1048576 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-03T02:06:49.111 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.disable_auto_compactions: 0 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.table_properties_collectors: 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.inplace_update_support: 0 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-03T02:06:49.112 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.bloom_locality: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.max_successive_merges: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.paranoid_file_checks: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.force_consistency_checks: 1 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.report_bg_io_stats: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.ttl: 2592000 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.enable_blob_files: false 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.min_blob_size: 0 2024-06-03T02:06:49.113 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.blob_file_size: 268435456 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.blob_file_starting_level: 0 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7c9ac657-e3ab-4b9b-ad5f-546a3c770e59 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717380408760958, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717380408761367, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717380408, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7c9ac657-e3ab-4b9b-ad5f-546a3c770e59", "db_session_id": "G1TBANN241ZM0Z0D5KBT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-03T02:06:49.114 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717380408761446, "job": 1, "event": "recovery_finished"} 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c257424000 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: DB pointer 0x55c25740e000 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout: ** DB Stats ** 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T02:06:49.115 INFO:journalctl@ceph.mon.c.smithi135.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: ** Compaction Stats [default] ** 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: ** Compaction Stats [default] ** 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-03T02:06:49.116 INFO:journalctl@ceph.mon.c.smithi135.stdout: 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: Block cache BinnedLRUCache@0x55c257335090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-03T02:06:49.117 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: using public_addr v2:172.21.15.135:0/0 -> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: starting mon.c rank -1 at public addrs [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] at bind addrs [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(???) e0 preinit fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).mds e1 new map 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout: e1 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout: legacy client fscid: -1 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout: 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout: No filesystems configured 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-03T02:06:49.118 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Added host smithi123 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Updating smithi123:/etc/ceph/ceph.conf 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Updating smithi123:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Updating smithi123:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T02:06:49.120 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Deploying cephadm binary to smithi135 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Added host smithi135 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.121 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Updating smithi135:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Updating smithi135:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.122 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4007887968' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4007887968' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi079:172.21.15.79=a;smithi123:172.21.15.123=b;smithi135:172.21.15.135=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Saving service mon spec with placement smithi079:172.21.15.79=a;smithi123:172.21.15.123=b;smithi135:172.21.15.135=c;count:3 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:49.123 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:06:49.124 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:49.124 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: Deploying daemon mon.c on smithi135 2024-06-03T02:06:49.124 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.124 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:49.124 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/1665215183' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T02:06:49.124 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 ceph-mon[36653]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-03T02:06:49.124 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 bash[36634]: abe230c9a7e3aba64925ecf47c45931d0c2767466f84fea2fdb8d5523c8f5c2c 2024-06-03T02:06:49.124 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:48 smithi135 systemd[1]: Started Ceph mon.c for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:06:49.351 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T02:06:49.351 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T02:06:49.581 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:06:54.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.016+0000 7f48c0e36700 1 -- 172.21.15.135:0/2053248758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48a40057d0 msgr2=0x7f48a4007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:54.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.016+0000 7f48c0e36700 1 --2- 172.21.15.135:0/2053248758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48a40057d0 0x7f48a4007c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f48bc100dc0 tx=0x7f48b000feb0 comp rx=0 tx=0).stop 2024-06-03T02:06:54.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.016+0000 7f48c0e36700 1 -- 172.21.15.135:0/2053248758 shutdown_connections 2024-06-03T02:06:54.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.016+0000 7f48c0e36700 1 --2- 172.21.15.135:0/2053248758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48a40057d0 0x7f48a4007c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:54.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.016+0000 7f48c0e36700 1 -- 172.21.15.135:0/2053248758 >> 172.21.15.135:0/2053248758 conn(0x7f48bc0fc980 msgr2=0x7f48bc0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:54.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.016+0000 7f48c0e36700 1 -- 172.21.15.135:0/2053248758 shutdown_connections 2024-06-03T02:06:54.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.017+0000 7f48c0e36700 1 -- 172.21.15.135:0/2053248758 wait complete. 2024-06-03T02:06:54.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48c0e36700 1 Processor -- start 2024-06-03T02:06:54.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48c0e36700 1 -- start start 2024-06-03T02:06:54.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48c0e36700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48a40057d0 0x7f48bc199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:54.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48c0e36700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 0x7f48bc19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:54.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48c0e36700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc107ef0 con 0x7f48a40057d0 2024-06-03T02:06:54.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48c0e36700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc107d70 con 0x7f48bc19a260 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48b9d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 0x7f48bc19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48b9d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 0x7f48bc19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51440/0 (socket says 172.21.15.135:51440) 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48b9d9b700 1 -- 172.21.15.135:0/1721229355 learned_addr learned my addr 172.21.15.135:0/1721229355 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48b9d9b700 1 -- 172.21.15.135:0/1721229355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 msgr2=0x7f48bc19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48b9d9b700 1 -- 172.21.15.135:0/1721229355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 msgr2=0x7f48bc19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48b9d9b700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 0x7f48bc19e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48b9d9b700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 0x7f48bc19e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.018+0000 7f48ba59c700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48a40057d0 0x7f48bc199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.019+0000 7f48ba59c700 1 -- 172.21.15.135:0/1721229355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 msgr2=0x7f48bc19e5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.019+0000 7f48ba59c700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 0x7f48bc19e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.019+0000 7f48ba59c700 1 -- 172.21.15.135:0/1721229355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc19a8a0 con 0x7f48a40057d0 2024-06-03T02:06:54.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.020+0000 7f48ba59c700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48a40057d0 0x7f48bc199d20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f48b00059d0 tx=0x7f48b00057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:54.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.020+0000 7f48ab7fe700 1 -- 172.21.15.135:0/1721229355 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f48b0004b20 con 0x7f48a40057d0 2024-06-03T02:06:54.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.020+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48bc19eb30 con 0x7f48a40057d0 2024-06-03T02:06:54.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.020+0000 7f48ab7fe700 1 -- 172.21.15.135:0/1721229355 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b0013e40 con 0x7f48a40057d0 2024-06-03T02:06:54.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.020+0000 7f48ab7fe700 1 -- 172.21.15.135:0/1721229355 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f48b0011cd0 con 0x7f48a40057d0 2024-06-03T02:06:54.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.020+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48bc19f090 con 0x7f48a40057d0 2024-06-03T02:06:54.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.021+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48bc1025d0 con 0x7f48a40057d0 2024-06-03T02:06:54.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.022+0000 7f48ab7fe700 1 -- 172.21.15.135:0/1721229355 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f48b0023070 con 0x7f48a40057d0 2024-06-03T02:06:54.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.022+0000 7f48ab7fe700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f489c038e90 0x7f489c03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:54.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.022+0000 7f48ab7fe700 1 -- 172.21.15.135:0/1721229355 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f48b0020050 con 0x7f48a40057d0 2024-06-03T02:06:54.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.023+0000 7f48b9d9b700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f489c038e90 0x7f489c03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:54.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.024+0000 7f48b9d9b700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f489c038e90 0x7f489c03b350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009180 tx=0x7f48ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:54.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.038+0000 7f48ab7fe700 1 -- 172.21.15.135:0/1721229355 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f48b0031020 con 0x7f48a40057d0 2024-06-03T02:06:54.192 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: mon.a calling monitor election 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: mon.c calling monitor election 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T02:06:54.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: monmap epoch 2 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: last_changed 2024-06-03T02:06:48.799139+0000 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: min_mon_release 19 (squid) 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: election_strategy: 1 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: fsmap 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: mgrmap e13: x(active, since 61s) 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: overall HEALTH_OK 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:54.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:54.195 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:54.195 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:06:54.195 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:54.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.316+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f48bc19f340 con 0x7f48a40057d0 2024-06-03T02:06:54.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:06:54.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: mon.a calling monitor election 2024-06-03T02:06:54.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: mon.c calling monitor election 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: monmap epoch 2 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: last_changed 2024-06-03T02:06:48.799139+0000 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: min_mon_release 19 (squid) 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: election_strategy: 1 2024-06-03T02:06:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: fsmap 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: mgrmap e13: x(active, since 61s) 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: overall HEALTH_OK 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:06:54.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:06:54.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.317+0000 7f48ab7fe700 1 -- 172.21.15.135:0/1721229355 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f48b0028030 con 0x7f48a40057d0 2024-06-03T02:06:54.319 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:06:54.319 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":2,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","modified":"2024-06-03T02:06:48.799139Z","created":"2024-06-03T02:04:38.900256Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f489c038e90 msgr2=0x7f489c03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f489c038e90 0x7f489c03b350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009180 tx=0x7f48ac008040 comp rx=0 tx=0).stop 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48a40057d0 msgr2=0x7f48bc199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48a40057d0 0x7f48bc199d20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f48b00059d0 tx=0x7f48b00057d0 comp rx=0 tx=0).stop 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 shutdown_connections 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f489c038e90 0x7f489c03b350 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48a40057d0 0x7f48bc199d20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 --2- 172.21.15.135:0/1721229355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48bc19a260 0x7f48bc19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 >> 172.21.15.135:0/1721229355 conn(0x7f48bc0fc980 msgr2=0x7f48bc0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:54.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 shutdown_connections 2024-06-03T02:06:54.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:54.319+0000 7f48c0e36700 1 -- 172.21.15.135:0/1721229355 wait complete. 2024-06-03T02:06:54.321 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 2 2024-06-03T02:06:55.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:54 smithi079 ceph-mon[31819]: Deploying daemon mon.b on smithi123 2024-06-03T02:06:55.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:54 smithi079 ceph-mon[31819]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:55.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:54 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/1721229355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T02:06:55.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:54 smithi135 ceph-mon[36653]: Deploying daemon mon.b on smithi123 2024-06-03T02:06:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:54 smithi135 ceph-mon[36653]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:54 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/1721229355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T02:06:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:06:55.876 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T02:06:55.876 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T02:06:56.108 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:06:57.233 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:06:57 smithi135 ceph-mon[36653]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:57.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:06:57 smithi079 ceph-mon[31819]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:06:57.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.442+0000 7ffa2b274700 1 -- 172.21.15.135:0/1922301414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa24101100 msgr2=0x7ffa241014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.442+0000 7ffa2b274700 1 --2- 172.21.15.135:0/1922301414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa24101100 0x7ffa241014e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000b0a0 tx=0x7ffa2000b3b0 comp rx=0 tx=0).stop 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.442+0000 7ffa2b274700 1 -- 172.21.15.135:0/1922301414 shutdown_connections 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.442+0000 7ffa2b274700 1 --2- 172.21.15.135:0/1922301414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa24101100 0x7ffa241014e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.442+0000 7ffa2b274700 1 -- 172.21.15.135:0/1922301414 >> 172.21.15.135:0/1922301414 conn(0x7ffa240fc9d0 msgr2=0x7ffa240fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.442+0000 7ffa2b274700 1 -- 172.21.15.135:0/1922301414 shutdown_connections 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.442+0000 7ffa2b274700 1 -- 172.21.15.135:0/1922301414 wait complete. 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.443+0000 7ffa2b274700 1 Processor -- start 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.443+0000 7ffa2b274700 1 -- start start 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.443+0000 7ffa2b274700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa24101100 0x7ffa24198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:57.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.443+0000 7ffa2b274700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 0x7ffa241994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.443+0000 7ffa2b274700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2410b4d0 con 0x7ffa24101100 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.443+0000 7ffa2b274700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2410b350 con 0x7ffa24199060 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.443+0000 7ffa2880f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 0x7ffa241994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa2880f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 0x7ffa241994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51448/0 (socket says 172.21.15.135:51448) 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa2880f700 1 -- 172.21.15.135:0/3657086162 learned_addr learned my addr 172.21.15.135:0/3657086162 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa2880f700 1 -- 172.21.15.135:0/3657086162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 msgr2=0x7ffa241994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa2880f700 1 -- 172.21.15.135:0/3657086162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 msgr2=0x7ffa241994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa2880f700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 0x7ffa241994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa2880f700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 0x7ffa241994e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T02:06:57.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa29010700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa24101100 0x7ffa24198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:57.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa29010700 1 -- 172.21.15.135:0/3657086162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 msgr2=0x7ffa241994e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:06:57.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa29010700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 0x7ffa241994e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:06:57.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.444+0000 7ffa29010700 1 -- 172.21.15.135:0/3657086162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa241a1b90 con 0x7ffa24101100 2024-06-03T02:06:57.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.445+0000 7ffa29010700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa24101100 0x7ffa24198b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000b830 tx=0x7ffa20007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:57.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.445+0000 7ffa1a7fc700 1 -- 172.21.15.135:0/3657086162 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffa20007680 con 0x7ffa24101100 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.445+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa241a1d60 con 0x7ffa24101100 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.445+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa241a2230 con 0x7ffa24101100 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.445+0000 7ffa1a7fc700 1 -- 172.21.15.135:0/3657086162 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa20022070 con 0x7ffa24101100 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.445+0000 7ffa1a7fc700 1 -- 172.21.15.135:0/3657086162 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffa2001d620 con 0x7ffa24101100 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.446+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa08005330 con 0x7ffa24101100 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.447+0000 7ffa1a7fc700 1 -- 172.21.15.135:0/3657086162 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ffa200279d0 con 0x7ffa24101100 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.447+0000 7ffa1a7fc700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffa10038eb0 0x7ffa1003b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.447+0000 7ffa1a7fc700 1 -- 172.21.15.135:0/3657086162 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffa2000f050 con 0x7ffa24101100 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.447+0000 7ffa2880f700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffa10038eb0 0x7ffa1003b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:06:57.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.448+0000 7ffa2880f700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffa10038eb0 0x7ffa1003b370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009180 tx=0x7ffa14008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:06:57.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.463+0000 7ffa1a7fc700 1 -- 172.21.15.135:0/3657086162 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffa2001ce80 con 0x7ffa24101100 2024-06-03T02:06:57.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.733+0000 7ffa1a7fc700 1 -- 172.21.15.135:0/3657086162 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20026e30 con 0x7ffa24101100 2024-06-03T02:06:57.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:06:57.741+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ffa08005620 con 0x7ffa24101100 2024-06-03T02:06:57.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:06:57 smithi123 systemd[1]: Started Ceph mon.b for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:07:02.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.751+0000 7ffa1a7fc700 1 -- 172.21.15.135:0/3657086162 <== mon.0 v2:172.21.15.79:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7ffa2001d7c0 con 0x7ffa24101100 2024-06-03T02:07:02.753 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-03T02:07:02.753 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":3,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","modified":"2024-06-03T02:06:57.731594Z","created":"2024-06-03T02:04:38.900256Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-03T02:07:02.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffa10038eb0 msgr2=0x7ffa1003b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:02.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffa10038eb0 0x7ffa1003b370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009180 tx=0x7ffa14008040 comp rx=0 tx=0).stop 2024-06-03T02:07:02.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa24101100 msgr2=0x7ffa24198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:02.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa24101100 0x7ffa24198b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000b830 tx=0x7ffa20007b60 comp rx=0 tx=0).stop 2024-06-03T02:07:02.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 shutdown_connections 2024-06-03T02:07:02.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffa10038eb0 0x7ffa1003b370 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:02.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa24101100 0x7ffa24198b20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:02.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 --2- 172.21.15.135:0/3657086162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa24199060 0x7ffa241994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:02.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 >> 172.21.15.135:0/3657086162 conn(0x7ffa240fc9d0 msgr2=0x7ffa24102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:07:02.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 shutdown_connections 2024-06-03T02:07:02.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:02.755+0000 7ffa2b274700 1 -- 172.21.15.135:0/3657086162 wait complete. 2024-06-03T02:07:02.756 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 3 2024-06-03T02:07:03.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: mon.c calling monitor election 2024-06-03T02:07:03.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:07:03.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: mon.a calling monitor election 2024-06-03T02:07:03.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:07:03.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/3657086162' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T02:07:03.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:03.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: monmap epoch 3 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: last_changed 2024-06-03T02:06:57.731594+0000 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: min_mon_release 19 (squid) 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: election_strategy: 1 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:07:03.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-03T02:07:03.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-03T02:07:03.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: fsmap 2024-06-03T02:07:03.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:07:03.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: mgrmap e13: x(active, since 70s) 2024-06-03T02:07:03.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: overall HEALTH_OK 2024-06-03T02:07:03.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:03.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:03.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:03.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: mon.c calling monitor election 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: mon.a calling monitor election 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/3657086162' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: monmap epoch 3 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: last_changed 2024-06-03T02:06:57.731594+0000 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: min_mon_release 19 (squid) 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: election_strategy: 1 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-03T02:07:03.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: fsmap 2024-06-03T02:07:03.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:07:03.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: mgrmap e13: x(active, since 70s) 2024-06-03T02:07:03.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: overall HEALTH_OK 2024-06-03T02:07:03.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:03.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:03.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:03.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:03.289 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-03T02:07:03.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-03T02:07:03.718 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:07:04.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:04.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:04.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:04.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:05.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: mon.c calling monitor election 2024-06-03T02:07:05.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:07:05.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: mon.a calling monitor election 2024-06-03T02:07:05.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='client.? 172.21.15.135:0/3657086162' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: monmap epoch 3 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: last_changed 2024-06-03T02:06:57.731594+0000 2024-06-03T02:07:05.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: min_mon_release 19 (squid) 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: election_strategy: 1 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: fsmap 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: mgrmap e13: x(active, since 70s) 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: overall HEALTH_OK 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:05.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: mon.b calling monitor election 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: Updating smithi079:/etc/ceph/ceph.conf 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: Updating smithi123:/etc/ceph/ceph.conf 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: Updating smithi123:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: Updating smithi135:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: mon.b calling monitor election 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: mon.c calling monitor election 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: mon.a calling monitor election 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: monmap epoch 3 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:07:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: last_changed 2024-06-03T02:06:57.731594+0000 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: min_mon_release 19 (squid) 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: election_strategy: 1 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: fsmap 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: mgrmap e13: x(active, since 72s) 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: overall HEALTH_OK 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:05 smithi079 ceph-mon[31819]: Updating smithi079:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: mon.b calling monitor election 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: Updating smithi079:/etc/ceph/ceph.conf 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: Updating smithi123:/etc/ceph/ceph.conf 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: Updating smithi123:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: Updating smithi135:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: mon.b calling monitor election 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: mon.c calling monitor election 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: mon.a calling monitor election 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-03T02:07:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: monmap epoch 3 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: last_changed 2024-06-03T02:06:57.731594+0000 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: min_mon_release 19 (squid) 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: election_strategy: 1 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: fsmap 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: mgrmap e13: x(active, since 72s) 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: overall HEALTH_OK 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:05 smithi135 ceph-mon[36653]: Updating smithi079:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:07:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: mon.b calling monitor election 2024-06-03T02:07:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: Updating smithi079:/etc/ceph/ceph.conf 2024-06-03T02:07:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: Updating smithi123:/etc/ceph/ceph.conf 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: Updating smithi123:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: Updating smithi135:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: mon.b calling monitor election 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: mon.c calling monitor election 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: mon.a calling monitor election 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: monmap epoch 3 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: last_changed 2024-06-03T02:06:57.731594+0000 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: created 2024-06-03T02:04:38.900256+0000 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: min_mon_release 19 (squid) 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: election_strategy: 1 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: 0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-06-03T02:07:05.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: fsmap 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: mgrmap e13: x(active, since 72s) 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: overall HEALTH_OK 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:05.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:05 smithi123 ceph-mon[36640]: Updating smithi079:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T02:07:05.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.854+0000 7f923ee00700 1 -- 172.21.15.79:0/3217540753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 msgr2=0x7f923810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:05.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.854+0000 7f923ee00700 1 --2- 172.21.15.79:0/3217540753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 0x7f923810d8c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9228009910 tx=0x7f9228009c20 comp rx=0 tx=0).stop 2024-06-03T02:07:05.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.854+0000 7f923ee00700 1 -- 172.21.15.79:0/3217540753 shutdown_connections 2024-06-03T02:07:05.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.854+0000 7f923ee00700 1 --2- 172.21.15.79:0/3217540753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 0x7f923810d8c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:05.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.854+0000 7f923ee00700 1 -- 172.21.15.79:0/3217540753 >> 172.21.15.79:0/3217540753 conn(0x7f9238077d20 msgr2=0x7f923807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:07:05.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.854+0000 7f923ee00700 1 -- 172.21.15.79:0/3217540753 shutdown_connections 2024-06-03T02:07:05.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.854+0000 7f923ee00700 1 -- 172.21.15.79:0/3217540753 wait complete. 2024-06-03T02:07:05.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.855+0000 7f923ee00700 1 Processor -- start 2024-06-03T02:07:05.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.855+0000 7f923ee00700 1 -- start start 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f923ee00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 0x7f923819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f923ee00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f923819e710 0x7f92381a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f923ee00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92381a2fe0 0x7f92381a3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f923ee00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9238112b50 con 0x7f923810d4e0 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f923ee00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92381129d0 con 0x7f92381a2fe0 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f923ee00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9238112cd0 con 0x7f923819e710 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f9237e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 0x7f923819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f9237e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 0x7f923819e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34004/0 (socket says 172.21.15.79:34004) 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f9237e00700 1 -- 172.21.15.79:0/1793157832 learned_addr learned my addr 172.21.15.79:0/1793157832 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f923ce00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92381a2fe0 0x7f92381a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f9237400700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f923819e710 0x7f92381a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f9237e00700 1 -- 172.21.15.79:0/1793157832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f923819e710 msgr2=0x7f92381a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f9237e00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f923819e710 0x7f92381a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:05.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f9237e00700 1 -- 172.21.15.79:0/1793157832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92381a2fe0 msgr2=0x7f92381a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:05.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.856+0000 7f9237e00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92381a2fe0 0x7f92381a3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:05.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.857+0000 7f9237e00700 1 -- 172.21.15.79:0/1793157832 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92381aad50 con 0x7f923810d4e0 2024-06-03T02:07:05.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.857+0000 7f9237e00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 0x7f923819e1d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9228004f80 tx=0x7f9228004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:07:05.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.857+0000 7f9234c00700 1 -- 172.21.15.79:0/1793157832 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228026070 con 0x7f923810d4e0 2024-06-03T02:07:05.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.857+0000 7f9234c00700 1 -- 172.21.15.79:0/1793157832 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f922800b750 con 0x7f923810d4e0 2024-06-03T02:07:05.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.857+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92381aaf20 con 0x7f923810d4e0 2024-06-03T02:07:05.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.857+0000 7f9234c00700 1 -- 172.21.15.79:0/1793157832 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92280038e0 con 0x7f923810d4e0 2024-06-03T02:07:05.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.857+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92381ab390 con 0x7f923810d4e0 2024-06-03T02:07:05.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.858+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9238063af0 con 0x7f923810d4e0 2024-06-03T02:07:05.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.858+0000 7f9234c00700 1 -- 172.21.15.79:0/1793157832 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9228029070 con 0x7f923810d4e0 2024-06-03T02:07:05.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.859+0000 7f9234c00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f921c039580 0x7f921c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:05.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.859+0000 7f9234c00700 1 -- 172.21.15.79:0/1793157832 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f922804cdb0 con 0x7f923810d4e0 2024-06-03T02:07:05.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.859+0000 7f9237400700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f921c039580 0x7f921c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:05.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.860+0000 7f9237400700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f921c039580 0x7f921c03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9224009ad0 tx=0x7f9224009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:07:05.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:05.865+0000 7f9234c00700 1 -- 172.21.15.79:0/1793157832 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9228013070 con 0x7f923810d4e0 2024-06-03T02:07:06.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.105+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f92381ab6a0 con 0x7f923810d4e0 2024-06-03T02:07:06.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.106+0000 7f9234c00700 1 -- 172.21.15.79:0/1793157832 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f9228030020 con 0x7f923810d4e0 2024-06-03T02:07:06.106 INFO:teuthology.orchestra.run.smithi079.stdout:# minimal ceph.conf for 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:07:06.106 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2024-06-03T02:07:06.106 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:07:06.107 INFO:teuthology.orchestra.run.smithi079.stdout: mon_host = [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] 2024-06-03T02:07:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f921c039580 msgr2=0x7f921c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f921c039580 0x7f921c03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9224009ad0 tx=0x7f9224009210 comp rx=0 tx=0).stop 2024-06-03T02:07:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 msgr2=0x7f923819e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 0x7f923819e1d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9228004f80 tx=0x7f9228004fb0 comp rx=0 tx=0).stop 2024-06-03T02:07:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 shutdown_connections 2024-06-03T02:07:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92381a2fe0 0x7f92381a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f921c039580 0x7f921c03ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f923810d4e0 0x7f923819e1d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 --2- 172.21.15.79:0/1793157832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f923819e710 0x7f92381a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 >> 172.21.15.79:0/1793157832 conn(0x7f9238077d20 msgr2=0x7f923810a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:07:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 shutdown_connections 2024-06-03T02:07:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:06.109+0000 7f923ee00700 1 -- 172.21.15.79:0/1793157832 wait complete. 2024-06-03T02:07:06.884 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-03T02:07:06.884 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:07:06.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T02:07:06.942 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.942 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.942 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.942 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:06.942 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.942 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.942 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.942 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.942 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: Reconfiguring mon.a (unknown last config time)... 2024-06-03T02:07:06.943 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:07:06.943 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T02:07:06.943 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:06.943 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: Reconfiguring daemon mon.a on smithi079 2024-06-03T02:07:06.943 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:06.943 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:06 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1793157832' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:06.954 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:07:06.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:07:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:06.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: Reconfiguring mon.a (unknown last config time)... 2024-06-03T02:07:06.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:07:06.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T02:07:06.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:06.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: Reconfiguring daemon mon.a on smithi079 2024-06-03T02:07:06.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:06.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:06 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1793157832' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:07.061 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:07:07.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T02:07:07.092 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:07:07.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:07:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:07:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: Reconfiguring mon.a (unknown last config time)... 2024-06-03T02:07:07.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:07:07.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T02:07:07.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:07.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: Reconfiguring daemon mon.a on smithi079 2024-06-03T02:07:07.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:07.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:06 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1793157832' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:07.159 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:07:07.159 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T02:07:07.191 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:07:07.191 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T02:07:07.258 INFO:tasks.cephadm:Adding mgr.x on smithi079 2024-06-03T02:07:07.258 INFO:tasks.cephadm:Adding mgr.y on smithi123 2024-06-03T02:07:07.258 INFO:tasks.cephadm:Adding mgr.z on smithi135 2024-06-03T02:07:07.258 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi079=x;smithi123=y;smithi135=z' 2024-06-03T02:07:07.528 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:07:08.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:08 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:08.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:08 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:08.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:09.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.001+0000 7f7c8e49b700 1 -- 172.21.15.135:0/75844800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c88102ad0 msgr2=0x7f7c88104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:09.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.001+0000 7f7c8e49b700 1 --2- 172.21.15.135:0/75844800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c88102ad0 0x7f7c88104ec0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009910 tx=0x7f7c78009c20 comp rx=0 tx=0).stop 2024-06-03T02:07:09.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.001+0000 7f7c8e49b700 1 -- 172.21.15.135:0/75844800 shutdown_connections 2024-06-03T02:07:09.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.001+0000 7f7c8e49b700 1 --2- 172.21.15.135:0/75844800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c88102ad0 0x7f7c88104ec0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:09.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.001+0000 7f7c8e49b700 1 -- 172.21.15.135:0/75844800 >> 172.21.15.135:0/75844800 conn(0x7f7c880fc4c0 msgr2=0x7f7c880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.001+0000 7f7c8e49b700 1 -- 172.21.15.135:0/75844800 shutdown_connections 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.001+0000 7f7c8e49b700 1 -- 172.21.15.135:0/75844800 wait complete. 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.002+0000 7f7c8e49b700 1 Processor -- start 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.002+0000 7f7c8e49b700 1 -- start start 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.002+0000 7f7c8e49b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c88102ad0 0x7f7c881977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.002+0000 7f7c8e49b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c88197ce0 0x7f7c8819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.002+0000 7f7c8e49b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c88198160 0x7f7c8819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.002+0000 7f7c8e49b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88108020 con 0x7f7c88197ce0 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.002+0000 7f7c8e49b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88107ea0 con 0x7f7c88102ad0 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.002+0000 7f7c8e49b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881081a0 con 0x7f7c88198160 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8ca38700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c88198160 0x7f7c8819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:09.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8ca38700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c88198160 0x7f7c8819f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57876/0 (socket says 172.21.15.135:57876) 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8ca38700 1 -- 172.21.15.135:0/1541783038 learned_addr learned my addr 172.21.15.135:0/1541783038 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c87fff700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c88102ad0 0x7f7c881977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8ca38700 1 -- 172.21.15.135:0/1541783038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c88102ad0 msgr2=0x7f7c881977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8ca38700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c88102ad0 0x7f7c881977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8ca38700 1 -- 172.21.15.135:0/1541783038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c88197ce0 msgr2=0x7f7c8819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8ca38700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c88197ce0 0x7f7c8819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8ca38700 1 -- 172.21.15.135:0/1541783038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8819fbb0 con 0x7f7c88198160 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8ca38700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c88198160 0x7f7c8819f2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000ba50 tx=0x7f7c8000be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c857fa700 1 -- 172.21.15.135:0/1541783038 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80019070 con 0x7f7c88198160 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c857fa700 1 -- 172.21.15.135:0/1541783038 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c80014440 con 0x7f7c88198160 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8819fea0 con 0x7f7c88198160 2024-06-03T02:07:09.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c857fa700 1 -- 172.21.15.135:0/1541783038 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80007590 con 0x7f7c88198160 2024-06-03T02:07:09.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.003+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c881a0430 con 0x7f7c88198160 2024-06-03T02:07:09.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.004+0000 7f7c857fa700 1 -- 172.21.15.135:0/1541783038 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7c8001c070 con 0x7f7c88198160 2024-06-03T02:07:09.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.004+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c88063af0 con 0x7f7c88198160 2024-06-03T02:07:09.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.004+0000 7f7c857fa700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c70039580 0x7f7c7003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:09.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.004+0000 7f7c857fa700 1 -- 172.21.15.135:0/1541783038 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7c8004b890 con 0x7f7c88198160 2024-06-03T02:07:09.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.005+0000 7f7c87fff700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c70039580 0x7f7c7003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:09.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.006+0000 7f7c87fff700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c70039580 0x7f7c7003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800b5c0 tx=0x7f7c7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:07:09.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.010+0000 7f7c857fa700 1 -- 172.21.15.135:0/1541783038 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c800112a0 con 0x7f7c88198160 2024-06-03T02:07:09.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.217+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi079=x;smithi123=y;smithi135=z", "target": ["mon-mgr", ""]}) -- 0x7f7c88062960 con 0x7f7c70039580 2024-06-03T02:07:09.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:09 smithi123 ceph-mon[36640]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:09.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:09.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:09.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:07:09.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T02:07:09.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:09.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.230+0000 7f7c857fa700 1 -- 172.21.15.135:0/1541783038 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7c88062960 con 0x7f7c70039580 2024-06-03T02:07:09.229 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mgr update... 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c70039580 msgr2=0x7f7c7003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c70039580 0x7f7c7003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800b5c0 tx=0x7f7c7801a040 comp rx=0 tx=0).stop 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c88198160 msgr2=0x7f7c8819f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c88198160 0x7f7c8819f2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000ba50 tx=0x7f7c8000be10 comp rx=0 tx=0).stop 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 shutdown_connections 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c88198160 0x7f7c8819f2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c88102ad0 0x7f7c881977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c70039580 0x7f7c7003ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 --2- 172.21.15.135:0/1541783038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c88197ce0 0x7f7c8819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 >> 172.21.15.135:0/1541783038 conn(0x7f7c880fc4c0 msgr2=0x7f7c880fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 shutdown_connections 2024-06-03T02:07:09.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:07:09.232+0000 7f7c8e49b700 1 -- 172.21.15.135:0/1541783038 wait complete. 2024-06-03T02:07:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:09 smithi079 ceph-mon[31819]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:07:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T02:07:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:09.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:09 smithi135 ceph-mon[36653]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:07:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T02:07:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:09.811 DEBUG:teuthology.orchestra.run.smithi123:mgr.y> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-03T02:07:09.814 DEBUG:teuthology.orchestra.run.smithi135:mgr.z> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-03T02:07:09.816 INFO:tasks.cephadm:Deploying OSDs... 2024-06-03T02:07:09.816 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:07:09.816 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T02:07:09.879 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T02:07:09.879 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-06-03T02:07:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T02:07:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:09.932 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-06-03T02:07:09.932 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:09.932 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:09.932 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-06-03 02:06:08.715579764 +0000 2024-06-03T02:07:09.932 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-06-03 02:00:43.584174052 +0000 2024-06-03T02:07:09.932 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-06-03 02:00:43.584174052 +0000 2024-06-03T02:07:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-06-03 02:00:43.584174052 +0000 2024-06-03T02:07:09.932 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T02:07:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-06-03T02:07:10.008 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-06-03T02:07:10.008 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000372919 s, 1.4 MB/s 2024-06-03T02:07:10.011 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T02:07:10.060 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-06-03T02:07:10.111 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T02:07:10.111 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:10.111 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-06-03T02:07:10.111 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:10.111 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:10.111 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-06-03 02:06:08.792578042 +0000 2024-06-03T02:07:10.111 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-06-03 02:00:44.195160091 +0000 2024-06-03T02:07:10.111 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-06-03 02:00:44.195160091 +0000 2024-06-03T02:07:10.111 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-06-03 02:00:44.195160091 +0000 2024-06-03T02:07:10.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T02:07:10.184 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-06-03T02:07:10.184 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-06-03T02:07:10.184 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000315155 s, 1.6 MB/s 2024-06-03T02:07:10.188 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T02:07:10.240 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-06-03T02:07:10.290 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T02:07:10.290 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:10.290 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-06-03T02:07:10.290 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:10.290 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:10.290 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-06-03 02:06:08.849576766 +0000 2024-06-03T02:07:10.290 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-06-03 02:00:44.778146771 +0000 2024-06-03T02:07:10.290 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-06-03 02:00:44.778146771 +0000 2024-06-03T02:07:10.291 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-06-03 02:00:44.778146771 +0000 2024-06-03T02:07:10.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T02:07:10.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:10 smithi079 ceph-mon[31819]: Reconfiguring mon.b (monmap changed)... 2024-06-03T02:07:10.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:10 smithi079 ceph-mon[31819]: Reconfiguring daemon mon.b on smithi123 2024-06-03T02:07:10.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:10.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:10.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:10.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:07:10.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T02:07:10.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:10.372 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-06-03T02:07:10.373 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-06-03T02:07:10.373 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000398067 s, 1.3 MB/s 2024-06-03T02:07:10.376 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T02:07:10.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:10 smithi123 ceph-mon[36640]: Reconfiguring mon.b (monmap changed)... 2024-06-03T02:07:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:10 smithi123 ceph-mon[36640]: Reconfiguring daemon mon.b on smithi123 2024-06-03T02:07:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:07:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T02:07:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:10.474 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-06-03T02:07:10.501 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:10 smithi135 ceph-mon[36653]: Reconfiguring mon.b (monmap changed)... 2024-06-03T02:07:10.501 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:10 smithi135 ceph-mon[36653]: Reconfiguring daemon mon.b on smithi123 2024-06-03T02:07:10.501 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:10.501 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:10.501 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:10.502 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T02:07:10.502 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T02:07:10.502 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T02:07:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:10.526 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-06-03T02:07:10.526 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:10.527 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:10.527 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-06-03 02:06:08.903575558 +0000 2024-06-03T02:07:10.527 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-06-03 02:00:45.353133632 +0000 2024-06-03T02:07:10.527 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-06-03 02:00:45.353133632 +0000 2024-06-03T02:07:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-06-03 02:00:45.353133632 +0000 2024-06-03T02:07:10.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T02:07:10.610 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-06-03T02:07:10.610 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-06-03T02:07:10.610 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000515523 s, 993 kB/s 2024-06-03T02:07:10.614 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T02:07:10.670 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-03T02:07:10.670 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T02:07:10.688 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T02:07:10.688 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-06-03T02:07:10.746 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T02:07:10.747 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:10.747 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-03T02:07:10.747 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:10.747 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:10.747 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-06-03 02:06:57.196850036 +0000 2024-06-03T02:07:10.747 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-06-03 01:59:01.605360418 +0000 2024-06-03T02:07:10.747 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-06-03 01:59:01.605360418 +0000 2024-06-03T02:07:10.747 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-06-03 01:59:01.605360418 +0000 2024-06-03T02:07:10.747 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T02:07:10.813 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-06-03T02:07:10.813 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-06-03T02:07:10.813 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000201516 s, 2.5 MB/s 2024-06-03T02:07:10.815 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T02:07:10.872 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-06-03T02:07:10.929 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T02:07:10.929 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:10.929 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-03T02:07:10.929 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:10.929 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:10.929 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-06-03 02:06:57.196850036 +0000 2024-06-03T02:07:10.929 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-06-03 01:59:01.939352268 +0000 2024-06-03T02:07:10.930 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-06-03 01:59:01.939352268 +0000 2024-06-03T02:07:10.930 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-06-03 01:59:01.939352268 +0000 2024-06-03T02:07:10.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T02:07:10.994 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-06-03T02:07:10.994 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-06-03T02:07:10.995 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000296539 s, 1.7 MB/s 2024-06-03T02:07:10.996 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T02:07:11.052 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-06-03T02:07:11.108 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T02:07:11.109 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:11.109 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-03T02:07:11.109 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:11.109 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:11.109 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-06-03 02:06:57.196850036 +0000 2024-06-03T02:07:11.109 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-06-03 01:59:02.288343752 +0000 2024-06-03T02:07:11.109 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-06-03 01:59:02.288343752 +0000 2024-06-03T02:07:11.109 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-06-03 01:59:02.288343752 +0000 2024-06-03T02:07:11.109 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T02:07:11.172 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-06-03T02:07:11.173 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-06-03T02:07:11.173 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000250845 s, 2.0 MB/s 2024-06-03T02:07:11.174 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T02:07:11.231 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-06-03T02:07:11.286 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T02:07:11.286 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:11.286 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-03T02:07:11.286 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:11.286 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:11.286 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-06-03 02:06:57.197850011 +0000 2024-06-03T02:07:11.286 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-06-03 01:59:02.622335602 +0000 2024-06-03T02:07:11.286 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-06-03 01:59:02.622335602 +0000 2024-06-03T02:07:11.286 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-06-03 01:59:02.622335602 +0000 2024-06-03T02:07:11.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T02:07:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-06-03T02:07:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-06-03T02:07:11.349 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000206105 s, 2.5 MB/s 2024-06-03T02:07:11.350 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T02:07:11.406 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-03T02:07:11.406 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T02:07:11.422 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T02:07:11.422 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-06-03T02:07:11.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:11 smithi123 ceph-mon[36640]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi079=x;smithi123=y;smithi135=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:07:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:11 smithi123 ceph-mon[36640]: Saving service mgr spec with placement smithi079=x;smithi123=y;smithi135=z;count:3 2024-06-03T02:07:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:11 smithi123 ceph-mon[36640]: Reconfiguring mon.c (monmap changed)... 2024-06-03T02:07:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:11 smithi123 ceph-mon[36640]: Reconfiguring daemon mon.c on smithi135 2024-06-03T02:07:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:11 smithi123 ceph-mon[36640]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:11.479 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T02:07:11.479 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:11.479 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-06-03T02:07:11.479 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:11.479 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:11.479 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-03 02:06:47.830761828 +0000 2024-06-03T02:07:11.479 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-03 01:59:02.411212894 +0000 2024-06-03T02:07:11.479 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-03 01:59:02.411212894 +0000 2024-06-03T02:07:11.479 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-03 01:59:02.411212894 +0000 2024-06-03T02:07:11.480 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T02:07:11.541 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:11 smithi135 ceph-mon[36653]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi079=x;smithi123=y;smithi135=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:07:11.541 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:11 smithi135 ceph-mon[36653]: Saving service mgr spec with placement smithi079=x;smithi123=y;smithi135=z;count:3 2024-06-03T02:07:11.542 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:11 smithi135 ceph-mon[36653]: Reconfiguring mon.c (monmap changed)... 2024-06-03T02:07:11.542 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:11 smithi135 ceph-mon[36653]: Reconfiguring daemon mon.c on smithi135 2024-06-03T02:07:11.542 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:11 smithi135 ceph-mon[36653]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:11.544 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-03T02:07:11.544 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-03T02:07:11.544 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000266758 s, 1.9 MB/s 2024-06-03T02:07:11.546 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T02:07:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:11 smithi079 ceph-mon[31819]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi079=x;smithi123=y;smithi135=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:07:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:11 smithi079 ceph-mon[31819]: Saving service mgr spec with placement smithi079=x;smithi123=y;smithi135=z;count:3 2024-06-03T02:07:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:11 smithi079 ceph-mon[31819]: Reconfiguring mon.c (monmap changed)... 2024-06-03T02:07:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:11 smithi079 ceph-mon[31819]: Reconfiguring daemon mon.c on smithi135 2024-06-03T02:07:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:11 smithi079 ceph-mon[31819]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:11.602 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-06-03T02:07:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T02:07:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:11.657 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-03T02:07:11.657 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:11.657 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:11.657 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-03 02:06:47.831761806 +0000 2024-06-03T02:07:11.657 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-03 01:59:02.738205699 +0000 2024-06-03T02:07:11.657 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-03 01:59:02.738205699 +0000 2024-06-03T02:07:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-03 01:59:02.738205699 +0000 2024-06-03T02:07:11.658 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T02:07:11.719 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-03T02:07:11.720 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-03T02:07:11.720 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000190578 s, 2.7 MB/s 2024-06-03T02:07:11.721 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T02:07:11.777 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-06-03T02:07:11.832 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T02:07:11.832 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:11.832 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-06-03T02:07:11.832 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:11.832 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:11.832 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-03 02:06:47.831761806 +0000 2024-06-03T02:07:11.832 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-03 01:59:03.069198416 +0000 2024-06-03T02:07:11.832 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-03 01:59:03.069198416 +0000 2024-06-03T02:07:11.832 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-03 01:59:03.069198416 +0000 2024-06-03T02:07:11.833 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T02:07:11.896 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-03T02:07:11.896 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-03T02:07:11.896 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000187865 s, 2.7 MB/s 2024-06-03T02:07:11.897 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T02:07:11.953 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-06-03T02:07:12.008 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T02:07:12.008 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T02:07:12.009 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-03T02:07:12.009 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T02:07:12.009 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T02:07:12.009 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-03 02:06:47.832761784 +0000 2024-06-03T02:07:12.009 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-03 01:59:03.389191375 +0000 2024-06-03T02:07:12.009 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-03 01:59:03.389191375 +0000 2024-06-03T02:07:12.009 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-03 01:59:03.389191375 +0000 2024-06-03T02:07:12.009 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T02:07:12.071 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-03T02:07:12.071 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-03T02:07:12.071 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000197279 s, 2.6 MB/s 2024-06-03T02:07:12.072 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T02:07:12.129 INFO:tasks.cephadm:Deploying osd.0 on smithi079 with /dev/vg_nvme/lv_4... 2024-06-03T02:07:12.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T02:07:12.558 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T02:07:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:12.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: Deploying daemon mgr.z on smithi135 2024-06-03T02:07:12.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:12 smithi079 ceph-mon[31819]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: Deploying daemon mgr.z on smithi135 2024-06-03T02:07:12.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:12 smithi135 ceph-mon[36653]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:12.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: Deploying daemon mgr.z on smithi135 2024-06-03T02:07:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:12 smithi123 ceph-mon[36640]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:13.595 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:13 smithi135 systemd[1]: Starting Ceph mgr.z for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:07:13.942 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:13 smithi135 podman[38479]: 2024-06-03 02:07:13.681583246 +0000 UTC m=+0.101432795 container create 5f6977c5c1792e5d4a1a48dcef3ae391ce89ccbd68b8c2262db7c1e48d1945ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-03T02:07:13.942 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:13 smithi135 podman[38479]: 2024-06-03 02:07:13.597655699 +0000 UTC m=+0.017505254 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:07:13.942 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:13 smithi135 podman[38479]: 2024-06-03 02:07:13.875800895 +0000 UTC m=+0.295650451 container init 5f6977c5c1792e5d4a1a48dcef3ae391ce89ccbd68b8c2262db7c1e48d1945ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-03T02:07:13.942 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:13 smithi135 podman[38479]: 2024-06-03 02:07:13.87837888 +0000 UTC m=+0.298228438 container start 5f6977c5c1792e5d4a1a48dcef3ae391ce89ccbd68b8c2262db7c1e48d1945ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-03T02:07:13.942 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:13 smithi135 ceph-mgr[38498]: -- 172.21.15.135:0/374328586 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b114ca5a0 con 0x561b1149ec00 2024-06-03T02:07:13.942 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:13 smithi135 bash[38479]: 5f6977c5c1792e5d4a1a48dcef3ae391ce89ccbd68b8c2262db7c1e48d1945ad 2024-06-03T02:07:14.216 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:13 smithi135 systemd[1]: Started Ceph mgr.z for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:07:14.216 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:14.032+0000 7fd6454ae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T02:07:14.216 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:14.216+0000 7fd6454ae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:07:14.595 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:14.318+0000 7fd6454ae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T02:07:15.014 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:14.624+0000 7fd6454ae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T02:07:15.015 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:14.708+0000 7fd6454ae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T02:07:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:15 smithi123 ceph-mon[36640]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T02:07:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T02:07:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T02:07:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:15 smithi123 ceph-mon[36640]: Deploying daemon mgr.y on smithi123 2024-06-03T02:07:15.252 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T02:07:15.252 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T02:07:15.252 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-06-03T02:07:15.252 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-06-03T02:07:15.252 INFO:teuthology.orchestra.run.smithi079.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0360877 s, 291 MB/s 2024-06-03T02:07:15.252 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 2024-06-03T02:07:15.252 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-06-03T02:07:15.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:15.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:15.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:15 smithi079 ceph-mon[31819]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:15.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:15.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T02:07:15.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T02:07:15.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T02:07:15.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:15.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:15 smithi079 ceph-mon[31819]: Deploying daemon mgr.y on smithi123 2024-06-03T02:07:15.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:15.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:15.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-mon[36653]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:15.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:15.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T02:07:15.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T02:07:15.307 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T02:07:15.307 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:15.307 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-mon[36653]: Deploying daemon mgr.y on smithi123 2024-06-03T02:07:16.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_4 2024-06-03T02:07:16.184 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:15 smithi123 systemd[1]: Starting Ceph mgr.y for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:07:16.258 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:15.945+0000 7fd6454ae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T02:07:16.259 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:16.117+0000 7fd6454ae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T02:07:16.435 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:07:16.461 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 podman[37993]: 2024-06-03 02:07:16.184948942 +0000 UTC m=+0.099921830 container create a4a1e043d3c42b73136cafb43f39b692c602fdbb0f185a22092c6806a29b3efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-03T02:07:16.461 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 podman[37993]: 2024-06-03 02:07:16.101962702 +0000 UTC m=+0.016935605 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:07:16.461 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 podman[37993]: 2024-06-03 02:07:16.387331306 +0000 UTC m=+0.302304204 container init a4a1e043d3c42b73136cafb43f39b692c602fdbb0f185a22092c6806a29b3efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-03T02:07:16.461 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 podman[37993]: 2024-06-03 02:07:16.390483284 +0000 UTC m=+0.305456180 container start a4a1e043d3c42b73136cafb43f39b692c602fdbb0f185a22092c6806a29b3efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:07:16.461 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 ceph-mgr[38013]: -- 172.21.15.123:0/3258644710 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8fd8845a0 con 0x55a8fd859000 2024-06-03T02:07:16.461 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 bash[37993]: a4a1e043d3c42b73136cafb43f39b692c602fdbb0f185a22092c6806a29b3efd 2024-06-03T02:07:16.581 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:16.294+0000 7fd6454ae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T02:07:16.581 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:16.581+0000 7fd6454ae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T02:07:16.723 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 systemd[1]: Started Ceph mgr.y for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:07:16.723 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:16.548+0000 7ffbf473b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T02:07:16.845 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:16.673+0000 7fd6454ae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T02:07:17.090 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:16.744+0000 7ffbf473b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:07:17.090 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:16 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:16.848+0000 7ffbf473b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T02:07:17.345 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:16.882+0000 7fd6454ae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T02:07:17.345 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:17 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:17.083+0000 7fd6454ae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:07:17.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:17 smithi135 ceph-mon[36653]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:17.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:17.473 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:17 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:17.177+0000 7ffbf473b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T02:07:17.473 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:17 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:17.266+0000 7ffbf473b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T02:07:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:17 smithi123 ceph-mon[36640]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:17.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:17 smithi079 ceph-mon[31819]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:17.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:17.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:18.095 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:17 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:17.720+0000 7fd6454ae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T02:07:18.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:18 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:18.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.327+0000 7f2a1aa00700 1 -- 172.21.15.79:0/3534136179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141025b0 msgr2=0x7f2a14102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:18.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.327+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/3534136179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141025b0 0x7f2a14102a10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0098f0 tx=0x7f29fc009c00 comp rx=0 tx=0).stop 2024-06-03T02:07:18.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.327+0000 7f2a1aa00700 1 -- 172.21.15.79:0/3534136179 shutdown_connections 2024-06-03T02:07:18.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.327+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/3534136179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a14102f50 0x7f2a14109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:18.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.327+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/3534136179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a141013b0 0x7f2a141017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:18.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.327+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/3534136179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141025b0 0x7f2a14102a10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:18.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.327+0000 7f2a1aa00700 1 -- 172.21.15.79:0/3534136179 >> 172.21.15.79:0/3534136179 conn(0x7f2a140fc950 msgr2=0x7f2a140fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:07:18.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.327+0000 7f2a1aa00700 1 -- 172.21.15.79:0/3534136179 shutdown_connections 2024-06-03T02:07:18.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.327+0000 7f2a1aa00700 1 -- 172.21.15.79:0/3534136179 wait complete. 2024-06-03T02:07:18.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.328+0000 7f2a1aa00700 1 Processor -- start 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.328+0000 7f2a1aa00700 1 -- start start 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141013b0 0x7f2a14198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a141025b0 0x7f2a14198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a14102f50 0x7f2a1419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a1aa00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1410cf70 con 0x7f2a141025b0 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a1aa00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1410cdf0 con 0x7f2a14102f50 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a1aa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1410d0f0 con 0x7f2a141013b0 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a13e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141013b0 0x7f2a14198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a13e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141013b0 0x7f2a14198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42020/0 (socket says 172.21.15.79:42020) 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a13e00700 1 -- 172.21.15.79:0/2034961213 learned_addr learned my addr 172.21.15.79:0/2034961213 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a0b400700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a141025b0 0x7f2a14198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a18a00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a14102f50 0x7f2a1419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:18.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a13e00700 1 -- 172.21.15.79:0/2034961213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a14102f50 msgr2=0x7f2a1419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:18.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a13e00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a14102f50 0x7f2a1419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:18.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a13e00700 1 -- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a141025b0 msgr2=0x7f2a14198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:18.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a13e00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a141025b0 0x7f2a14198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:18.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.329+0000 7f2a13e00700 1 -- 172.21.15.79:0/2034961213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a141a0340 con 0x7f2a141013b0 2024-06-03T02:07:18.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.330+0000 7f2a13e00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141013b0 0x7f2a14198730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e9a0 tx=0x7f2a0400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:07:18.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.330+0000 7f2a11600700 1 -- 172.21.15.79:0/2034961213 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04004220 con 0x7f2a141013b0 2024-06-03T02:07:18.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.330+0000 7f2a11600700 1 -- 172.21.15.79:0/2034961213 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a040043c0 con 0x7f2a141013b0 2024-06-03T02:07:18.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.330+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a141a0630 con 0x7f2a141013b0 2024-06-03T02:07:18.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.330+0000 7f2a11600700 1 -- 172.21.15.79:0/2034961213 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04003df0 con 0x7f2a141013b0 2024-06-03T02:07:18.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.330+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a141a0bc0 con 0x7f2a141013b0 2024-06-03T02:07:18.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.331+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a14069180 con 0x7f2a141013b0 2024-06-03T02:07:18.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.331+0000 7f2a11600700 1 -- 172.21.15.79:0/2034961213 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2a0400c810 con 0x7f2a141013b0 2024-06-03T02:07:18.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.332+0000 7f2a11600700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f29f8039330 0x7f29f803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:18.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.332+0000 7f2a11600700 1 -- 172.21.15.79:0/2034961213 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2a0404c560 con 0x7f2a141013b0 2024-06-03T02:07:18.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.332+0000 7f2a0b400700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f29f8039330 0x7f29f803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:18.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.333+0000 7f2a0b400700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f29f8039330 0x7f29f803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009fd0 tx=0x7f29fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:07:18.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.341+0000 7f2a11600700 1 -- 172.21.15.79:0/2034961213 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a0401fb80 con 0x7f2a141013b0 2024-06-03T02:07:18.464 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:18 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:18.464 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:18 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:18.161+0000 7fd6454ae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T02:07:18.464 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:18 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:18.258+0000 7fd6454ae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T02:07:18.464 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:18 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:18.464+0000 7fd6454ae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T02:07:18.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:18 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:18.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:18.595+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2a1406b0f0 con 0x7f29f8039330 2024-06-03T02:07:18.854 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:18 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:18.519+0000 7ffbf473b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T02:07:18.854 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:18 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:18.688+0000 7ffbf473b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T02:07:18.854 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:18 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:18.854+0000 7ffbf473b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T02:07:19.319 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:18 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:18.987+0000 7fd6454ae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T02:07:19.319 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:19.078+0000 7fd6454ae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T02:07:19.319 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T02:07:19.320 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:19.421 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:19.133+0000 7ffbf473b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T02:07:19.421 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:19.223+0000 7ffbf473b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T02:07:19.421 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:19.420+0000 7ffbf473b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T02:07:19.421 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:19.421 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.421 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.421 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.421 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.421 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:19.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:19.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:07:19.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:07:19.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:19.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T02:07:19.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T02:07:19.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:19.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:19.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:19.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:19.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:19.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:07:19.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:07:19.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:19.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T02:07:19.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T02:07:19.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:19.595 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:19.319+0000 7fd6454ae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T02:07:19.595 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:19.452+0000 7fd6454ae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T02:07:19.595 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:19.545+0000 7fd6454ae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T02:07:19.722 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:19.608+0000 7ffbf473b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T02:07:19.961 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:19.659+0000 7fd6454ae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T02:07:19.961 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:19.961+0000 7fd6454ae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T02:07:20.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:20 smithi079 ceph-mon[31819]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:07:20.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:20 smithi079 ceph-mon[31819]: Reconfiguring mgr.x (unknown last config time)... 2024-06-03T02:07:20.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:20 smithi079 ceph-mon[31819]: Reconfiguring daemon mgr.x on smithi079 2024-06-03T02:07:20.344 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:20 smithi135 ceph-mon[36653]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:07:20.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:20 smithi135 ceph-mon[36653]: Reconfiguring mgr.x (unknown last config time)... 2024-06-03T02:07:20.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:20 smithi135 ceph-mon[36653]: Reconfiguring daemon mgr.x on smithi079 2024-06-03T02:07:20.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:20 smithi123 ceph-mon[36640]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:07:20.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:20 smithi123 ceph-mon[36640]: Reconfiguring mgr.x (unknown last config time)... 2024-06-03T02:07:20.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:20 smithi123 ceph-mon[36640]: Reconfiguring daemon mgr.x on smithi079 2024-06-03T02:07:20.473 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:20.195+0000 7ffbf473b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T02:07:20.845 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:20 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:20.558+0000 7fd6454ae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T02:07:20.845 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:07:20 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z[38494]: 2024-06-03T02:07:20.773+0000 7fd6454ae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T02:07:20.891 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:20.603+0000 7ffbf473b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T02:07:20.891 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:20.697+0000 7ffbf473b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T02:07:20.891 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:20.891+0000 7ffbf473b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T02:07:21.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:21.165+0000 7f2a11600700 1 -- 172.21.15.79:0/2034961213 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f2a04018950 con 0x7f2a141013b0 2024-06-03T02:07:21.473 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:21.402+0000 7ffbf473b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T02:07:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-mon[36640]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-mon[36640]: Standby manager daemon z started 2024-06-03T02:07:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-mon[36640]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T02:07:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-mon[36640]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T02:07:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-mon[36640]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T02:07:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-mon[36640]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T02:07:21.484 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:21 smithi079 ceph-mon[31819]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:21.484 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:21 smithi079 ceph-mon[31819]: Standby manager daemon z started 2024-06-03T02:07:21.484 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:21 smithi079 ceph-mon[31819]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T02:07:21.485 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:21 smithi079 ceph-mon[31819]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T02:07:21.485 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:21 smithi079 ceph-mon[31819]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T02:07:21.485 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:21 smithi079 ceph-mon[31819]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T02:07:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:21 smithi135 ceph-mon[36653]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:21 smithi135 ceph-mon[36653]: Standby manager daemon z started 2024-06-03T02:07:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:21 smithi135 ceph-mon[36653]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T02:07:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:21 smithi135 ceph-mon[36653]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T02:07:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:21 smithi135 ceph-mon[36653]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T02:07:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:21 smithi135 ceph-mon[36653]: from='mgr.? 172.21.15.135:0/2738660098' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T02:07:21.865 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:21.492+0000 7ffbf473b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T02:07:21.865 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:21.732+0000 7ffbf473b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T02:07:21.865 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:21.865+0000 7ffbf473b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T02:07:22.164 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:21.960+0000 7ffbf473b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T02:07:22.164 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:22.075+0000 7ffbf473b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T02:07:22.317 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:22 smithi079 ceph-mon[31819]: mgrmap e14: x(active, since 89s), standbys: z 2024-06-03T02:07:22.317 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:22 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T02:07:22.317 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:22 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:22.317 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:22 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:22.317 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:22 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:22.317 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:22 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:22.317 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:22 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:22.317 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:22 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:22.318 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:22 smithi079 ceph-mon[31819]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:22.472 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:22.396+0000 7ffbf473b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T02:07:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-mon[36640]: mgrmap e14: x(active, since 89s), standbys: z 2024-06-03T02:07:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T02:07:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-mon[36640]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:22 smithi135 ceph-mon[36653]: mgrmap e14: x(active, since 89s), standbys: z 2024-06-03T02:07:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:22 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T02:07:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:22 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:22 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:22 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:22 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:22 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:22 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:22 smithi135 ceph-mon[36653]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:23.473 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:22 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:22.997+0000 7ffbf473b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T02:07:23.473 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y[38008]: 2024-06-03T02:07:23.214+0000 7ffbf473b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T02:07:23.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:23.821+0000 7f2a11600700 1 -- 172.21.15.79:0/2034961213 <== mon.1 v2:172.21.15.135:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f2a0400f3c0 con 0x7f2a141013b0 2024-06-03T02:07:24.066 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: Standby manager daemon y started 2024-06-03T02:07:24.066 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T02:07:24.066 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T02:07:24.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T02:07:24.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T02:07:24.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2052751236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ff0fc4b-c2d4-4408-b6cf-827bba928e4b"}]: dispatch 2024-06-03T02:07:24.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ff0fc4b-c2d4-4408-b6cf-827bba928e4b"}]: dispatch 2024-06-03T02:07:24.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ff0fc4b-c2d4-4408-b6cf-827bba928e4b"}]': finished 2024-06-03T02:07:24.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: osdmap e5: 1 total, 0 up, 1 in 2024-06-03T02:07:24.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: Standby manager daemon y started 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2052751236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ff0fc4b-c2d4-4408-b6cf-827bba928e4b"}]: dispatch 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ff0fc4b-c2d4-4408-b6cf-827bba928e4b"}]: dispatch 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ff0fc4b-c2d4-4408-b6cf-827bba928e4b"}]': finished 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: osdmap e5: 1 total, 0 up, 1 in 2024-06-03T02:07:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: Standby manager daemon y started 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: from='mgr.? 172.21.15.123:0/3237288599' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2052751236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ff0fc4b-c2d4-4408-b6cf-827bba928e4b"}]: dispatch 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ff0fc4b-c2d4-4408-b6cf-827bba928e4b"}]: dispatch 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ff0fc4b-c2d4-4408-b6cf-827bba928e4b"}]': finished 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: osdmap e5: 1 total, 0 up, 1 in 2024-06-03T02:07:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:25.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:24 smithi079 ceph-mon[31819]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-06-03T02:07:25.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T02:07:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:24 smithi079 ceph-mon[31819]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:24 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4217318371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:07:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:24 smithi135 ceph-mon[36653]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-06-03T02:07:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:24 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T02:07:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:24 smithi135 ceph-mon[36653]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:24 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4217318371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:07:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:24 smithi123 ceph-mon[36640]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-06-03T02:07:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:24 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T02:07:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:24 smithi123 ceph-mon[36640]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:24 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4217318371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:07:27.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:27 smithi135 ceph-mon[36653]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:27.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:27 smithi123 ceph-mon[36640]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:27.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:27 smithi079 ceph-mon[31819]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:29.416 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:29 smithi135 ceph-mon[36653]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:29 smithi123 ceph-mon[36640]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:29.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:29 smithi079 ceph-mon[31819]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:31.238 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:31 smithi079 ceph-mon[31819]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:31.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:31 smithi123 ceph-mon[36640]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:31.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:31 smithi135 ceph-mon[36653]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:33 smithi123 ceph-mon[36640]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T02:07:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:33 smithi079 ceph-mon[31819]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T02:07:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:33 smithi135 ceph-mon[36653]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T02:07:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:34 smithi079 ceph-mon[31819]: Deploying daemon osd.0 on smithi079 2024-06-03T02:07:34.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:34 smithi123 ceph-mon[36640]: Deploying daemon osd.0 on smithi079 2024-06-03T02:07:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:34 smithi135 ceph-mon[36653]: Deploying daemon osd.0 on smithi079 2024-06-03T02:07:35.409 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:35 smithi079 ceph-mon[31819]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:35.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:35 smithi123 ceph-mon[36640]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:35 smithi135 ceph-mon[36653]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:37.220 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:37 smithi079 ceph-mon[31819]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:37 smithi123 ceph-mon[36640]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:37 smithi135 ceph-mon[36653]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:39.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:39 smithi079 ceph-mon[31819]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:39 smithi123 ceph-mon[36640]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:39 smithi135 ceph-mon[36653]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:41.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:41 smithi079 ceph-mon[31819]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:41.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:41 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:41.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:41 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:41.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:41 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:41.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:41 smithi123 ceph-mon[36640]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:41.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:41 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:41.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:41 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:41.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:41 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:41 smithi135 ceph-mon[36653]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:41 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:41 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:41 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:43.451 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:43 smithi079 ceph-mon[31819]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:43.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:43 smithi123 ceph-mon[36640]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:43 smithi135 ceph-mon[36653]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:44.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:44.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:44.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:44.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:44.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:44.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:44 smithi079 ceph-mon[31819]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:44.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:44 smithi079 ceph-mon[31819]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T02:07:44.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:44.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:44.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:44.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:44.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:44.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:44 smithi123 ceph-mon[36640]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:44.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:44 smithi123 ceph-mon[36640]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T02:07:45.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:45.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:45.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:45.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:45.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:45.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:44 smithi135 ceph-mon[36653]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:45.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:44 smithi135 ceph-mon[36653]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T02:07:45.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.790+0000 7f2a11600700 1 -- 172.21.15.79:0/2034961213 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2a1406b0f0 con 0x7f29f8039330 2024-06-03T02:07:45.791 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 0 on host 'smithi079' 2024-06-03T02:07:45.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f29f8039330 msgr2=0x7f29f803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:45.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f29f8039330 0x7f29f803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009fd0 tx=0x7f29fc01a040 comp rx=0 tx=0).stop 2024-06-03T02:07:45.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141013b0 msgr2=0x7f2a14198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:45.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141013b0 0x7f2a14198730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e9a0 tx=0x7f2a0400ecb0 comp rx=0 tx=0).stop 2024-06-03T02:07:45.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 shutdown_connections 2024-06-03T02:07:45.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a14102f50 0x7f2a1419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:45.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a141013b0 0x7f2a14198730 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:45.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f29f8039330 0x7f29f803b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:45.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 --2- 172.21.15.79:0/2034961213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a141025b0 0x7f2a14198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:45.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 >> 172.21.15.79:0/2034961213 conn(0x7f2a140fc950 msgr2=0x7f2a1406a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:07:45.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 shutdown_connections 2024-06-03T02:07:45.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:07:45.794+0000 7f2a1aa00700 1 -- 172.21.15.79:0/2034961213 wait complete. 2024-06-03T02:07:45.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:45 smithi123 ceph-mon[36640]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T02:07:45.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:45 smithi123 ceph-mon[36640]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T02:07:45.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:45 smithi123 ceph-mon[36640]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:07:45.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:45 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:46.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:45 smithi079 ceph-mon[31819]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T02:07:46.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:45 smithi079 ceph-mon[31819]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T02:07:46.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:45 smithi079 ceph-mon[31819]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:07:46.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:45 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:46.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:45 smithi135 ceph-mon[36653]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T02:07:46.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:45 smithi135 ceph-mon[36653]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T02:07:46.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:45 smithi135 ceph-mon[36653]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:07:46.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:45 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:46.592 DEBUG:teuthology.orchestra.run.smithi079:osd.0> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.0.service 2024-06-03T02:07:46.598 INFO:tasks.cephadm:Deploying osd.1 on smithi123 with /dev/vg_nvme/lv_4... 2024-06-03T02:07:46.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T02:07:46.829 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:07:46.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:46 smithi079 ceph-mon[31819]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:07:46.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:46 smithi079 ceph-mon[31819]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T02:07:46.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:46 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:46.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:46 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:46.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:46 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:46.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:46 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:46.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:46 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:46.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:46 smithi079 ceph-mon[31819]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:46.950 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:46 smithi123 ceph-mon[36640]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:07:46.950 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:46 smithi123 ceph-mon[36640]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T02:07:46.950 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:46 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:46.951 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:46 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:46.951 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:46 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:46.951 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:46 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:46.951 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:46 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:46.951 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:46 smithi123 ceph-mon[36640]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:47.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:46 smithi135 ceph-mon[36653]: from='osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:07:47.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:46 smithi135 ceph-mon[36653]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T02:07:47.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:46 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:47.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:46 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:47.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:46 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:47.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:46 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:47.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:46 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:07:47.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:46 smithi135 ceph-mon[36653]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T02:07:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:47 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:07:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:47 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:07:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:47 smithi123 ceph-mon[36640]: osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] boot 2024-06-03T02:07:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:47 smithi123 ceph-mon[36640]: osdmap e8: 1 total, 1 up, 1 in 2024-06-03T02:07:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:48.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:47 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:07:48.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:47 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:07:48.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:48.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:47 smithi079 ceph-mon[31819]: osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] boot 2024-06-03T02:07:48.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:47 smithi079 ceph-mon[31819]: osdmap e8: 1 total, 1 up, 1 in 2024-06-03T02:07:48.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:48.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:47 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:07:48.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:47 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:07:48.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:48.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:47 smithi135 ceph-mon[36653]: osd.0 [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] boot 2024-06-03T02:07:48.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:47 smithi135 ceph-mon[36653]: osdmap e8: 1 total, 1 up, 1 in 2024-06-03T02:07:48.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T02:07:48.634 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T02:07:48.634 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T02:07:48.634 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-06-03T02:07:48.634 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-06-03T02:07:48.634 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114464 s, 916 MB/s 2024-06-03T02:07:48.635 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-06-03T02:07:48.972 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:48 smithi123 ceph-mon[36640]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T02:07:48.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:48 smithi123 ceph-mon[36640]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:49.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:48 smithi079 ceph-mon[31819]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T02:07:49.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:48 smithi079 ceph-mon[31819]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:49.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:48 smithi135 ceph-mon[36653]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T02:07:49.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:48 smithi135 ceph-mon[36653]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:49.219 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-06-03T02:07:49.451 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:07:50.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 -- 172.21.15.123:0/815923972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c100b80 msgr2=0x7f8a7c100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:50.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 --2- 172.21.15.123:0/815923972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c100b80 0x7f8a7c100fa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800aad0 tx=0x7f8a7800ade0 comp rx=0 tx=0).stop 2024-06-03T02:07:50.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 -- 172.21.15.123:0/815923972 shutdown_connections 2024-06-03T02:07:50.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 --2- 172.21.15.123:0/815923972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a7c102720 0x7f8a7c108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:50.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 --2- 172.21.15.123:0/815923972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c100b80 0x7f8a7c100fa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:50.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 --2- 172.21.15.123:0/815923972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a7c101d80 0x7f8a7c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:50.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 -- 172.21.15.123:0/815923972 >> 172.21.15.123:0/815923972 conn(0x7f8a7c0fc100 msgr2=0x7f8a7c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 -- 172.21.15.123:0/815923972 shutdown_connections 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 -- 172.21.15.123:0/815923972 wait complete. 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 Processor -- start 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 -- start start 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.845+0000 7f8a8450f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a7c100b80 0x7f8a7c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a8450f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c101d80 0x7f8a7c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a8450f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a7c102720 0x7f8a7c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a8450f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c10bf90 con 0x7f8a7c100b80 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a8450f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c10be10 con 0x7f8a7c101d80 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a8450f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c10c110 con 0x7f8a7c102720 2024-06-03T02:07:50.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a81aaa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c101d80 0x7f8a7c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a81aaa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c101d80 0x7f8a7c198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48622/0 (socket says 172.21.15.123:48622) 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a81aaa700 1 -- 172.21.15.123:0/3088194713 learned_addr learned my addr 172.21.15.123:0/3088194713 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a82aac700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a7c102720 0x7f8a7c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a81aaa700 1 -- 172.21.15.123:0/3088194713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a7c102720 msgr2=0x7f8a7c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a81aaa700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a7c102720 0x7f8a7c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a81aaa700 1 -- 172.21.15.123:0/3088194713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a7c100b80 msgr2=0x7f8a7c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a81aaa700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a7c100b80 0x7f8a7c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a81aaa700 1 -- 172.21.15.123:0/3088194713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7c1a0440 con 0x7f8a7c101d80 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a82aac700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a7c102720 0x7f8a7c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a81aaa700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c101d80 0x7f8a7c198d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00b530 tx=0x7f8a6c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:07:50.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a737fe700 1 -- 172.21.15.123:0/3088194713 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c015020 con 0x7f8a7c101d80 2024-06-03T02:07:50.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a737fe700 1 -- 172.21.15.123:0/3088194713 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a6c011d10 con 0x7f8a7c101d80 2024-06-03T02:07:50.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a737fe700 1 -- 172.21.15.123:0/3088194713 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00f2e0 con 0x7f8a7c101d80 2024-06-03T02:07:50.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7c1a0730 con 0x7f8a7c101d80 2024-06-03T02:07:50.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.846+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7c1a0d40 con 0x7f8a7c101d80 2024-06-03T02:07:50.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.847+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a7c069180 con 0x7f8a7c101d80 2024-06-03T02:07:50.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.848+0000 7f8a737fe700 1 -- 172.21.15.123:0/3088194713 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f8a6c018020 con 0x7f8a7c101d80 2024-06-03T02:07:50.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.849+0000 7f8a737fe700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a680a2c10 0x7f8a680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:07:50.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.849+0000 7f8a737fe700 1 -- 172.21.15.123:0/3088194713 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f8a6c0cd1b0 con 0x7f8a7c101d80 2024-06-03T02:07:50.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.850+0000 7f8a822ab700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a680a2c10 0x7f8a680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:07:50.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.850+0000 7f8a822ab700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a680a2c10 0x7f8a680a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a78003d20 tx=0x7f8a78000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:07:50.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:50.853+0000 7f8a737fe700 1 -- 172.21.15.123:0/3088194713 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8a6c01f020 con 0x7f8a7c101d80 2024-06-03T02:07:51.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:07:51.057+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8a7c108950 con 0x7f8a680a2c10 2024-06-03T02:07:51.323 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:51 smithi079 ceph-mon[31819]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:51.323 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:51 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:07:51.323 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:51 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:07:51.323 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:51 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:51.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:51 smithi123 ceph-mon[36640]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:51.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:51 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:07:51.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:51 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:07:51.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:51 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:51 smithi135 ceph-mon[36653]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:51 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:07:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:51 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:07:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:51 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:52.347 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:52 smithi123 ceph-mon[36640]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:07:52.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:07:52.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:52.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:52.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:52 smithi079 ceph-mon[31819]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:07:52.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:07:52.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:52.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:52.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:52 smithi135 ceph-mon[36653]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:07:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:07:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:07:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:07:52.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:07:53.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:53 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi079 2024-06-03T02:07:53.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:53 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi079 to 17089M 2024-06-03T02:07:53.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:53 smithi123 ceph-mon[36640]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:53 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi079 2024-06-03T02:07:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:53 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi079 to 17089M 2024-06-03T02:07:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:53 smithi079 ceph-mon[31819]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:53 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi079 2024-06-03T02:07:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:53 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi079 to 17089M 2024-06-03T02:07:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:53 smithi135 ceph-mon[36653]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:54 smithi123 ceph-mon[36640]: from='client.? 172.21.15.123:0/1823608781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc15c092-e416-48ac-8568-c7f12ae5a56d"}]: dispatch 2024-06-03T02:07:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:54 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc15c092-e416-48ac-8568-c7f12ae5a56d"}]: dispatch 2024-06-03T02:07:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:54 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc15c092-e416-48ac-8568-c7f12ae5a56d"}]': finished 2024-06-03T02:07:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:54 smithi123 ceph-mon[36640]: osdmap e10: 2 total, 1 up, 2 in 2024-06-03T02:07:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:07:54.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:54 smithi079 ceph-mon[31819]: from='client.? 172.21.15.123:0/1823608781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc15c092-e416-48ac-8568-c7f12ae5a56d"}]: dispatch 2024-06-03T02:07:54.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:54 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc15c092-e416-48ac-8568-c7f12ae5a56d"}]: dispatch 2024-06-03T02:07:54.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:54 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc15c092-e416-48ac-8568-c7f12ae5a56d"}]': finished 2024-06-03T02:07:54.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:54 smithi079 ceph-mon[31819]: osdmap e10: 2 total, 1 up, 2 in 2024-06-03T02:07:54.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:07:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:54 smithi135 ceph-mon[36653]: from='client.? 172.21.15.123:0/1823608781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc15c092-e416-48ac-8568-c7f12ae5a56d"}]: dispatch 2024-06-03T02:07:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:54 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc15c092-e416-48ac-8568-c7f12ae5a56d"}]: dispatch 2024-06-03T02:07:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:54 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc15c092-e416-48ac-8568-c7f12ae5a56d"}]': finished 2024-06-03T02:07:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:54 smithi135 ceph-mon[36653]: osdmap e10: 2 total, 1 up, 2 in 2024-06-03T02:07:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:07:55.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:55 smithi123 ceph-mon[36640]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:55.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:55 smithi123 ceph-mon[36640]: from='client.? 172.21.15.123:0/906670056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:07:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:55 smithi079 ceph-mon[31819]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:55 smithi079 ceph-mon[31819]: from='client.? 172.21.15.123:0/906670056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:07:55.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:55 smithi135 ceph-mon[36653]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:55 smithi135 ceph-mon[36653]: from='client.? 172.21.15.123:0/906670056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:07:57.373 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:57 smithi123 ceph-mon[36640]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:57.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:57 smithi079 ceph-mon[31819]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:57.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:57 smithi135 ceph-mon[36653]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:59.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:07:59 smithi135 ceph-mon[36653]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:59.461 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:07:59 smithi123 ceph-mon[36640]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:07:59.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:07:59 smithi079 ceph-mon[31819]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:01.392 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:01 smithi123 ceph-mon[36640]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:01.392 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T02:08:01.392 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:01.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:01 smithi079 ceph-mon[31819]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:01.593 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T02:08:01.593 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:01 smithi135 ceph-mon[36653]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T02:08:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:02.361 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:02 smithi123 ceph-mon[36640]: Deploying daemon osd.1 on smithi123 2024-06-03T02:08:02.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:02 smithi079 ceph-mon[31819]: Deploying daemon osd.1 on smithi123 2024-06-03T02:08:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:02 smithi135 ceph-mon[36653]: Deploying daemon osd.1 on smithi123 2024-06-03T02:08:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:03 smithi123 ceph-mon[36640]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:03.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:03 smithi079 ceph-mon[31819]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:03 smithi135 ceph-mon[36653]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:05 smithi123 ceph-mon[36640]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:05.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:05 smithi079 ceph-mon[31819]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:05 smithi135 ceph-mon[36653]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:06 smithi123 ceph-mon[36640]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:07.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:07.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:07.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:07.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:06 smithi079 ceph-mon[31819]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:07.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:06 smithi135 ceph-mon[36653]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:09.214 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:09 smithi123 ceph-mon[36640]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:09.214 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:09.214 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:09.214 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:09.214 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:09.214 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.291+0000 7f8a737fe700 1 -- 172.21.15.123:0/3088194713 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8a7c108950 con 0x7f8a680a2c10 2024-06-03T02:08:09.292 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 1 on host 'smithi123' 2024-06-03T02:08:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a680a2c10 msgr2=0x7f8a680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a680a2c10 0x7f8a680a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a78003d20 tx=0x7f8a78000bc0 comp rx=0 tx=0).stop 2024-06-03T02:08:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c101d80 msgr2=0x7f8a7c198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c101d80 0x7f8a7c198d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00b530 tx=0x7f8a6c00b8f0 comp rx=0 tx=0).stop 2024-06-03T02:08:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 shutdown_connections 2024-06-03T02:08:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a7c102720 0x7f8a7c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a680a2c10 0x7f8a680a50d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:09.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a7c100b80 0x7f8a7c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 --2- 172.21.15.123:0/3088194713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a7c101d80 0x7f8a7c198d20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 >> 172.21.15.123:0/3088194713 conn(0x7f8a7c0fc100 msgr2=0x7f8a7c107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:08:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 shutdown_connections 2024-06-03T02:08:09.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:08:09.294+0000 7f8a8450f700 1 -- 172.21.15.123:0/3088194713 wait complete. 2024-06-03T02:08:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:09 smithi079 ceph-mon[31819]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:09 smithi135 ceph-mon[36653]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:09.885 DEBUG:teuthology.orchestra.run.smithi123:osd.1> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.1.service 2024-06-03T02:08:09.887 INFO:tasks.cephadm:Deploying osd.2 on smithi135 with /dev/vg_nvme/lv_4... 2024-06-03T02:08:09.887 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T02:08:10.127 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:08:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:10 smithi123 ceph-mon[36640]: from='osd.1 [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T02:08:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:10 smithi123 ceph-mon[36640]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T02:08:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:10 smithi123 ceph-mon[36640]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:10.509 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:10.509 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:10.509 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:10.509 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:10 smithi135 ceph-mon[36653]: from='osd.1 [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T02:08:10.509 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:10 smithi135 ceph-mon[36653]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T02:08:10.509 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:10 smithi135 ceph-mon[36653]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:10 smithi079 ceph-mon[31819]: from='osd.1 [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T02:08:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:10 smithi079 ceph-mon[31819]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T02:08:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:10 smithi079 ceph-mon[31819]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:11.341 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:11 smithi123 ceph-mon[36640]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T02:08:11.341 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:11 smithi123 ceph-mon[36640]: from='osd.1 [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:08:11.341 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:11 smithi123 ceph-mon[36640]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T02:08:11.341 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:11 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:11.341 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:11 smithi123 ceph-mon[36640]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:08:11.342 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1[41221]: 2024-06-03T02:08:11.280+0000 7fb3f3011700 -1 osd.1 0 waiting for initial osdmap 2024-06-03T02:08:11.342 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1[41221]: 2024-06-03T02:08:11.284+0000 7fb3eee04700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:08:11.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:11 smithi135 ceph-mon[36653]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T02:08:11.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:11 smithi135 ceph-mon[36653]: from='osd.1 [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:08:11.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:11 smithi135 ceph-mon[36653]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T02:08:11.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:11 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:11.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:11 smithi135 ceph-mon[36653]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:08:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:11 smithi079 ceph-mon[31819]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T02:08:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:11 smithi079 ceph-mon[31819]: from='osd.1 [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:08:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:11 smithi079 ceph-mon[31819]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T02:08:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:11 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:11 smithi079 ceph-mon[31819]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:08:12.020 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T02:08:12.020 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T02:08:12.020 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-03T02:08:12.020 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-03T02:08:12.021 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112107 s, 935 MB/s 2024-06-03T02:08:12.021 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-03T02:08:12.414 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:12 smithi123 ceph-mon[36640]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:08:12.414 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:12 smithi123 ceph-mon[36640]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T02:08:12.415 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:12.415 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:12.415 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:12 smithi123 ceph-mon[36640]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:12.415 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:12.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:12 smithi079 ceph-mon[31819]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:08:12.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:12 smithi079 ceph-mon[31819]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T02:08:12.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:12.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:12.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:12 smithi079 ceph-mon[31819]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:12.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:12.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:12 smithi135 ceph-mon[36653]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:08:12.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:12 smithi135 ceph-mon[36653]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T02:08:12.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:12.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:12.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:12 smithi135 ceph-mon[36653]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T02:08:12.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:12.620 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_4 2024-06-03T02:08:12.860 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:08:13.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:08:13.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:08:13.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: osd.1 [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] boot 2024-06-03T02:08:13.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: osdmap e13: 2 total, 2 up, 2 in 2024-06-03T02:08:13.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:13.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi123 2024-06-03T02:08:13.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:08:13.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi123 to 17088M 2024-06-03T02:08:13.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:13.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:13.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:08:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:08:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: osd.1 [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] boot 2024-06-03T02:08:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: osdmap e13: 2 total, 2 up, 2 in 2024-06-03T02:08:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi123 2024-06-03T02:08:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:08:13.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi123 to 17088M 2024-06-03T02:08:13.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:13.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:13.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: osd.1 [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] boot 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: osdmap e13: 2 total, 2 up, 2 in 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi123 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi123 to 17088M 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:13.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:13.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:14.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.314+0000 7f867c5b8700 1 -- 172.21.15.135:0/747071888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 msgr2=0x7f867410a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.314+0000 7f867c5b8700 1 --2- 172.21.15.135:0/747071888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 0x7f867410a610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f866c0098f0 tx=0x7f866c009c00 comp rx=0 tx=0).stop 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.314+0000 7f867c5b8700 1 -- 172.21.15.135:0/747071888 shutdown_connections 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.314+0000 7f867c5b8700 1 --2- 172.21.15.135:0/747071888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 0x7f867410a610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.314+0000 7f867c5b8700 1 --2- 172.21.15.135:0/747071888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8674102f40 0x7f8674105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.314+0000 7f867c5b8700 1 --2- 172.21.15.135:0/747071888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8674105870 0x7f8674107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.314+0000 7f867c5b8700 1 -- 172.21.15.135:0/747071888 >> 172.21.15.135:0/747071888 conn(0x7f86740fc950 msgr2=0x7f86740fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.314+0000 7f867c5b8700 1 -- 172.21.15.135:0/747071888 shutdown_connections 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.314+0000 7f867c5b8700 1 -- 172.21.15.135:0/747071888 wait complete. 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867c5b8700 1 Processor -- start 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867c5b8700 1 -- start start 2024-06-03T02:08:14.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867c5b8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8674102f40 0x7f867419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867c5b8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8674105870 0x7f867419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867c5b8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 0x7f86741a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867c5b8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f867410e0e0 con 0x7f8674105870 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867c5b8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f867410df60 con 0x7f8674102f40 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867c5b8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f867410e260 con 0x7f86741081a0 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867ab55700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 0x7f86741a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867ab55700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 0x7f86741a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58410/0 (socket says 172.21.15.135:58410) 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867ab55700 1 -- 172.21.15.135:0/1830770659 learned_addr learned my addr 172.21.15.135:0/1830770659 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867a354700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8674102f40 0x7f867419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867ab55700 1 -- 172.21.15.135:0/1830770659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8674102f40 msgr2=0x7f867419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:14.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867ab55700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8674102f40 0x7f867419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:14.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867ab55700 1 -- 172.21.15.135:0/1830770659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8674105870 msgr2=0x7f867419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:14.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.315+0000 7f867ab55700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8674105870 0x7f867419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:14.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.316+0000 7f867ab55700 1 -- 172.21.15.135:0/1830770659 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86741a2580 con 0x7f86741081a0 2024-06-03T02:08:14.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.316+0000 7f867ab55700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 0x7f86741a1ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f866c009230 tx=0x7f866c005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:08:14.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.317+0000 7f866b7fe700 1 -- 172.21.15.135:0/1830770659 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c025070 con 0x7f86741081a0 2024-06-03T02:08:14.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.317+0000 7f866b7fe700 1 -- 172.21.15.135:0/1830770659 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f866c00ba40 con 0x7f86741081a0 2024-06-03T02:08:14.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.317+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86741a2810 con 0x7f86741081a0 2024-06-03T02:08:14.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.317+0000 7f866b7fe700 1 -- 172.21.15.135:0/1830770659 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c003e40 con 0x7f86741081a0 2024-06-03T02:08:14.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.317+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86741a2ce0 con 0x7f86741081a0 2024-06-03T02:08:14.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.318+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867402dca0 con 0x7f86741081a0 2024-06-03T02:08:14.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.318+0000 7f866b7fe700 1 -- 172.21.15.135:0/1830770659 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f866c028070 con 0x7f86741081a0 2024-06-03T02:08:14.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.319+0000 7f866b7fe700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f86600a2b90 0x7f86600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:08:14.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.319+0000 7f866b7fe700 1 -- 172.21.15.135:0/1830770659 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f866c0ce060 con 0x7f86741081a0 2024-06-03T02:08:14.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.319+0000 7f867a354700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f86600a2b90 0x7f86600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:08:14.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.321+0000 7f867a354700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f86600a2b90 0x7f86600a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f867000a400 tx=0x7f867000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:08:14.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.324+0000 7f866b7fe700 1 -- 172.21.15.135:0/1830770659 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f866c013030 con 0x7f86741081a0 2024-06-03T02:08:14.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:14.527+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f86740628d0 con 0x7f86600a2b90 2024-06-03T02:08:14.528 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:14 smithi135 ceph-mon[36653]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T02:08:14.528 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:14 smithi135 ceph-mon[36653]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:14.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:14 smithi079 ceph-mon[31819]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T02:08:14.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:14 smithi079 ceph-mon[31819]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:14.625 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:14 smithi123 ceph-mon[36640]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T02:08:14.625 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:14 smithi123 ceph-mon[36640]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:15.414 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:15 smithi135 ceph-mon[36653]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:08:15.414 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:08:15.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:08:15.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:15.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:15 smithi079 ceph-mon[31819]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:08:15.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:08:15.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:08:15.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:15.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:15 smithi123 ceph-mon[36640]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:08:15.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:08:15.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:08:15.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:16.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:16 smithi079 ceph-mon[31819]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:16 smithi135 ceph-mon[36653]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:16.688 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:16 smithi123 ceph-mon[36640]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:17.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:17 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/1786719782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf1db324-1336-4e1b-950d-3b07e158b5eb"}]: dispatch 2024-06-03T02:08:17.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:17 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf1db324-1336-4e1b-950d-3b07e158b5eb"}]: dispatch 2024-06-03T02:08:17.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:17 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf1db324-1336-4e1b-950d-3b07e158b5eb"}]': finished 2024-06-03T02:08:17.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:17 smithi135 ceph-mon[36653]: osdmap e15: 3 total, 2 up, 3 in 2024-06-03T02:08:17.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:17.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:17 smithi123 ceph-mon[36640]: from='client.? 172.21.15.135:0/1786719782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf1db324-1336-4e1b-950d-3b07e158b5eb"}]: dispatch 2024-06-03T02:08:17.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:17 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf1db324-1336-4e1b-950d-3b07e158b5eb"}]: dispatch 2024-06-03T02:08:17.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:17 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf1db324-1336-4e1b-950d-3b07e158b5eb"}]': finished 2024-06-03T02:08:17.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:17 smithi123 ceph-mon[36640]: osdmap e15: 3 total, 2 up, 3 in 2024-06-03T02:08:17.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:17.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:17 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/1786719782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf1db324-1336-4e1b-950d-3b07e158b5eb"}]: dispatch 2024-06-03T02:08:17.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:17 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf1db324-1336-4e1b-950d-3b07e158b5eb"}]: dispatch 2024-06-03T02:08:17.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:17 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf1db324-1336-4e1b-950d-3b07e158b5eb"}]': finished 2024-06-03T02:08:17.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:17 smithi079 ceph-mon[31819]: osdmap e15: 3 total, 2 up, 3 in 2024-06-03T02:08:17.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:18 smithi123 ceph-mon[36640]: from='client.? 172.21.15.135:0/1862546239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:08:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:18 smithi123 ceph-mon[36640]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:18.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:18 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/1862546239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:08:18.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:18 smithi079 ceph-mon[31819]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:18.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:18 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/1862546239' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:08:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:18 smithi135 ceph-mon[36653]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:21.417 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:21 smithi135 ceph-mon[36653]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:21.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:21 smithi123 ceph-mon[36640]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:21 smithi079 ceph-mon[31819]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:23.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:23 smithi135 ceph-mon[36653]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:23.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:23 smithi123 ceph-mon[36640]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:23 smithi079 ceph-mon[31819]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:25.266 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:25 smithi135 ceph-mon[36653]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:25.266 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T02:08:25.266 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:25.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:25 smithi123 ceph-mon[36640]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:25.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T02:08:25.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:25 smithi079 ceph-mon[31819]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T02:08:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:26.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:26 smithi123 ceph-mon[36640]: Deploying daemon osd.2 on smithi135 2024-06-03T02:08:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:26 smithi135 ceph-mon[36653]: Deploying daemon osd.2 on smithi135 2024-06-03T02:08:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:26 smithi079 ceph-mon[31819]: Deploying daemon osd.2 on smithi135 2024-06-03T02:08:27.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:27 smithi135 ceph-mon[36653]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:27.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:27 smithi123 ceph-mon[36640]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:27.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:27 smithi079 ceph-mon[31819]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:29 smithi135 ceph-mon[36653]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:29 smithi123 ceph-mon[36640]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:29.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:29 smithi079 ceph-mon[31819]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:30.206 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:30.206 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:30.207 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:31.432 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:31 smithi135 ceph-mon[36653]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:31.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:31 smithi123 ceph-mon[36640]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:31.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:31 smithi079 ceph-mon[31819]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:32.965 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 2 on host 'smithi135' 2024-06-03T02:08:32.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.965+0000 7f866b7fe700 1 -- 172.21.15.135:0/1830770659 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f86740628d0 con 0x7f86600a2b90 2024-06-03T02:08:32.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f86600a2b90 msgr2=0x7f86600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:32.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f86600a2b90 0x7f86600a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f867000a400 tx=0x7f867000d040 comp rx=0 tx=0).stop 2024-06-03T02:08:32.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 msgr2=0x7f86741a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:32.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 0x7f86741a1ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f866c009230 tx=0x7f866c005250 comp rx=0 tx=0).stop 2024-06-03T02:08:32.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 shutdown_connections 2024-06-03T02:08:32.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741081a0 0x7f86741a1ee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:32.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8674102f40 0x7f867419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:32.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f86600a2b90 0x7f86600a5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:32.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 --2- 172.21.15.135:0/1830770659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8674105870 0x7f867419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:32.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 >> 172.21.15.135:0/1830770659 conn(0x7f86740fc950 msgr2=0x7f8674102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:08:32.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 shutdown_connections 2024-06-03T02:08:32.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:08:32.967+0000 7f867c5b8700 1 -- 172.21.15.135:0/1830770659 wait complete. 2024-06-03T02:08:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:33 smithi123 ceph-mon[36640]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:33.521 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:33 smithi135 ceph-mon[36653]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:33.521 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.521 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.521 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:33.521 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:33.521 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.521 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.522 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.522 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:33.547 DEBUG:teuthology.orchestra.run.smithi135:osd.2> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.2.service 2024-06-03T02:08:33.551 INFO:tasks.cephadm:Deploying osd.3 on smithi079 with /dev/vg_nvme/lv_3... 2024-06-03T02:08:33.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T02:08:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:33 smithi079 ceph-mon[31819]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:08:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:08:34.406 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:34 smithi079 ceph-mon[31819]: from='osd.2 [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T02:08:34.406 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:34 smithi079 ceph-mon[31819]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T02:08:34.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:34 smithi123 ceph-mon[36640]: from='osd.2 [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T02:08:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:34 smithi123 ceph-mon[36640]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T02:08:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:34 smithi135 ceph-mon[36653]: from='osd.2 [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T02:08:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:34 smithi135 ceph-mon[36653]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T02:08:35.246 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:35 smithi079 ceph-mon[31819]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:35.246 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:35 smithi079 ceph-mon[31819]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T02:08:35.247 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:35 smithi079 ceph-mon[31819]: from='osd.2 [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:08:35.247 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:35 smithi079 ceph-mon[31819]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T02:08:35.247 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:35.247 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:35 smithi079 ceph-mon[31819]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:08:35.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:35 smithi135 ceph-mon[36653]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:35.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:35 smithi135 ceph-mon[36653]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T02:08:35.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:35 smithi135 ceph-mon[36653]: from='osd.2 [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:08:35.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:35 smithi135 ceph-mon[36653]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T02:08:35.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:35.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:35 smithi135 ceph-mon[36653]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:08:35.347 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2[41736]: 2024-06-03T02:08:35.163+0000 7f0d742a1700 -1 osd.2 0 waiting for initial osdmap 2024-06-03T02:08:35.347 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2[41736]: 2024-06-03T02:08:35.170+0000 7f0d70094700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:08:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:35 smithi123 ceph-mon[36640]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:35 smithi123 ceph-mon[36640]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T02:08:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:35 smithi123 ceph-mon[36640]: from='osd.2 [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:08:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:35 smithi123 ceph-mon[36640]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T02:08:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:35 smithi123 ceph-mon[36640]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:08:36.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:36 smithi135 ceph-mon[36653]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:08:36.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:36 smithi135 ceph-mon[36653]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T02:08:36.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:36 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:36.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:36 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:36.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:36 smithi135 ceph-mon[36653]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:36.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:36 smithi123 ceph-mon[36640]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:08:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:36 smithi123 ceph-mon[36640]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T02:08:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:36 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:36 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:36 smithi123 ceph-mon[36640]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:36.584 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:36 smithi079 ceph-mon[31819]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:08:36.584 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:36 smithi079 ceph-mon[31819]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T02:08:36.584 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:36 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:36.585 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:36 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:36.585 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:36 smithi079 ceph-mon[31819]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T02:08:36.828 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T02:08:36.828 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T02:08:36.828 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-06-03T02:08:36.828 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-06-03T02:08:36.828 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0324328 s, 323 MB/s 2024-06-03T02:08:36.828 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: osd.2 [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] boot 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: osdmap e18: 3 total, 3 up, 3 in 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi135 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:08:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi135 to 17088M 2024-06-03T02:08:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: osd.2 [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] boot 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: osdmap e18: 3 total, 3 up, 3 in 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi135 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi135 to 17088M 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:37.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:37.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:08:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:08:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: osd.2 [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] boot 2024-06-03T02:08:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: osdmap e18: 3 total, 3 up, 3 in 2024-06-03T02:08:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T02:08:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi135 2024-06-03T02:08:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:08:37.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi135 to 17088M 2024-06-03T02:08:37.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:37.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:08:37.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:37.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_3 2024-06-03T02:08:38.214 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:08:38.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:38 smithi079 ceph-mon[31819]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T02:08:38.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:38 smithi123 ceph-mon[36640]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T02:08:39.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:38 smithi135 ceph-mon[36653]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T02:08:39.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:39 smithi079 ceph-mon[31819]: pgmap v92: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:08:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T02:08:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:39 smithi079 ceph-mon[31819]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T02:08:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:08:39.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:39 smithi123 ceph-mon[36640]: pgmap v92: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:39.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:39 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:08:39.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:39 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T02:08:39.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:39 smithi123 ceph-mon[36640]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T02:08:39.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:39 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:08:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:39 smithi135 ceph-mon[36653]: pgmap v92: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:39 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:08:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:39 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T02:08:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:39 smithi135 ceph-mon[36653]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T02:08:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:39 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:08:40.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.311+0000 7f2a7da00700 1 -- 172.21.15.79:0/376365468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a78074e70 msgr2=0x7f2a780752f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:40.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.311+0000 7f2a7da00700 1 --2- 172.21.15.79:0/376365468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a78074e70 0x7f2a780752f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009990 tx=0x7f2a64009ca0 comp rx=0 tx=0).stop 2024-06-03T02:08:40.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.311+0000 7f2a7da00700 1 -- 172.21.15.79:0/376365468 shutdown_connections 2024-06-03T02:08:40.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.311+0000 7f2a7da00700 1 --2- 172.21.15.79:0/376365468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a7810a3f0 0x7f2a7810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:40.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.311+0000 7f2a7da00700 1 --2- 172.21.15.79:0/376365468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a78076440 0x7f2a780748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:40.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.311+0000 7f2a7da00700 1 --2- 172.21.15.79:0/376365468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a78074e70 0x7f2a780752f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:40.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.311+0000 7f2a7da00700 1 -- 172.21.15.79:0/376365468 >> 172.21.15.79:0/376365468 conn(0x7f2a780fda90 msgr2=0x7f2a780ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:08:40.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.311+0000 7f2a7da00700 1 -- 172.21.15.79:0/376365468 shutdown_connections 2024-06-03T02:08:40.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.311+0000 7f2a7da00700 1 -- 172.21.15.79:0/376365468 wait complete. 2024-06-03T02:08:40.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.312+0000 7f2a7da00700 1 Processor -- start 2024-06-03T02:08:40.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.312+0000 7f2a7da00700 1 -- start start 2024-06-03T02:08:40.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a7da00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a78074e70 0x7f2a7819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:08:40.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a7da00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a78076440 0x7f2a7819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:08:40.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a7da00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a7810a3f0 0x7f2a781a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:08:40.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a7da00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781102f0 con 0x7f2a7810a3f0 2024-06-03T02:08:40.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a7da00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78110170 con 0x7f2a78074e70 2024-06-03T02:08:40.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a7da00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78110470 con 0x7f2a78076440 2024-06-03T02:08:40.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a76000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a78076440 0x7f2a7819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:08:40.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a76a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a78074e70 0x7f2a7819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:08:40.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a76000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a78076440 0x7f2a7819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:57864/0 (socket says 172.21.15.79:57864) 2024-06-03T02:08:40.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a76000700 1 -- 172.21.15.79:0/4199660147 learned_addr learned my addr 172.21.15.79:0/4199660147 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:08:40.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.313+0000 7f2a77400700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a7810a3f0 0x7f2a781a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a76000700 1 -- 172.21.15.79:0/4199660147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a78074e70 msgr2=0x7f2a7819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a76000700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a78074e70 0x7f2a7819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a76000700 1 -- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a7810a3f0 msgr2=0x7f2a781a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a76000700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a7810a3f0 0x7f2a781a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a76000700 1 -- 172.21.15.79:0/4199660147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a781a2590 con 0x7f2a78076440 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a76a00700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a78074e70 0x7f2a7819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a76000700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a78076440 0x7f2a7819ae70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a640095c0 tx=0x7f2a6400bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a6f400700 1 -- 172.21.15.79:0/4199660147 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64026070 con 0x7f2a78076440 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a6f400700 1 -- 172.21.15.79:0/4199660147 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a64003d70 con 0x7f2a78076440 2024-06-03T02:08:40.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a781a2820 con 0x7f2a78076440 2024-06-03T02:08:40.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a6f400700 1 -- 172.21.15.79:0/4199660147 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64021410 con 0x7f2a78076440 2024-06-03T02:08:40.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.314+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a781a2c70 con 0x7f2a78076440 2024-06-03T02:08:40.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.315+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a78069180 con 0x7f2a78076440 2024-06-03T02:08:40.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.317+0000 7f2a6f400700 1 -- 172.21.15.79:0/4199660147 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f2a64029070 con 0x7f2a78076440 2024-06-03T02:08:40.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.319+0000 7f2a6f400700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a5c0a2b90 0x7f2a5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:08:40.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.319+0000 7f2a6f400700 1 -- 172.21.15.79:0/4199660147 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f2a640cea50 con 0x7f2a78076440 2024-06-03T02:08:40.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.319+0000 7f2a76a00700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a5c0a2b90 0x7f2a5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:08:40.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.319+0000 7f2a76a00700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a5c0a2b90 0x7f2a5c0a5050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009680 tx=0x7f2a68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:08:40.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.323+0000 7f2a6f400700 1 -- 172.21.15.79:0/4199660147 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a6405d850 con 0x7f2a78076440 2024-06-03T02:08:40.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.557+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2a780628d0 con 0x7f2a5c0a2b90 2024-06-03T02:08:40.766 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:08:40 smithi079 sudo[51283]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T02:08:40.766 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:08:40 smithi079 sudo[51283]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:40.766 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:08:40 smithi079 sudo[51283]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:40.766 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:08:40 smithi079 sudo[51283]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:40.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:08:40.776+0000 7f2a6f400700 1 -- 172.21.15.79:0/4199660147 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2a64009d10 con 0x7f2a78076440 2024-06-03T02:08:41.018 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T02:08:41.021 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:40 smithi135 ceph-mon[36653]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T02:08:41.021 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:40 smithi135 ceph-mon[36653]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:41.022 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:08:41.022 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:08:41.022 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:41.035 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T02:08:41.035 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:40 smithi079 ceph-mon[31819]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T02:08:41.035 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:40 smithi079 ceph-mon[31819]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:41.035 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:08:41.035 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:08:41.035 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:41.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T02:08:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:40 smithi123 ceph-mon[36640]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T02:08:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:40 smithi123 ceph-mon[36640]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:08:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:08:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:41.337 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:08:41 smithi079 sudo[51353]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T02:08:41.337 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:08:41 smithi079 sudo[51353]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:41.337 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:08:41 smithi079 sudo[51353]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:41.337 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:08:41 smithi079 sudo[51353]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:41.577 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:41 smithi123 sudo[45206]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T02:08:41.578 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:41 smithi123 sudo[45206]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:41.578 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:41 smithi123 sudo[45206]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:41.578 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:41 smithi123 sudo[45206]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:41.868 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:41 smithi135 ceph-mon[36653]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:08:41.868 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:41 smithi135 ceph-mon[36653]: osdmap e22: 3 total, 3 up, 3 in 2024-06-03T02:08:41.868 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:41 smithi135 ceph-mon[36653]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-03T02:08:41.868 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:41 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:41.868 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:41 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:41.973 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:41 smithi123 sudo[45209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T02:08:41.973 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:41 smithi123 sudo[45209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:41.973 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:41 smithi123 sudo[45209]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:41.973 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:08:41 smithi123 sudo[45209]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:41.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:41 smithi123 ceph-mon[36640]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:08:41.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:41 smithi123 ceph-mon[36640]: osdmap e22: 3 total, 3 up, 3 in 2024-06-03T02:08:41.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:41 smithi123 ceph-mon[36640]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-03T02:08:41.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:41 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:41.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:41 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:41 smithi079 ceph-mon[31819]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:08:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:41 smithi079 ceph-mon[31819]: osdmap e22: 3 total, 3 up, 3 in 2024-06-03T02:08:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:41 smithi079 ceph-mon[31819]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-03T02:08:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:41 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:41 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:42.122 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:41 smithi135 sudo[45794]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T02:08:42.122 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:41 smithi135 sudo[45794]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:42.122 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:41 smithi135 sudo[45794]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:42.123 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:41 smithi135 sudo[45794]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:42.595 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:42 smithi135 sudo[45797]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T02:08:42.595 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:42 smithi135 sudo[45797]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:42.595 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:42 smithi135 sudo[45797]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:42.595 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:08:42 smithi135 sudo[45797]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:43.052 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:42 smithi079 sudo[51433]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T02:08:43.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:42 smithi079 sudo[51433]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:43.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:42 smithi079 sudo[51433]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:43.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:42 smithi079 sudo[51433]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:43.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:43 smithi079 sudo[51461]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T02:08:43.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:43 smithi079 sudo[51461]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:43.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:43 smithi079 sudo[51461]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:43.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:43 smithi079 sudo[51461]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:43.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:43 smithi079 ceph-mon[31819]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:43.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:43 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:43.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:43 smithi079 ceph-mon[31819]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T02:08:43.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:43 smithi079 ceph-mon[31819]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T02:08:43.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 ceph-mon[36653]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:43.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:43.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T02:08:43.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T02:08:43.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 sudo[45800]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T02:08:43.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 sudo[45800]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:43.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 sudo[45800]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:43.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 sudo[45800]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:43 smithi123 ceph-mon[36640]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:43 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:08:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:43 smithi123 ceph-mon[36640]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T02:08:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:43 smithi123 ceph-mon[36640]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T02:08:43.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 sudo[45803]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T02:08:43.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 sudo[45803]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:43.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 sudo[45803]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:43.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:43 smithi135 sudo[45803]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:44.340 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 sudo[45212]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T02:08:44.340 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 sudo[45212]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:44.340 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 sudo[45212]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:44.340 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 sudo[45212]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:44.340 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 sudo[45215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T02:08:44.340 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 sudo[45215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T02:08:44.340 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 sudo[45215]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T02:08:44.340 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 sudo[45215]: pam_unix(sudo:session): session closed for user root 2024-06-03T02:08:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:08:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:08:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:08:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T02:08:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:08:44.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:08:44.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:08:44.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T02:08:44.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T02:08:44.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:08:44.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:08:44.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:08:44.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:44.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:44 smithi135 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T02:08:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:08:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:08:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:08:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T02:08:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:08:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:08:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:08:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T02:08:44.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T02:08:44.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:08:44.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:08:44.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:08:44.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:44.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:44 smithi123 ceph-mon[36640]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T02:08:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:08:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:08:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:08:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T02:08:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:08:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:08:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:08:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T02:08:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T02:08:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T02:08:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T02:08:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T02:08:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:44 smithi079 ceph-mon[31819]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T02:08:45.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:45 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2796713157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1394880-b33f-4511-a8df-b80c7d0273fd"}]: dispatch 2024-06-03T02:08:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:45 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1394880-b33f-4511-a8df-b80c7d0273fd"}]: dispatch 2024-06-03T02:08:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:45 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1394880-b33f-4511-a8df-b80c7d0273fd"}]': finished 2024-06-03T02:08:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:45 smithi079 ceph-mon[31819]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T02:08:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:45 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:08:45.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:45 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2796713157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1394880-b33f-4511-a8df-b80c7d0273fd"}]: dispatch 2024-06-03T02:08:45.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:45 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1394880-b33f-4511-a8df-b80c7d0273fd"}]: dispatch 2024-06-03T02:08:45.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:45 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1394880-b33f-4511-a8df-b80c7d0273fd"}]': finished 2024-06-03T02:08:45.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:45 smithi123 ceph-mon[36640]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T02:08:45.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:45 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:08:45.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:45 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2796713157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1394880-b33f-4511-a8df-b80c7d0273fd"}]: dispatch 2024-06-03T02:08:45.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:45 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1394880-b33f-4511-a8df-b80c7d0273fd"}]: dispatch 2024-06-03T02:08:45.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:45 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1394880-b33f-4511-a8df-b80c7d0273fd"}]': finished 2024-06-03T02:08:45.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:45 smithi135 ceph-mon[36653]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T02:08:45.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:45 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:08:46.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:46 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3236439956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:08:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:46 smithi123 ceph-mon[36640]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:46.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:46 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3236439956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:08:46.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:46 smithi079 ceph-mon[31819]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:46.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:46 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3236439956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:08:46.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:46 smithi135 ceph-mon[36653]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:49.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:49 smithi079 ceph-mon[31819]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:49.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:49 smithi123 ceph-mon[36640]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:49 smithi135 ceph-mon[36653]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:08:51.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:51 smithi123 ceph-mon[36640]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-03T02:08:51.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:51 smithi079 ceph-mon[31819]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-03T02:08:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:51 smithi135 ceph-mon[36653]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-03T02:08:53.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:53 smithi123 ceph-mon[36640]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-03T02:08:53.555 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:53 smithi079 ceph-mon[31819]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-03T02:08:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:53 smithi135 ceph-mon[36653]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-03T02:08:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T02:08:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T02:08:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T02:08:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:08:55.245 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:55 smithi079 ceph-mon[31819]: Deploying daemon osd.3 on smithi079 2024-06-03T02:08:55.245 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:55 smithi079 ceph-mon[31819]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-03T02:08:55.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:55 smithi123 ceph-mon[36640]: Deploying daemon osd.3 on smithi079 2024-06-03T02:08:55.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:55 smithi123 ceph-mon[36640]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-03T02:08:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:55 smithi135 ceph-mon[36653]: Deploying daemon osd.3 on smithi079 2024-06-03T02:08:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:55 smithi135 ceph-mon[36653]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-03T02:08:57.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:57 smithi079 ceph-mon[31819]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 804 B/s rd, 23 KiB/s wr, 2 op/s 2024-06-03T02:08:57.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:57 smithi123 ceph-mon[36640]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 804 B/s rd, 23 KiB/s wr, 2 op/s 2024-06-03T02:08:57.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:57 smithi135 ceph-mon[36653]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 804 B/s rd, 23 KiB/s wr, 2 op/s 2024-06-03T02:08:59.218 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:08:59 smithi079 ceph-mon[31819]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-03T02:08:59.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:08:59 smithi135 ceph-mon[36653]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-03T02:08:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:08:59 smithi123 ceph-mon[36640]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-03T02:09:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:01 smithi123 ceph-mon[36640]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-03T02:09:01.505 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:01 smithi079 ceph-mon[31819]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-03T02:09:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:01 smithi135 ceph-mon[36653]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-03T02:09:03.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:03 smithi123 ceph-mon[36640]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:03.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:03.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:03.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:03.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:03 smithi079 ceph-mon[31819]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:03.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:03.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:03.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:03 smithi135 ceph-mon[36653]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:05.322 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:05 smithi079 ceph-mon[31819]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:05 smithi123 ceph-mon[36640]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:05 smithi135 ceph-mon[36653]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:07.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.207+0000 7f2a6f400700 1 -- 172.21.15.79:0/4199660147 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2a780628d0 con 0x7f2a5c0a2b90 2024-06-03T02:09:07.207 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 3 on host 'smithi079' 2024-06-03T02:09:07.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a5c0a2b90 msgr2=0x7f2a5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a5c0a2b90 0x7f2a5c0a5050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009680 tx=0x7f2a68009260 comp rx=0 tx=0).stop 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a78076440 msgr2=0x7f2a7819ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a78076440 0x7f2a7819ae70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a640095c0 tx=0x7f2a6400bb90 comp rx=0 tx=0).stop 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 shutdown_connections 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a5c0a2b90 0x7f2a5c0a5050 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a7810a3f0 0x7f2a781a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a78074e70 0x7f2a7819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 --2- 172.21.15.79:0/4199660147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a78076440 0x7f2a7819ae70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 >> 172.21.15.79:0/4199660147 conn(0x7f2a780fda90 msgr2=0x7f2a78106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 shutdown_connections 2024-06-03T02:09:07.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:09:07.211+0000 7f2a7da00700 1 -- 172.21.15.79:0/4199660147 wait complete. 2024-06-03T02:09:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:07 smithi123 ceph-mon[36640]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:07 smithi123 ceph-mon[36640]: from='osd.3 [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T02:09:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:07 smithi123 ceph-mon[36640]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T02:09:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:07 smithi079 ceph-mon[31819]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:07 smithi079 ceph-mon[31819]: from='osd.3 [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T02:09:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:07 smithi079 ceph-mon[31819]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T02:09:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:07 smithi135 ceph-mon[36653]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:07 smithi135 ceph-mon[36653]: from='osd.3 [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T02:09:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:07 smithi135 ceph-mon[36653]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T02:09:08.065 DEBUG:teuthology.orchestra.run.smithi079:osd.3> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.3.service 2024-06-03T02:09:08.072 INFO:tasks.cephadm:Deploying osd.4 on smithi123 with /dev/vg_nvme/lv_3... 2024-06-03T02:09:08.072 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T02:09:08.316 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:09:08.338 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3[53533]: 2024-06-03T02:09:08.193+0000 7efeeac00700 -1 osd.3 0 waiting for initial osdmap 2024-06-03T02:09:08.338 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3[53533]: 2024-06-03T02:09:08.203+0000 7efee5a00700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:09:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-mon[31819]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T02:09:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-mon[31819]: from='osd.3 [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:09:08.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-mon[31819]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T02:09:08.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:08.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-mon[31819]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:09:08.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:08.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:08.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:08.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:08 smithi079 ceph-mon[31819]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:08.438 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:08 smithi123 ceph-mon[36640]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T02:09:08.438 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:08 smithi123 ceph-mon[36640]: from='osd.3 [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:09:08.438 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:08 smithi123 ceph-mon[36640]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T02:09:08.439 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:08 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:08.439 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:08 smithi123 ceph-mon[36640]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:09:08.439 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:08 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:08.439 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:08 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:08.439 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:08 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:08.439 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:08 smithi123 ceph-mon[36640]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:08 smithi135 ceph-mon[36653]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T02:09:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:08 smithi135 ceph-mon[36653]: from='osd.3 [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:09:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:08 smithi135 ceph-mon[36653]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T02:09:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:08 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:08 smithi135 ceph-mon[36653]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:09:08.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:08 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:08.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:08 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:08.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:08 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:08.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:08 smithi135 ceph-mon[36653]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-06-03T02:09:09.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:09 smithi123 ceph-mon[36640]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:09:09.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:09 smithi123 ceph-mon[36640]: osdmap e25: 4 total, 3 up, 4 in 2024-06-03T02:09:09.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:09.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:09.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:09 smithi123 ceph-mon[36640]: osd.3 [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] boot 2024-06-03T02:09:09.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:09 smithi123 ceph-mon[36640]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T02:09:09.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:09.448 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:09 smithi079 ceph-mon[31819]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:09:09.448 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:09 smithi079 ceph-mon[31819]: osdmap e25: 4 total, 3 up, 4 in 2024-06-03T02:09:09.449 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:09.449 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:09.449 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:09 smithi079 ceph-mon[31819]: osd.3 [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] boot 2024-06-03T02:09:09.449 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:09 smithi079 ceph-mon[31819]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T02:09:09.449 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:09 smithi135 ceph-mon[36653]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:09:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:09 smithi135 ceph-mon[36653]: osdmap e25: 4 total, 3 up, 4 in 2024-06-03T02:09:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:09 smithi135 ceph-mon[36653]: osd.3 [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] boot 2024-06-03T02:09:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:09 smithi135 ceph-mon[36653]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T02:09:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T02:09:10.237 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:10 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:09:10.237 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:10 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:09:10.237 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:10 smithi123 ceph-mon[36640]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T02:09:10.237 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:10 smithi123 ceph-mon[36640]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:10.238 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T02:09:10.238 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T02:09:10.238 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-06-03T02:09:10.238 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-06-03T02:09:10.238 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114961 s, 912 MB/s 2024-06-03T02:09:10.238 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-06-03T02:09:10.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:10 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:09:10.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:10 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:09:10.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:10 smithi079 ceph-mon[31819]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T02:09:10.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:10 smithi079 ceph-mon[31819]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:10 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:09:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:10 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:09:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:10 smithi135 ceph-mon[36653]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T02:09:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:10 smithi135 ceph-mon[36653]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:10.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-06-03T02:09:11.123 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:09:12.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:11 smithi123 ceph-mon[36640]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T02:09:12.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:11 smithi079 ceph-mon[31819]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T02:09:12.344 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:11 smithi135 ceph-mon[36653]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T02:09:12.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 -- 172.21.15.123:0/242328107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 msgr2=0x7f67a8109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/242328107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 0x7f67a8109840 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f67a000b0d0 tx=0x7f67a000b3e0 comp rx=0 tx=0).stop 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 -- 172.21.15.123:0/242328107 shutdown_connections 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/242328107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 0x7f67a8109840 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/242328107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a81013f0 0x7f67a8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/242328107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67a81025f0 0x7f67a8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 -- 172.21.15.123:0/242328107 >> 172.21.15.123:0/242328107 conn(0x7f67a80fc990 msgr2=0x7f67a80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 -- 172.21.15.123:0/242328107 shutdown_connections 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 -- 172.21.15.123:0/242328107 wait complete. 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.820+0000 7f67ae7d5700 1 Processor -- start 2024-06-03T02:09:12.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67ae7d5700 1 -- start start 2024-06-03T02:09:12.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67ae7d5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a81013f0 0x7f67a8105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67ae7d5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67a81025f0 0x7f67a81039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67ae7d5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 0x7f67a8103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67ae7d5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f67a810d050 con 0x7f67a81025f0 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67ae7d5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67a810ced0 con 0x7f67a8102f90 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67ae7d5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67a810d1d0 con 0x7f67a81013f0 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67acd72700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 0x7f67a8103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67acd72700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 0x7f67a8103f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60636/0 (socket says 172.21.15.123:60636) 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67acd72700 1 -- 172.21.15.123:0/103457339 learned_addr learned my addr 172.21.15.123:0/103457339 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67a7fff700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a81013f0 0x7f67a8105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67acd72700 1 -- 172.21.15.123:0/103457339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a81013f0 msgr2=0x7f67a8105380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67acd72700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a81013f0 0x7f67a8105380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67acd72700 1 -- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67a81025f0 msgr2=0x7f67a81039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:12.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67a77fe700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67a81025f0 0x7f67a81039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:09:12.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67acd72700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67a81025f0 0x7f67a81039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:12.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.821+0000 7f67acd72700 1 -- 172.21.15.123:0/103457339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a81047c0 con 0x7f67a8102f90 2024-06-03T02:09:12.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.822+0000 7f67acd72700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 0x7f67a8103f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006070 tx=0x7f67a0007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:09:12.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.822+0000 7f67a57fa700 1 -- 172.21.15.123:0/103457339 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0021020 con 0x7f67a8102f90 2024-06-03T02:09:12.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.822+0000 7f67a57fa700 1 -- 172.21.15.123:0/103457339 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a001d950 con 0x7f67a8102f90 2024-06-03T02:09:12.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.822+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a81a2890 con 0x7f67a8102f90 2024-06-03T02:09:12.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.822+0000 7f67a57fa700 1 -- 172.21.15.123:0/103457339 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0005450 con 0x7f67a8102f90 2024-06-03T02:09:12.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.822+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a81a2e50 con 0x7f67a8102f90 2024-06-03T02:09:12.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.823+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a8069180 con 0x7f67a8102f90 2024-06-03T02:09:12.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.824+0000 7f67a57fa700 1 -- 172.21.15.123:0/103457339 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f67a0024020 con 0x7f67a8102f90 2024-06-03T02:09:12.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.824+0000 7f67a57fa700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f67900a2ca0 0x7f67900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:09:12.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.825+0000 7f67a57fa700 1 -- 172.21.15.123:0/103457339 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f67a000f020 con 0x7f67a8102f90 2024-06-03T02:09:12.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.825+0000 7f67a7fff700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f67900a2ca0 0x7f67900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:09:12.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.825+0000 7f67a7fff700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f67900a2ca0 0x7f67900a5160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67a8102450 tx=0x7f679800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:09:12.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:12.828+0000 7f67a57fa700 1 -- 172.21.15.123:0/103457339 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f67a009da80 con 0x7f67a8102f90 2024-06-03T02:09:13.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:13.033+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f67a80feb20 con 0x7f67900a2ca0 2024-06-03T02:09:13.159 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:12 smithi079 ceph-mon[31819]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T02:09:13.159 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:12 smithi079 ceph-mon[31819]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:13.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:12 smithi123 ceph-mon[36640]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T02:09:13.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:12 smithi123 ceph-mon[36640]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:13.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:12 smithi135 ceph-mon[36653]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T02:09:13.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:12 smithi135 ceph-mon[36653]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:13 smithi123 ceph-mon[36640]: osdmap e30: 4 total, 4 up, 4 in 2024-06-03T02:09:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:13 smithi123 ceph-mon[36640]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:09:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:09:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:09:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:14.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:13 smithi079 ceph-mon[31819]: osdmap e30: 4 total, 4 up, 4 in 2024-06-03T02:09:14.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:13 smithi079 ceph-mon[31819]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:09:14.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:09:14.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:09:14.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:14.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:13 smithi135 ceph-mon[36653]: osdmap e30: 4 total, 4 up, 4 in 2024-06-03T02:09:14.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:13 smithi135 ceph-mon[36653]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:09:14.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:09:14.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:09:14.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi079 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi079 to 8544M 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-06-03T02:09:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi079 2024-06-03T02:09:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi079 to 8544M 2024-06-03T02:09:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:15.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:15.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi079 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi079 to 8544M 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:16 smithi123 ceph-mon[36640]: from='client.? 172.21.15.123:0/1082000363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c407e697-6fbf-4447-b7ae-6a2079134052"}]: dispatch 2024-06-03T02:09:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:16 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c407e697-6fbf-4447-b7ae-6a2079134052"}]: dispatch 2024-06-03T02:09:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:16 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c407e697-6fbf-4447-b7ae-6a2079134052"}]': finished 2024-06-03T02:09:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:16 smithi123 ceph-mon[36640]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T02:09:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:16 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:16.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:16 smithi079 ceph-mon[31819]: from='client.? 172.21.15.123:0/1082000363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c407e697-6fbf-4447-b7ae-6a2079134052"}]: dispatch 2024-06-03T02:09:16.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:16 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c407e697-6fbf-4447-b7ae-6a2079134052"}]: dispatch 2024-06-03T02:09:16.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:16 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c407e697-6fbf-4447-b7ae-6a2079134052"}]': finished 2024-06-03T02:09:16.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:16 smithi079 ceph-mon[31819]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T02:09:16.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:16 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:16.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:16 smithi135 ceph-mon[36653]: from='client.? 172.21.15.123:0/1082000363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c407e697-6fbf-4447-b7ae-6a2079134052"}]: dispatch 2024-06-03T02:09:16.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:16 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c407e697-6fbf-4447-b7ae-6a2079134052"}]: dispatch 2024-06-03T02:09:16.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:16 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c407e697-6fbf-4447-b7ae-6a2079134052"}]': finished 2024-06-03T02:09:16.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:16 smithi135 ceph-mon[36653]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T02:09:16.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:16 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:17.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:17 smithi079 ceph-mon[31819]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-03T02:09:17.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:17 smithi079 ceph-mon[31819]: from='client.? 172.21.15.123:0/1408976371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:09:17.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:17 smithi135 ceph-mon[36653]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-03T02:09:17.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:17 smithi135 ceph-mon[36653]: from='client.? 172.21.15.123:0/1408976371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:09:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:17 smithi123 ceph-mon[36640]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-06-03T02:09:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:17 smithi123 ceph-mon[36640]: from='client.? 172.21.15.123:0/1408976371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:09:19.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:19 smithi079 ceph-mon[31819]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-03T02:09:19.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:19 smithi135 ceph-mon[36653]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-03T02:09:19.441 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:19 smithi123 ceph-mon[36640]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-03T02:09:21.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:21 smithi079 ceph-mon[31819]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T02:09:21.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:21 smithi135 ceph-mon[36653]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T02:09:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:21 smithi123 ceph-mon[36640]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T02:09:23.213 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:23 smithi123 ceph-mon[36640]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-03T02:09:23.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:23 smithi135 ceph-mon[36653]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-03T02:09:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:23 smithi079 ceph-mon[31819]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-03T02:09:24.208 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:24 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T02:09:24.208 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:24 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T02:09:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:24.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:24 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T02:09:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:24 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:25 smithi123 ceph-mon[36640]: Deploying daemon osd.4 on smithi123 2024-06-03T02:09:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:25 smithi123 ceph-mon[36640]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:25 smithi079 ceph-mon[31819]: Deploying daemon osd.4 on smithi123 2024-06-03T02:09:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:25 smithi079 ceph-mon[31819]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:25 smithi135 ceph-mon[36653]: Deploying daemon osd.4 on smithi123 2024-06-03T02:09:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:25 smithi135 ceph-mon[36653]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:27.382 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:27 smithi123 ceph-mon[36640]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:27.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:27 smithi079 ceph-mon[31819]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:27.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:27 smithi135 ceph-mon[36653]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:29.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:29 smithi135 ceph-mon[36653]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:29.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:29.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:29.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:29.470 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:29 smithi123 ceph-mon[36640]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:29.470 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:29.470 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:29.470 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:29.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:29 smithi079 ceph-mon[31819]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:29.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:29.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:29.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:31.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:31 smithi123 ceph-mon[36640]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:31.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:31 smithi079 ceph-mon[31819]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:31.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:31 smithi135 ceph-mon[36653]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:33.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.064+0000 7f67a57fa700 1 -- 172.21.15.123:0/103457339 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f67a80feb20 con 0x7f67900a2ca0 2024-06-03T02:09:33.065 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 4 on host 'smithi123' 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.066+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f67900a2ca0 msgr2=0x7f67900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.066+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f67900a2ca0 0x7f67900a5160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67a8102450 tx=0x7f679800b410 comp rx=0 tx=0).stop 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.066+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 msgr2=0x7f67a8103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.066+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 0x7f67a8103f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006070 tx=0x7f67a0007d90 comp rx=0 tx=0).stop 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.067+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 shutdown_connections 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.067+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67a8102f90 0x7f67a8103f10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.067+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a81013f0 0x7f67a8105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.067+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f67900a2ca0 0x7f67900a5160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.067+0000 7f67ae7d5700 1 --2- 172.21.15.123:0/103457339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67a81025f0 0x7f67a81039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.067+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 >> 172.21.15.123:0/103457339 conn(0x7f67a80fc990 msgr2=0x7f67a80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:09:33.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.067+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 shutdown_connections 2024-06-03T02:09:33.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:09:33.067+0000 7f67ae7d5700 1 -- 172.21.15.123:0/103457339 wait complete. 2024-06-03T02:09:33.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:33.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:33.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:33.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='osd.4 [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T02:09:33.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T02:09:33.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='osd.4 [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T02:09:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T02:09:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='osd.4 [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T02:09:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T02:09:33.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:33.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:33.726 DEBUG:teuthology.orchestra.run.smithi123:osd.4> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.4.service 2024-06-03T02:09:33.729 INFO:tasks.cephadm:Deploying osd.5 on smithi135 with /dev/vg_nvme/lv_3... 2024-06-03T02:09:33.729 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T02:09:33.971 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:09:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:34 smithi135 ceph-mon[36653]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T02:09:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:34 smithi135 ceph-mon[36653]: from='osd.4 [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:09:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:34 smithi135 ceph-mon[36653]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T02:09:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:34 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:34 smithi135 ceph-mon[36653]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:09:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:34 smithi123 ceph-mon[36640]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T02:09:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:34 smithi123 ceph-mon[36640]: from='osd.4 [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:09:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:34 smithi123 ceph-mon[36640]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T02:09:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:34 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:34 smithi123 ceph-mon[36640]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:09:34.473 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:09:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:09:34.159+0000 7ff11c05c700 -1 osd.4 0 waiting for initial osdmap 2024-06-03T02:09:34.473 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:09:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:09:34.166+0000 7ff117e4f700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:09:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:34 smithi079 ceph-mon[31819]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T02:09:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:34 smithi079 ceph-mon[31819]: from='osd.4 [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:09:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:34 smithi079 ceph-mon[31819]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T02:09:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:34 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:34 smithi079 ceph-mon[31819]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:09:35.446 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:35 smithi135 ceph-mon[36653]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:35.446 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:35 smithi135 ceph-mon[36653]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:09:35.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:35 smithi135 ceph-mon[36653]: osdmap e33: 5 total, 4 up, 5 in 2024-06-03T02:09:35.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:35.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:35 smithi123 ceph-mon[36640]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:35 smithi123 ceph-mon[36640]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:09:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:35 smithi123 ceph-mon[36640]: osdmap e33: 5 total, 4 up, 5 in 2024-06-03T02:09:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:35.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:35 smithi079 ceph-mon[31819]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:35.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:35 smithi079 ceph-mon[31819]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:09:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:35 smithi079 ceph-mon[31819]: osdmap e33: 5 total, 4 up, 5 in 2024-06-03T02:09:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:35.996 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T02:09:35.996 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T02:09:35.996 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-03T02:09:35.997 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-03T02:09:35.997 INFO:teuthology.orchestra.run.smithi135.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114326 s, 917 MB/s 2024-06-03T02:09:35.997 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-03T02:09:36.349 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:36 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:09:36.350 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:36 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:09:36.350 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:36 smithi123 ceph-mon[36640]: osd.4 [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] boot 2024-06-03T02:09:36.350 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:36 smithi123 ceph-mon[36640]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T02:09:36.350 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:36 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:36.350 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:36 smithi123 ceph-mon[36640]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:36.461 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:36 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:09:36.461 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:36 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:09:36.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:36 smithi135 ceph-mon[36653]: osd.4 [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] boot 2024-06-03T02:09:36.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:36 smithi135 ceph-mon[36653]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T02:09:36.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:36 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:36.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:36 smithi135 ceph-mon[36653]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:36 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:09:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:36 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:09:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:36 smithi079 ceph-mon[31819]: osd.4 [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] boot 2024-06-03T02:09:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:36 smithi079 ceph-mon[31819]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T02:09:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:36 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T02:09:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:36 smithi079 ceph-mon[31819]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-03T02:09:36.612 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_3 2024-06-03T02:09:36.856 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:09:37.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: osdmap e35: 5 total, 5 up, 5 in 2024-06-03T02:09:37.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi123 2024-06-03T02:09:37.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:37.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:37.265 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi123 to 8544M 2024-06-03T02:09:37.265 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.265 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:37.265 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:37.265 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: osdmap e35: 5 total, 5 up, 5 in 2024-06-03T02:09:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi123 2024-06-03T02:09:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi123 to 8544M 2024-06-03T02:09:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:37.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:37.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: osdmap e35: 5 total, 5 up, 5 in 2024-06-03T02:09:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi123 2024-06-03T02:09:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:09:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi123 to 8544M 2024-06-03T02:09:37.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:37.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:37.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:37.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:38.217 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:38 smithi135 ceph-mon[36653]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:38.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:38 smithi123 ceph-mon[36640]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:38.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.522+0000 7fc6c2494700 1 -- 172.21.15.135:0/2570460758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc1005a0 msgr2=0x7fc6bc100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.522+0000 7fc6c2494700 1 --2- 172.21.15.135:0/2570460758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc1005a0 0x7fc6bc100a00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009990 tx=0x7fc6b0009ca0 comp rx=0 tx=0).stop 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.522+0000 7fc6c2494700 1 -- 172.21.15.135:0/2570460758 shutdown_connections 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.522+0000 7fc6c2494700 1 --2- 172.21.15.135:0/2570460758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6bc100f40 0x7fc6bc10ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.522+0000 7fc6c2494700 1 --2- 172.21.15.135:0/2570460758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc103b20 0x7fc6bc103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.522+0000 7fc6c2494700 1 --2- 172.21.15.135:0/2570460758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc1005a0 0x7fc6bc100a00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.522+0000 7fc6c2494700 1 -- 172.21.15.135:0/2570460758 >> 172.21.15.135:0/2570460758 conn(0x7fc6bc0fc160 msgr2=0x7fc6bc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.522+0000 7fc6c2494700 1 -- 172.21.15.135:0/2570460758 shutdown_connections 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.523+0000 7fc6c2494700 1 -- 172.21.15.135:0/2570460758 wait complete. 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.523+0000 7fc6c2494700 1 Processor -- start 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.523+0000 7fc6c2494700 1 -- start start 2024-06-03T02:09:38.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6c2494700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6bc1005a0 0x7fc6bc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6c2494700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc100f40 0x7fc6bc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6c2494700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc103b20 0x7fc6bc19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6c2494700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10e150 con 0x7fc6bc103b20 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6c2494700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10dfd0 con 0x7fc6bc1005a0 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6c2494700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10e2d0 con 0x7fc6bc100f40 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bb7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc100f40 0x7fc6bc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bb7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc100f40 0x7fc6bc198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42342/0 (socket says 172.21.15.135:42342) 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bb7fe700 1 -- 172.21.15.135:0/4137386649 learned_addr learned my addr 172.21.15.135:0/4137386649 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bbfff700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6bc1005a0 0x7fc6bc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:09:38.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bb7fe700 1 -- 172.21.15.135:0/4137386649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6bc1005a0 msgr2=0x7fc6bc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bb7fe700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6bc1005a0 0x7fc6bc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bb7fe700 1 -- 172.21.15.135:0/4137386649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc103b20 msgr2=0x7fc6bc19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bb7fe700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc103b20 0x7fc6bc19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bb7fe700 1 -- 172.21.15.135:0/4137386649 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6bc1a0360 con 0x7fc6bc100f40 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bbfff700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6bc1005a0 0x7fc6bc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6bb7fe700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc100f40 0x7fc6bc198c40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00059d0 tx=0x7fc6b000bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6b97fa700 1 -- 172.21.15.135:0/4137386649 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0027070 con 0x7fc6bc100f40 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6b97fa700 1 -- 172.21.15.135:0/4137386649 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6b0022470 con 0x7fc6bc100f40 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6bc1a05f0 con 0x7fc6bc100f40 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6b97fa700 1 -- 172.21.15.135:0/4137386649 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000f6c0 con 0x7fc6bc100f40 2024-06-03T02:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.524+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6bc1a0b70 con 0x7fc6bc100f40 2024-06-03T02:09:38.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.526+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6bc069180 con 0x7fc6bc100f40 2024-06-03T02:09:38.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.527+0000 7fc6b97fa700 1 -- 172.21.15.135:0/4137386649 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc6b002a070 con 0x7fc6bc100f40 2024-06-03T02:09:38.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.527+0000 7fc6b97fa700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc6a40a2ca0 0x7fc6a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.527+0000 7fc6b97fa700 1 -- 172.21.15.135:0/4137386649 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fc6b00cede0 con 0x7fc6bc100f40 2024-06-03T02:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.528+0000 7fc6bbfff700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc6a40a2ca0 0x7fc6a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.529+0000 7fc6bbfff700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc6a40a2ca0 0x7fc6a40a5160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06a900 tx=0x7fc6ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.532+0000 7fc6b97fa700 1 -- 172.21.15.135:0/4137386649 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc6b0013070 con 0x7fc6bc100f40 2024-06-03T02:09:38.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:38 smithi079 ceph-mon[31819]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:38.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:38.737+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc6bc0628d0 con 0x7fc6a40a2ca0 2024-06-03T02:09:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:39 smithi123 ceph-mon[36640]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:09:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:39 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:09:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:39 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:09:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:39 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:39.511 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:39 smithi135 ceph-mon[36653]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:09:39.511 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:39 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:09:39.511 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:39 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:09:39.511 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:39 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:39 smithi079 ceph-mon[31819]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:09:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:09:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:09:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:40.466 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:40 smithi135 ceph-mon[36653]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:40.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:40 smithi123 ceph-mon[36640]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:40.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:40 smithi079 ceph-mon[31819]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:41 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/1794905492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3e5d988-4303-49db-9740-c3e87c443c76"}]: dispatch 2024-06-03T02:09:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:41 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3e5d988-4303-49db-9740-c3e87c443c76"}]: dispatch 2024-06-03T02:09:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:41 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3e5d988-4303-49db-9740-c3e87c443c76"}]': finished 2024-06-03T02:09:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:41 smithi079 ceph-mon[31819]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T02:09:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:41 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:09:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:41 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/1794905492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3e5d988-4303-49db-9740-c3e87c443c76"}]: dispatch 2024-06-03T02:09:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:41 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3e5d988-4303-49db-9740-c3e87c443c76"}]: dispatch 2024-06-03T02:09:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:41 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3e5d988-4303-49db-9740-c3e87c443c76"}]': finished 2024-06-03T02:09:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:41 smithi135 ceph-mon[36653]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T02:09:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:41 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:09:41.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:41 smithi123 ceph-mon[36640]: from='client.? 172.21.15.135:0/1794905492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3e5d988-4303-49db-9740-c3e87c443c76"}]: dispatch 2024-06-03T02:09:41.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:41 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3e5d988-4303-49db-9740-c3e87c443c76"}]: dispatch 2024-06-03T02:09:41.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:41 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3e5d988-4303-49db-9740-c3e87c443c76"}]': finished 2024-06-03T02:09:41.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:41 smithi123 ceph-mon[36640]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T02:09:41.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:41 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:09:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:42 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/2818312353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:09:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:42 smithi135 ceph-mon[36653]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:42.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:42 smithi123 ceph-mon[36640]: from='client.? 172.21.15.135:0/2818312353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:09:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:42 smithi123 ceph-mon[36640]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:42.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:42 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/2818312353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:09:42.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:42 smithi079 ceph-mon[31819]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:45.233 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:45 smithi135 ceph-mon[36653]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:45.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:45 smithi123 ceph-mon[36640]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:45.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:45 smithi079 ceph-mon[31819]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:47.278 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:47 smithi135 ceph-mon[36653]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:47.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:47 smithi123 ceph-mon[36640]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:47.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:47 smithi079 ceph-mon[31819]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:49.383 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:49 smithi135 ceph-mon[36653]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:49.383 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:49 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T02:09:49.383 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:49 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:49.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:49 smithi123 ceph-mon[36640]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:49.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:49 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T02:09:49.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:49 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:49 smithi079 ceph-mon[31819]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:49 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T02:09:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:49 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:50.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:50 smithi135 ceph-mon[36653]: Deploying daemon osd.5 on smithi135 2024-06-03T02:09:50.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:50 smithi123 ceph-mon[36640]: Deploying daemon osd.5 on smithi135 2024-06-03T02:09:50.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:50 smithi079 ceph-mon[31819]: Deploying daemon osd.5 on smithi135 2024-06-03T02:09:51.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:51 smithi123 ceph-mon[36640]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:51.533 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:51 smithi135 ceph-mon[36653]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:51.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:51 smithi079 ceph-mon[31819]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:53.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:53 smithi135 ceph-mon[36653]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:53.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:53 smithi123 ceph-mon[36640]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:53.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:53 smithi079 ceph-mon[31819]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:55.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:55.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:55.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:55.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:54 smithi079 ceph-mon[31819]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:54 smithi135 ceph-mon[36653]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:55.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:55.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:55.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:55.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:54 smithi123 ceph-mon[36640]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:57.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:57 smithi135 ceph-mon[36653]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:57.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:57 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:57.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:57 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:57.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:57 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:57.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:57 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:57.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:57 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:57.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:57 smithi123 ceph-mon[36640]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:57.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:57 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:57.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:57 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:57.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:57 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:57.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:57 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:57.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:57 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:57.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:57 smithi079 ceph-mon[31819]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:57.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:57 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:57.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:57 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:57.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:57 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:09:57.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:57 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:09:57.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:57 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.826+0000 7fc6b97fa700 1 -- 172.21.15.135:0/4137386649 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc6bc0628d0 con 0x7fc6a40a2ca0 2024-06-03T02:09:57.826 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 5 on host 'smithi135' 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc6a40a2ca0 msgr2=0x7fc6a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc6a40a2ca0 0x7fc6a40a5160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06a900 tx=0x7fc6ac008040 comp rx=0 tx=0).stop 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc100f40 msgr2=0x7fc6bc198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc100f40 0x7fc6bc198c40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00059d0 tx=0x7fc6b000bfa0 comp rx=0 tx=0).stop 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 shutdown_connections 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc6a40a2ca0 0x7fc6a40a5160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6bc103b20 0x7fc6bc19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6bc1005a0 0x7fc6bc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 --2- 172.21.15.135:0/4137386649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc100f40 0x7fc6bc198c40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:09:57.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 >> 172.21.15.135:0/4137386649 conn(0x7fc6bc0fc160 msgr2=0x7fc6bc106b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:09:57.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 shutdown_connections 2024-06-03T02:09:57.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:09:57.829+0000 7fc6c2494700 1 -- 172.21.15.135:0/4137386649 wait complete. 2024-06-03T02:09:58.465 DEBUG:teuthology.orchestra.run.smithi135:osd.5> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.5.service 2024-06-03T02:09:58.468 INFO:tasks.cephadm:Deploying osd.6 on smithi079 with /dev/vg_nvme/lv_2... 2024-06-03T02:09:58.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T02:09:58.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:58.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:58.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:58.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:58 smithi079 ceph-mon[31819]: from='osd.5 [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T02:09:58.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:58 smithi079 ceph-mon[31819]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T02:09:58.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:58 smithi079 ceph-mon[31819]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:58.907 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:09:59.028 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:59.028 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:59.028 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:59.028 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:58 smithi135 ceph-mon[36653]: from='osd.5 [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T02:09:59.028 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:58 smithi135 ceph-mon[36653]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T02:09:59.028 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:58 smithi135 ceph-mon[36653]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:09:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:09:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:58 smithi123 ceph-mon[36640]: from='osd.5 [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T02:09:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:58 smithi123 ceph-mon[36640]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T02:09:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:58 smithi123 ceph-mon[36640]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:09:59.929 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:59 smithi135 ceph-mon[36653]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T02:09:59.929 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:59 smithi135 ceph-mon[36653]: from='osd.5 [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:09:59.929 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:59 smithi135 ceph-mon[36653]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T02:09:59.929 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:59 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:09:59.929 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:09:59 smithi135 ceph-mon[36653]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:10:00.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:59 smithi079 ceph-mon[31819]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T02:10:00.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:59 smithi079 ceph-mon[31819]: from='osd.5 [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:10:00.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:59 smithi079 ceph-mon[31819]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T02:10:00.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:59 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:00.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:09:59 smithi079 ceph-mon[31819]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:10:00.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:59 smithi123 ceph-mon[36640]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T02:10:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:59 smithi123 ceph-mon[36640]: from='osd.5 [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:10:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:59 smithi123 ceph-mon[36640]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T02:10:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:59 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:09:59 smithi123 ceph-mon[36640]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:10:00.991 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:00 smithi079 ceph-mon[31819]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:10:00.991 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:00 smithi079 ceph-mon[31819]: osdmap e38: 6 total, 5 up, 6 in 2024-06-03T02:10:00.991 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:00.991 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:00 smithi079 ceph-mon[31819]: osdmap e39: 6 total, 5 up, 6 in 2024-06-03T02:10:00.991 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:00.991 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:00 smithi079 ceph-mon[31819]: overall HEALTH_OK 2024-06-03T02:10:00.991 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:00 smithi079 ceph-mon[31819]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:10:00.991 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:00 smithi079 ceph-mon[31819]: from='osd.5 ' entity='osd.5' 2024-06-03T02:10:00.992 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:01.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-mon[36653]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:10:01.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-mon[36653]: osdmap e38: 6 total, 5 up, 6 in 2024-06-03T02:10:01.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:01.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-mon[36653]: osdmap e39: 6 total, 5 up, 6 in 2024-06-03T02:10:01.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:01.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-mon[36653]: overall HEALTH_OK 2024-06-03T02:10:01.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-mon[36653]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:10:01.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-mon[36653]: from='osd.5 ' entity='osd.5' 2024-06-03T02:10:01.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:01.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:10:00.670+0000 7fac6704d700 -1 osd.5 0 waiting for initial osdmap 2024-06-03T02:10:01.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:10:00 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:10:00.680+0000 7fac5ee38700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:10:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:00 smithi123 ceph-mon[36640]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:10:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:00 smithi123 ceph-mon[36640]: osdmap e38: 6 total, 5 up, 6 in 2024-06-03T02:10:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:00 smithi123 ceph-mon[36640]: osdmap e39: 6 total, 5 up, 6 in 2024-06-03T02:10:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:00 smithi123 ceph-mon[36640]: overall HEALTH_OK 2024-06-03T02:10:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:00 smithi123 ceph-mon[36640]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-03T02:10:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:00 smithi123 ceph-mon[36640]: from='osd.5 ' entity='osd.5' 2024-06-03T02:10:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:01.961 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T02:10:01.961 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T02:10:01.961 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-06-03T02:10:01.962 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-06-03T02:10:01.962 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0484689 s, 216 MB/s 2024-06-03T02:10:01.962 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-06-03T02:10:01.962 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:10:01.962 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:10:01.962 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: osd.5 [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] boot 2024-06-03T02:10:01.962 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T02:10:01.962 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:01.962 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:01.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:01.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:01.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:01.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:01.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:01.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:10:01.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:02.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:10:02.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:10:02.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: osd.5 [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] boot 2024-06-03T02:10:02.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T02:10:02.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:02.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:02.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:02.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:02.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:02.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:02.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:02.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:10:02.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:10:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:10:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: osd.5 [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] boot 2024-06-03T02:10:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T02:10:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T02:10:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:02.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:02.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:02.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:10:02.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:02.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_2 2024-06-03T02:10:03.122 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:10:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:02 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi135 2024-06-03T02:10:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:02 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi135 to 8544M 2024-06-03T02:10:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:02 smithi123 ceph-mon[36640]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T02:10:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:02 smithi123 ceph-mon[36640]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:03.267 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:02 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi135 2024-06-03T02:10:03.267 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:02 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi135 to 8544M 2024-06-03T02:10:03.268 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:02 smithi079 ceph-mon[31819]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T02:10:03.268 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:02 smithi079 ceph-mon[31819]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:02 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi135 2024-06-03T02:10:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:02 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi135 to 8544M 2024-06-03T02:10:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:02 smithi135 ceph-mon[36653]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T02:10:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:02 smithi135 ceph-mon[36653]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:04.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:03 smithi123 ceph-mon[36640]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T02:10:04.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:03 smithi079 ceph-mon[31819]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T02:10:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:03 smithi135 ceph-mon[36653]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T02:10:05.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:04 smithi079 ceph-mon[31819]: osdmap e43: 6 total, 6 up, 6 in 2024-06-03T02:10:05.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:04 smithi079 ceph-mon[31819]: pgmap v159: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-06-03T02:10:05.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:04 smithi123 ceph-mon[36640]: osdmap e43: 6 total, 6 up, 6 in 2024-06-03T02:10:05.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:04 smithi123 ceph-mon[36640]: pgmap v159: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-06-03T02:10:05.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:04 smithi135 ceph-mon[36653]: osdmap e43: 6 total, 6 up, 6 in 2024-06-03T02:10:05.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:04 smithi135 ceph-mon[36653]: pgmap v159: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-06-03T02:10:05.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.354+0000 7ff88b400700 1 -- 172.21.15.79:0/1473001019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff88410a6d0 msgr2=0x7ff884110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:05.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.354+0000 7ff88b400700 1 --2- 172.21.15.79:0/1473001019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff88410a6d0 0x7ff884110f80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009f90 tx=0x7ff87c00b330 comp rx=0 tx=0).stop 2024-06-03T02:10:05.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.354+0000 7ff88b400700 1 -- 172.21.15.79:0/1473001019 shutdown_connections 2024-06-03T02:10:05.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.354+0000 7ff88b400700 1 --2- 172.21.15.79:0/1473001019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff88410a6d0 0x7ff884110f80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:05.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.354+0000 7ff88b400700 1 --2- 172.21.15.79:0/1473001019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff884108b30 0x7ff884108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:05.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.354+0000 7ff88b400700 1 --2- 172.21.15.79:0/1473001019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff884109d30 0x7ff88410a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:05.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.354+0000 7ff88b400700 1 -- 172.21.15.79:0/1473001019 >> 172.21.15.79:0/1473001019 conn(0x7ff884077b00 msgr2=0x7ff884079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:10:05.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.354+0000 7ff88b400700 1 -- 172.21.15.79:0/1473001019 shutdown_connections 2024-06-03T02:10:05.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.354+0000 7ff88b400700 1 -- 172.21.15.79:0/1473001019 wait complete. 2024-06-03T02:10:05.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.356+0000 7ff88b400700 1 Processor -- start 2024-06-03T02:10:05.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.356+0000 7ff88b400700 1 -- start start 2024-06-03T02:10:05.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.356+0000 7ff88b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff884108b30 0x7ff88419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:10:05.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.356+0000 7ff88b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff884109d30 0x7ff88419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:10:05.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.356+0000 7ff88b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff88410a6d0 0x7ff8841a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:10:05.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.356+0000 7ff88b400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff884114770 con 0x7ff88410a6d0 2024-06-03T02:10:05.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.356+0000 7ff88b400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841145f0 con 0x7ff884108b30 2024-06-03T02:10:05.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.356+0000 7ff88b400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841148f0 con 0x7ff884109d30 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff883e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff884109d30 0x7ff88419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff883e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff884109d30 0x7ff88419f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:50626/0 (socket says 172.21.15.79:50626) 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff883e00700 1 -- 172.21.15.79:0/522802146 learned_addr learned my addr 172.21.15.79:0/522802146 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff888a00700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff884108b30 0x7ff88419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff889400700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff88410a6d0 0x7ff8841a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff888a00700 1 -- 172.21.15.79:0/522802146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff884109d30 msgr2=0x7ff88419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff888a00700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff884109d30 0x7ff88419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff888a00700 1 -- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff88410a6d0 msgr2=0x7ff8841a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff888a00700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff88410a6d0 0x7ff8841a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:05.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.357+0000 7ff888a00700 1 -- 172.21.15.79:0/522802146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8841a6930 con 0x7ff884108b30 2024-06-03T02:10:05.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.358+0000 7ff888a00700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff884108b30 0x7ff88419ece0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff87800e9c0 tx=0x7ff87800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:10:05.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.358+0000 7ff881600700 1 -- 172.21.15.79:0/522802146 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878004230 con 0x7ff884108b30 2024-06-03T02:10:05.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.358+0000 7ff881600700 1 -- 172.21.15.79:0/522802146 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8780043d0 con 0x7ff884108b30 2024-06-03T02:10:05.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.358+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8841a6c20 con 0x7ff884108b30 2024-06-03T02:10:05.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.358+0000 7ff881600700 1 -- 172.21.15.79:0/522802146 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878010430 con 0x7ff884108b30 2024-06-03T02:10:05.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.358+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8841a7160 con 0x7ff884108b30 2024-06-03T02:10:05.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.359+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff884069180 con 0x7ff884108b30 2024-06-03T02:10:05.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.361+0000 7ff881600700 1 -- 172.21.15.79:0/522802146 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff87800c7c0 con 0x7ff884108b30 2024-06-03T02:10:05.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.362+0000 7ff881600700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff86c0a2ca0 0x7ff86c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:10:05.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.362+0000 7ff881600700 1 -- 172.21.15.79:0/522802146 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4533+0+0 (secure 0 0 0) 0x7ff8780ce530 con 0x7ff884108b30 2024-06-03T02:10:05.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.362+0000 7ff883e00700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff86c0a2ca0 0x7ff86c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:10:05.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.363+0000 7ff883e00700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff86c0a2ca0 0x7ff86c0a5160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff870005d10 tx=0x7ff870009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:10:05.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.367+0000 7ff881600700 1 -- 172.21.15.79:0/522802146 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff87809c2b0 con 0x7ff884108b30 2024-06-03T02:10:05.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:05.601+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff8840628d0 con 0x7ff86c0a2ca0 2024-06-03T02:10:06.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:05 smithi123 ceph-mon[36640]: osdmap e44: 6 total, 6 up, 6 in 2024-06-03T02:10:06.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:05 smithi123 ceph-mon[36640]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-06-03T02:10:06.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:10:06.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:10:06.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:06.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:05 smithi079 ceph-mon[31819]: osdmap e44: 6 total, 6 up, 6 in 2024-06-03T02:10:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:05 smithi079 ceph-mon[31819]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-06-03T02:10:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:10:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:10:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:05 smithi135 ceph-mon[36653]: osdmap e44: 6 total, 6 up, 6 in 2024-06-03T02:10:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:05 smithi135 ceph-mon[36653]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-06-03T02:10:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:10:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:10:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:07.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:06 smithi123 ceph-mon[36640]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:10:07.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:06 smithi123 ceph-mon[36640]: pgmap v161: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-06-03T02:10:07.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:06 smithi079 ceph-mon[31819]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:10:07.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:06 smithi079 ceph-mon[31819]: pgmap v161: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-06-03T02:10:07.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:06 smithi135 ceph-mon[36653]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:10:07.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:06 smithi135 ceph-mon[36653]: pgmap v161: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-06-03T02:10:09.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:09 smithi123 ceph-mon[36640]: pgmap v162: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-06-03T02:10:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:09 smithi079 ceph-mon[31819]: pgmap v162: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-06-03T02:10:09.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:09 smithi135 ceph-mon[36653]: pgmap v162: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-06-03T02:10:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:10 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4006801428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eabb74df-81e5-4e78-81b7-0d925a8317e5"}]: dispatch 2024-06-03T02:10:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:10 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4006801428' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eabb74df-81e5-4e78-81b7-0d925a8317e5"}]': finished 2024-06-03T02:10:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:10 smithi123 ceph-mon[36640]: osdmap e45: 7 total, 6 up, 7 in 2024-06-03T02:10:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:10 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4006801428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eabb74df-81e5-4e78-81b7-0d925a8317e5"}]: dispatch 2024-06-03T02:10:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:10 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4006801428' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eabb74df-81e5-4e78-81b7-0d925a8317e5"}]': finished 2024-06-03T02:10:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:10 smithi079 ceph-mon[31819]: osdmap e45: 7 total, 6 up, 7 in 2024-06-03T02:10:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:10 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4006801428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eabb74df-81e5-4e78-81b7-0d925a8317e5"}]: dispatch 2024-06-03T02:10:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:10 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4006801428' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eabb74df-81e5-4e78-81b7-0d925a8317e5"}]': finished 2024-06-03T02:10:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:10 smithi135 ceph-mon[36653]: osdmap e45: 7 total, 6 up, 7 in 2024-06-03T02:10:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:11.443 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:11 smithi135 ceph-mon[36653]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-03T02:10:11.443 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:11 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1437804333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:10:11.443 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:11 smithi135 ceph-mon[36653]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-06-03T02:10:11.443 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:11 smithi135 ceph-mon[36653]: Cluster is now healthy 2024-06-03T02:10:11.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:11 smithi123 ceph-mon[36640]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-03T02:10:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:11 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1437804333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:10:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:11 smithi123 ceph-mon[36640]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-06-03T02:10:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:11 smithi123 ceph-mon[36640]: Cluster is now healthy 2024-06-03T02:10:11.489 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:11 smithi079 ceph-mon[31819]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-03T02:10:11.490 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:11 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1437804333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:10:11.490 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:11 smithi079 ceph-mon[31819]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-06-03T02:10:11.490 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:11 smithi079 ceph-mon[31819]: Cluster is now healthy 2024-06-03T02:10:13.397 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:13 smithi079 ceph-mon[31819]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T02:10:13.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:13 smithi123 ceph-mon[36640]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T02:10:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:13 smithi135 ceph-mon[36653]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T02:10:15.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:15 smithi123 ceph-mon[36640]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-03T02:10:15.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:15 smithi079 ceph-mon[31819]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-03T02:10:15.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:15 smithi135 ceph-mon[36653]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-03T02:10:17.471 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:17 smithi123 ceph-mon[36640]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:10:17.510 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:17 smithi079 ceph-mon[31819]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:10:17.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:17 smithi135 ceph-mon[36653]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:10:19.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:19 smithi123 ceph-mon[36640]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:10:19.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:19 smithi079 ceph-mon[31819]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:10:19.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:19 smithi135 ceph-mon[36653]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:10:20.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:20 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-03T02:10:20.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:20 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:20.502 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:20 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-03T02:10:20.502 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:20 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:20 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-03T02:10:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:20 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:21.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:21 smithi123 ceph-mon[36640]: Deploying daemon osd.6 on smithi079 2024-06-03T02:10:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:21 smithi123 ceph-mon[36640]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:21 smithi079 ceph-mon[31819]: Deploying daemon osd.6 on smithi079 2024-06-03T02:10:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:21 smithi079 ceph-mon[31819]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:21.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:21 smithi135 ceph-mon[36653]: Deploying daemon osd.6 on smithi079 2024-06-03T02:10:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:21 smithi135 ceph-mon[36653]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:23 smithi123 ceph-mon[36640]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:23 smithi079 ceph-mon[31819]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:23 smithi135 ceph-mon[36653]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:25.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:25 smithi123 ceph-mon[36640]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:25.503 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:25 smithi079 ceph-mon[31819]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:25 smithi135 ceph-mon[36653]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:27.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:27 smithi123 ceph-mon[36640]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:27.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:27 smithi079 ceph-mon[31819]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:27.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:27 smithi135 ceph-mon[36653]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:28.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:28 smithi079 ceph-mon[31819]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:28.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:28 smithi135 ceph-mon[36653]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:28 smithi123 ceph-mon[36640]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:29.451 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:29.451 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:10:29.496 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:29.496 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:10:29.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:29.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:10:30.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:30 smithi079 ceph-mon[31819]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:30.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:30 smithi135 ceph-mon[36653]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:30 smithi123 ceph-mon[36640]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:32.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.901+0000 7ff881600700 1 -- 172.21.15.79:0/522802146 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff8840628d0 con 0x7ff86c0a2ca0 2024-06-03T02:10:32.903 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 6 on host 'smithi079' 2024-06-03T02:10:32.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.905+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff86c0a2ca0 msgr2=0x7ff86c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:32.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.905+0000 7ff88b400700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff86c0a2ca0 0x7ff86c0a5160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff870005d10 tx=0x7ff870009310 comp rx=0 tx=0).stop 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.905+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff884108b30 msgr2=0x7ff88419ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.905+0000 7ff88b400700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff884108b30 0x7ff88419ece0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff87800e9c0 tx=0x7ff87800ecd0 comp rx=0 tx=0).stop 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.906+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 shutdown_connections 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.906+0000 7ff88b400700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff86c0a2ca0 0x7ff86c0a5160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.906+0000 7ff88b400700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff88410a6d0 0x7ff8841a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.906+0000 7ff88b400700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff884108b30 0x7ff88419ece0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.906+0000 7ff88b400700 1 --2- 172.21.15.79:0/522802146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff884109d30 0x7ff88419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.906+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 >> 172.21.15.79:0/522802146 conn(0x7ff884077b00 msgr2=0x7ff88410cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.906+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 shutdown_connections 2024-06-03T02:10:32.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:10:32.906+0000 7ff88b400700 1 -- 172.21.15.79:0/522802146 wait complete. 2024-06-03T02:10:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:33 smithi123 ceph-mon[36640]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:33 smithi123 ceph-mon[36640]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T02:10:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:33 smithi079 ceph-mon[31819]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:33.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:33 smithi079 ceph-mon[31819]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T02:10:33.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:33 smithi135 ceph-mon[36653]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:33 smithi135 ceph-mon[36653]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T02:10:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:33.852 DEBUG:teuthology.orchestra.run.smithi079:osd.6> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.6.service 2024-06-03T02:10:33.857 INFO:tasks.cephadm:Deploying osd.7 on smithi123 with /dev/vg_nvme/lv_2... 2024-06-03T02:10:33.857 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T02:10:34.101 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:10:34.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:34 smithi123 ceph-mon[36640]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-03T02:10:34.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:34 smithi123 ceph-mon[36640]: osdmap e46: 7 total, 6 up, 7 in 2024-06-03T02:10:34.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:34 smithi123 ceph-mon[36640]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:10:34.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:34 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:34.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:34 smithi123 ceph-mon[36640]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:10:34.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:34 smithi123 ceph-mon[36640]: osdmap e47: 7 total, 6 up, 7 in 2024-06-03T02:10:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:34 smithi079 ceph-mon[31819]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-03T02:10:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:34 smithi079 ceph-mon[31819]: osdmap e46: 7 total, 6 up, 7 in 2024-06-03T02:10:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:34 smithi079 ceph-mon[31819]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:10:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:34 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:34 smithi079 ceph-mon[31819]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:10:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:34 smithi079 ceph-mon[31819]: osdmap e47: 7 total, 6 up, 7 in 2024-06-03T02:10:34.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:10:34 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:10:34.175+0000 7f86f4000700 -1 osd.6 0 waiting for initial osdmap 2024-06-03T02:10:34.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:10:34 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:10:34.203+0000 7f86eee00700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:10:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:34 smithi135 ceph-mon[36653]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-03T02:10:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:34 smithi135 ceph-mon[36653]: osdmap e46: 7 total, 6 up, 7 in 2024-06-03T02:10:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:34 smithi135 ceph-mon[36653]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:10:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:34 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:34 smithi135 ceph-mon[36653]: from='osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:10:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:34 smithi135 ceph-mon[36653]: osdmap e47: 7 total, 6 up, 7 in 2024-06-03T02:10:35.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:35 smithi079 ceph-mon[31819]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:35.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:35.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:35.417 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:35 smithi123 ceph-mon[36640]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:35.417 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:35.418 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:35 smithi135 ceph-mon[36653]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:36.114 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T02:10:36.114 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T02:10:36.114 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-06-03T02:10:36.114 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-06-03T02:10:36.114 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126843 s, 827 MB/s 2024-06-03T02:10:36.114 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-06-03T02:10:36.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:36 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:10:36.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:36 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:10:36.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:36 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:36.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:36 smithi123 ceph-mon[36640]: osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] boot 2024-06-03T02:10:36.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:36 smithi123 ceph-mon[36640]: osdmap e48: 7 total, 7 up, 7 in 2024-06-03T02:10:36.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:36 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:36.574 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:36 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:10:36.574 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:36 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:10:36.574 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:36 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:36.574 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:36 smithi079 ceph-mon[31819]: osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] boot 2024-06-03T02:10:36.575 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:36 smithi079 ceph-mon[31819]: osdmap e48: 7 total, 7 up, 7 in 2024-06-03T02:10:36.575 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:36 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:36 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:10:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:36 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:10:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:36 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:36 smithi135 ceph-mon[36653]: osd.6 [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] boot 2024-06-03T02:10:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:36 smithi135 ceph-mon[36653]: osdmap e48: 7 total, 7 up, 7 in 2024-06-03T02:10:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:36 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T02:10:36.748 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-06-03T02:10:36.991 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:10:37.389 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:37 smithi123 ceph-mon[36640]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:37 smithi079 ceph-mon[31819]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:37.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:37 smithi135 ceph-mon[36653]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T02:10:38.377 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:38 smithi123 ceph-mon[36640]: osdmap e49: 7 total, 7 up, 7 in 2024-06-03T02:10:38.377 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:38 smithi123 ceph-mon[36640]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:38.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:38 smithi079 ceph-mon[31819]: osdmap e49: 7 total, 7 up, 7 in 2024-06-03T02:10:38.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:38 smithi079 ceph-mon[31819]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:38.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:38 smithi135 ceph-mon[36653]: osdmap e49: 7 total, 7 up, 7 in 2024-06-03T02:10:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:38 smithi135 ceph-mon[36653]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:38.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.625+0000 7fd78bfcb700 1 -- 172.21.15.123:0/1655474775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd784103720 msgr2=0x7fd784103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.625+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/1655474775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd784103720 0x7fd784103b80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd7800099c0 tx=0x7fd780009cd0 comp rx=0 tx=0).stop 2024-06-03T02:10:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.625+0000 7fd78bfcb700 1 -- 172.21.15.123:0/1655474775 shutdown_connections 2024-06-03T02:10:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.625+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/1655474775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7841040c0 0x7fd78410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.625+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/1655474775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd784102520 0x7fd784102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.625+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/1655474775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd784103720 0x7fd784103b80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.625+0000 7fd78bfcb700 1 -- 172.21.15.123:0/1655474775 >> 172.21.15.123:0/1655474775 conn(0x7fd7840fdae0 msgr2=0x7fd7840fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:10:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 -- 172.21.15.123:0/1655474775 shutdown_connections 2024-06-03T02:10:38.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 -- 172.21.15.123:0/1655474775 wait complete. 2024-06-03T02:10:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 Processor -- start 2024-06-03T02:10:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 -- start start 2024-06-03T02:10:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd784102520 0x7fd784198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:10:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd784103720 0x7fd784198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:10:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7841040c0 0x7fd78419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:10:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd78410e200 con 0x7fd784102520 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd78410e080 con 0x7fd7841040c0 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78bfcb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd78410e380 con 0x7fd784103720 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78a568700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7841040c0 0x7fd78419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78a568700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7841040c0 0x7fd78419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47458/0 (socket says 172.21.15.123:47458) 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.626+0000 7fd78a568700 1 -- 172.21.15.123:0/3136159099 learned_addr learned my addr 172.21.15.123:0/3136159099 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd789566700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd784103720 0x7fd784198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd789d67700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd784102520 0x7fd784198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd78a568700 1 -- 172.21.15.123:0/3136159099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd784103720 msgr2=0x7fd784198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd78a568700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd784103720 0x7fd784198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd78a568700 1 -- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd784102520 msgr2=0x7fd784198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd78a568700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd784102520 0x7fd784198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd78a568700 1 -- 172.21.15.123:0/3136159099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7841a04b0 con 0x7fd7841040c0 2024-06-03T02:10:38.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd78a568700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7841040c0 0x7fd78419fe10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00b970 tx=0x7fd77c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:10:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd77affd700 1 -- 172.21.15.123:0/3136159099 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c019070 con 0x7fd7841040c0 2024-06-03T02:10:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7841a07a0 con 0x7fd7841040c0 2024-06-03T02:10:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd77affd700 1 -- 172.21.15.123:0/3136159099 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77c014440 con 0x7fd7841040c0 2024-06-03T02:10:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd77affd700 1 -- 172.21.15.123:0/3136159099 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0075d0 con 0x7fd7841040c0 2024-06-03T02:10:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.627+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7841a0d10 con 0x7fd7841040c0 2024-06-03T02:10:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.628+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd784069180 con 0x7fd7841040c0 2024-06-03T02:10:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.630+0000 7fd77affd700 1 -- 172.21.15.123:0/3136159099 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd77c01c070 con 0x7fd7841040c0 2024-06-03T02:10:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.630+0000 7fd77affd700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7700a2c20 0x7fd7700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:10:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.631+0000 7fd77affd700 1 -- 172.21.15.123:0/3136159099 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7fd77c0cdbc0 con 0x7fd7841040c0 2024-06-03T02:10:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.631+0000 7fd789d67700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7700a2c20 0x7fd7700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:10:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.631+0000 7fd789d67700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7700a2c20 0x7fd7700a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd774006fd0 tx=0x7fd774008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:10:38.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.634+0000 7fd77affd700 1 -- 172.21.15.123:0/3136159099 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd77c09b830 con 0x7fd7841040c0 2024-06-03T02:10:38.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:38.837+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd7840628d0 con 0x7fd7700a2c20 2024-06-03T02:10:39.459 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:39 smithi123 ceph-mon[36640]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:10:39.459 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:39 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:10:39.459 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:39 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:10:39.459 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:39 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:39 smithi079 ceph-mon[31819]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:10:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:10:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:10:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:39 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:39.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:39 smithi135 ceph-mon[36653]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:10:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:39 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:10:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:39 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:10:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:39 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi079 2024-06-03T02:10:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi079 to 5696M 2024-06-03T02:10:41.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:41.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:10:41.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:41.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:10:41.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:41.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:10:41.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi079 2024-06-03T02:10:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi079 to 5696M 2024-06-03T02:10:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:41.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:10:41.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:41.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:10:41.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:41.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:10:41.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi079 2024-06-03T02:10:41.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:41.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi079 to 5696M 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:10:41.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:42.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:41 smithi123 ceph-mon[36640]: from='client.? 172.21.15.123:0/3108036895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed"}]: dispatch 2024-06-03T02:10:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:41 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed"}]: dispatch 2024-06-03T02:10:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:41 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed"}]': finished 2024-06-03T02:10:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:41 smithi123 ceph-mon[36640]: osdmap e50: 8 total, 7 up, 8 in 2024-06-03T02:10:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:41 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:10:42.239 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:41 smithi079 ceph-mon[31819]: from='client.? 172.21.15.123:0/3108036895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed"}]: dispatch 2024-06-03T02:10:42.239 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:41 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed"}]: dispatch 2024-06-03T02:10:42.239 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:41 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed"}]': finished 2024-06-03T02:10:42.240 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:41 smithi079 ceph-mon[31819]: osdmap e50: 8 total, 7 up, 8 in 2024-06-03T02:10:42.240 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:41 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:10:42.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:41 smithi135 ceph-mon[36653]: from='client.? 172.21.15.123:0/3108036895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed"}]: dispatch 2024-06-03T02:10:42.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:41 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed"}]: dispatch 2024-06-03T02:10:42.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:41 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed"}]': finished 2024-06-03T02:10:42.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:41 smithi135 ceph-mon[36653]: osdmap e50: 8 total, 7 up, 8 in 2024-06-03T02:10:42.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:41 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:10:43.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:42 smithi079 ceph-mon[31819]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:43.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:42 smithi079 ceph-mon[31819]: from='client.? 172.21.15.123:0/2342373387' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:10:43.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:42 smithi135 ceph-mon[36653]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:43.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:42 smithi135 ceph-mon[36653]: from='client.? 172.21.15.123:0/2342373387' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:10:43.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:42 smithi123 ceph-mon[36640]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:43.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:42 smithi123 ceph-mon[36640]: from='client.? 172.21.15.123:0/2342373387' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:10:45.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:45 smithi123 ceph-mon[36640]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:45 smithi079 ceph-mon[31819]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:45 smithi135 ceph-mon[36653]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:47.451 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:47 smithi123 ceph-mon[36640]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:47.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:47 smithi079 ceph-mon[31819]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:47.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:47 smithi135 ceph-mon[36653]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:49.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:49 smithi123 ceph-mon[36640]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:49.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:49 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-03T02:10:49.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:49 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:49.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:49 smithi079 ceph-mon[31819]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:49 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-03T02:10:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:49 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:49 smithi135 ceph-mon[36653]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:49 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-03T02:10:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:49 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:10:50.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:50 smithi123 ceph-mon[36640]: Deploying daemon osd.7 on smithi123 2024-06-03T02:10:50.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:50 smithi079 ceph-mon[31819]: Deploying daemon osd.7 on smithi123 2024-06-03T02:10:50.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:50 smithi135 ceph-mon[36653]: Deploying daemon osd.7 on smithi123 2024-06-03T02:10:51.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:51 smithi123 ceph-mon[36640]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:51.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:51 smithi079 ceph-mon[31819]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:51 smithi135 ceph-mon[36653]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:53.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:53 smithi123 ceph-mon[36640]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:53 smithi079 ceph-mon[31819]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:53 smithi135 ceph-mon[36653]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:55.397 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:55 smithi123 ceph-mon[36640]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:55.398 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:55 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:55.398 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:55 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:55.398 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:55 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:10:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:55 smithi079 ceph-mon[31819]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:55 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:55 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:55 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:10:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:55 smithi135 ceph-mon[36653]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:55 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:55 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:55 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:10:57.467 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:57 smithi123 ceph-mon[36640]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:57.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:57 smithi079 ceph-mon[31819]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:57.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:57 smithi135 ceph-mon[36653]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:58.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.371+0000 7fd77affd700 1 -- 172.21.15.123:0/3136159099 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd7840628d0 con 0x7fd7700a2c20 2024-06-03T02:10:58.372 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 7 on host 'smithi123' 2024-06-03T02:10:58.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7700a2c20 msgr2=0x7fd7700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7700a2c20 0x7fd7700a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd774006fd0 tx=0x7fd774008040 comp rx=0 tx=0).stop 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7841040c0 msgr2=0x7fd78419fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7841040c0 0x7fd78419fe10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00b970 tx=0x7fd77c00bd30 comp rx=0 tx=0).stop 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 shutdown_connections 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7841040c0 0x7fd78419fe10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7700a2c20 0x7fd7700a50e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd784102520 0x7fd784198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 --2- 172.21.15.123:0/3136159099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd784103720 0x7fd784198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 >> 172.21.15.123:0/3136159099 conn(0x7fd7840fdae0 msgr2=0x7fd784106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 shutdown_connections 2024-06-03T02:10:58.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:10:58.373+0000 7fd78bfcb700 1 -- 172.21.15.123:0/3136159099 wait complete. 2024-06-03T02:10:59.020 DEBUG:teuthology.orchestra.run.smithi123:osd.7> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.7.service 2024-06-03T02:10:59.023 INFO:tasks.cephadm:Deploying osd.8 on smithi135 with /dev/vg_nvme/lv_2... 2024-06-03T02:10:59.023 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T02:10:59.271 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:10:59.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:59 smithi135 ceph-mon[36653]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:59.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:59 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:59.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:59 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:59.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:59 smithi135 ceph-mon[36653]: from='osd.7 [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T02:10:59.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:10:59 smithi135 ceph-mon[36653]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T02:10:59.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:59 smithi123 ceph-mon[36640]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:59 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:59 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:59 smithi123 ceph-mon[36640]: from='osd.7 [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T02:10:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:10:59 smithi123 ceph-mon[36640]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T02:10:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:59 smithi079 ceph-mon[31819]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:10:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:59 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:59 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:10:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:59 smithi079 ceph-mon[31819]: from='osd.7 [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T02:10:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:10:59 smithi079 ceph-mon[31819]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T02:11:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-mon[36640]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-03T02:11:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-mon[36640]: from='osd.7 [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:11:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-mon[36640]: osdmap e51: 8 total, 7 up, 8 in 2024-06-03T02:11:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-mon[36640]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:11:00.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-mon[36640]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:11:00.474 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:11:00.366+0000 7fa1d3c24700 -1 osd.7 0 waiting for initial osdmap 2024-06-03T02:11:00.474 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:11:00 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:11:00.375+0000 7fa1cfa17700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:11:00.474 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:00.475 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:00.475 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:00 smithi135 ceph-mon[36653]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-03T02:11:00.475 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:00 smithi135 ceph-mon[36653]: from='osd.7 [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:11:00.475 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:00 smithi135 ceph-mon[36653]: osdmap e51: 8 total, 7 up, 8 in 2024-06-03T02:11:00.475 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:00.475 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:00 smithi135 ceph-mon[36653]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:11:00.475 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:00 smithi135 ceph-mon[36653]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:11:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:00 smithi079 ceph-mon[31819]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-03T02:11:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:00 smithi079 ceph-mon[31819]: from='osd.7 [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:11:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:00 smithi079 ceph-mon[31819]: osdmap e51: 8 total, 7 up, 8 in 2024-06-03T02:11:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:00 smithi079 ceph-mon[31819]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:11:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:00 smithi079 ceph-mon[31819]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:11:01.378 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T02:11:01.378 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T02:11:01.378 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-03T02:11:01.378 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-03T02:11:01.378 INFO:teuthology.orchestra.run.smithi135.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116756 s, 898 MB/s 2024-06-03T02:11:01.378 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-03T02:11:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:01 smithi123 ceph-mon[36640]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:11:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:01 smithi123 ceph-mon[36640]: osdmap e52: 8 total, 7 up, 8 in 2024-06-03T02:11:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:01.708 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:01 smithi135 ceph-mon[36653]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:11:01.709 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:01 smithi135 ceph-mon[36653]: osdmap e52: 8 total, 7 up, 8 in 2024-06-03T02:11:01.709 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:01.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:01 smithi079 ceph-mon[31819]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:11:01.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:01 smithi079 ceph-mon[31819]: osdmap e52: 8 total, 7 up, 8 in 2024-06-03T02:11:01.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:02.037 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_2 2024-06-03T02:11:02.278 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:11:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:02 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:11:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:02 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:11:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:02 smithi135 ceph-mon[36653]: osd.7 [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] boot 2024-06-03T02:11:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:02 smithi135 ceph-mon[36653]: osdmap e53: 8 total, 8 up, 8 in 2024-06-03T02:11:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:02 smithi135 ceph-mon[36653]: pgmap v198: 1 pgs: 1 remapped; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:11:02.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:02 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:11:02.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:02 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:11:02.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:02.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:02 smithi123 ceph-mon[36640]: osd.7 [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] boot 2024-06-03T02:11:02.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:02 smithi123 ceph-mon[36640]: osdmap e53: 8 total, 8 up, 8 in 2024-06-03T02:11:02.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:02.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:02 smithi123 ceph-mon[36640]: pgmap v198: 1 pgs: 1 remapped; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:11:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:02 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:11:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:02 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:11:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:02 smithi079 ceph-mon[31819]: osd.7 [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] boot 2024-06-03T02:11:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:02 smithi079 ceph-mon[31819]: osdmap e53: 8 total, 8 up, 8 in 2024-06-03T02:11:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T02:11:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:02 smithi079 ceph-mon[31819]: pgmap v198: 1 pgs: 1 remapped; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-06-03T02:11:03.453 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-03T02:11:03.453 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: osdmap e54: 8 total, 8 up, 8 in 2024-06-03T02:11:03.453 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi123 2024-06-03T02:11:03.453 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.453 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.453 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:03.453 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi123 to 5696M 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:03.454 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-03T02:11:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: osdmap e54: 8 total, 8 up, 8 in 2024-06-03T02:11:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi123 2024-06-03T02:11:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:03.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi123 to 5696M 2024-06-03T02:11:03.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:03.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:03.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:03.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:03.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:03.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-03T02:11:03.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: osdmap e54: 8 total, 8 up, 8 in 2024-06-03T02:11:03.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi123 2024-06-03T02:11:03.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi123 to 5696M 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:03.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.977+0000 7fbef3799700 1 -- 172.21.15.135:0/3932507855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec1040c0 msgr2=0x7fbeec10a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:03.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.977+0000 7fbef3799700 1 --2- 172.21.15.135:0/3932507855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec1040c0 0x7fbeec10a970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbee400b0a0 tx=0x7fbee400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:11:03.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.977+0000 7fbef3799700 1 -- 172.21.15.135:0/3932507855 shutdown_connections 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.977+0000 7fbef3799700 1 --2- 172.21.15.135:0/3932507855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec1040c0 0x7fbeec10a970 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.977+0000 7fbef3799700 1 --2- 172.21.15.135:0/3932507855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec102520 0x7fbeec102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.977+0000 7fbef3799700 1 --2- 172.21.15.135:0/3932507855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec103720 0x7fbeec103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.977+0000 7fbef3799700 1 -- 172.21.15.135:0/3932507855 >> 172.21.15.135:0/3932507855 conn(0x7fbeec0fdae0 msgr2=0x7fbeec0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.978+0000 7fbef3799700 1 -- 172.21.15.135:0/3932507855 shutdown_connections 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.978+0000 7fbef3799700 1 -- 172.21.15.135:0/3932507855 wait complete. 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.978+0000 7fbef3799700 1 Processor -- start 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.978+0000 7fbef3799700 1 -- start start 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.978+0000 7fbef3799700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec102520 0x7fbeec1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef3799700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec103720 0x7fbeec198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef3799700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec1040c0 0x7fbeec19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:11:03.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef3799700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10e160 con 0x7fbeec103720 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef3799700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10dfe0 con 0x7fbeec1040c0 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef3799700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10e2e0 con 0x7fbeec102520 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1535700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec102520 0x7fbeec1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1535700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec102520 0x7fbeec1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41480/0 (socket says 172.21.15.135:41480) 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1535700 1 -- 172.21.15.135:0/3221473430 learned_addr learned my addr 172.21.15.135:0/3221473430 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1d36700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec1040c0 0x7fbeec19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1535700 1 -- 172.21.15.135:0/3221473430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec1040c0 msgr2=0x7fbeec19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1535700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec1040c0 0x7fbeec19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1535700 1 -- 172.21.15.135:0/3221473430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec103720 msgr2=0x7fbeec198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1535700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec103720 0x7fbeec198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1535700 1 -- 172.21.15.135:0/3221473430 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec1a0310 con 0x7fbeec102520 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef1535700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec102520 0x7fbeec1986b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbee800d9b0 tx=0x7fbee800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:11:03.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbee27fc700 1 -- 172.21.15.135:0/3221473430 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8017030 con 0x7fbeec102520 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbee27fc700 1 -- 172.21.15.135:0/3221473430 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee8004500 con 0x7fbeec102520 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec1a0600 con 0x7fbeec102520 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbee27fc700 1 -- 172.21.15.135:0/3221473430 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8003aa0 con 0x7fbeec102520 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.979+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec1a0c10 con 0x7fbeec102520 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.980+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec069180 con 0x7fbeec102520 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.981+0000 7fbee27fc700 1 -- 172.21.15.135:0/3221473430 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbee801a070 con 0x7fbeec102520 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.982+0000 7fbee27fc700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbed80a2ca0 0x7fbed80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.982+0000 7fbee27fc700 1 -- 172.21.15.135:0/3221473430 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7fbee80ce160 con 0x7fbeec102520 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.983+0000 7fbef0d34700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbed80a2ca0 0x7fbed80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.983+0000 7fbef0d34700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbed80a2ca0 0x7fbed80a5160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0097b0 tx=0x7fbedc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:11:03.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:03.986+0000 7fbee27fc700 1 -- 172.21.15.135:0/3221473430 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbee800e350 con 0x7fbeec102520 2024-06-03T02:11:04.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:04.192+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbeec0628d0 con 0x7fbed80a2ca0 2024-06-03T02:11:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:04 smithi123 ceph-mon[36640]: osdmap e55: 8 total, 8 up, 8 in 2024-06-03T02:11:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:04 smithi123 ceph-mon[36640]: pgmap v201: 1 pgs: 1 remapped; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:11:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:11:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:04.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:04 smithi079 ceph-mon[31819]: osdmap e55: 8 total, 8 up, 8 in 2024-06-03T02:11:04.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:04 smithi079 ceph-mon[31819]: pgmap v201: 1 pgs: 1 remapped; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:04.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:04 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:11:04.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:04 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:11:04.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:04 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:04.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:04 smithi135 ceph-mon[36653]: osdmap e55: 8 total, 8 up, 8 in 2024-06-03T02:11:04.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:04 smithi135 ceph-mon[36653]: pgmap v201: 1 pgs: 1 remapped; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:04.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:04 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:11:04.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:04 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:11:04.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:04 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:05 smithi123 ceph-mon[36640]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:11:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:05 smithi123 ceph-mon[36640]: osdmap e56: 8 total, 8 up, 8 in 2024-06-03T02:11:05.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:05 smithi079 ceph-mon[31819]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:11:05.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:05 smithi079 ceph-mon[31819]: osdmap e56: 8 total, 8 up, 8 in 2024-06-03T02:11:05.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:05 smithi135 ceph-mon[36653]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:11:05.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:05 smithi135 ceph-mon[36653]: osdmap e56: 8 total, 8 up, 8 in 2024-06-03T02:11:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:06 smithi135 ceph-mon[36653]: osdmap e57: 8 total, 8 up, 8 in 2024-06-03T02:11:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:06 smithi135 ceph-mon[36653]: pgmap v204: 1 pgs: 1 remapped; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:06.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:06 smithi123 ceph-mon[36640]: osdmap e57: 8 total, 8 up, 8 in 2024-06-03T02:11:06.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:06 smithi123 ceph-mon[36640]: pgmap v204: 1 pgs: 1 remapped; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:06.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:06 smithi079 ceph-mon[31819]: osdmap e57: 8 total, 8 up, 8 in 2024-06-03T02:11:06.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:06 smithi079 ceph-mon[31819]: pgmap v204: 1 pgs: 1 remapped; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:07 smithi123 ceph-mon[36640]: from='client.? 172.21.15.135:0/282535992' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd056a07-1f3b-450a-8da4-9a8c8edf22e2"}]: dispatch 2024-06-03T02:11:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:07 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd056a07-1f3b-450a-8da4-9a8c8edf22e2"}]: dispatch 2024-06-03T02:11:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:07 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd056a07-1f3b-450a-8da4-9a8c8edf22e2"}]': finished 2024-06-03T02:11:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:07 smithi123 ceph-mon[36640]: osdmap e58: 9 total, 8 up, 9 in 2024-06-03T02:11:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:07.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:07 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/282535992' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd056a07-1f3b-450a-8da4-9a8c8edf22e2"}]: dispatch 2024-06-03T02:11:07.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:07 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd056a07-1f3b-450a-8da4-9a8c8edf22e2"}]: dispatch 2024-06-03T02:11:07.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:07 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd056a07-1f3b-450a-8da4-9a8c8edf22e2"}]': finished 2024-06-03T02:11:07.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:07 smithi079 ceph-mon[31819]: osdmap e58: 9 total, 8 up, 9 in 2024-06-03T02:11:07.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:07.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:07 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/282535992' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd056a07-1f3b-450a-8da4-9a8c8edf22e2"}]: dispatch 2024-06-03T02:11:07.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:07 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd056a07-1f3b-450a-8da4-9a8c8edf22e2"}]: dispatch 2024-06-03T02:11:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:07 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd056a07-1f3b-450a-8da4-9a8c8edf22e2"}]': finished 2024-06-03T02:11:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:07 smithi135 ceph-mon[36653]: osdmap e58: 9 total, 8 up, 9 in 2024-06-03T02:11:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:08.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:08 smithi123 ceph-mon[36640]: from='client.? 172.21.15.135:0/3780208545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:11:08.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:08 smithi123 ceph-mon[36640]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:08.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:08 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/3780208545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:11:08.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:08 smithi079 ceph-mon[31819]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:08.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:08 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/3780208545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:11:08.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:08 smithi135 ceph-mon[36653]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:09.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:09 smithi123 ceph-mon[36640]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-03T02:11:09.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:09 smithi123 ceph-mon[36640]: Cluster is now healthy 2024-06-03T02:11:09.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:09 smithi079 ceph-mon[31819]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-03T02:11:09.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:09 smithi079 ceph-mon[31819]: Cluster is now healthy 2024-06-03T02:11:09.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:09 smithi135 ceph-mon[36653]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-03T02:11:09.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:09 smithi135 ceph-mon[36653]: Cluster is now healthy 2024-06-03T02:11:10.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:10 smithi123 ceph-mon[36640]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:10.736 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:10 smithi135 ceph-mon[36653]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:10.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:10 smithi079 ceph-mon[31819]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:13.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:13 smithi135 ceph-mon[36653]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:13.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:13 smithi123 ceph-mon[36640]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:13.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:13 smithi079 ceph-mon[31819]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:15.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:15 smithi123 ceph-mon[36640]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:15.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-03T02:11:15.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:15.519 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:15 smithi135 ceph-mon[36653]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:15.519 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-03T02:11:15.519 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:15.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:15 smithi079 ceph-mon[31819]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:15.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-03T02:11:15.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:16.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:16 smithi123 ceph-mon[36640]: Deploying daemon osd.8 on smithi135 2024-06-03T02:11:16.516 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:16 smithi135 ceph-mon[36653]: Deploying daemon osd.8 on smithi135 2024-06-03T02:11:16.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:16 smithi079 ceph-mon[31819]: Deploying daemon osd.8 on smithi135 2024-06-03T02:11:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:17 smithi123 ceph-mon[36640]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:17.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:17 smithi135 ceph-mon[36653]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:17.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:17 smithi079 ceph-mon[31819]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:19.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:19 smithi135 ceph-mon[36653]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:19.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:19 smithi123 ceph-mon[36640]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:19.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:19 smithi079 ceph-mon[31819]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:21.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:21 smithi135 ceph-mon[36653]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:21.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:21.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:21.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:21 smithi123 ceph-mon[36640]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:21 smithi079 ceph-mon[31819]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:23.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:23 smithi135 ceph-mon[36653]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:23 smithi123 ceph-mon[36640]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:23.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:23 smithi079 ceph-mon[31819]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:23.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.831+0000 7fbee27fc700 1 -- 172.21.15.135:0/3221473430 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbeec0628d0 con 0x7fbed80a2ca0 2024-06-03T02:11:23.831 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 8 on host 'smithi135' 2024-06-03T02:11:23.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbed80a2ca0 msgr2=0x7fbed80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:23.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbed80a2ca0 0x7fbed80a5160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0097b0 tx=0x7fbedc006d20 comp rx=0 tx=0).stop 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec102520 msgr2=0x7fbeec1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec102520 0x7fbeec1986b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbee800d9b0 tx=0x7fbee800dcc0 comp rx=0 tx=0).stop 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 shutdown_connections 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec1040c0 0x7fbeec19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbeec102520 0x7fbeec1986b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbed80a2ca0 0x7fbed80a5160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 --2- 172.21.15.135:0/3221473430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbeec103720 0x7fbeec198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 >> 172.21.15.135:0/3221473430 conn(0x7fbeec0fdae0 msgr2=0x7fbeec1069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 shutdown_connections 2024-06-03T02:11:23.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:11:23.834+0000 7fbef3799700 1 -- 172.21.15.135:0/3221473430 wait complete. 2024-06-03T02:11:24.527 DEBUG:teuthology.orchestra.run.smithi135:osd.8> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.8.service 2024-06-03T02:11:24.529 INFO:tasks.cephadm:Deploying osd.9 on smithi079 with /dev/vg_nvme/lv_1... 2024-06-03T02:11:24.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-03T02:11:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:11:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:24 smithi079 ceph-mon[31819]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:24 smithi079 ceph-mon[31819]: from='osd.8 [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T02:11:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:24 smithi079 ceph-mon[31819]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T02:11:25.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:24 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:24 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:24 smithi135 ceph-mon[36653]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:24 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:24 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:24 smithi135 ceph-mon[36653]: from='osd.8 [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T02:11:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:24 smithi135 ceph-mon[36653]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T02:11:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:24 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:24 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:24 smithi123 ceph-mon[36640]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:24 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:24 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:24 smithi123 ceph-mon[36640]: from='osd.8 [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T02:11:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:24 smithi123 ceph-mon[36640]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T02:11:26.237 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:26 smithi123 ceph-mon[36640]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-03T02:11:26.237 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:26 smithi123 ceph-mon[36640]: from='osd.8 [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:11:26.237 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:26 smithi123 ceph-mon[36640]: osdmap e59: 9 total, 8 up, 9 in 2024-06-03T02:11:26.237 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:26.238 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:26 smithi123 ceph-mon[36640]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:11:26.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:26 smithi079 ceph-mon[31819]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-03T02:11:26.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:26 smithi079 ceph-mon[31819]: from='osd.8 [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:11:26.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:26 smithi079 ceph-mon[31819]: osdmap e59: 9 total, 8 up, 9 in 2024-06-03T02:11:26.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:26.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:26 smithi079 ceph-mon[31819]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:11:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:26 smithi135 ceph-mon[36653]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-03T02:11:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:26 smithi135 ceph-mon[36653]: from='osd.8 [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:11:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:26 smithi135 ceph-mon[36653]: osdmap e59: 9 total, 8 up, 9 in 2024-06-03T02:11:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:26 smithi135 ceph-mon[36653]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:11:26.345 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:11:26 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:11:26.143+0000 7eff9b778700 -1 osd.8 0 waiting for initial osdmap 2024-06-03T02:11:26.346 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:11:26 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:11:26.154+0000 7eff9756b700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:11:27.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:27 smithi135 ceph-mon[36653]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:27.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:27 smithi135 ceph-mon[36653]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:11:27.265 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:27 smithi135 ceph-mon[36653]: osdmap e60: 9 total, 8 up, 9 in 2024-06-03T02:11:27.265 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:27 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:27.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:27 smithi079 ceph-mon[31819]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:27.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:27 smithi079 ceph-mon[31819]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:11:27.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:27 smithi079 ceph-mon[31819]: osdmap e60: 9 total, 8 up, 9 in 2024-06-03T02:11:27.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:27.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:27 smithi123 ceph-mon[36640]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-06-03T02:11:27.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:27 smithi123 ceph-mon[36640]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:11:27.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:27 smithi123 ceph-mon[36640]: osdmap e60: 9 total, 8 up, 9 in 2024-06-03T02:11:27.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:27 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:28.029 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-03T02:11:28.029 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-03T02:11:28.030 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-06-03T02:11:28.030 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-06-03T02:11:28.030 INFO:teuthology.orchestra.run.smithi079.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0356064 s, 294 MB/s 2024-06-03T02:11:28.030 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-06-03T02:11:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:11:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:11:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: osd.8 [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] boot 2024-06-03T02:11:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: osdmap e61: 9 total, 9 up, 9 in 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:28 smithi079 ceph-mon[31819]: osdmap e62: 9 total, 9 up, 9 in 2024-06-03T02:11:28.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:11:28.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:11:28.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: osd.8 [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] boot 2024-06-03T02:11:28.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: osdmap e61: 9 total, 9 up, 9 in 2024-06-03T02:11:28.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:28.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:28 smithi123 ceph-mon[36640]: osdmap e62: 9 total, 9 up, 9 in 2024-06-03T02:11:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:11:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:11:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: osd.8 [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] boot 2024-06-03T02:11:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: osdmap e61: 9 total, 9 up, 9 in 2024-06-03T02:11:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T02:11:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:28 smithi135 ceph-mon[36653]: osdmap e62: 9 total, 9 up, 9 in 2024-06-03T02:11:28.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_1 2024-06-03T02:11:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:11:29.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:29 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi135 2024-06-03T02:11:29.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:29 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi135 to 5696M 2024-06-03T02:11:29.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:29 smithi079 ceph-mon[31819]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:29.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:29 smithi079 ceph-mon[31819]: osdmap e63: 9 total, 9 up, 9 in 2024-06-03T02:11:29.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:29 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi135 2024-06-03T02:11:29.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:29 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi135 to 5696M 2024-06-03T02:11:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:29 smithi123 ceph-mon[36640]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:29 smithi123 ceph-mon[36640]: osdmap e63: 9 total, 9 up, 9 in 2024-06-03T02:11:29.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:29 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi135 2024-06-03T02:11:29.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:29 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi135 to 5696M 2024-06-03T02:11:29.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:29 smithi135 ceph-mon[36653]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:29.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:29 smithi135 ceph-mon[36653]: osdmap e63: 9 total, 9 up, 9 in 2024-06-03T02:11:31.042 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:30 smithi079 ceph-mon[31819]: osdmap e64: 9 total, 9 up, 9 in 2024-06-03T02:11:31.042 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:30 smithi079 ceph-mon[31819]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:31.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:30 smithi135 ceph-mon[36653]: osdmap e64: 9 total, 9 up, 9 in 2024-06-03T02:11:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:30 smithi135 ceph-mon[36653]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:31.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:30 smithi123 ceph-mon[36640]: osdmap e64: 9 total, 9 up, 9 in 2024-06-03T02:11:31.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:30 smithi123 ceph-mon[36640]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:31.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.585+0000 7f242e400700 1 -- 172.21.15.79:0/4014356230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2428104740 msgr2=0x7f2428104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:31.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.585+0000 7f242e400700 1 --2- 172.21.15.79:0/4014356230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2428104740 0x7f2428104b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2418009990 tx=0x7f2418009ca0 comp rx=0 tx=0).stop 2024-06-03T02:11:31.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.585+0000 7f242e400700 1 -- 172.21.15.79:0/4014356230 shutdown_connections 2024-06-03T02:11:31.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.585+0000 7f242e400700 1 --2- 172.21.15.79:0/4014356230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24281062e0 0x7f242810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:31.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.585+0000 7f242e400700 1 --2- 172.21.15.79:0/4014356230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2428104740 0x7f2428104b60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:31.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.585+0000 7f242e400700 1 --2- 172.21.15.79:0/4014356230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2428105940 0x7f2428105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:31.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.585+0000 7f242e400700 1 -- 172.21.15.79:0/4014356230 >> 172.21.15.79:0/4014356230 conn(0x7f24280ffce0 msgr2=0x7f2428102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:11:31.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.586+0000 7f242e400700 1 -- 172.21.15.79:0/4014356230 shutdown_connections 2024-06-03T02:11:31.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.586+0000 7f242e400700 1 -- 172.21.15.79:0/4014356230 wait complete. 2024-06-03T02:11:31.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.587+0000 7f242e400700 1 Processor -- start 2024-06-03T02:11:31.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.588+0000 7f242e400700 1 -- start start 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.588+0000 7f242e400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2428104740 0x7f242819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.588+0000 7f242e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428105940 0x7f242819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.588+0000 7f242e400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24281062e0 0x7f24281a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.588+0000 7f242e400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2428110370 con 0x7f2428104740 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.588+0000 7f242e400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24281101f0 con 0x7f2428105940 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.588+0000 7f242e400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24281104f0 con 0x7f24281062e0 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2426a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428105940 0x7f242819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2427e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24281062e0 0x7f24281a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2426a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428105940 0x7f242819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43934/0 (socket says 172.21.15.79:43934) 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2426a00700 1 -- 172.21.15.79:0/963849001 learned_addr learned my addr 172.21.15.79:0/963849001 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:11:31.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2427400700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2428104740 0x7f242819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:11:31.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2427e00700 1 -- 172.21.15.79:0/963849001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428105940 msgr2=0x7f242819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:31.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2427e00700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428105940 0x7f242819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2427e00700 1 -- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2428104740 msgr2=0x7f242819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2427e00700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2428104740 0x7f242819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2427e00700 1 -- 172.21.15.79:0/963849001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24281a25e0 con 0x7f24281062e0 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2426a00700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428105940 0x7f242819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.589+0000 7f2427e00700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24281062e0 0x7f24281a1f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2420009f60 tx=0x7f2420009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.590+0000 7f241fe00700 1 -- 172.21.15.79:0/963849001 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24200092e0 con 0x7f24281062e0 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.590+0000 7f241fe00700 1 -- 172.21.15.79:0/963849001 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2420004bb0 con 0x7f24281062e0 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.590+0000 7f241fe00700 1 -- 172.21.15.79:0/963849001 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420004280 con 0x7f24281062e0 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.590+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24281a2870 con 0x7f24281062e0 2024-06-03T02:11:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.590+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24281a2d60 con 0x7f24281062e0 2024-06-03T02:11:31.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.591+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2428069180 con 0x7f24281062e0 2024-06-03T02:11:31.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.593+0000 7f241fe00700 1 -- 172.21.15.79:0/963849001 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f242001b030 con 0x7f24281062e0 2024-06-03T02:11:31.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.595+0000 7f241fe00700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f240c0a2ca0 0x7f240c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:11:31.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.596+0000 7f241fe00700 1 -- 172.21.15.79:0/963849001 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5829+0+0 (secure 0 0 0) 0x7f24200ceac0 con 0x7f24281062e0 2024-06-03T02:11:31.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.596+0000 7f2427400700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f240c0a2ca0 0x7f240c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:11:31.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.596+0000 7f2427400700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f240c0a2ca0 0x7f240c0a5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f241800b820 tx=0x7f2418005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:11:31.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.597+0000 7f241fe00700 1 -- 172.21.15.79:0/963849001 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2420022020 con 0x7f24281062e0 2024-06-03T02:11:31.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:31.831+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f24280628d0 con 0x7f240c0a2ca0 2024-06-03T02:11:32.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:31 smithi079 ceph-mon[31819]: osdmap e65: 9 total, 9 up, 9 in 2024-06-03T02:11:32.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:31 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:11:32.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:31 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:11:32.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:31 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:31 smithi135 ceph-mon[36653]: osdmap e65: 9 total, 9 up, 9 in 2024-06-03T02:11:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:31 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:11:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:31 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:11:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:31 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:32.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:31 smithi123 ceph-mon[36640]: osdmap e65: 9 total, 9 up, 9 in 2024-06-03T02:11:32.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:31 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:11:32.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:31 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:11:32.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:31 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:33.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:32 smithi079 ceph-mon[31819]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:11:33.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:32 smithi079 ceph-mon[31819]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:33.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:32 smithi123 ceph-mon[36640]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:11:33.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:32 smithi123 ceph-mon[36640]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:33.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:32 smithi135 ceph-mon[36653]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:11:33.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:32 smithi135 ceph-mon[36653]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:35 smithi123 ceph-mon[36640]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-03T02:11:35.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:35 smithi079 ceph-mon[31819]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-03T02:11:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:35 smithi135 ceph-mon[36653]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-03T02:11:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:36 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2490373922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbad302f-a304-4c5b-96c0-3e619ba42f9d"}]: dispatch 2024-06-03T02:11:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:36 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2490373922' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbad302f-a304-4c5b-96c0-3e619ba42f9d"}]': finished 2024-06-03T02:11:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:36 smithi123 ceph-mon[36640]: osdmap e66: 10 total, 9 up, 10 in 2024-06-03T02:11:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:36 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:11:36.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:36 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2490373922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbad302f-a304-4c5b-96c0-3e619ba42f9d"}]: dispatch 2024-06-03T02:11:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:36 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2490373922' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbad302f-a304-4c5b-96c0-3e619ba42f9d"}]': finished 2024-06-03T02:11:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:36 smithi079 ceph-mon[31819]: osdmap e66: 10 total, 9 up, 10 in 2024-06-03T02:11:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:36 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:11:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:36 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2490373922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbad302f-a304-4c5b-96c0-3e619ba42f9d"}]: dispatch 2024-06-03T02:11:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:36 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2490373922' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbad302f-a304-4c5b-96c0-3e619ba42f9d"}]': finished 2024-06-03T02:11:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:36 smithi135 ceph-mon[36653]: osdmap e66: 10 total, 9 up, 10 in 2024-06-03T02:11:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:36 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:11:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:37 smithi123 ceph-mon[36640]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-03T02:11:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:37 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1098004476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:11:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:37 smithi079 ceph-mon[31819]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-03T02:11:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:37 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1098004476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:11:37.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:37 smithi135 ceph-mon[36653]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-03T02:11:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:37 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1098004476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:11:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:39 smithi123 ceph-mon[36640]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T02:11:39.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:39 smithi079 ceph-mon[31819]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T02:11:39.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:39 smithi135 ceph-mon[36653]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T02:11:41.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:41 smithi079 ceph-mon[31819]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-03T02:11:41.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:41 smithi123 ceph-mon[36640]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-03T02:11:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:41 smithi135 ceph-mon[36653]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-03T02:11:43.446 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:43 smithi079 ceph-mon[31819]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:11:43.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:43 smithi123 ceph-mon[36640]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:11:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:43 smithi135 ceph-mon[36653]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:11:45.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:45 smithi079 ceph-mon[31819]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:45.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:45 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-03T02:11:45.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:45 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:45.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:45 smithi123 ceph-mon[36640]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:45.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:45 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-03T02:11:45.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:45 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:45 smithi135 ceph-mon[36653]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:45 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-03T02:11:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:45 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:11:46.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:46 smithi123 ceph-mon[36640]: Deploying daemon osd.9 on smithi079 2024-06-03T02:11:46.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:46 smithi079 ceph-mon[31819]: Deploying daemon osd.9 on smithi079 2024-06-03T02:11:46.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:46 smithi135 ceph-mon[36653]: Deploying daemon osd.9 on smithi079 2024-06-03T02:11:47.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:47 smithi079 ceph-mon[31819]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:47.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:47 smithi123 ceph-mon[36640]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:47.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:47 smithi135 ceph-mon[36653]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:48.474 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:48 smithi079 ceph-mon[31819]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:48 smithi135 ceph-mon[36653]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:48 smithi123 ceph-mon[36640]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:51.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:51 smithi123 ceph-mon[36640]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:51.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:51 smithi079 ceph-mon[31819]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:51 smithi135 ceph-mon[36653]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:53.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:53 smithi123 ceph-mon[36640]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:53.498 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:53 smithi079 ceph-mon[31819]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:53 smithi135 ceph-mon[36653]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:55.035 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:55.035 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:55.036 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:55.036 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:54 smithi079 ceph-mon[31819]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:55.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:55.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:55.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:55.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:54 smithi135 ceph-mon[36653]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:11:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:54 smithi123 ceph-mon[36640]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:57.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:57 smithi123 ceph-mon[36640]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:57.495 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:57 smithi079 ceph-mon[31819]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:57.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:57 smithi135 ceph-mon[36653]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:58.421 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:58 smithi079 ceph-mon[31819]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T02:11:58.421 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:58.422 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:58.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:58 smithi123 ceph-mon[36640]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T02:11:58.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:58.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:58 smithi135 ceph-mon[36653]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T02:11:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:58.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.882+0000 7f241fe00700 1 -- 172.21.15.79:0/963849001 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f24280628d0 con 0x7f240c0a2ca0 2024-06-03T02:11:58.883 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 9 on host 'smithi079' 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f240c0a2ca0 msgr2=0x7f240c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f240c0a2ca0 0x7f240c0a5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f241800b820 tx=0x7f2418005fd0 comp rx=0 tx=0).stop 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24281062e0 msgr2=0x7f24281a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24281062e0 0x7f24281a1f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2420009f60 tx=0x7f2420009f90 comp rx=0 tx=0).stop 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 shutdown_connections 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24281062e0 0x7f24281a1f40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f240c0a2ca0 0x7f240c0a5160 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2428104740 0x7f242819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 --2- 172.21.15.79:0/963849001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428105940 0x7f242819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.886+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 >> 172.21.15.79:0/963849001 conn(0x7f24280ffce0 msgr2=0x7f2428108cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.887+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 shutdown_connections 2024-06-03T02:11:58.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:11:58.887+0000 7f242e400700 1 -- 172.21.15.79:0/963849001 wait complete. 2024-06-03T02:11:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:59 smithi123 ceph-mon[36640]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:59 smithi123 ceph-mon[36640]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-03T02:11:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:59 smithi123 ceph-mon[36640]: osdmap e67: 10 total, 9 up, 10 in 2024-06-03T02:11:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:59 smithi123 ceph-mon[36640]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:11:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:59 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:11:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:59 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:11:59 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:59.496 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:59 smithi135 ceph-mon[36653]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:59.496 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:59 smithi135 ceph-mon[36653]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-03T02:11:59.496 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:59 smithi135 ceph-mon[36653]: osdmap e67: 10 total, 9 up, 10 in 2024-06-03T02:11:59.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:59 smithi135 ceph-mon[36653]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:11:59.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:59 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:11:59.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:59 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:59.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:11:59 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:59.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:59 smithi079 ceph-mon[31819]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:11:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:59 smithi079 ceph-mon[31819]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-03T02:11:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:59 smithi079 ceph-mon[31819]: osdmap e67: 10 total, 9 up, 10 in 2024-06-03T02:11:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:59 smithi079 ceph-mon[31819]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-06-03T02:11:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:59 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:11:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:59 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:11:59 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:11:59.720 DEBUG:teuthology.orchestra.run.smithi079:osd.9> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.9.service 2024-06-03T02:11:59.725 INFO:tasks.cephadm:Deploying osd.10 on smithi123 with /dev/vg_nvme/lv_1... 2024-06-03T02:11:59.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-03T02:11:59.971 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:12:00.345 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:00 smithi123 ceph-mon[36640]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:12:00.345 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:00 smithi123 ceph-mon[36640]: osdmap e68: 10 total, 9 up, 10 in 2024-06-03T02:12:00.345 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:12:00.345 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:12:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:00 smithi079 ceph-mon[31819]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:12:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:00 smithi079 ceph-mon[31819]: osdmap e68: 10 total, 9 up, 10 in 2024-06-03T02:12:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:12:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:12:00.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:00 smithi135 ceph-mon[36653]: from='osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-06-03T02:12:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:00 smithi135 ceph-mon[36653]: osdmap e68: 10 total, 9 up, 10 in 2024-06-03T02:12:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:12:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:12:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:01 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:12:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:01 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:12:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:01 smithi123 ceph-mon[36640]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:12:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:01 smithi123 ceph-mon[36640]: osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] boot 2024-06-03T02:12:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:01 smithi123 ceph-mon[36640]: osdmap e69: 10 total, 10 up, 10 in 2024-06-03T02:12:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:12:01.539 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:01 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:12:01.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:01 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:12:01.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:01 smithi079 ceph-mon[31819]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:12:01.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:01 smithi079 ceph-mon[31819]: osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] boot 2024-06-03T02:12:01.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:01 smithi079 ceph-mon[31819]: osdmap e69: 10 total, 10 up, 10 in 2024-06-03T02:12:01.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:12:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:01 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:12:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:01 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:12:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:01 smithi135 ceph-mon[36653]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-06-03T02:12:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:01 smithi135 ceph-mon[36653]: osd.9 [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] boot 2024-06-03T02:12:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:01 smithi135 ceph-mon[36653]: osdmap e69: 10 total, 10 up, 10 in 2024-06-03T02:12:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T02:12:02.056 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-03T02:12:02.056 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-03T02:12:02.057 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-06-03T02:12:02.057 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-06-03T02:12:02.057 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113803 s, 921 MB/s 2024-06-03T02:12:02.057 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-06-03T02:12:02.526 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:02 smithi123 ceph-mon[36640]: osdmap e70: 10 total, 10 up, 10 in 2024-06-03T02:12:02.527 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:02 smithi123 ceph-mon[36640]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:02.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:02 smithi079 ceph-mon[31819]: osdmap e70: 10 total, 10 up, 10 in 2024-06-03T02:12:02.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:02 smithi079 ceph-mon[31819]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:02 smithi135 ceph-mon[36653]: osdmap e70: 10 total, 10 up, 10 in 2024-06-03T02:12:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:02 smithi135 ceph-mon[36653]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:02.775 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-06-03T02:12:03.011 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:12:04.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.596+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/560275196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 msgr2=0x7f3fd4105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:04.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.596+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/560275196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 0x7f3fd4105330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009990 tx=0x7f3fc4009ca0 comp rx=0 tx=0).stop 2024-06-03T02:12:04.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.596+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/560275196 shutdown_connections 2024-06-03T02:12:04.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.596+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/560275196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fd41081a0 0x7f3fd410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:04.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.596+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/560275196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 0x7f3fd4105330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:04.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.596+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/560275196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fd4105870 0x7f3fd4107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:04.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.596+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/560275196 >> 172.21.15.123:0/560275196 conn(0x7f3fd40fc990 msgr2=0x7f3fd40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:12:04.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.596+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/560275196 shutdown_connections 2024-06-03T02:12:04.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.596+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/560275196 wait complete. 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd8f1f700 1 Processor -- start 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd8f1f700 1 -- start start 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd8f1f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 0x7f3fd4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd8f1f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fd4105870 0x7f3fd4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd8f1f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fd41081a0 0x7f3fd419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd8f1f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd410e150 con 0x7f3fd4105870 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd8f1f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd410dfd0 con 0x7f3fd4102f40 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd8f1f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd410e2d0 con 0x7f3fd41081a0 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 0x7f3fd4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 0x7f3fd4198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39188/0 (socket says 172.21.15.123:39188) 2024-06-03T02:12:04.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd259c700 1 -- 172.21.15.123:0/3034374821 learned_addr learned my addr 172.21.15.123:0/3034374821 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.597+0000 7f3fd2d9d700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fd41081a0 0x7f3fd419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fd259c700 1 -- 172.21.15.123:0/3034374821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fd41081a0 msgr2=0x7f3fd419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fd259c700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fd41081a0 0x7f3fd419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fd259c700 1 -- 172.21.15.123:0/3034374821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fd4105870 msgr2=0x7f3fd4198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fd259c700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fd4105870 0x7f3fd4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fd259c700 1 -- 172.21.15.123:0/3034374821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd41a0360 con 0x7f3fd4102f40 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fd2d9d700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fd41081a0 0x7f3fd419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fd259c700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 0x7f3fd4198700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4003890 tx=0x7f3fc4003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fc37fe700 1 -- 172.21.15.123:0/3034374821 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4027070 con 0x7f3fd4102f40 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd41a05f0 con 0x7f3fd4102f40 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fc37fe700 1 -- 172.21.15.123:0/3034374821 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fc4022470 con 0x7f3fd4102f40 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fc37fe700 1 -- 172.21.15.123:0/3034374821 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400f6a0 con 0x7f3fd4102f40 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.598+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd4100db0 con 0x7f3fd4102f40 2024-06-03T02:12:04.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.599+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd4069180 con 0x7f3fd4102f40 2024-06-03T02:12:04.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.600+0000 7f3fc37fe700 1 -- 172.21.15.123:0/3034374821 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3fc402a070 con 0x7f3fd4102f40 2024-06-03T02:12:04.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.600+0000 7f3fc37fe700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fbc0a7030 0x7f3fbc0a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:04.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.601+0000 7f3fc37fe700 1 -- 172.21.15.123:0/3034374821 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7f3fc40cf940 con 0x7f3fd4102f40 2024-06-03T02:12:04.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.601+0000 7f3fd1d9b700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fbc0a7030 0x7f3fbc0a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:04.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.602+0000 7f3fd1d9b700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fbc0a7030 0x7f3fbc0a94f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80097b0 tx=0x7f3fc8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:12:04.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.605+0000 7f3fc37fe700 1 -- 172.21.15.123:0/3034374821 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3fc405e850 con 0x7f3fd4102f40 2024-06-03T02:12:04.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:04.813+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3fd40628d0 con 0x7f3fbc0a7030 2024-06-03T02:12:05.467 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:05.467 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.467 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:05.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:05.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:05.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:12:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:12:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:05.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:06 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi079 2024-06-03T02:12:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:06 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi079 to 4272M 2024-06-03T02:12:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:06 smithi123 ceph-mon[36640]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:12:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:06 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi079 2024-06-03T02:12:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:06 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi079 to 4272M 2024-06-03T02:12:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:06 smithi079 ceph-mon[31819]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:12:06.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:06 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi079 2024-06-03T02:12:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:06 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi079 to 4272M 2024-06-03T02:12:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:06 smithi135 ceph-mon[36653]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:12:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:07 smithi123 ceph-mon[36640]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:07.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:07 smithi079 ceph-mon[31819]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:07.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:07 smithi135 ceph-mon[36653]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:08.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:08 smithi123 ceph-mon[36640]: from='client.? 172.21.15.123:0/4148312670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66fbdf38-f92c-4cb6-8724-4b1df4d472f8"}]: dispatch 2024-06-03T02:12:08.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:08 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66fbdf38-f92c-4cb6-8724-4b1df4d472f8"}]: dispatch 2024-06-03T02:12:08.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:08 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66fbdf38-f92c-4cb6-8724-4b1df4d472f8"}]': finished 2024-06-03T02:12:08.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:08 smithi123 ceph-mon[36640]: osdmap e71: 11 total, 10 up, 11 in 2024-06-03T02:12:08.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:08 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:08.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:08 smithi079 ceph-mon[31819]: from='client.? 172.21.15.123:0/4148312670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66fbdf38-f92c-4cb6-8724-4b1df4d472f8"}]: dispatch 2024-06-03T02:12:08.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:08 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66fbdf38-f92c-4cb6-8724-4b1df4d472f8"}]: dispatch 2024-06-03T02:12:08.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:08 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66fbdf38-f92c-4cb6-8724-4b1df4d472f8"}]': finished 2024-06-03T02:12:08.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:08 smithi079 ceph-mon[31819]: osdmap e71: 11 total, 10 up, 11 in 2024-06-03T02:12:08.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:08 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:08 smithi135 ceph-mon[36653]: from='client.? 172.21.15.123:0/4148312670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66fbdf38-f92c-4cb6-8724-4b1df4d472f8"}]: dispatch 2024-06-03T02:12:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:08 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66fbdf38-f92c-4cb6-8724-4b1df4d472f8"}]: dispatch 2024-06-03T02:12:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:08 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66fbdf38-f92c-4cb6-8724-4b1df4d472f8"}]': finished 2024-06-03T02:12:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:08 smithi135 ceph-mon[36653]: osdmap e71: 11 total, 10 up, 11 in 2024-06-03T02:12:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:08 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:09.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:09 smithi123 ceph-mon[36640]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:09.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:09 smithi123 ceph-mon[36640]: from='client.? 172.21.15.123:0/2597264596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:12:09.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:09 smithi079 ceph-mon[31819]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:09.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:09 smithi079 ceph-mon[31819]: from='client.? 172.21.15.123:0/2597264596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:12:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:09 smithi135 ceph-mon[36653]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:09 smithi135 ceph-mon[36653]: from='client.? 172.21.15.123:0/2597264596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:12:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:11 smithi123 ceph-mon[36640]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:11 smithi079 ceph-mon[31819]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:11.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:11 smithi135 ceph-mon[36653]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:13.370 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:13 smithi123 ceph-mon[36640]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:13.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:13 smithi079 ceph-mon[31819]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:13.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:13 smithi135 ceph-mon[36653]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:15 smithi079 ceph-mon[31819]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:15.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:15 smithi123 ceph-mon[36640]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:15.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:15 smithi135 ceph-mon[36653]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:16.337 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:16 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-03T02:12:16.337 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:16 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:16.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:16 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-03T02:12:16.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:16 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:16 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-03T02:12:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:16 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:17.322 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:17 smithi123 ceph-mon[36640]: Deploying daemon osd.10 on smithi123 2024-06-03T02:12:17.323 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:17 smithi123 ceph-mon[36640]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:17.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:17 smithi079 ceph-mon[31819]: Deploying daemon osd.10 on smithi123 2024-06-03T02:12:17.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:17 smithi079 ceph-mon[31819]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:17.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:17 smithi135 ceph-mon[36653]: Deploying daemon osd.10 on smithi123 2024-06-03T02:12:17.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:17 smithi135 ceph-mon[36653]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:18.378 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:18 smithi123 ceph-mon[36640]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:18.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:18 smithi079 ceph-mon[31819]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:18.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:18 smithi135 ceph-mon[36653]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:21 smithi123 ceph-mon[36640]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:21 smithi079 ceph-mon[31819]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:21 smithi135 ceph-mon[36653]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:23 smithi123 ceph-mon[36640]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:23.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:23 smithi079 ceph-mon[31819]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:23 smithi135 ceph-mon[36653]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:24.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.803+0000 7f3fc37fe700 1 -- 172.21.15.123:0/3034374821 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3fd40628d0 con 0x7f3fbc0a7030 2024-06-03T02:12:24.804 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 10 on host 'smithi123' 2024-06-03T02:12:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fbc0a7030 msgr2=0x7f3fbc0a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fbc0a7030 0x7f3fbc0a94f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80097b0 tx=0x7f3fc8006d20 comp rx=0 tx=0).stop 2024-06-03T02:12:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 msgr2=0x7f3fd4198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 0x7f3fd4198700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4003890 tx=0x7f3fc4003730 comp rx=0 tx=0).stop 2024-06-03T02:12:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 shutdown_connections 2024-06-03T02:12:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fd41081a0 0x7f3fd419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fd4102f40 0x7f3fd4198700 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:24.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fbc0a7030 0x7f3fbc0a94f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:24.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 --2- 172.21.15.123:0/3034374821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fd4105870 0x7f3fd4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:24.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 >> 172.21.15.123:0/3034374821 conn(0x7f3fd40fc990 msgr2=0x7f3fd4103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:12:24.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 shutdown_connections 2024-06-03T02:12:24.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-03T02:12:24.805+0000 7f3fd8f1f700 1 -- 172.21.15.123:0/3034374821 wait complete. 2024-06-03T02:12:25.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:25 smithi079 ceph-mon[31819]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:25.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:25.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:25.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:25 smithi079 ceph-mon[31819]: from='osd.10 [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T02:12:25.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:25 smithi079 ceph-mon[31819]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T02:12:25.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:25 smithi123 ceph-mon[36640]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:25.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:25.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:25.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:25 smithi123 ceph-mon[36640]: from='osd.10 [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T02:12:25.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:25 smithi123 ceph-mon[36640]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T02:12:25.533 DEBUG:teuthology.orchestra.run.smithi123:osd.10> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.10.service 2024-06-03T02:12:25.536 INFO:tasks.cephadm:Deploying osd.11 on smithi135 with /dev/vg_nvme/lv_1... 2024-06-03T02:12:25.536 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-03T02:12:25.562 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:25 smithi135 ceph-mon[36653]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:25.563 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:25.563 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:25.563 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:25 smithi135 ceph-mon[36653]: from='osd.10 [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T02:12:25.563 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:25 smithi135 ceph-mon[36653]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T02:12:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:12:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:26 smithi123 ceph-mon[36640]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-03T02:12:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:26 smithi123 ceph-mon[36640]: from='osd.10 [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:12:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:26 smithi123 ceph-mon[36640]: osdmap e72: 11 total, 10 up, 11 in 2024-06-03T02:12:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:26 smithi123 ceph-mon[36640]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:12:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:26 smithi123 ceph-mon[36640]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:27.013 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:27.013 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:27.013 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:26 smithi135 ceph-mon[36653]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-03T02:12:27.013 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:26 smithi135 ceph-mon[36653]: from='osd.10 [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:12:27.013 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:26 smithi135 ceph-mon[36653]: osdmap e72: 11 total, 10 up, 11 in 2024-06-03T02:12:27.013 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:27.013 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:26 smithi135 ceph-mon[36653]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:12:27.013 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:26 smithi135 ceph-mon[36653]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:27.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:27.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:27.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:26 smithi079 ceph-mon[31819]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-03T02:12:27.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:26 smithi079 ceph-mon[31819]: from='osd.10 [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:12:27.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:26 smithi079 ceph-mon[31819]: osdmap e72: 11 total, 10 up, 11 in 2024-06-03T02:12:27.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:27.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:26 smithi079 ceph-mon[31819]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-03T02:12:27.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:26 smithi079 ceph-mon[31819]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:27.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:12:27 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:12:27.470+0000 7f8572412700 -1 osd.10 0 waiting for initial osdmap 2024-06-03T02:12:27.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:12:27 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:12:27.483+0000 7f856a1fd700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:12:27.891 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:27 smithi135 ceph-mon[36653]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:12:27.891 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:27 smithi135 ceph-mon[36653]: osdmap e73: 11 total, 10 up, 11 in 2024-06-03T02:12:27.892 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:27 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:27.892 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:27 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:27.892 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:27 smithi135 ceph-mon[36653]: from='osd.10 ' entity='osd.10' 2024-06-03T02:12:27.892 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-03T02:12:27.892 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-03T02:12:27.892 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-03T02:12:27.892 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-03T02:12:27.892 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134023 s, 782 MB/s 2024-06-03T02:12:27.892 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-03T02:12:28.068 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:27 smithi123 ceph-mon[36640]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:12:28.068 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:27 smithi123 ceph-mon[36640]: osdmap e73: 11 total, 10 up, 11 in 2024-06-03T02:12:28.068 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:27 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:28.068 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:27 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:28.068 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:27 smithi123 ceph-mon[36640]: from='osd.10 ' entity='osd.10' 2024-06-03T02:12:28.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:27 smithi079 ceph-mon[31819]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-03T02:12:28.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:27 smithi079 ceph-mon[31819]: osdmap e73: 11 total, 10 up, 11 in 2024-06-03T02:12:28.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:28.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:27 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:28.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:27 smithi079 ceph-mon[31819]: from='osd.10 ' entity='osd.10' 2024-06-03T02:12:28.557 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_1 2024-06-03T02:12:28.847 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:12:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:28 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:12:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:28 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:12:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:28 smithi123 ceph-mon[36640]: osd.10 [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] boot 2024-06-03T02:12:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:28 smithi123 ceph-mon[36640]: osdmap e74: 11 total, 11 up, 11 in 2024-06-03T02:12:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:28 smithi123 ceph-mon[36640]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:28 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:12:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:28 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:12:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:28 smithi079 ceph-mon[31819]: osd.10 [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] boot 2024-06-03T02:12:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:28 smithi079 ceph-mon[31819]: osdmap e74: 11 total, 11 up, 11 in 2024-06-03T02:12:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:29.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:28 smithi079 ceph-mon[31819]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:29.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:28 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:12:29.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:28 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:12:29.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:29.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:28 smithi135 ceph-mon[36653]: osd.10 [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] boot 2024-06-03T02:12:29.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:28 smithi135 ceph-mon[36653]: osdmap e74: 11 total, 11 up, 11 in 2024-06-03T02:12:29.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T02:12:29.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:28 smithi135 ceph-mon[36653]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 281 MiB used, 894 GiB / 894 GiB avail 2024-06-03T02:12:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: osdmap e75: 11 total, 11 up, 11 in 2024-06-03T02:12:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:29 smithi079 ceph-mon[31819]: osdmap e76: 11 total, 11 up, 11 in 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: osdmap e75: 11 total, 11 up, 11 in 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:30.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:30.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:30.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:29 smithi135 ceph-mon[36653]: osdmap e76: 11 total, 11 up, 11 in 2024-06-03T02:12:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: osdmap e75: 11 total, 11 up, 11 in 2024-06-03T02:12:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:29 smithi123 ceph-mon[36640]: osdmap e76: 11 total, 11 up, 11 in 2024-06-03T02:12:30.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.503+0000 7f817004c700 1 -- 172.21.15.135:0/3203895828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168100f20 msgr2=0x7f816810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:30.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.503+0000 7f817004c700 1 --2- 172.21.15.135:0/3203895828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168100f20 0x7f816810aad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f815800b0a0 tx=0x7f815800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:12:30.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.503+0000 7f817004c700 1 -- 172.21.15.135:0/3203895828 shutdown_connections 2024-06-03T02:12:30.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.503+0000 7f817004c700 1 --2- 172.21.15.135:0/3203895828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168100f20 0x7f816810aad0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:30.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.503+0000 7f817004c700 1 --2- 172.21.15.135:0/3203895828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8168103ae0 0x7f8168103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:30.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.503+0000 7f817004c700 1 --2- 172.21.15.135:0/3203895828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8168100560 0x7f81681009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:30.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.503+0000 7f817004c700 1 -- 172.21.15.135:0/3203895828 >> 172.21.15.135:0/3203895828 conn(0x7f81680fc120 msgr2=0x7f81680fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:12:30.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.503+0000 7f817004c700 1 -- 172.21.15.135:0/3203895828 shutdown_connections 2024-06-03T02:12:30.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.503+0000 7f817004c700 1 -- 172.21.15.135:0/3203895828 wait complete. 2024-06-03T02:12:30.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f817004c700 1 Processor -- start 2024-06-03T02:12:30.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f817004c700 1 -- start start 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f817004c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8168100560 0x7f81681986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f817004c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8168100f20 0x7f8168198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f817004c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168103ae0 0x7f816819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f817004c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f816810e190 con 0x7f8168100560 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f817004c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f816810e010 con 0x7f8168100f20 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f817004c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f816810e310 con 0x7f8168103ae0 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816e5e9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168103ae0 0x7f816819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816e5e9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168103ae0 0x7f816819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47570/0 (socket says 172.21.15.135:47570) 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816e5e9700 1 -- 172.21.15.135:0/3243911109 learned_addr learned my addr 172.21.15.135:0/3243911109 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816d5e7700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8168100f20 0x7f8168198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816e5e9700 1 -- 172.21.15.135:0/3243911109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8168100f20 msgr2=0x7f8168198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:30.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816e5e9700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8168100f20 0x7f8168198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816e5e9700 1 -- 172.21.15.135:0/3243911109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8168100560 msgr2=0x7f81681986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816e5e9700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8168100560 0x7f81681986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816e5e9700 1 -- 172.21.15.135:0/3243911109 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81681a0340 con 0x7f8168103ae0 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.504+0000 7f816d5e7700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8168100f20 0x7f8168198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.505+0000 7f816e5e9700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168103ae0 0x7f816819fca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8158009800 tx=0x7f815800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.505+0000 7f8166ffd700 1 -- 172.21.15.135:0/3243911109 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158021070 con 0x7f8168103ae0 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.505+0000 7f8166ffd700 1 -- 172.21.15.135:0/3243911109 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8158005520 con 0x7f8168103ae0 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.505+0000 7f8166ffd700 1 -- 172.21.15.135:0/3243911109 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815801b930 con 0x7f8168103ae0 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.505+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81681a05d0 con 0x7f8168103ae0 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.505+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81681a0b00 con 0x7f8168103ae0 2024-06-03T02:12:30.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.506+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8168069180 con 0x7f8168103ae0 2024-06-03T02:12:30.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.507+0000 7f8166ffd700 1 -- 172.21.15.135:0/3243911109 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8158024070 con 0x7f8168103ae0 2024-06-03T02:12:30.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.507+0000 7f8166ffd700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81500a2ca0 0x7f81500a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:30.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.508+0000 7f8166ffd700 1 -- 172.21.15.135:0/3243911109 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7f815800f020 con 0x7f8168103ae0 2024-06-03T02:12:30.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.508+0000 7f816dde8700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81500a2ca0 0x7f81500a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:30.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.509+0000 7f816dde8700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81500a2ca0 0x7f81500a5160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8168069e70 tx=0x7f8160006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:12:30.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.512+0000 7f8166ffd700 1 -- 172.21.15.135:0/3243911109 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f815802b020 con 0x7f8168103ae0 2024-06-03T02:12:30.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:30.717+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f81680628d0 con 0x7f81500a2ca0 2024-06-03T02:12:31.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:30 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi123 2024-06-03T02:12:31.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:30 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi123 to 4272M 2024-06-03T02:12:31.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:30 smithi079 ceph-mon[31819]: pgmap v265: 1 pgs: 1 peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:31.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:12:31.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:12:31.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:31.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:30 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi123 2024-06-03T02:12:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:30 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi123 to 4272M 2024-06-03T02:12:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:30 smithi135 ceph-mon[36653]: pgmap v265: 1 pgs: 1 peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:12:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:12:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:31.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:30 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi123 2024-06-03T02:12:31.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:30 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi123 to 4272M 2024-06-03T02:12:31.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:30 smithi123 ceph-mon[36640]: pgmap v265: 1 pgs: 1 peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:31.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T02:12:31.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T02:12:31.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:31 smithi135 ceph-mon[36653]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:12:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:31 smithi135 ceph-mon[36653]: osdmap e77: 11 total, 11 up, 11 in 2024-06-03T02:12:32.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:31 smithi123 ceph-mon[36640]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:12:32.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:31 smithi123 ceph-mon[36640]: osdmap e77: 11 total, 11 up, 11 in 2024-06-03T02:12:32.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:31 smithi079 ceph-mon[31819]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:12:32.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:31 smithi079 ceph-mon[31819]: osdmap e77: 11 total, 11 up, 11 in 2024-06-03T02:12:33.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:32 smithi123 ceph-mon[36640]: osdmap e78: 11 total, 11 up, 11 in 2024-06-03T02:12:33.226 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:32 smithi123 ceph-mon[36640]: pgmap v268: 1 pgs: 1 peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:33.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:32 smithi079 ceph-mon[31819]: osdmap e78: 11 total, 11 up, 11 in 2024-06-03T02:12:33.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:32 smithi079 ceph-mon[31819]: pgmap v268: 1 pgs: 1 peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:33.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:32 smithi135 ceph-mon[36653]: osdmap e78: 11 total, 11 up, 11 in 2024-06-03T02:12:33.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:32 smithi135 ceph-mon[36653]: pgmap v268: 1 pgs: 1 peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:33 smithi123 ceph-mon[36640]: from='client.? 172.21.15.135:0/706575522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd600f8-5366-44e9-8827-e7f9fc41e2ba"}]: dispatch 2024-06-03T02:12:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:33 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd600f8-5366-44e9-8827-e7f9fc41e2ba"}]: dispatch 2024-06-03T02:12:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:33 smithi123 ceph-mon[36640]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fd600f8-5366-44e9-8827-e7f9fc41e2ba"}]': finished 2024-06-03T02:12:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:33 smithi123 ceph-mon[36640]: osdmap e79: 12 total, 11 up, 12 in 2024-06-03T02:12:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:33 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/706575522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd600f8-5366-44e9-8827-e7f9fc41e2ba"}]: dispatch 2024-06-03T02:12:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:33 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd600f8-5366-44e9-8827-e7f9fc41e2ba"}]: dispatch 2024-06-03T02:12:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:33 smithi079 ceph-mon[31819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fd600f8-5366-44e9-8827-e7f9fc41e2ba"}]': finished 2024-06-03T02:12:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:33 smithi079 ceph-mon[31819]: osdmap e79: 12 total, 11 up, 12 in 2024-06-03T02:12:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:34.344 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:33 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/706575522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd600f8-5366-44e9-8827-e7f9fc41e2ba"}]: dispatch 2024-06-03T02:12:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:33 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fd600f8-5366-44e9-8827-e7f9fc41e2ba"}]: dispatch 2024-06-03T02:12:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:33 smithi135 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fd600f8-5366-44e9-8827-e7f9fc41e2ba"}]': finished 2024-06-03T02:12:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:33 smithi135 ceph-mon[36653]: osdmap e79: 12 total, 11 up, 12 in 2024-06-03T02:12:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:35.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:34 smithi123 ceph-mon[36640]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-06-03T02:12:35.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:34 smithi123 ceph-mon[36640]: from='client.? 172.21.15.135:0/2727566858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:12:35.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:34 smithi079 ceph-mon[31819]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-06-03T02:12:35.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:34 smithi079 ceph-mon[31819]: from='client.? 172.21.15.135:0/2727566858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:12:35.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:34 smithi135 ceph-mon[36653]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-06-03T02:12:35.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:34 smithi135 ceph-mon[36653]: from='client.? 172.21.15.135:0/2727566858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T02:12:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:37 smithi123 ceph-mon[36640]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T02:12:37.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:37 smithi135 ceph-mon[36653]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T02:12:37.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:37 smithi079 ceph-mon[31819]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T02:12:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:39 smithi123 ceph-mon[36640]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-06-03T02:12:39.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:39 smithi079 ceph-mon[31819]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-06-03T02:12:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:39 smithi135 ceph-mon[36653]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-06-03T02:12:41.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:41 smithi135 ceph-mon[36653]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-06-03T02:12:41.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:41 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-03T02:12:41.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:41 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:41.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:41 smithi123 ceph-mon[36640]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-06-03T02:12:41.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:41 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-03T02:12:41.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:41 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:41.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:41 smithi079 ceph-mon[31819]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-06-03T02:12:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:41 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-03T02:12:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:41 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:42.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:42 smithi135 ceph-mon[36653]: Deploying daemon osd.11 on smithi135 2024-06-03T02:12:42.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:42 smithi123 ceph-mon[36640]: Deploying daemon osd.11 on smithi135 2024-06-03T02:12:42.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:42 smithi079 ceph-mon[31819]: Deploying daemon osd.11 on smithi135 2024-06-03T02:12:43.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:43 smithi135 ceph-mon[36653]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:12:43.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:43 smithi123 ceph-mon[36640]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:12:43.488 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:43 smithi079 ceph-mon[31819]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T02:12:45.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:45 smithi123 ceph-mon[36640]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:45.504 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:45 smithi135 ceph-mon[36653]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:45 smithi079 ceph-mon[31819]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:47.411 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:47 smithi135 ceph-mon[36653]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:47.411 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:47.411 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:47.411 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:47.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:47 smithi123 ceph-mon[36640]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:47.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:47.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:47.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:47.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:47 smithi079 ceph-mon[31819]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:47.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:47.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:47.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:48.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:48 smithi079 ceph-mon[31819]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:48 smithi135 ceph-mon[36653]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:48 smithi123 ceph-mon[36640]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:50.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.801+0000 7f8166ffd700 1 -- 172.21.15.135:0/3243911109 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f81680628d0 con 0x7f81500a2ca0 2024-06-03T02:12:50.803 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 11 on host 'smithi135' 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81500a2ca0 msgr2=0x7f81500a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81500a2ca0 0x7f81500a5160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8168069e70 tx=0x7f8160006d20 comp rx=0 tx=0).stop 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168103ae0 msgr2=0x7f816819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168103ae0 0x7f816819fca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8158009800 tx=0x7f815800be20 comp rx=0 tx=0).stop 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 shutdown_connections 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8168103ae0 0x7f816819fca0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81500a2ca0 0x7f81500a5160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8168100560 0x7f81681986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 --2- 172.21.15.135:0/3243911109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8168100f20 0x7f8168198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 >> 172.21.15.135:0/3243911109 conn(0x7f81680fc120 msgr2=0x7f8168106b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:12:50.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 shutdown_connections 2024-06-03T02:12:50.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:12:50.804+0000 7f817004c700 1 -- 172.21.15.135:0/3243911109 wait complete. 2024-06-03T02:12:51.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:51.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:51.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:51.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='osd.11 [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T02:12:51.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T02:12:51.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:51.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:51.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:51.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:51.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='osd.11 [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T02:12:51.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T02:12:51.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:51.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:51.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:51.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:51.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='osd.11 [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T02:12:51.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T02:12:51.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:12:51.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:51.419 DEBUG:teuthology.orchestra.run.smithi135:osd.11> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.11.service 2024-06-03T02:12:51.422 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-03T02:12:51.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-03T02:12:51.876 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:12:51.964 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:51 smithi135 ceph-mon[36653]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-03T02:12:51.964 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:51 smithi135 ceph-mon[36653]: from='osd.11 [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:12:51.965 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:51 smithi135 ceph-mon[36653]: osdmap e80: 12 total, 11 up, 12 in 2024-06-03T02:12:51.965 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:51 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:51.965 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:51 smithi135 ceph-mon[36653]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:12:52.209 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:51 smithi079 ceph-mon[31819]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-03T02:12:52.209 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:51 smithi079 ceph-mon[31819]: from='osd.11 [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:12:52.209 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:51 smithi079 ceph-mon[31819]: osdmap e80: 12 total, 11 up, 12 in 2024-06-03T02:12:52.209 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:51 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:52.209 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:51 smithi079 ceph-mon[31819]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:12:52.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:51 smithi123 ceph-mon[36640]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-03T02:12:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:51 smithi123 ceph-mon[36640]: from='osd.11 [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:12:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:51 smithi123 ceph-mon[36640]: osdmap e80: 12 total, 11 up, 12 in 2024-06-03T02:12:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:51 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:51 smithi123 ceph-mon[36640]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-03T02:12:52.595 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:12:52 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:12:52.549+0000 7f46286e2700 -1 osd.11 0 waiting for initial osdmap 2024-06-03T02:12:52.595 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:12:52 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:12:52.563+0000 7f46204cd700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T02:12:52.949 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:52 smithi135 ceph-mon[36653]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:12:52.949 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:52 smithi135 ceph-mon[36653]: osdmap e81: 12 total, 11 up, 12 in 2024-06-03T02:12:52.949 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:52.949 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:52.949 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:52 smithi135 ceph-mon[36653]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:52.949 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:52 smithi135 ceph-mon[36653]: from='osd.11 ' entity='osd.11' 2024-06-03T02:12:53.056 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:52 smithi079 ceph-mon[31819]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:12:53.057 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:52 smithi079 ceph-mon[31819]: osdmap e81: 12 total, 11 up, 12 in 2024-06-03T02:12:53.057 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:53.057 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:53.057 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:52 smithi079 ceph-mon[31819]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:53.057 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:52 smithi079 ceph-mon[31819]: from='osd.11 ' entity='osd.11' 2024-06-03T02:12:53.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:52 smithi123 ceph-mon[36640]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-03T02:12:53.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:52 smithi123 ceph-mon[36640]: osdmap e81: 12 total, 11 up, 12 in 2024-06-03T02:12:53.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:53.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:53.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:52 smithi123 ceph-mon[36640]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-06-03T02:12:53.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:52 smithi123 ceph-mon[36640]: from='osd.11 ' entity='osd.11' 2024-06-03T02:12:54.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:53 smithi135 ceph-mon[36653]: purged_snaps scrub starts 2024-06-03T02:12:54.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:53 smithi135 ceph-mon[36653]: purged_snaps scrub ok 2024-06-03T02:12:54.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:54.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:53 smithi135 ceph-mon[36653]: osd.11 [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] boot 2024-06-03T02:12:54.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:53 smithi135 ceph-mon[36653]: osdmap e82: 12 total, 12 up, 12 in 2024-06-03T02:12:54.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:54.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.128+0000 7f53e8a00700 1 -- 172.21.15.79:0/652366260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e41081e0 msgr2=0x7f53e410a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:54.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.128+0000 7f53e8a00700 1 --2- 172.21.15.79:0/652366260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e41081e0 0x7f53e410a650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00b0a0 tx=0x7f53dc00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.128+0000 7f53e8a00700 1 -- 172.21.15.79:0/652366260 shutdown_connections 2024-06-03T02:12:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.128+0000 7f53e8a00700 1 --2- 172.21.15.79:0/652366260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e41081e0 0x7f53e410a650 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.128+0000 7f53e8a00700 1 --2- 172.21.15.79:0/652366260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53e4102f80 0x7f53e4105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.128+0000 7f53e8a00700 1 --2- 172.21.15.79:0/652366260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53e41058b0 0x7f53e4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.128+0000 7f53e8a00700 1 -- 172.21.15.79:0/652366260 >> 172.21.15.79:0/652366260 conn(0x7f53e40fc990 msgr2=0x7f53e40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:12:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.128+0000 7f53e8a00700 1 -- 172.21.15.79:0/652366260 shutdown_connections 2024-06-03T02:12:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.128+0000 7f53e8a00700 1 -- 172.21.15.79:0/652366260 wait complete. 2024-06-03T02:12:54.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.129+0000 7f53e8a00700 1 Processor -- start 2024-06-03T02:12:54.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.129+0000 7f53e8a00700 1 -- start start 2024-06-03T02:12:54.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e8a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53e4102f80 0x7f53e419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:54.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e8a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53e41058b0 0x7f53e419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:54.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e8a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e41081e0 0x7f53e41a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:54.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e8a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410e100 con 0x7f53e41058b0 2024-06-03T02:12:54.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e8a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410df80 con 0x7f53e41081e0 2024-06-03T02:12:54.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e8a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410e280 con 0x7f53e4102f80 2024-06-03T02:12:54.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e1e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53e4102f80 0x7f53e419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:54.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e1e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53e4102f80 0x7f53e419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37626/0 (socket says 172.21.15.79:37626) 2024-06-03T02:12:54.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e1e00700 1 -- 172.21.15.79:0/3313216986 learned_addr learned my addr 172.21.15.79:0/3313216986 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:12:54.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e1400700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53e41058b0 0x7f53e419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:54.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e2800700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e41081e0 0x7f53e41a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:54.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e1e00700 1 -- 172.21.15.79:0/3313216986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e41081e0 msgr2=0x7f53e41a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:54.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e1e00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e41081e0 0x7f53e41a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e1e00700 1 -- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53e41058b0 msgr2=0x7f53e419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:54.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e1e00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53e41058b0 0x7f53e419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.130+0000 7f53e1e00700 1 -- 172.21.15.79:0/3313216986 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e41a2520 con 0x7f53e4102f80 2024-06-03T02:12:54.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.131+0000 7f53e1e00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53e4102f80 0x7f53e419a8c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53d400e9f0 tx=0x7f53d400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:12:54.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.131+0000 7f53daa00700 1 -- 172.21.15.79:0/3313216986 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d400cbd0 con 0x7f53e4102f80 2024-06-03T02:12:54.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.131+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e41a2810 con 0x7f53e4102f80 2024-06-03T02:12:54.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.131+0000 7f53daa00700 1 -- 172.21.15.79:0/3313216986 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d400ee90 con 0x7f53e4102f80 2024-06-03T02:12:54.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.131+0000 7f53daa00700 1 -- 172.21.15.79:0/3313216986 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d40056c0 con 0x7f53e4102f80 2024-06-03T02:12:54.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.131+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e41a2d50 con 0x7f53e4102f80 2024-06-03T02:12:54.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.132+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e4069180 con 0x7f53e4102f80 2024-06-03T02:12:54.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.134+0000 7f53daa00700 1 -- 172.21.15.79:0/3313216986 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f53d4004750 con 0x7f53e4102f80 2024-06-03T02:12:54.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.135+0000 7f53daa00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f53c80a2ca0 0x7f53c80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:54.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.135+0000 7f53daa00700 1 -- 172.21.15.79:0/3313216986 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f53d40cee10 con 0x7f53e4102f80 2024-06-03T02:12:54.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.136+0000 7f53e1400700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f53c80a2ca0 0x7f53c80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:54.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.136+0000 7f53e1400700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f53c80a2ca0 0x7f53c80a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009c60 tx=0x7f53d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:12:54.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.140+0000 7f53daa00700 1 -- 172.21.15.79:0/3313216986 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f53d40d2030 con 0x7f53e4102f80 2024-06-03T02:12:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:53 smithi123 ceph-mon[36640]: purged_snaps scrub starts 2024-06-03T02:12:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:53 smithi123 ceph-mon[36640]: purged_snaps scrub ok 2024-06-03T02:12:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:53 smithi123 ceph-mon[36640]: osd.11 [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] boot 2024-06-03T02:12:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:53 smithi123 ceph-mon[36640]: osdmap e82: 12 total, 12 up, 12 in 2024-06-03T02:12:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:53 smithi079 ceph-mon[31819]: purged_snaps scrub starts 2024-06-03T02:12:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:53 smithi079 ceph-mon[31819]: purged_snaps scrub ok 2024-06-03T02:12:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:53 smithi079 ceph-mon[31819]: osd.11 [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] boot 2024-06-03T02:12:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:53 smithi079 ceph-mon[31819]: osdmap e82: 12 total, 12 up, 12 in 2024-06-03T02:12:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T02:12:54.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.361+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f53e40630a0 con 0x7f53e4102f80 2024-06-03T02:12:54.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.362+0000 7f53daa00700 1 -- 172.21.15.79:0/3313216986 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f53d409c170 con 0x7f53e4102f80 2024-06-03T02:12:54.363 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:12:54.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f53c80a2ca0 msgr2=0x7f53c80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:54.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f53c80a2ca0 0x7f53c80a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009c60 tx=0x7f53d0009260 comp rx=0 tx=0).stop 2024-06-03T02:12:54.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53e4102f80 msgr2=0x7f53e419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:54.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53e4102f80 0x7f53e419a8c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53d400e9f0 tx=0x7f53d400ed00 comp rx=0 tx=0).stop 2024-06-03T02:12:54.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 shutdown_connections 2024-06-03T02:12:54.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e41081e0 0x7f53e41a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53e4102f80 0x7f53e419a8c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f53c80a2ca0 0x7f53c80a5160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 --2- 172.21.15.79:0/3313216986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53e41058b0 0x7f53e419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:54.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 >> 172.21.15.79:0/3313216986 conn(0x7f53e40fc990 msgr2=0x7f53e41026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:12:54.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.365+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 shutdown_connections 2024-06-03T02:12:54.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:54.366+0000 7f53e8a00700 1 -- 172.21.15.79:0/3313216986 wait complete. 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3313216986' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:55.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3313216986' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.256 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1717380773,"num_in_osds":12,"osd_in_since":1717380753,"num_remapped_pgs":0} 2024-06-03T02:12:55.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:12:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:12:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3313216986' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T02:12:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:12:55.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:12:55.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:12:55.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:12:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:12:56.164 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:55 smithi079 ceph-mon[31819]: Detected new or changed devices on smithi135 2024-06-03T02:12:56.164 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:55 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi135 to 4272M 2024-06-03T02:12:56.164 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:55 smithi079 ceph-mon[31819]: osdmap e83: 12 total, 12 up, 12 in 2024-06-03T02:12:56.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:55 smithi123 ceph-mon[36640]: Detected new or changed devices on smithi135 2024-06-03T02:12:56.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:55 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi135 to 4272M 2024-06-03T02:12:56.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:55 smithi123 ceph-mon[36640]: osdmap e83: 12 total, 12 up, 12 in 2024-06-03T02:12:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:55 smithi135 ceph-mon[36653]: Detected new or changed devices on smithi135 2024-06-03T02:12:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:55 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi135 to 4272M 2024-06-03T02:12:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:55 smithi135 ceph-mon[36653]: osdmap e83: 12 total, 12 up, 12 in 2024-06-03T02:12:57.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:56 smithi079 ceph-mon[31819]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:12:57.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:56 smithi123 ceph-mon[36640]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:12:57.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:56 smithi135 ceph-mon[36653]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:12:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.972+0000 7fefd5c00700 1 -- 172.21.15.79:0/4264045754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd0074fe0 msgr2=0x7fefd010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.972+0000 7fefd5c00700 1 --2- 172.21.15.79:0/4264045754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd0074fe0 0x7fefd010e9b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009990 tx=0x7fefb8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:12:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.972+0000 7fefd5c00700 1 -- 172.21.15.79:0/4264045754 shutdown_connections 2024-06-03T02:12:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.972+0000 7fefd5c00700 1 --2- 172.21.15.79:0/4264045754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefd010eef0 0x7fefd01112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.972+0000 7fefd5c00700 1 --2- 172.21.15.79:0/4264045754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefd00746c0 0x7fefd0074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.972+0000 7fefd5c00700 1 --2- 172.21.15.79:0/4264045754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd0074fe0 0x7fefd010e9b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.972+0000 7fefd5c00700 1 -- 172.21.15.79:0/4264045754 >> 172.21.15.79:0/4264045754 conn(0x7fefd00795f0 msgr2=0x7fefd0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:12:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.972+0000 7fefd5c00700 1 -- 172.21.15.79:0/4264045754 shutdown_connections 2024-06-03T02:12:57.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.972+0000 7fefd5c00700 1 -- 172.21.15.79:0/4264045754 wait complete. 2024-06-03T02:12:57.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.974+0000 7fefd5c00700 1 Processor -- start 2024-06-03T02:12:57.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.974+0000 7fefd5c00700 1 -- start start 2024-06-03T02:12:57.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.974+0000 7fefd5c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefd00746c0 0x7fefd019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:57.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.974+0000 7fefd5c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefd0074fe0 0x7fefd019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.974+0000 7fefd5c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd010eef0 0x7fefd019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.974+0000 7fefd5c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0114a30 con 0x7fefd010eef0 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.974+0000 7fefd5c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fefd01148b0 con 0x7fefd00746c0 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.974+0000 7fefd5c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0114bb0 con 0x7fefd0074fe0 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcf400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd010eef0 0x7fefd019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefd00746c0 0x7fefd019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefc6000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefd0074fe0 0x7fefd019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcf400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd010eef0 0x7fefd019f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41586/0 (socket says 172.21.15.79:41586) 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefd00746c0 0x7fefd019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59346/0 (socket says 172.21.15.79:59346) 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcf400700 1 -- 172.21.15.79:0/1633571087 learned_addr learned my addr 172.21.15.79:0/1633571087 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcf400700 1 -- 172.21.15.79:0/1633571087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefd0074fe0 msgr2=0x7fefd019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcf400700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefd0074fe0 0x7fefd019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:57.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcf400700 1 -- 172.21.15.79:0/1633571087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefd00746c0 msgr2=0x7fefd019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:57.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcf400700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefd00746c0 0x7fefd019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:57.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcf400700 1 -- 172.21.15.79:0/1633571087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd019fe20 con 0x7fefd010eef0 2024-06-03T02:12:57.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.975+0000 7fefcf400700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd010eef0 0x7fefd019f6e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fefc8007f10 tx=0x7fefc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:12:57.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.976+0000 7fefc7e00700 1 -- 172.21.15.79:0/1633571087 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8004bb0 con 0x7fefd010eef0 2024-06-03T02:12:57.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.976+0000 7fefc7e00700 1 -- 172.21.15.79:0/1633571087 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc8004d50 con 0x7fefd010eef0 2024-06-03T02:12:57.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.976+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd01a0110 con 0x7fefd010eef0 2024-06-03T02:12:57.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.976+0000 7fefc7e00700 1 -- 172.21.15.79:0/1633571087 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8004480 con 0x7fefd010eef0 2024-06-03T02:12:57.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.976+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd01a7990 con 0x7fefd010eef0 2024-06-03T02:12:57.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.977+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd0069180 con 0x7fefd010eef0 2024-06-03T02:12:57.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.979+0000 7fefc7e00700 1 -- 172.21.15.79:0/1633571087 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fefc801a030 con 0x7fefd010eef0 2024-06-03T02:12:57.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.980+0000 7fefc7e00700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fefb40a2ca0 0x7fefb40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:12:57.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.980+0000 7fefc7e00700 1 -- 172.21.15.79:0/1633571087 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fefc80ce090 con 0x7fefd010eef0 2024-06-03T02:12:57.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.981+0000 7fefcea00700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fefb40a2ca0 0x7fefb40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:12:57.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.981+0000 7fefcea00700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fefb40a2ca0 0x7fefb40a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fefc0005950 tx=0x7fefc000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:12:57.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:57.986+0000 7fefc7e00700 1 -- 172.21.15.79:0/1633571087 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fefc8021020 con 0x7fefd010eef0 2024-06-03T02:12:58.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.213+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fefd01a7bc0 con 0x7fefd010eef0 2024-06-03T02:12:58.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.215+0000 7fefc7e00700 1 -- 172.21.15.79:0/1633571087 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18462 (secure 0 0 0) 0x7fefc8010070 con 0x7fefd010eef0 2024-06-03T02:12:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:12:58.216 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":83,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:12:54.894292+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fefb40a2ca0 msgr2=0x7fefb40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fefb40a2ca0 0x7fefb40a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fefc0005950 tx=0x7fefc000b410 comp rx=0 tx=0).stop 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd010eef0 msgr2=0x7fefd019f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd010eef0 0x7fefd019f6e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fefc8007f10 tx=0x7fefc800d3b0 comp rx=0 tx=0).stop 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 shutdown_connections 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fefb40a2ca0 0x7fefb40a5160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefd010eef0 0x7fefd019f6e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fefd00746c0 0x7fefd019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 --2- 172.21.15.79:0/1633571087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefd0074fe0 0x7fefd019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.218+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 >> 172.21.15.79:0/1633571087 conn(0x7fefd00795f0 msgr2=0x7fefd00ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.219+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 shutdown_connections 2024-06-03T02:12:58.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:12:58.219+0000 7fefd5c00700 1 -- 172.21.15.79:0/1633571087 wait complete. 2024-06-03T02:12:59.108 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-03T02:08:38.699457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-03T02:12:59.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-03T02:12:59.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:59 smithi079 ceph-mon[31819]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:12:59.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:12:59 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1633571087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:12:59.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:59 smithi123 ceph-mon[36640]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:12:59.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:12:59 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1633571087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:12:59.551 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:12:59.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:59 smithi135 ceph-mon[36653]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:12:59.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:12:59 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1633571087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:13:01.439 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:01 smithi079 ceph-mon[31819]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:01.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:01 smithi123 ceph-mon[36640]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:01 smithi135 ceph-mon[36653]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.710+0000 7fe8a0600700 1 -- 172.21.15.79:0/233161273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe898102740 msgr2=0x7fe898102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.710+0000 7fe8a0600700 1 --2- 172.21.15.79:0/233161273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe898102740 0x7fe898102b20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0099c0 tx=0x7fe88c009cd0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.711+0000 7fe8a0600700 1 -- 172.21.15.79:0/233161273 shutdown_connections 2024-06-03T02:13:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.711+0000 7fe8a0600700 1 --2- 172.21.15.79:0/233161273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe898107840 0x7fe89810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.711+0000 7fe8a0600700 1 --2- 172.21.15.79:0/233161273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe898102740 0x7fe898102b20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.711+0000 7fe8a0600700 1 --2- 172.21.15.79:0/233161273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8981030f0 0x7fe898107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.711+0000 7fe8a0600700 1 -- 172.21.15.79:0/233161273 >> 172.21.15.79:0/233161273 conn(0x7fe8980795f0 msgr2=0x7fe898079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:01.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.711+0000 7fe8a0600700 1 -- 172.21.15.79:0/233161273 shutdown_connections 2024-06-03T02:13:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.711+0000 7fe8a0600700 1 -- 172.21.15.79:0/233161273 wait complete. 2024-06-03T02:13:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.712+0000 7fe8a0600700 1 Processor -- start 2024-06-03T02:13:01.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.713+0000 7fe8a0600700 1 -- start start 2024-06-03T02:13:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.713+0000 7fe8a0600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe898102740 0x7fe89819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.713+0000 7fe8a0600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8981030f0 0x7fe89819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.713+0000 7fe8a0600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe898107840 0x7fe89819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.713+0000 7fe8a0600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981127b0 con 0x7fe898102740 2024-06-03T02:13:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.713+0000 7fe8a0600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe898112630 con 0x7fe8981030f0 2024-06-03T02:13:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.713+0000 7fe8a0600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe898112930 con 0x7fe898107840 2024-06-03T02:13:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.713+0000 7fe89dc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe898102740 0x7fe89819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89dc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe898102740 0x7fe89819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41606/0 (socket says 172.21.15.79:41606) 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89dc00700 1 -- 172.21.15.79:0/17440149 learned_addr learned my addr 172.21.15.79:0/17440149 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89e600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe898107840 0x7fe89819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89d200700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8981030f0 0x7fe89819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89e600700 1 -- 172.21.15.79:0/17440149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8981030f0 msgr2=0x7fe89819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89e600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8981030f0 0x7fe89819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89e600700 1 -- 172.21.15.79:0/17440149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe898102740 msgr2=0x7fe89819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89e600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe898102740 0x7fe89819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89e600700 1 -- 172.21.15.79:0/17440149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89819fe10 con 0x7fe898107840 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89d200700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8981030f0 0x7fe89819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.714+0000 7fe89e600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe898107840 0x7fe89819f6d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe89000efb0 tx=0x7fe89000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:01.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.715+0000 7fe896000700 1 -- 172.21.15.79:0/17440149 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890003a00 con 0x7fe898107840 2024-06-03T02:13:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.715+0000 7fe896000700 1 -- 172.21.15.79:0/17440149 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe890003ba0 con 0x7fe898107840 2024-06-03T02:13:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.715+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8981a0100 con 0x7fe898107840 2024-06-03T02:13:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.715+0000 7fe896000700 1 -- 172.21.15.79:0/17440149 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890004850 con 0x7fe898107840 2024-06-03T02:13:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.715+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981a7980 con 0x7fe898107840 2024-06-03T02:13:01.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.716+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe898069180 con 0x7fe898107840 2024-06-03T02:13:01.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.718+0000 7fe896000700 1 -- 172.21.15.79:0/17440149 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe890012050 con 0x7fe898107840 2024-06-03T02:13:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.719+0000 7fe896000700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe8800a2c20 0x7fe8800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.719+0000 7fe896000700 1 -- 172.21.15.79:0/17440149 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe8900ce500 con 0x7fe898107840 2024-06-03T02:13:01.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.719+0000 7fe89dc00700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe8800a2c20 0x7fe8800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:01.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.720+0000 7fe89dc00700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe8800a2c20 0x7fe8800a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009990 tx=0x7fe88c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:01.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.724+0000 7fe896000700 1 -- 172.21.15.79:0/17440149 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe890022060 con 0x7fe898107840 2024-06-03T02:13:01.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.945+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8981a7b50 con 0x7fe898107840 2024-06-03T02:13:01.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.946+0000 7fe896000700 1 -- 172.21.15.79:0/17440149 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fe890010040 con 0x7fe898107840 2024-06-03T02:13:01.946 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-06-03T02:13:01.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe8800a2c20 msgr2=0x7fe8800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:01.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe8800a2c20 0x7fe8800a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009990 tx=0x7fe88c005ab0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe898107840 msgr2=0x7fe89819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:01.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe898107840 0x7fe89819f6d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe89000efb0 tx=0x7fe89000c3a0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 shutdown_connections 2024-06-03T02:13:01.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe898107840 0x7fe89819f6d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe8800a2c20 0x7fe8800a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe898102740 0x7fe89819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 --2- 172.21.15.79:0/17440149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8981030f0 0x7fe89819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:01.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 >> 172.21.15.79:0/17440149 conn(0x7fe8980795f0 msgr2=0x7fe898108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:01.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 shutdown_connections 2024-06-03T02:13:01.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:01.949+0000 7fe8a0600700 1 -- 172.21.15.79:0/17440149 wait complete. 2024-06-03T02:13:02.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:02 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/17440149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T02:13:02.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:02 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/17440149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T02:13:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:02 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/17440149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T02:13:02.834 INFO:tasks.cephadm:Adding mds.a on smithi079 2024-06-03T02:13:02.834 INFO:tasks.cephadm:Adding mds.d on smithi079 2024-06-03T02:13:02.834 INFO:tasks.cephadm:Adding mds.g on smithi079 2024-06-03T02:13:02.834 INFO:tasks.cephadm:Adding mds.j on smithi079 2024-06-03T02:13:02.834 INFO:tasks.cephadm:Adding mds.b on smithi123 2024-06-03T02:13:02.834 INFO:tasks.cephadm:Adding mds.e on smithi123 2024-06-03T02:13:02.834 INFO:tasks.cephadm:Adding mds.h on smithi123 2024-06-03T02:13:02.835 INFO:tasks.cephadm:Adding mds.k on smithi123 2024-06-03T02:13:02.835 INFO:tasks.cephadm:Adding mds.c on smithi135 2024-06-03T02:13:02.835 INFO:tasks.cephadm:Adding mds.f on smithi135 2024-06-03T02:13:02.835 INFO:tasks.cephadm:Adding mds.i on smithi135 2024-06-03T02:13:02.835 INFO:tasks.cephadm:Adding mds.l on smithi135 2024-06-03T02:13:02.835 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi079=a;smithi079=d;smithi079=g;smithi079=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l' 2024-06-03T02:13:03.084 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:13:03.455 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:03 smithi135 ceph-mon[36653]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:03.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:03 smithi123 ceph-mon[36640]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:03.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:03 smithi079 ceph-mon[31819]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:04.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 -- 172.21.15.135:0/2334860431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 msgr2=0x7f6df0102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:04.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 --2- 172.21.15.135:0/2334860431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 0x7f6df0102a50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009990 tx=0x7f6de4009ca0 comp rx=0 tx=0).stop 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 -- 172.21.15.135:0/2334860431 shutdown_connections 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 --2- 172.21.15.135:0/2334860431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6df0102f90 0x7f6df0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 --2- 172.21.15.135:0/2334860431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df01013f0 0x7f6df0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 --2- 172.21.15.135:0/2334860431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 0x7f6df0102a50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 -- 172.21.15.135:0/2334860431 >> 172.21.15.135:0/2334860431 conn(0x7f6df00fc990 msgr2=0x7f6df00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 -- 172.21.15.135:0/2334860431 shutdown_connections 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 -- 172.21.15.135:0/2334860431 wait complete. 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.771+0000 7f6df7e49700 1 Processor -- start 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df7e49700 1 -- start start 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df7e49700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df01013f0 0x7f6df0105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:04.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df7e49700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 0x7f6df01039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df7e49700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6df0102f90 0x7f6df0103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df7e49700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010d030 con 0x7f6df0102f90 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df7e49700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010ceb0 con 0x7f6df01013f0 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df7e49700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010d1b0 con 0x7f6df01025f0 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df53e4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 0x7f6df01039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df53e4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 0x7f6df01039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37648/0 (socket says 172.21.15.135:37648) 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df53e4700 1 -- 172.21.15.135:0/1433906268 learned_addr learned my addr 172.21.15.135:0/1433906268 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df5be5700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df01013f0 0x7f6df0105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df53e4700 1 -- 172.21.15.135:0/1433906268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df01013f0 msgr2=0x7f6df0105380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:04.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df53e4700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df01013f0 0x7f6df0105380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df53e4700 1 -- 172.21.15.135:0/1433906268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6df0102f90 msgr2=0x7f6df0103f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df53e4700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6df0102f90 0x7f6df0103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df53e4700 1 -- 172.21.15.135:0/1433906268 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df01047c0 con 0x7f6df01025f0 2024-06-03T02:13:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df53e4700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 0x7f6df01039d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6de40098e0 tx=0x7f6de4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6deeffd700 1 -- 172.21.15.135:0/1433906268 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4027070 con 0x7f6df01025f0 2024-06-03T02:13:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6deeffd700 1 -- 172.21.15.135:0/1433906268 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de40059c0 con 0x7f6df01025f0 2024-06-03T02:13:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df01a2900 con 0x7f6df01025f0 2024-06-03T02:13:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.772+0000 7f6deeffd700 1 -- 172.21.15.135:0/1433906268 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4022e40 con 0x7f6df01025f0 2024-06-03T02:13:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.773+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df01a2dd0 con 0x7f6df01025f0 2024-06-03T02:13:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.774+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df01a3000 con 0x7f6df01025f0 2024-06-03T02:13:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.775+0000 7f6deeffd700 1 -- 172.21.15.135:0/1433906268 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6de402a070 con 0x7f6df01025f0 2024-06-03T02:13:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.775+0000 7f6deeffd700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dd80a2c20 0x7f6dd80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.775+0000 7f6deeffd700 1 -- 172.21.15.135:0/1433906268 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f6de40cfad0 con 0x7f6df01025f0 2024-06-03T02:13:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.776+0000 7f6df5be5700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dd80a2c20 0x7f6dd80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.777+0000 7f6df5be5700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dd80a2c20 0x7f6dd80a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009550 tx=0x7f6de00091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.779+0000 7f6deeffd700 1 -- 172.21.15.135:0/1433906268 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6de4013070 con 0x7f6df01025f0 2024-06-03T02:13:04.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:04.987+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi079=a;smithi079=d;smithi079=g;smithi079=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}) -- 0x7f6df00feca0 con 0x7f6dd80a2c20 2024-06-03T02:13:05.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.003+0000 7f6deeffd700 1 -- 172.21.15.135:0/1433906268 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6df00feca0 con 0x7f6dd80a2c20 2024-06-03T02:13:05.003 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mds.all update... 2024-06-03T02:13:05.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dd80a2c20 msgr2=0x7f6dd80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:05.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dd80a2c20 0x7f6dd80a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009550 tx=0x7f6de00091b0 comp rx=0 tx=0).stop 2024-06-03T02:13:05.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 msgr2=0x7f6df01039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:05.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 0x7f6df01039d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6de40098e0 tx=0x7f6de4005e80 comp rx=0 tx=0).stop 2024-06-03T02:13:05.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 shutdown_connections 2024-06-03T02:13:05.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dd80a2c20 0x7f6dd80a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:05.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6df0102f90 0x7f6df0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:05.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df01013f0 0x7f6df0105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:05.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 --2- 172.21.15.135:0/1433906268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6df01025f0 0x7f6df01039d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:05.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 >> 172.21.15.135:0/1433906268 conn(0x7f6df00fc990 msgr2=0x7f6df00fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:05.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 shutdown_connections 2024-06-03T02:13:05.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-03T02:13:05.005+0000 7f6df7e49700 1 -- 172.21.15.135:0/1433906268 wait complete. 2024-06-03T02:13:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:05 smithi123 ceph-mon[36640]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:13:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:05 smithi079 ceph-mon[31819]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:05.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:13:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:05 smithi135 ceph-mon[36653]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:13:05.697 DEBUG:teuthology.orchestra.run.smithi079:mds.a> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.a.service 2024-06-03T02:13:05.702 DEBUG:teuthology.orchestra.run.smithi079:mds.d> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.d.service 2024-06-03T02:13:05.747 DEBUG:teuthology.orchestra.run.smithi079:mds.g> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.g.service 2024-06-03T02:13:05.793 DEBUG:teuthology.orchestra.run.smithi079:mds.j> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.j.service 2024-06-03T02:13:05.838 DEBUG:teuthology.orchestra.run.smithi123:mds.b> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.b.service 2024-06-03T02:13:05.841 DEBUG:teuthology.orchestra.run.smithi123:mds.e> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.e.service 2024-06-03T02:13:05.884 DEBUG:teuthology.orchestra.run.smithi123:mds.h> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.h.service 2024-06-03T02:13:05.927 DEBUG:teuthology.orchestra.run.smithi123:mds.k> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.k.service 2024-06-03T02:13:05.969 DEBUG:teuthology.orchestra.run.smithi135:mds.c> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.c.service 2024-06-03T02:13:05.972 DEBUG:teuthology.orchestra.run.smithi135:mds.f> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.f.service 2024-06-03T02:13:06.014 DEBUG:teuthology.orchestra.run.smithi135:mds.i> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.i.service 2024-06-03T02:13:06.057 DEBUG:teuthology.orchestra.run.smithi135:mds.l> sudo journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.l.service 2024-06-03T02:13:06.099 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-03T02:13:06.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:13:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:06 smithi135 ceph-mon[36653]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi079=a;smithi079=d;smithi079=g;smithi079=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:13:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:06 smithi135 ceph-mon[36653]: Saving service mds.all spec with placement smithi079=a;smithi079=d;smithi079=g;smithi079=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-06-03T02:13:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:13:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:06 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:06 smithi123 ceph-mon[36640]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi079=a;smithi079=d;smithi079=g;smithi079=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:13:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:06 smithi123 ceph-mon[36640]: Saving service mds.all spec with placement smithi079=a;smithi079=d;smithi079=g;smithi079=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-06-03T02:13:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:13:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:06 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:06.502 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:13:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:06 smithi079 ceph-mon[31819]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi079=a;smithi079=d;smithi079=g;smithi079=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:13:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:06 smithi079 ceph-mon[31819]: Saving service mds.all spec with placement smithi079=a;smithi079=d;smithi079=g;smithi079=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-06-03T02:13:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:13:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:06 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:07.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:07 smithi135 ceph-mon[36653]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:07.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:07 smithi135 ceph-mon[36653]: Deploying daemon mds.l on smithi135 2024-06-03T02:13:07.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:07 smithi123 ceph-mon[36640]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:07.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:07 smithi123 ceph-mon[36640]: Deploying daemon mds.l on smithi135 2024-06-03T02:13:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:07 smithi079 ceph-mon[31819]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:07 smithi079 ceph-mon[31819]: Deploying daemon mds.l on smithi135 2024-06-03T02:13:08.345 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:13:08 smithi135 systemd[1]: Starting Ceph mds.l for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:13:08.696 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:13:08 smithi135 podman[66577]: 2024-06-03 02:13:08.441877047 +0000 UTC m=+0.114466116 container create 037a71e766e0435a82133c46b769771d8b01fe1517edd7b0825865ff540e53e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-03T02:13:08.696 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:13:08 smithi135 podman[66577]: 2024-06-03 02:13:08.344678033 +0000 UTC m=+0.017267126 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:13:08.696 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:13:08 smithi135 podman[66577]: 2024-06-03 02:13:08.670631771 +0000 UTC m=+0.343220846 container init 037a71e766e0435a82133c46b769771d8b01fe1517edd7b0825865ff540e53e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-03T02:13:08.696 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:13:08 smithi135 podman[66577]: 2024-06-03 02:13:08.673104539 +0000 UTC m=+0.345693619 container start 037a71e766e0435a82133c46b769771d8b01fe1517edd7b0825865ff540e53e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:13:08.696 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:13:08 smithi135 ceph-mds[66596]: -- 172.21.15.135:0/4227795043 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db09e5fa40 con 0x55db09e33000 2024-06-03T02:13:08.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.782+0000 7ff31e800700 1 -- 172.21.15.79:0/3162159332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff31810a680 msgr2=0x7ff31810aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:08.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.782+0000 7ff31e800700 1 --2- 172.21.15.79:0/3162159332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff31810a680 0x7ff31810aa60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff308009990 tx=0x7ff308009ca0 comp rx=0 tx=0).stop 2024-06-03T02:13:08.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.782+0000 7ff31e800700 1 -- 172.21.15.79:0/3162159332 shutdown_connections 2024-06-03T02:13:08.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.782+0000 7ff31e800700 1 --2- 172.21.15.79:0/3162159332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318105040 0x7ff318111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:08.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.782+0000 7ff31e800700 1 --2- 172.21.15.79:0/3162159332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff31810a680 0x7ff31810aa60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:08.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.782+0000 7ff31e800700 1 --2- 172.21.15.79:0/3162159332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff318104680 0x7ff318104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:08.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.782+0000 7ff31e800700 1 -- 172.21.15.79:0/3162159332 >> 172.21.15.79:0/3162159332 conn(0x7ff318100170 msgr2=0x7ff318102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:08.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.782+0000 7ff31e800700 1 -- 172.21.15.79:0/3162159332 shutdown_connections 2024-06-03T02:13:08.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.782+0000 7ff31e800700 1 -- 172.21.15.79:0/3162159332 wait complete. 2024-06-03T02:13:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.783+0000 7ff31e800700 1 Processor -- start 2024-06-03T02:13:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.783+0000 7ff31e800700 1 -- start start 2024-06-03T02:13:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff31e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318104680 0x7ff31819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff31e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff318105040 0x7ff31819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff31e800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff31810a680 0x7ff31819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff31e800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff318114a20 con 0x7ff318104680 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff31e800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181148a0 con 0x7ff31810a680 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff31e800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff318114ba0 con 0x7ff318105040 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff317400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318104680 0x7ff31819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff316a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff318105040 0x7ff31819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff317400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318104680 0x7ff31819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51652/0 (socket says 172.21.15.79:51652) 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff317400700 1 -- 172.21.15.79:0/712616309 learned_addr learned my addr 172.21.15.79:0/712616309 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.784+0000 7ff317e00700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff31810a680 0x7ff31819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff316a00700 1 -- 172.21.15.79:0/712616309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff31810a680 msgr2=0x7ff31819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff316a00700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff31810a680 0x7ff31819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff316a00700 1 -- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318104680 msgr2=0x7ff31819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff316a00700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318104680 0x7ff31819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff316a00700 1 -- 172.21.15.79:0/712616309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31819fe80 con 0x7ff318105040 2024-06-03T02:13:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff317400700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318104680 0x7ff31819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:13:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff316a00700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff318105040 0x7ff31819b3b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff30400e9f0 tx=0x7ff30400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff30fe00700 1 -- 172.21.15.79:0/712616309 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304004220 con 0x7ff318105040 2024-06-03T02:13:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff30fe00700 1 -- 172.21.15.79:0/712616309 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff304004d10 con 0x7ff318105040 2024-06-03T02:13:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3181a0170 con 0x7ff318105040 2024-06-03T02:13:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff30fe00700 1 -- 172.21.15.79:0/712616309 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304005020 con 0x7ff318105040 2024-06-03T02:13:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.785+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3181a79f0 con 0x7ff318105040 2024-06-03T02:13:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.787+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31810cc10 con 0x7ff318105040 2024-06-03T02:13:08.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.789+0000 7ff30fe00700 1 -- 172.21.15.79:0/712616309 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff3040053d0 con 0x7ff318105040 2024-06-03T02:13:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.791+0000 7ff30fe00700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff2fc0a2c20 0x7ff2fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.791+0000 7ff30fe00700 1 -- 172.21.15.79:0/712616309 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff3040ced90 con 0x7ff318105040 2024-06-03T02:13:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.791+0000 7ff317400700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff2fc0a2c20 0x7ff2fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.792+0000 7ff317400700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff2fc0a2c20 0x7ff2fc0a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff308006010 tx=0x7ff30800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:08.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:08.795+0000 7ff30fe00700 1 -- 172.21.15.79:0/712616309 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff30409c0f0 con 0x7ff318105040 2024-06-03T02:13:09.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.070+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff318069180 con 0x7ff318105040 2024-06-03T02:13:09.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.071+0000 7ff30fe00700 1 -- 172.21.15.79:0/712616309 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7ff30409b9f0 con 0x7ff318105040 2024-06-03T02:13:09.072 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:13:09.072 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-03T02:13:09.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff2fc0a2c20 msgr2=0x7ff2fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:09.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff2fc0a2c20 0x7ff2fc0a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff308006010 tx=0x7ff30800b260 comp rx=0 tx=0).stop 2024-06-03T02:13:09.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff318105040 msgr2=0x7ff31819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:09.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff318105040 0x7ff31819b3b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff30400e9f0 tx=0x7ff30400ed00 comp rx=0 tx=0).stop 2024-06-03T02:13:09.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 shutdown_connections 2024-06-03T02:13:09.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff31810a680 0x7ff31819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:09.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff2fc0a2c20 0x7ff2fc0a50e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:09.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318104680 0x7ff31819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:09.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 --2- 172.21.15.79:0/712616309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff318105040 0x7ff31819b3b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:09.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 >> 172.21.15.79:0/712616309 conn(0x7ff318100170 msgr2=0x7ff318100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:09.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 shutdown_connections 2024-06-03T02:13:09.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:09.075+0000 7ff31e800700 1 -- 172.21.15.79:0/712616309 wait complete. 2024-06-03T02:13:09.077 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 2 2024-06-03T02:13:09.095 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:13:08 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l[66592]: starting mds.l at 2024-06-03T02:13:09.095 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:13:08 smithi135 bash[66577]: 037a71e766e0435a82133c46b769771d8b01fe1517edd7b0825865ff540e53e4 2024-06-03T02:13:09.095 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:13:08 smithi135 systemd[1]: Started Ceph mds.l for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:09.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.135:6832/1231185643,v1:172.21.15.135:6833/1231185643] up:boot 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: fsmap 1 up:standby 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:09.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:09 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/712616309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:09.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:09.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.135:6832/1231185643,v1:172.21.15.135:6833/1231185643] up:boot 2024-06-03T02:13:09.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: fsmap 1 up:standby 2024-06-03T02:13:09.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T02:13:09.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:09.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:09.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:09.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:09.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:09.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:09.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:09 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/712616309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.135:6832/1231185643,v1:172.21.15.135:6833/1231185643] up:boot 2024-06-03T02:13:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: fsmap 1 up:standby 2024-06-03T02:13:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T02:13:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:09.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:09.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:09.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:09.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:09.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:09 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/712616309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:10 smithi123 ceph-mon[36640]: Deploying daemon mds.k on smithi123 2024-06-03T02:13:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:10 smithi123 ceph-mon[36640]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:10 smithi079 ceph-mon[31819]: Deploying daemon mds.k on smithi123 2024-06-03T02:13:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:10 smithi079 ceph-mon[31819]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:10 smithi135 ceph-mon[36653]: Deploying daemon mds.k on smithi123 2024-06-03T02:13:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:10 smithi135 ceph-mon[36653]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:11.223 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:13:11 smithi123 systemd[1]: Starting Ceph mds.k for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:13:11.575 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:13:11 smithi123 podman[66563]: 2024-06-03 02:13:11.210486681 +0000 UTC m=+0.017594067 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:13:11.575 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:13:11 smithi123 podman[66563]: 2024-06-03 02:13:11.332359363 +0000 UTC m=+0.139466726 container create cd3843bbaf1ddfc1bdc1ed045c53acad17613b0f6d899cd8fed8f5279ed173ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-03T02:13:11.576 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:13:11 smithi123 podman[66563]: 2024-06-03 02:13:11.576558706 +0000 UTC m=+0.383666069 container init cd3843bbaf1ddfc1bdc1ed045c53acad17613b0f6d899cd8fed8f5279ed173ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-03T02:13:11.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:13:11.973 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:13:11 smithi123 podman[66563]: 2024-06-03 02:13:11.579681802 +0000 UTC m=+0.386789168 container start cd3843bbaf1ddfc1bdc1ed045c53acad17613b0f6d899cd8fed8f5279ed173ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-03T02:13:11.973 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:13:11 smithi123 ceph-mds[66582]: -- 172.21.15.123:0/436085184 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556828d67a40 con 0x556828d3b000 2024-06-03T02:13:11.973 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:13:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k[66578]: starting mds.k at 2024-06-03T02:13:11.973 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:13:11 smithi123 bash[66563]: cd3843bbaf1ddfc1bdc1ed045c53acad17613b0f6d899cd8fed8f5279ed173ee 2024-06-03T02:13:11.973 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:13:11 smithi123 systemd[1]: Started Ceph mds.k for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:12.361 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:13:12.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6832/296204924,v1:172.21.15.123:6833/296204924] up:boot 2024-06-03T02:13:12.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: fsmap 2 up:standby 2024-06-03T02:13:12.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T02:13:12.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:12.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:12.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:12.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:12.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:12.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:12.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: Deploying daemon mds.j on smithi079 2024-06-03T02:13:12.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:12 smithi079 ceph-mon[31819]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6832/296204924,v1:172.21.15.123:6833/296204924] up:boot 2024-06-03T02:13:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: fsmap 2 up:standby 2024-06-03T02:13:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T02:13:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:12.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:12.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: Deploying daemon mds.j on smithi079 2024-06-03T02:13:12.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:12 smithi123 ceph-mon[36640]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:13.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6832/296204924,v1:172.21.15.123:6833/296204924] up:boot 2024-06-03T02:13:13.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: fsmap 2 up:standby 2024-06-03T02:13:13.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T02:13:13.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:13.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:13.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:13.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:13.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:13.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:13.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: Deploying daemon mds.j on smithi079 2024-06-03T02:13:13.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:12 smithi135 ceph-mon[36653]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:15.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.182+0000 7f68b2800700 1 -- 172.21.15.79:0/700139377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68ac102ee0 msgr2=0x7f68ac10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:15.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.182+0000 7f68b2800700 1 --2- 172.21.15.79:0/700139377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68ac102ee0 0x7f68ac10f3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009f90 tx=0x7f68a400b330 comp rx=0 tx=0).stop 2024-06-03T02:13:15.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.182+0000 7f68b2800700 1 -- 172.21.15.79:0/700139377 shutdown_connections 2024-06-03T02:13:15.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.182+0000 7f68b2800700 1 --2- 172.21.15.79:0/700139377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68ac102ee0 0x7f68ac10f3c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.182+0000 7f68b2800700 1 --2- 172.21.15.79:0/700139377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac107450 0x7f68ac107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.182+0000 7f68b2800700 1 --2- 172.21.15.79:0/700139377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac102540 0x7f68ac1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.182+0000 7f68b2800700 1 -- 172.21.15.79:0/700139377 >> 172.21.15.79:0/700139377 conn(0x7f68ac0795f0 msgr2=0x7f68ac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:15.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.182+0000 7f68b2800700 1 -- 172.21.15.79:0/700139377 shutdown_connections 2024-06-03T02:13:15.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.182+0000 7f68b2800700 1 -- 172.21.15.79:0/700139377 wait complete. 2024-06-03T02:13:15.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.184+0000 7f68b2800700 1 Processor -- start 2024-06-03T02:13:15.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.184+0000 7f68b2800700 1 -- start start 2024-06-03T02:13:15.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.184+0000 7f68b2800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68ac102540 0x7f68ac19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:15.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68b2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac102ee0 0x7f68ac19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:15.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68b2800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac107450 0x7f68ac19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:15.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68b2800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac112830 con 0x7f68ac107450 2024-06-03T02:13:15.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68b2800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1126b0 con 0x7f68ac102540 2024-06-03T02:13:15.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68b2800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1129b0 con 0x7f68ac102ee0 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68a2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac102ee0 0x7f68ac19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68a2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac102ee0 0x7f68ac19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52044/0 (socket says 172.21.15.79:52044) 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68ab400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68ac102540 0x7f68ac19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68a2a00700 1 -- 172.21.15.79:0/4211302038 learned_addr learned my addr 172.21.15.79:0/4211302038 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68abe00700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac107450 0x7f68ac19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68a2a00700 1 -- 172.21.15.79:0/4211302038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68ac102540 msgr2=0x7f68ac19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68a2a00700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68ac102540 0x7f68ac19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68a2a00700 1 -- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac107450 msgr2=0x7f68ac19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68a2a00700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac107450 0x7f68ac19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68a2a00700 1 -- 172.21.15.79:0/4211302038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac19fdd0 con 0x7f68ac102ee0 2024-06-03T02:13:15.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.185+0000 7f68abe00700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac107450 0x7f68ac19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:13:15.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.186+0000 7f68a2a00700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac102ee0 0x7f68ac19b300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f689400e9c0 tx=0x7f689400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:15.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.186+0000 7f68a8c00700 1 -- 172.21.15.79:0/4211302038 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689400cbd0 con 0x7f68ac102ee0 2024-06-03T02:13:15.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.186+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ac1a00c0 con 0x7f68ac102ee0 2024-06-03T02:13:15.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.186+0000 7f68a8c00700 1 -- 172.21.15.79:0/4211302038 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f689400cd70 con 0x7f68ac102ee0 2024-06-03T02:13:15.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.186+0000 7f68a8c00700 1 -- 172.21.15.79:0/4211302038 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894005660 con 0x7f68ac102ee0 2024-06-03T02:13:15.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.186+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ac1a7940 con 0x7f68ac102ee0 2024-06-03T02:13:15.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.187+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ac069180 con 0x7f68ac102ee0 2024-06-03T02:13:15.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.191+0000 7f68a8c00700 1 -- 172.21.15.79:0/4211302038 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6894004750 con 0x7f68ac102ee0 2024-06-03T02:13:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.192+0000 7f68a8c00700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f68900a2c20 0x7f68900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.192+0000 7f68a8c00700 1 -- 172.21.15.79:0/4211302038 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f68940ced00 con 0x7f68ac102ee0 2024-06-03T02:13:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.192+0000 7f68ab400700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f68900a2c20 0x7f68900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.193+0000 7f68ab400700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f68900a2c20 0x7f68900a50e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f68ac103720 tx=0x7f689c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.196+0000 7f68a8c00700 1 -- 172.21.15.79:0/4211302038 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f689409c100 con 0x7f68ac102ee0 2024-06-03T02:13:15.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:15 smithi135 ceph-mon[36653]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:15.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.465+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68ac19c120 con 0x7f68ac102ee0 2024-06-03T02:13:15.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:15 smithi079 ceph-mon[31819]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:15.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.466+0000 7f68a8c00700 1 -- 172.21.15.79:0/4211302038 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f68ac19c120 con 0x7f68ac102ee0 2024-06-03T02:13:15.465 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:13:15.465 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-03T02:13:15.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f68900a2c20 msgr2=0x7f68900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:15.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f68900a2c20 0x7f68900a50e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f68ac103720 tx=0x7f689c009260 comp rx=0 tx=0).stop 2024-06-03T02:13:15.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac102ee0 msgr2=0x7f68ac19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:15.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac102ee0 0x7f68ac19b300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f689400e9c0 tx=0x7f689400ecd0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 shutdown_connections 2024-06-03T02:13:15.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f68900a2c20 0x7f68900a50e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68ac107450 0x7f68ac19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68ac102540 0x7f68ac19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 --2- 172.21.15.79:0/4211302038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac102ee0 0x7f68ac19b300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:15.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 >> 172.21.15.79:0/4211302038 conn(0x7f68ac0795f0 msgr2=0x7f68ac1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:15.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 shutdown_connections 2024-06-03T02:13:15.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:15.469+0000 7f68b2800700 1 -- 172.21.15.79:0/4211302038 wait complete. 2024-06-03T02:13:15.470 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 3 2024-06-03T02:13:15.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:15 smithi123 ceph-mon[36640]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:16.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:16 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4211302038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:16.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:16 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4211302038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:16 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4211302038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:17.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:17 smithi079 ceph-mon[31819]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:17.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:17 smithi123 ceph-mon[36640]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:17.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:17 smithi135 ceph-mon[36653]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:17.837 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:13:17 smithi079 systemd[1]: Started Ceph mds.j for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:18.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:13:18.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:18.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:18.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:18.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:18.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:18.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:18.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: Deploying daemon mds.d on smithi079 2024-06-03T02:13:18.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:18.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.79:6834/859282294,v1:172.21.15.79:6835/859282294] up:boot 2024-06-03T02:13:18.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: fsmap 3 up:standby 2024-06-03T02:13:18.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:18 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: Deploying daemon mds.d on smithi079 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.79:6834/859282294,v1:172.21.15.79:6835/859282294] up:boot 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: fsmap 3 up:standby 2024-06-03T02:13:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:18 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: Deploying daemon mds.d on smithi079 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.79:6834/859282294,v1:172.21.15.79:6835/859282294] up:boot 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: fsmap 3 up:standby 2024-06-03T02:13:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:18 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T02:13:20.025 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:13:21.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:21 smithi079 ceph-mon[31819]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:21.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:21 smithi123 ceph-mon[36640]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:21 smithi135 ceph-mon[36653]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:22.089 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:13:21 smithi079 podman[73203]: 2024-06-03 02:13:21.998828752 +0000 UTC m=+0.157033521 container create 4f6fb36aef5945b21fd19cc2ab5b74bfd07d7dfe8d84e61e683c91dfa1483a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:13:22.588 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:13:22 smithi079 podman[73203]: 2024-06-03 02:13:22.404278129 +0000 UTC m=+0.562482899 container init 4f6fb36aef5945b21fd19cc2ab5b74bfd07d7dfe8d84e61e683c91dfa1483a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:13:22.588 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:13:22 smithi079 podman[73203]: 2024-06-03 02:13:22.414633422 +0000 UTC m=+0.572838119 container start 4f6fb36aef5945b21fd19cc2ab5b74bfd07d7dfe8d84e61e683c91dfa1483a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-06-03T02:13:22.588 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:13:22 smithi079 ceph-mds[73240]: -- 172.21.15.79:0/2896236467 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc9b0eda40 con 0x55fc9b0c1000 2024-06-03T02:13:22.588 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:13:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d[73234]: starting mds.d at 2024-06-03T02:13:22.588 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:13:22 smithi079 bash[73203]: 4f6fb36aef5945b21fd19cc2ab5b74bfd07d7dfe8d84e61e683c91dfa1483a2a 2024-06-03T02:13:22.589 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:13:22 smithi079 systemd[1]: Started Ceph mds.d for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:23.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.062+0000 7fb7d3e00700 1 -- 172.21.15.79:0/4219618621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d4104f70 msgr2=0x7fb7d4111430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:23.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.063+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/4219618621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d4104f70 0x7fb7d4111430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b0d0 tx=0x7fb7cc00b3e0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.063+0000 7fb7d3e00700 1 -- 172.21.15.79:0/4219618621 shutdown_connections 2024-06-03T02:13:23.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.063+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/4219618621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d4104f70 0x7fb7d4111430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.063+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/4219618621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d410a660 0x7fb7d410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.063+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/4219618621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d41045b0 0x7fb7d4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.063+0000 7fb7d3e00700 1 -- 172.21.15.79:0/4219618621 >> 172.21.15.79:0/4219618621 conn(0x7fb7d4100080 msgr2=0x7fb7d41024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:23.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.063+0000 7fb7d3e00700 1 -- 172.21.15.79:0/4219618621 shutdown_connections 2024-06-03T02:13:23.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.063+0000 7fb7d3e00700 1 -- 172.21.15.79:0/4219618621 wait complete. 2024-06-03T02:13:23.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.064+0000 7fb7d3e00700 1 Processor -- start 2024-06-03T02:13:23.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.064+0000 7fb7d3e00700 1 -- start start 2024-06-03T02:13:23.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.064+0000 7fb7d3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d41045b0 0x7fb7d41a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:23.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.064+0000 7fb7d3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d4104f70 0x7fb7d41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.064+0000 7fb7d3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d410a660 0x7fb7d419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.064+0000 7fb7d3e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4114960 con 0x7fb7d4104f70 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.064+0000 7fb7d3e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41147e0 con 0x7fb7d410a660 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.064+0000 7fb7d3e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4114ae0 con 0x7fb7d41045b0 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d410a660 0x7fb7d419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d2000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d4104f70 0x7fb7d41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d410a660 0x7fb7d419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:39278/0 (socket says 172.21.15.79:39278) 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d3400700 1 -- 172.21.15.79:0/3470385604 learned_addr learned my addr 172.21.15.79:0/3470385604 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d2a00700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d41045b0 0x7fb7d41a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d3400700 1 -- 172.21.15.79:0/3470385604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d41045b0 msgr2=0x7fb7d41a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d3400700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d41045b0 0x7fb7d41a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d3400700 1 -- 172.21.15.79:0/3470385604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d4104f70 msgr2=0x7fb7d41a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:23.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d3400700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d4104f70 0x7fb7d41a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d3400700 1 -- 172.21.15.79:0/3470385604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d419b1a0 con 0x7fb7d410a660 2024-06-03T02:13:23.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.065+0000 7fb7d3400700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d410a660 0x7fb7d419aa40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00d010 tx=0x7fb7cc0079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:23.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.066+0000 7fb7cb400700 1 -- 172.21.15.79:0/3470385604 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc003c00 con 0x7fb7d410a660 2024-06-03T02:13:23.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.066+0000 7fb7cb400700 1 -- 172.21.15.79:0/3470385604 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc022070 con 0x7fb7d410a660 2024-06-03T02:13:23.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.066+0000 7fb7cb400700 1 -- 172.21.15.79:0/3470385604 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc01b6c0 con 0x7fb7d410a660 2024-06-03T02:13:23.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.066+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d419b430 con 0x7fb7d410a660 2024-06-03T02:13:23.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.066+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d41a7750 con 0x7fb7d410a660 2024-06-03T02:13:23.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.067+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d4105cd0 con 0x7fb7d410a660 2024-06-03T02:13:23.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.069+0000 7fb7cb400700 1 -- 172.21.15.79:0/3470385604 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb7cc009ba0 con 0x7fb7d410a660 2024-06-03T02:13:23.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.070+0000 7fb7cb400700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7b80a2ca0 0x7fb7b80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:23.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.071+0000 7fb7cb400700 1 -- 172.21.15.79:0/3470385604 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb7cc00f020 con 0x7fb7d410a660 2024-06-03T02:13:23.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.071+0000 7fb7d2a00700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7b80a2ca0 0x7fb7b80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:23.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.071+0000 7fb7d2a00700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7b80a2ca0 0x7fb7b80a5160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb7d419bb80 tx=0x7fb7c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:23.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.072+0000 7fb7cb400700 1 -- 172.21.15.79:0/3470385604 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb7cc024310 con 0x7fb7d410a660 2024-06-03T02:13:23.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:23 smithi079 ceph-mon[31819]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:23.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:23.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:23.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:23.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:23.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:23.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:23.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.355+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7d4069180 con 0x7fb7d410a660 2024-06-03T02:13:23.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.355+0000 7fb7cb400700 1 -- 172.21.15.79:0/3470385604 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7fb7d4069180 con 0x7fb7d410a660 2024-06-03T02:13:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:13:23.356 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-06-03T02:13:23.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7b80a2ca0 msgr2=0x7fb7b80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:23.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7b80a2ca0 0x7fb7b80a5160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb7d419bb80 tx=0x7fb7c4009260 comp rx=0 tx=0).stop 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d410a660 msgr2=0x7fb7d419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d410a660 0x7fb7d419aa40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00d010 tx=0x7fb7cc0079a0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 shutdown_connections 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7d410a660 0x7fb7d419aa40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d41045b0 0x7fb7d41a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7b80a2ca0 0x7fb7b80a5160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 --2- 172.21.15.79:0/3470385604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7d4104f70 0x7fb7d41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 >> 172.21.15.79:0/3470385604 conn(0x7fb7d4100080 msgr2=0x7fb7d41015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.358+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 shutdown_connections 2024-06-03T02:13:23.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:23.359+0000 7fb7d3e00700 1 -- 172.21.15.79:0/3470385604 wait complete. 2024-06-03T02:13:23.360 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 5 2024-06-03T02:13:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:23 smithi123 ceph-mon[36640]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:23 smithi135 ceph-mon[36653]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:24.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:24 smithi079 ceph-mon[31819]: Deploying daemon mds.e on smithi123 2024-06-03T02:13:24.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:24 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.79:6836/476881413,v1:172.21.15.79:6837/476881413] up:boot 2024-06-03T02:13:24.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:24 smithi079 ceph-mon[31819]: fsmap 4 up:standby 2024-06-03T02:13:24.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T02:13:24.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:24 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3470385604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:24 smithi123 ceph-mon[36640]: Deploying daemon mds.e on smithi123 2024-06-03T02:13:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:24 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.79:6836/476881413,v1:172.21.15.79:6837/476881413] up:boot 2024-06-03T02:13:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:24 smithi123 ceph-mon[36640]: fsmap 4 up:standby 2024-06-03T02:13:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:24 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T02:13:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:24 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3470385604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:24 smithi135 ceph-mon[36653]: Deploying daemon mds.e on smithi123 2024-06-03T02:13:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:24 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.79:6836/476881413,v1:172.21.15.79:6837/476881413] up:boot 2024-06-03T02:13:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:24 smithi135 ceph-mon[36653]: fsmap 4 up:standby 2024-06-03T02:13:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:24 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T02:13:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:24 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3470385604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:24.973 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:13:24 smithi123 systemd[1]: Starting Ceph mds.e for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:13:25.314 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:25 smithi123 ceph-mon[36640]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:25.315 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:13:25 smithi123 podman[66873]: 2024-06-03 02:13:24.924596714 +0000 UTC m=+0.016872528 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:13:25.315 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:13:25 smithi123 podman[66873]: 2024-06-03 02:13:25.037313425 +0000 UTC m=+0.129589214 container create 6b4da3e35fdd33b28cf4df8d0cc35a2b758126be51ba56acb2ba3eb55459a0c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-03T02:13:25.315 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:13:25 smithi123 podman[66873]: 2024-06-03 02:13:25.315408056 +0000 UTC m=+0.407683851 container init 6b4da3e35fdd33b28cf4df8d0cc35a2b758126be51ba56acb2ba3eb55459a0c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:13:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:25 smithi079 ceph-mon[31819]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:25.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:25 smithi135 ceph-mon[36653]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:25.723 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:13:25 smithi123 podman[66873]: 2024-06-03 02:13:25.318326604 +0000 UTC m=+0.410602399 container start 6b4da3e35fdd33b28cf4df8d0cc35a2b758126be51ba56acb2ba3eb55459a0c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:13:25.723 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:13:25 smithi123 ceph-mds[66893]: -- 172.21.15.123:0/2760738045 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a464199a40 con 0x55a46416cc00 2024-06-03T02:13:25.723 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:13:25 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e[66889]: starting mds.e at 2024-06-03T02:13:25.723 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:13:25 smithi123 bash[66873]: 6b4da3e35fdd33b28cf4df8d0cc35a2b758126be51ba56acb2ba3eb55459a0c7 2024-06-03T02:13:25.723 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:13:25 smithi123 systemd[1]: Started Ceph mds.e for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:26.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: Deploying daemon mds.f on smithi135 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] up:boot 2024-06-03T02:13:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: fsmap 5 up:standby 2024-06-03T02:13:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T02:13:26.589 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:13:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: Deploying daemon mds.f on smithi135 2024-06-03T02:13:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] up:boot 2024-06-03T02:13:26.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: fsmap 5 up:standby 2024-06-03T02:13:26.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T02:13:26.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:26.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:26.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:26.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:26.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:26.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:26.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: Deploying daemon mds.f on smithi135 2024-06-03T02:13:26.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:26.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] up:boot 2024-06-03T02:13:26.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: fsmap 5 up:standby 2024-06-03T02:13:26.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T02:13:27.845 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:13:27 smithi135 systemd[1]: Starting Ceph mds.f for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:13:28.235 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:13:27 smithi135 podman[66879]: 2024-06-03 02:13:27.869450568 +0000 UTC m=+0.017358535 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:13:28.235 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:13:27 smithi135 podman[66879]: 2024-06-03 02:13:27.998889512 +0000 UTC m=+0.146797505 container create db117653c628e33ee691f7b2817c6614cee5ae93133ed184185cac3c70b1ef28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-03T02:13:28.235 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:13:28 smithi135 podman[66879]: 2024-06-03 02:13:28.209980306 +0000 UTC m=+0.357888261 container init db117653c628e33ee691f7b2817c6614cee5ae93133ed184185cac3c70b1ef28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-03T02:13:28.236 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:13:28 smithi135 podman[66879]: 2024-06-03 02:13:28.212766046 +0000 UTC m=+0.360674009 container start db117653c628e33ee691f7b2817c6614cee5ae93133ed184185cac3c70b1ef28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-03T02:13:28.595 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:13:28 smithi135 ceph-mds[66898]: -- 172.21.15.135:0/3268360621 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b77e79a40 con 0x562b77e4cc00 2024-06-03T02:13:28.595 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:13:28 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f[66894]: starting mds.f at 2024-06-03T02:13:28.595 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:13:28 smithi135 bash[66879]: db117653c628e33ee691f7b2817c6614cee5ae93133ed184185cac3c70b1ef28 2024-06-03T02:13:28.595 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:13:28 smithi135 systemd[1]: Started Ceph mds.f for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:28.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.867+0000 7f7406800700 1 -- 172.21.15.79:0/3091445556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74001046b0 msgr2=0x7f7400104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:28.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.867+0000 7f7406800700 1 --2- 172.21.15.79:0/3091445556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74001046b0 0x7f7400104b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009990 tx=0x7f73ec009ca0 comp rx=0 tx=0).stop 2024-06-03T02:13:28.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.867+0000 7f7406800700 1 -- 172.21.15.79:0/3091445556 shutdown_connections 2024-06-03T02:13:28.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.867+0000 7f7406800700 1 --2- 172.21.15.79:0/3091445556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7400105070 0x7f7400111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:28.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.867+0000 7f7406800700 1 --2- 172.21.15.79:0/3091445556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f740010a6b0 0x7f740010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:28.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.867+0000 7f7406800700 1 --2- 172.21.15.79:0/3091445556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74001046b0 0x7f7400104b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:28.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.867+0000 7f7406800700 1 -- 172.21.15.79:0/3091445556 >> 172.21.15.79:0/3091445556 conn(0x7f74001001a0 msgr2=0x7f74001025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:28.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.867+0000 7f7406800700 1 -- 172.21.15.79:0/3091445556 shutdown_connections 2024-06-03T02:13:28.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.867+0000 7f7406800700 1 -- 172.21.15.79:0/3091445556 wait complete. 2024-06-03T02:13:28.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.868+0000 7f7406800700 1 Processor -- start 2024-06-03T02:13:28.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f7406800700 1 -- start start 2024-06-03T02:13:28.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f7406800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001046b0 0x7f740019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:28.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f7406800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7400105070 0x7f740019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:28.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f7406800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f740010a6b0 0x7f740019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:28.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f7406800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f74001149e0 con 0x7f74001046b0 2024-06-03T02:13:28.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f7406800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7400114860 con 0x7f740010a6b0 2024-06-03T02:13:28.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f7406800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7400114b60 con 0x7f7400105070 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f73fea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7400105070 0x7f740019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f73fea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7400105070 0x7f740019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46408/0 (socket says 172.21.15.79:46408) 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f73fea00700 1 -- 172.21.15.79:0/264071072 learned_addr learned my addr 172.21.15.79:0/264071072 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f73ff400700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001046b0 0x7f740019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.869+0000 7f73ffe00700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f740010a6b0 0x7f740019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f73fea00700 1 -- 172.21.15.79:0/264071072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f740010a6b0 msgr2=0x7f740019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f73fea00700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f740010a6b0 0x7f740019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f73fea00700 1 -- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001046b0 msgr2=0x7f740019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f73fea00700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001046b0 0x7f740019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f73fea00700 1 -- 172.21.15.79:0/264071072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740019fe80 con 0x7f7400105070 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f73fea00700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7400105070 0x7f740019b3b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73ec000c00 tx=0x7f73ec00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f73f7e00700 1 -- 172.21.15.79:0/264071072 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec027070 con 0x7f7400105070 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f73f7e00700 1 -- 172.21.15.79:0/264071072 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73ec003d70 con 0x7f7400105070 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74001a0110 con 0x7f7400105070 2024-06-03T02:13:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.870+0000 7f73f7e00700 1 -- 172.21.15.79:0/264071072 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00f6c0 con 0x7f7400105070 2024-06-03T02:13:28.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.871+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74001a79f0 con 0x7f7400105070 2024-06-03T02:13:28.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.872+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7400105df0 con 0x7f7400105070 2024-06-03T02:13:28.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.874+0000 7f73f7e00700 1 -- 172.21.15.79:0/264071072 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f73ec02a070 con 0x7f7400105070 2024-06-03T02:13:28.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.875+0000 7f73f7e00700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73e40a2c20 0x7f73e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:28.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.875+0000 7f73f7e00700 1 -- 172.21.15.79:0/264071072 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f73ec0d0100 con 0x7f7400105070 2024-06-03T02:13:28.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.875+0000 7f73ff400700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73e40a2c20 0x7f73e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:28.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.876+0000 7f73ff400700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73e40a2c20 0x7f73e40a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f73f0005950 tx=0x7f73f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:28.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:28.877+0000 7f73f7e00700 1 -- 172.21.15.79:0/264071072 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73ec031070 con 0x7f7400105070 2024-06-03T02:13:29.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.148+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f740019c1d0 con 0x7f7400105070 2024-06-03T02:13:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.149+0000 7f73f7e00700 1 -- 172.21.15.79:0/264071072 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f73ec013070 con 0x7f7400105070 2024-06-03T02:13:29.149 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:13:29.150 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-03T02:13:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73e40a2c20 msgr2=0x7f73e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73e40a2c20 0x7f73e40a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f73f0005950 tx=0x7f73f000b410 comp rx=0 tx=0).stop 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7400105070 msgr2=0x7f740019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7400105070 0x7f740019b3b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73ec000c00 tx=0x7f73ec00bab0 comp rx=0 tx=0).stop 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 shutdown_connections 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f740010a6b0 0x7f740019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73e40a2c20 0x7f73e40a50e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001046b0 0x7f740019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 --2- 172.21.15.79:0/264071072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7400105070 0x7f740019b3b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.152+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 >> 172.21.15.79:0/264071072 conn(0x7f74001001a0 msgr2=0x7f7400100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.153+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 shutdown_connections 2024-06-03T02:13:29.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:29.153+0000 7f7406800700 1 -- 172.21.15.79:0/264071072 wait complete. 2024-06-03T02:13:29.154 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 6 2024-06-03T02:13:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:29 smithi135 ceph-mon[36653]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:29 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:29 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/264071072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:29 smithi123 ceph-mon[36640]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:29 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:29 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/264071072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:29.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:29 smithi079 ceph-mon[31819]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:29.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:29.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:29.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:29.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:29.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:29.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:29 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:29.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:29 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/264071072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:30 smithi123 ceph-mon[36640]: Deploying daemon mds.c on smithi135 2024-06-03T02:13:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:30 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.135:6834/3846200406,v1:172.21.15.135:6835/3846200406] up:boot 2024-06-03T02:13:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:30 smithi123 ceph-mon[36640]: fsmap 6 up:standby 2024-06-03T02:13:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T02:13:30.551 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:30 smithi135 ceph-mon[36653]: Deploying daemon mds.c on smithi135 2024-06-03T02:13:30.552 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:30 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.135:6834/3846200406,v1:172.21.15.135:6835/3846200406] up:boot 2024-06-03T02:13:30.552 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:30 smithi135 ceph-mon[36653]: fsmap 6 up:standby 2024-06-03T02:13:30.552 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T02:13:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:30 smithi079 ceph-mon[31819]: Deploying daemon mds.c on smithi135 2024-06-03T02:13:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:30 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.135:6834/3846200406,v1:172.21.15.135:6835/3846200406] up:boot 2024-06-03T02:13:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:30 smithi079 ceph-mon[31819]: fsmap 6 up:standby 2024-06-03T02:13:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T02:13:30.845 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:13:30 smithi135 systemd[1]: Starting Ceph mds.c for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:13:31.216 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:13:30 smithi135 podman[67191]: 2024-06-03 02:13:30.856532439 +0000 UTC m=+0.016712123 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:13:31.216 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:13:30 smithi135 podman[67191]: 2024-06-03 02:13:30.972463595 +0000 UTC m=+0.132643256 container create e93d986aac4d05e726aaff8c484d266e1bea69b2b21a3a7ba884ea19587dad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-03T02:13:31.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:31 smithi123 ceph-mon[36640]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:31.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:31 smithi079 ceph-mon[31819]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:31.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:31 smithi135 ceph-mon[36653]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:31.595 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:13:31 smithi135 podman[67191]: 2024-06-03 02:13:31.250714644 +0000 UTC m=+0.410894315 container init e93d986aac4d05e726aaff8c484d266e1bea69b2b21a3a7ba884ea19587dad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:13:31.595 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:13:31 smithi135 podman[67191]: 2024-06-03 02:13:31.253313807 +0000 UTC m=+0.413493481 container start e93d986aac4d05e726aaff8c484d266e1bea69b2b21a3a7ba884ea19587dad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-03T02:13:31.595 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:13:31 smithi135 ceph-mds[67210]: -- 172.21.15.135:0/3493482522 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c10919a40 con 0x555c108ed000 2024-06-03T02:13:31.595 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:13:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c[67206]: starting mds.c at 2024-06-03T02:13:31.595 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:13:31 smithi135 bash[67191]: e93d986aac4d05e726aaff8c484d266e1bea69b2b21a3a7ba884ea19587dad19 2024-06-03T02:13:31.595 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:13:31 smithi135 systemd[1]: Started Ceph mds.c for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:31.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:13:32.362 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: Deploying daemon mds.h on smithi123 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.135:6836/755294241,v1:172.21.15.135:6837/755294241] up:boot 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: fsmap 7 up:standby 2024-06-03T02:13:32.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: Deploying daemon mds.h on smithi123 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.135:6836/755294241,v1:172.21.15.135:6837/755294241] up:boot 2024-06-03T02:13:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: fsmap 7 up:standby 2024-06-03T02:13:32.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T02:13:32.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:32.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:32.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:32.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:32.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:32.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:32.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: Deploying daemon mds.h on smithi123 2024-06-03T02:13:32.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:32.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.135:6836/755294241,v1:172.21.15.135:6837/755294241] up:boot 2024-06-03T02:13:32.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: fsmap 7 up:standby 2024-06-03T02:13:32.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T02:13:33.723 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:13:33 smithi123 systemd[1]: Starting Ceph mds.h for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:13:34.086 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:13:33 smithi123 podman[67178]: 2024-06-03 02:13:33.725712021 +0000 UTC m=+0.017742803 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:13:34.086 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:13:33 smithi123 podman[67178]: 2024-06-03 02:13:33.834264189 +0000 UTC m=+0.126295013 container create 47d4e72909ab5b2a6bcedd75fde733dbcb3bad032517a93f435b3ffa7d379e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:13:34.086 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:13:34 smithi123 podman[67178]: 2024-06-03 02:13:34.087131799 +0000 UTC m=+0.379162581 container init 47d4e72909ab5b2a6bcedd75fde733dbcb3bad032517a93f435b3ffa7d379e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-03T02:13:34.473 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:13:34 smithi123 podman[67178]: 2024-06-03 02:13:34.089842207 +0000 UTC m=+0.381872989 container start 47d4e72909ab5b2a6bcedd75fde733dbcb3bad032517a93f435b3ffa7d379e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-03T02:13:34.473 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:13:34 smithi123 ceph-mds[67197]: -- 172.21.15.123:0/3710799619 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557365caba40 con 0x557365c7f000 2024-06-03T02:13:34.473 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:13:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h[67193]: starting mds.h at 2024-06-03T02:13:34.473 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:13:34 smithi123 bash[67178]: 47d4e72909ab5b2a6bcedd75fde733dbcb3bad032517a93f435b3ffa7d379e43 2024-06-03T02:13:34.473 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:13:34 smithi123 systemd[1]: Started Ceph mds.h for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:34.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.592+0000 7fd911400700 1 -- 172.21.15.79:0/310782339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c106600 msgr2=0x7fd90c10e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:34.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.592+0000 7fd911400700 1 --2- 172.21.15.79:0/310782339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c106600 0x7fd90c10e1d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd90400b0d0 tx=0x7fd90400b3e0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.592+0000 7fd911400700 1 -- 172.21.15.79:0/310782339 shutdown_connections 2024-06-03T02:13:34.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.592+0000 7fd911400700 1 --2- 172.21.15.79:0/310782339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c106600 0x7fd90c10e1d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.592+0000 7fd911400700 1 --2- 172.21.15.79:0/310782339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd90c101520 0x7fd90c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.592+0000 7fd911400700 1 --2- 172.21.15.79:0/310782339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101ed0 0x7fd90c105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.592+0000 7fd911400700 1 -- 172.21.15.79:0/310782339 >> 172.21.15.79:0/310782339 conn(0x7fd90c0fcdb0 msgr2=0x7fd90c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.592+0000 7fd911400700 1 -- 172.21.15.79:0/310782339 shutdown_connections 2024-06-03T02:13:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.592+0000 7fd911400700 1 -- 172.21.15.79:0/310782339 wait complete. 2024-06-03T02:13:34.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd911400700 1 Processor -- start 2024-06-03T02:13:34.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd911400700 1 -- start start 2024-06-03T02:13:34.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd911400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd90c101520 0x7fd90c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:34.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd911400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101ed0 0x7fd90c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:34.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd911400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c106600 0x7fd90c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:34.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd911400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c111660 con 0x7fd90c101520 2024-06-03T02:13:34.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd911400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1114e0 con 0x7fd90c101ed0 2024-06-03T02:13:34.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd911400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1117e0 con 0x7fd90c106600 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd90be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd90c101520 0x7fd90c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd90be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd90c101520 0x7fd90c19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56276/0 (socket says 172.21.15.79:56276) 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd910a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c106600 0x7fd90c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd90be00700 1 -- 172.21.15.79:0/270164350 learned_addr learned my addr 172.21.15.79:0/270164350 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.594+0000 7fd90b400700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101ed0 0x7fd90c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.595+0000 7fd910a00700 1 -- 172.21.15.79:0/270164350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101ed0 msgr2=0x7fd90c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.595+0000 7fd910a00700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101ed0 0x7fd90c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.595+0000 7fd910a00700 1 -- 172.21.15.79:0/270164350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd90c101520 msgr2=0x7fd90c19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.595+0000 7fd910a00700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd90c101520 0x7fd90c19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.595+0000 7fd910a00700 1 -- 172.21.15.79:0/270164350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c19fd40 con 0x7fd90c106600 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.595+0000 7fd90b400700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101ed0 0x7fd90c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.595+0000 7fd910a00700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c106600 0x7fd90c19f600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd90400d010 tx=0x7fd904009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.595+0000 7fd908c00700 1 -- 172.21.15.79:0/270164350 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904003a00 con 0x7fd90c106600 2024-06-03T02:13:34.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.596+0000 7fd908c00700 1 -- 172.21.15.79:0/270164350 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd904022070 con 0x7fd90c106600 2024-06-03T02:13:34.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.596+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c19ffd0 con 0x7fd90c106600 2024-06-03T02:13:34.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.596+0000 7fd908c00700 1 -- 172.21.15.79:0/270164350 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90401b6c0 con 0x7fd90c106600 2024-06-03T02:13:34.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.596+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c1a78b0 con 0x7fd90c106600 2024-06-03T02:13:34.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.597+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90c069180 con 0x7fd90c106600 2024-06-03T02:13:34.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.599+0000 7fd908c00700 1 -- 172.21.15.79:0/270164350 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd904003ba0 con 0x7fd90c106600 2024-06-03T02:13:34.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.600+0000 7fd908c00700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd8f00a2ca0 0x7fd8f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:34.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.600+0000 7fd908c00700 1 -- 172.21.15.79:0/270164350 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd90400f020 con 0x7fd90c106600 2024-06-03T02:13:34.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.600+0000 7fd90be00700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd8f00a2ca0 0x7fd8f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:34.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.601+0000 7fd90be00700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd8f00a2ca0 0x7fd8f00a5160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009d20 tx=0x7fd8fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:34.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.604+0000 7fd908c00700 1 -- 172.21.15.79:0/270164350 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd90409cb90 con 0x7fd90c106600 2024-06-03T02:13:34.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.877+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd90c19c090 con 0x7fd90c106600 2024-06-03T02:13:34.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.878+0000 7fd908c00700 1 -- 172.21.15.79:0/270164350 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7fd90409c490 con 0x7fd90c106600 2024-06-03T02:13:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:13:34.878 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-06-03T02:13:34.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.881+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd8f00a2ca0 msgr2=0x7fd8f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:34.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.881+0000 7fd911400700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd8f00a2ca0 0x7fd8f00a5160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009d20 tx=0x7fd8fc009260 comp rx=0 tx=0).stop 2024-06-03T02:13:34.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.881+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c106600 msgr2=0x7fd90c19f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:34.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.881+0000 7fd911400700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c106600 0x7fd90c19f600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd90400d010 tx=0x7fd904009810 comp rx=0 tx=0).stop 2024-06-03T02:13:34.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.882+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 shutdown_connections 2024-06-03T02:13:34.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.882+0000 7fd911400700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c106600 0x7fd90c19f600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.882+0000 7fd911400700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd8f00a2ca0 0x7fd8f00a5160 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.882+0000 7fd911400700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd90c101520 0x7fd90c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.882+0000 7fd911400700 1 --2- 172.21.15.79:0/270164350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101ed0 0x7fd90c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:34.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.882+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 >> 172.21.15.79:0/270164350 conn(0x7fd90c0fcdb0 msgr2=0x7fd90c10cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:34.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.882+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 shutdown_connections 2024-06-03T02:13:34.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:34.882+0000 7fd911400700 1 -- 172.21.15.79:0/270164350 wait complete. 2024-06-03T02:13:34.883 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2024-06-03T02:13:35.279 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:35.279 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] up:boot 2024-06-03T02:13:35.279 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: fsmap 8 up:standby 2024-06-03T02:13:35.279 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T02:13:35.279 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:35.280 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:35.280 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:35.280 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:35.280 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:35.280 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:35.280 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:35 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/270164350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] up:boot 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: fsmap 8 up:standby 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:35 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/270164350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:35.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] up:boot 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: fsmap 8 up:standby 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:35.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:35 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/270164350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:36.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:36 smithi135 ceph-mon[36653]: Deploying daemon mds.i on smithi135 2024-06-03T02:13:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:36 smithi123 ceph-mon[36640]: Deploying daemon mds.i on smithi135 2024-06-03T02:13:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:36 smithi079 ceph-mon[31819]: Deploying daemon mds.i on smithi135 2024-06-03T02:13:36.761 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:13:36 smithi135 systemd[1]: Starting Ceph mds.i for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:13:37.015 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:13:36 smithi135 podman[67494]: 2024-06-03 02:13:36.665091925 +0000 UTC m=+0.017659043 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:13:37.015 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:13:36 smithi135 podman[67494]: 2024-06-03 02:13:36.788113155 +0000 UTC m=+0.140680287 container create 2232e148aa25dbadcff13804c3a8c2023dfdd8f201c1014ce061ba43eb6b3793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-03T02:13:37.015 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:13:37 smithi135 podman[67494]: 2024-06-03 02:13:37.016068005 +0000 UTC m=+0.368635123 container init 2232e148aa25dbadcff13804c3a8c2023dfdd8f201c1014ce061ba43eb6b3793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:13:37.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:37 smithi135 ceph-mon[36653]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:37.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:37 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:37.345 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:13:37 smithi135 podman[67494]: 2024-06-03 02:13:37.01885523 +0000 UTC m=+0.371422346 container start 2232e148aa25dbadcff13804c3a8c2023dfdd8f201c1014ce061ba43eb6b3793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-06-03T02:13:37.345 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:13:37 smithi135 ceph-mds[67513]: -- 172.21.15.135:0/3723582611 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c0c4d2fa40 con 0x55c0c4d02800 2024-06-03T02:13:37.346 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:13:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i[67509]: starting mds.i at 2024-06-03T02:13:37.346 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:13:37 smithi135 bash[67494]: 2232e148aa25dbadcff13804c3a8c2023dfdd8f201c1014ce061ba43eb6b3793 2024-06-03T02:13:37.346 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:13:37 smithi135 systemd[1]: Started Ceph mds.i for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:37 smithi123 ceph-mon[36640]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:37.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:37 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:37.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:37 smithi079 ceph-mon[31819]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:37 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:37.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:13:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:38 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:38 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] up:boot 2024-06-03T02:13:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:38 smithi123 ceph-mon[36640]: fsmap 9 up:standby 2024-06-03T02:13:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:38 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T02:13:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:38 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:38 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:38 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:38 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:38.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:38 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:38.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:38 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] up:boot 2024-06-03T02:13:38.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:38 smithi079 ceph-mon[31819]: fsmap 9 up:standby 2024-06-03T02:13:38.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:38 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T02:13:38.482 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:38 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:38.482 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:38 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:38.482 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:38 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:38.482 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:38 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:38 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:38 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] up:boot 2024-06-03T02:13:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:38 smithi135 ceph-mon[36653]: fsmap 9 up:standby 2024-06-03T02:13:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:38 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T02:13:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:38 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:38 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:38.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:38 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:38.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:38 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:39.039 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:13:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:39 smithi123 ceph-mon[36640]: Deploying daemon mds.g on smithi079 2024-06-03T02:13:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:39 smithi123 ceph-mon[36640]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:39 smithi079 ceph-mon[31819]: Deploying daemon mds.g on smithi079 2024-06-03T02:13:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:39 smithi079 ceph-mon[31819]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:39.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:39 smithi135 ceph-mon[36653]: Deploying daemon mds.g on smithi079 2024-06-03T02:13:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:39 smithi135 ceph-mon[36653]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:40.839 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:13:40 smithi079 systemd[1]: Starting Ceph mds.g for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:13:41.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:41 smithi123 ceph-mon[36640]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:41 smithi079 ceph-mon[31819]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:41 smithi135 ceph-mon[36653]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:41.883 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:13:41 smithi079 podman[73966]: 2024-06-03 02:13:41.526400652 +0000 UTC m=+0.032610960 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:13:41.883 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:13:41 smithi079 podman[73966]: 2024-06-03 02:13:41.631545311 +0000 UTC m=+0.137755655 container create 64f029655276317e6590527380b646d0cf543e83845851695461b79e1bd9cdcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-03T02:13:42.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.407+0000 7fd47b200700 1 -- 172.21.15.79:0/3189112679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd47410ee90 msgr2=0x7fd474111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:42.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.407+0000 7fd47b200700 1 --2- 172.21.15.79:0/3189112679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd47410ee90 0x7fd474111280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009f90 tx=0x7fd46c00b330 comp rx=0 tx=0).stop 2024-06-03T02:13:42.409 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:13:42 smithi079 podman[73966]: 2024-06-03 02:13:42.154841984 +0000 UTC m=+0.661052359 container init 64f029655276317e6590527380b646d0cf543e83845851695461b79e1bd9cdcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-03T02:13:42.409 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:13:42 smithi079 podman[73966]: 2024-06-03 02:13:42.165120227 +0000 UTC m=+0.671330561 container start 64f029655276317e6590527380b646d0cf543e83845851695461b79e1bd9cdcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-03T02:13:42.409 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:13:42 smithi079 ceph-mds[74007]: -- 172.21.15.79:0/1662872113 <== mon.0 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b68d417a40 con 0x55b68d3eb000 2024-06-03T02:13:42.409 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:13:42 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g[73997]: starting mds.g at 2024-06-03T02:13:42.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.407+0000 7fd47b200700 1 -- 172.21.15.79:0/3189112679 shutdown_connections 2024-06-03T02:13:42.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.407+0000 7fd47b200700 1 --2- 172.21.15.79:0/3189112679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd47410ee90 0x7fd474111280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.407+0000 7fd47b200700 1 --2- 172.21.15.79:0/3189112679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4740746c0 0x7fd474074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.407+0000 7fd47b200700 1 --2- 172.21.15.79:0/3189112679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd474074fe0 0x7fd47410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.407+0000 7fd47b200700 1 -- 172.21.15.79:0/3189112679 >> 172.21.15.79:0/3189112679 conn(0x7fd474079560 msgr2=0x7fd474079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:42.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.407+0000 7fd47b200700 1 -- 172.21.15.79:0/3189112679 shutdown_connections 2024-06-03T02:13:42.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.407+0000 7fd47b200700 1 -- 172.21.15.79:0/3189112679 wait complete. 2024-06-03T02:13:42.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.408+0000 7fd47b200700 1 Processor -- start 2024-06-03T02:13:42.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.409+0000 7fd47b200700 1 -- start start 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.409+0000 7fd47b200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740746c0 0x7fd47419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.409+0000 7fd47b200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd474074fe0 0x7fd47419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.409+0000 7fd47b200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd47410ee90 0x7fd47419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.409+0000 7fd47b200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741149d0 con 0x7fd474074fe0 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.409+0000 7fd47b200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd474114850 con 0x7fd47410ee90 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.409+0000 7fd47b200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd474114b50 con 0x7fd4740746c0 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.410+0000 7fd473400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd474074fe0 0x7fd47419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.410+0000 7fd479200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd47410ee90 0x7fd47419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.410+0000 7fd473400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd474074fe0 0x7fd47419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47442/0 (socket says 172.21.15.79:47442) 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.410+0000 7fd473400700 1 -- 172.21.15.79:0/140111745 learned_addr learned my addr 172.21.15.79:0/140111745 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.410+0000 7fd479200700 1 -- 172.21.15.79:0/140111745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740746c0 msgr2=0x7fd47419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.410+0000 7fd479200700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740746c0 0x7fd47419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.410+0000 7fd479200700 1 -- 172.21.15.79:0/140111745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd474074fe0 msgr2=0x7fd47419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.410+0000 7fd479200700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd474074fe0 0x7fd47419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.410+0000 7fd479200700 1 -- 172.21.15.79:0/140111745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47419fdc0 con 0x7fd47410ee90 2024-06-03T02:13:42.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.411+0000 7fd479200700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd47410ee90 0x7fd47419f680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd474069c20 tx=0x7fd46c00cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:42.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.411+0000 7fd470c00700 1 -- 172.21.15.79:0/140111745 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c021070 con 0x7fd47410ee90 2024-06-03T02:13:42.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.411+0000 7fd470c00700 1 -- 172.21.15.79:0/140111745 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd46c007ac0 con 0x7fd47410ee90 2024-06-03T02:13:42.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.411+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4741a0050 con 0x7fd47410ee90 2024-06-03T02:13:42.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.411+0000 7fd470c00700 1 -- 172.21.15.79:0/140111745 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0052a0 con 0x7fd47410ee90 2024-06-03T02:13:42.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.411+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4741a7930 con 0x7fd47410ee90 2024-06-03T02:13:42.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.412+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd474069180 con 0x7fd47410ee90 2024-06-03T02:13:42.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.414+0000 7fd470c00700 1 -- 172.21.15.79:0/140111745 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd46c024070 con 0x7fd47410ee90 2024-06-03T02:13:42.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.415+0000 7fd470c00700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4580a2ca0 0x7fd4580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.416+0000 7fd470c00700 1 -- 172.21.15.79:0/140111745 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd46c00f020 con 0x7fd47410ee90 2024-06-03T02:13:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.416+0000 7fd473e00700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4580a2ca0 0x7fd4580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.416+0000 7fd473e00700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4580a2ca0 0x7fd4580a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd464005950 tx=0x7fd4640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:42.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.420+0000 7fd470c00700 1 -- 172.21.15.79:0/140111745 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd46c02b020 con 0x7fd47410ee90 2024-06-03T02:13:42.706 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:13:42 smithi079 bash[73966]: 64f029655276317e6590527380b646d0cf543e83845851695461b79e1bd9cdcc 2024-06-03T02:13:42.706 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:13:42 smithi079 systemd[1]: Started Ceph mds.g for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:42.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.706+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4741a7b00 con 0x7fd47410ee90 2024-06-03T02:13:42.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.707+0000 7fd470c00700 1 -- 172.21.15.79:0/140111745 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7fd46c09d320 con 0x7fd47410ee90 2024-06-03T02:13:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:13:42.707 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.709+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4580a2ca0 msgr2=0x7fd4580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.709+0000 7fd47b200700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4580a2ca0 0x7fd4580a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd464005950 tx=0x7fd4640058e0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.709+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd47410ee90 msgr2=0x7fd47419f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.709+0000 7fd47b200700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd47410ee90 0x7fd47419f680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd474069c20 tx=0x7fd46c00cfa0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.710+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 shutdown_connections 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.710+0000 7fd47b200700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd47410ee90 0x7fd47419f680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.710+0000 7fd47b200700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740746c0 0x7fd47419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.710+0000 7fd47b200700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4580a2ca0 0x7fd4580a5160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.710+0000 7fd47b200700 1 --2- 172.21.15.79:0/140111745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd474074fe0 0x7fd47419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.710+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 >> 172.21.15.79:0/140111745 conn(0x7fd474079560 msgr2=0x7fd4740ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.710+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 shutdown_connections 2024-06-03T02:13:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:42.710+0000 7fd47b200700 1 -- 172.21.15.79:0/140111745 wait complete. 2024-06-03T02:13:42.711 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 10 2024-06-03T02:13:43.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:43 smithi079 ceph-mon[31819]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:43.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:43 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:43.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:43 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:43.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:43 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:43.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:43 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:43.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:43 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:43.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:43 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:43.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:43 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/140111745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:43 smithi135 ceph-mon[36653]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:43 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:43 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:43 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:43 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:43 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:43 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:43.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:43 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/140111745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:43 smithi123 ceph-mon[36640]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:43 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:43 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:43 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:43 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:43 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:43 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:43 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/140111745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:44.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:44 smithi079 ceph-mon[31819]: Deploying daemon mds.a on smithi079 2024-06-03T02:13:44.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:44 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] up:boot 2024-06-03T02:13:44.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:44 smithi079 ceph-mon[31819]: fsmap 10 up:standby 2024-06-03T02:13:44.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T02:13:44.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:44 smithi079 ceph-mon[31819]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:44 smithi135 ceph-mon[36653]: Deploying daemon mds.a on smithi079 2024-06-03T02:13:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:44 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] up:boot 2024-06-03T02:13:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:44 smithi135 ceph-mon[36653]: fsmap 10 up:standby 2024-06-03T02:13:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T02:13:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:44 smithi135 ceph-mon[36653]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:44 smithi123 ceph-mon[36640]: Deploying daemon mds.a on smithi079 2024-06-03T02:13:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:44 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] up:boot 2024-06-03T02:13:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:44 smithi123 ceph-mon[36640]: fsmap 10 up:standby 2024-06-03T02:13:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T02:13:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:44 smithi123 ceph-mon[36640]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:45.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:13:46.252 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:13:45 smithi079 podman[74369]: 2024-06-03 02:13:45.996983929 +0000 UTC m=+0.136431190 container create f1472a61b4ec51edac531e6906d0dd9af04979339d27185fc147834af747509e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-06-03T02:13:46.253 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:13:46 smithi079 podman[74369]: 2024-06-03 02:13:46.251654053 +0000 UTC m=+0.391101327 container init f1472a61b4ec51edac531e6906d0dd9af04979339d27185fc147834af747509e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-03T02:13:46.345 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:13:46.535 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:13:46 smithi079 podman[74369]: 2024-06-03 02:13:46.261662779 +0000 UTC m=+0.401110013 container start f1472a61b4ec51edac531e6906d0dd9af04979339d27185fc147834af747509e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-03T02:13:46.535 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:13:46 smithi079 ceph-mds[74413]: -- 172.21.15.79:0/2278572184 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f37b1ba40 con 0x562f37aeec00 2024-06-03T02:13:46.535 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:13:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a[74407]: starting mds.a at 2024-06-03T02:13:46.535 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:13:46 smithi079 bash[74369]: f1472a61b4ec51edac531e6906d0dd9af04979339d27185fc147834af747509e 2024-06-03T02:13:46.535 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:13:46 smithi079 systemd[1]: Started Ceph mds.a for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:47.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:47 smithi123 ceph-mon[36640]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:47.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:47.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:47.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:47.427 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:47.427 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:47.427 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:47 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:47.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:47 smithi079 ceph-mon[31819]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:47.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:47.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:47.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:47.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:47.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:47.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:47 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:47.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:47 smithi135 ceph-mon[36653]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:47.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:47.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:47.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:47.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T02:13:47.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T02:13:47.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:47 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:48.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:48 smithi079 ceph-mon[31819]: Deploying daemon mds.b on smithi123 2024-06-03T02:13:48.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:48 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.79:6840/1411940491,v1:172.21.15.79:6841/1411940491] up:boot 2024-06-03T02:13:48.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:48 smithi079 ceph-mon[31819]: fsmap 11 up:standby 2024-06-03T02:13:48.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:48 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T02:13:48.396 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:48 smithi123 ceph-mon[36640]: Deploying daemon mds.b on smithi123 2024-06-03T02:13:48.397 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:48 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.79:6840/1411940491,v1:172.21.15.79:6841/1411940491] up:boot 2024-06-03T02:13:48.397 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:48 smithi123 ceph-mon[36640]: fsmap 11 up:standby 2024-06-03T02:13:48.397 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:48 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T02:13:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:48 smithi135 ceph-mon[36653]: Deploying daemon mds.b on smithi123 2024-06-03T02:13:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:48 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.79:6840/1411940491,v1:172.21.15.79:6841/1411940491] up:boot 2024-06-03T02:13:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:48 smithi135 ceph-mon[36653]: fsmap 11 up:standby 2024-06-03T02:13:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T02:13:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.643+0000 7fe373200700 1 -- 172.21.15.79:0/1595436135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c10ca00 msgr2=0x7fe36c10ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.643+0000 7fe373200700 1 --2- 172.21.15.79:0/1595436135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c10ca00 0x7fe36c10ee70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe368009f90 tx=0x7fe36800b330 comp rx=0 tx=0).stop 2024-06-03T02:13:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.643+0000 7fe373200700 1 -- 172.21.15.79:0/1595436135 shutdown_connections 2024-06-03T02:13:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.643+0000 7fe373200700 1 --2- 172.21.15.79:0/1595436135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c10ca00 0x7fe36c10ee70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.643+0000 7fe373200700 1 --2- 172.21.15.79:0/1595436135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe36c103620 0x7fe36c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.643+0000 7fe373200700 1 --2- 172.21.15.79:0/1595436135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe36c103f40 0x7fe36c10c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.643+0000 7fe373200700 1 -- 172.21.15.79:0/1595436135 >> 172.21.15.79:0/1595436135 conn(0x7fe36c0fcdb0 msgr2=0x7fe36c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.643+0000 7fe373200700 1 -- 172.21.15.79:0/1595436135 shutdown_connections 2024-06-03T02:13:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.644+0000 7fe373200700 1 -- 172.21.15.79:0/1595436135 wait complete. 2024-06-03T02:13:48.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.645+0000 7fe373200700 1 Processor -- start 2024-06-03T02:13:48.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.645+0000 7fe373200700 1 -- start start 2024-06-03T02:13:48.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe373200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c103620 0x7fe36c101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe373200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe36c103f40 0x7fe36c1017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe373200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe36c10ca00 0x7fe36c101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe373200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c112720 con 0x7fe36c103620 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe373200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1125a0 con 0x7fe36c10ca00 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe373200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1128a0 con 0x7fe36c103f40 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe371e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c103620 0x7fe36c101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe371e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c103620 0x7fe36c101280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56116/0 (socket says 172.21.15.79:56116) 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe371e00700 1 -- 172.21.15.79:0/2686628866 learned_addr learned my addr 172.21.15.79:0/2686628866 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe372800700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe36c10ca00 0x7fe36c101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe371400700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe36c103f40 0x7fe36c1017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe372800700 1 -- 172.21.15.79:0/2686628866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe36c103f40 msgr2=0x7fe36c1017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe372800700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe36c103f40 0x7fe36c1017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe372800700 1 -- 172.21.15.79:0/2686628866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c103620 msgr2=0x7fe36c101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:48.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe372800700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c103620 0x7fe36c101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:48.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.646+0000 7fe372800700 1 -- 172.21.15.79:0/2686628866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c1a7060 con 0x7fe36c10ca00 2024-06-03T02:13:48.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.647+0000 7fe372800700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe36c10ca00 0x7fe36c101d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe368015000 tx=0x7fe368004f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:48.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.647+0000 7fe366a00700 1 -- 172.21.15.79:0/2686628866 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368021070 con 0x7fe36c10ca00 2024-06-03T02:13:48.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.647+0000 7fe366a00700 1 -- 172.21.15.79:0/2686628866 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe368007ac0 con 0x7fe36c10ca00 2024-06-03T02:13:48.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.647+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c1a72f0 con 0x7fe36c10ca00 2024-06-03T02:13:48.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.647+0000 7fe366a00700 1 -- 172.21.15.79:0/2686628866 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680052a0 con 0x7fe36c10ca00 2024-06-03T02:13:48.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.647+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c1a77e0 con 0x7fe36c10ca00 2024-06-03T02:13:48.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.648+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36c069180 con 0x7fe36c10ca00 2024-06-03T02:13:48.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.650+0000 7fe366a00700 1 -- 172.21.15.79:0/2686628866 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe368024070 con 0x7fe36c10ca00 2024-06-03T02:13:48.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.651+0000 7fe366a00700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3540a2c20 0x7fe3540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:48.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.652+0000 7fe366a00700 1 -- 172.21.15.79:0/2686628866 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe3680cfa60 con 0x7fe36c10ca00 2024-06-03T02:13:48.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.652+0000 7fe371e00700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3540a2c20 0x7fe3540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:48.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.652+0000 7fe371e00700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3540a2c20 0x7fe3540a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe360005d10 tx=0x7fe360009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:48.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.656+0000 7fe366a00700 1 -- 172.21.15.79:0/2686628866 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe36809ce50 con 0x7fe36c10ca00 2024-06-03T02:13:48.723 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:13:48 smithi123 systemd[1]: Starting Ceph mds.b for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:13:48.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.929+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe36c1a7a40 con 0x7fe36c10ca00 2024-06-03T02:13:48.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.930+0000 7fe366a00700 1 -- 172.21.15.79:0/2686628866 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7fe36809c750 con 0x7fe36c10ca00 2024-06-03T02:13:48.931 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:13:48.931 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-06-03T02:13:48.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3540a2c20 msgr2=0x7fe3540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:48.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3540a2c20 0x7fe3540a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe360005d10 tx=0x7fe360009310 comp rx=0 tx=0).stop 2024-06-03T02:13:48.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe36c10ca00 msgr2=0x7fe36c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:48.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe36c10ca00 0x7fe36c101d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe368015000 tx=0x7fe368004f00 comp rx=0 tx=0).stop 2024-06-03T02:13:48.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 shutdown_connections 2024-06-03T02:13:48.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe36c10ca00 0x7fe36c101d00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:48.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3540a2c20 0x7fe3540a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:48.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c103620 0x7fe36c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:48.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 --2- 172.21.15.79:0/2686628866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe36c103f40 0x7fe36c1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:48.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.933+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 >> 172.21.15.79:0/2686628866 conn(0x7fe36c0fcdb0 msgr2=0x7fe36c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:48.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.934+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 shutdown_connections 2024-06-03T02:13:48.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:48.934+0000 7fe373200700 1 -- 172.21.15.79:0/2686628866 wait complete. 2024-06-03T02:13:48.935 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 12 2024-06-03T02:13:49.049 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:13:48 smithi123 podman[67484]: 2024-06-03 02:13:48.692119073 +0000 UTC m=+0.017163833 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:13:49.049 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:13:48 smithi123 podman[67484]: 2024-06-03 02:13:48.805288846 +0000 UTC m=+0.130333633 container create a6f06cdcd13f55684dd6ba3a17cfb3e4c3e3ebc6cdf5728d87991308fbae1039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-03T02:13:49.049 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:13:49 smithi123 podman[67484]: 2024-06-03 02:13:49.024587139 +0000 UTC m=+0.349631870 container init a6f06cdcd13f55684dd6ba3a17cfb3e4c3e3ebc6cdf5728d87991308fbae1039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:13:49.049 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:13:49 smithi123 podman[67484]: 2024-06-03 02:13:49.027044826 +0000 UTC m=+0.352089574 container start a6f06cdcd13f55684dd6ba3a17cfb3e4c3e3ebc6cdf5728d87991308fbae1039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-03T02:13:49.315 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:49 smithi123 ceph-mon[36640]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:49.315 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:49 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2686628866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:49.316 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:49 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:49.316 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:13:49 smithi123 ceph-mds[67503]: -- 172.21.15.123:0/1869560172 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576a638ba40 con 0x5576a635ec00 2024-06-03T02:13:49.316 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:13:49 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b[67499]: starting mds.b at 2024-06-03T02:13:49.316 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:13:49 smithi123 bash[67484]: a6f06cdcd13f55684dd6ba3a17cfb3e4c3e3ebc6cdf5728d87991308fbae1039 2024-06-03T02:13:49.316 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:13:49 smithi123 systemd[1]: Started Ceph mds.b for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:13:49.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:49 smithi079 ceph-mon[31819]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:49 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2686628866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:49 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:49 smithi135 ceph-mon[36653]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:49 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2686628866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:49 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.530 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.530 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:50 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] up:boot 2024-06-03T02:13:50.530 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:50 smithi135 ceph-mon[36653]: fsmap 12 up:standby 2024-06-03T02:13:50.530 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T02:13:50.530 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.530 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.530 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.530 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:50 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:13:50.531 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:50 smithi135 ceph-mon[36653]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:50.532 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.533 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:50 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] up:boot 2024-06-03T02:13:50.533 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:50 smithi123 ceph-mon[36640]: fsmap 12 up:standby 2024-06-03T02:13:50.533 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T02:13:50.533 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.533 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.533 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.533 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:50 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:13:50.533 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:50 smithi123 ceph-mon[36640]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:50 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] up:boot 2024-06-03T02:13:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:50 smithi079 ceph-mon[31819]: fsmap 12 up:standby 2024-06-03T02:13:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T02:13:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:50 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:13:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:50 smithi079 ceph-mon[31819]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:51.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:13:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:13:52.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:52 smithi079 ceph-mon[31819]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:52 smithi135 ceph-mon[36653]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:52 smithi123 ceph-mon[36640]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:13:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:13:53.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:53.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:53.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:53.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:13:53.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:53.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:13:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:13:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:13:54.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:54 smithi079 ceph-mon[31819]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.567+0000 7f69a9600700 1 -- 172.21.15.79:0/2140079474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 msgr2=0x7f69a4104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.567+0000 7f69a9600700 1 --2- 172.21.15.79:0/2140079474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 0x7f69a4104b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6990009990 tx=0x7f6990009ca0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.567+0000 7f69a9600700 1 -- 172.21.15.79:0/2140079474 shutdown_connections 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.567+0000 7f69a9600700 1 --2- 172.21.15.79:0/2140079474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69a4105090 0x7f69a41115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.567+0000 7f69a9600700 1 --2- 172.21.15.79:0/2140079474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69a410a6d0 0x7f69a410aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.567+0000 7f69a9600700 1 --2- 172.21.15.79:0/2140079474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 0x7f69a4104b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.567+0000 7f69a9600700 1 -- 172.21.15.79:0/2140079474 >> 172.21.15.79:0/2140079474 conn(0x7f69a41001c0 msgr2=0x7f69a41025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.568+0000 7f69a9600700 1 -- 172.21.15.79:0/2140079474 shutdown_connections 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.568+0000 7f69a9600700 1 -- 172.21.15.79:0/2140079474 wait complete. 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.569+0000 7f69a9600700 1 Processor -- start 2024-06-03T02:13:54.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.569+0000 7f69a9600700 1 -- start start 2024-06-03T02:13:54.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.569+0000 7f69a9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 0x7f69a419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:54.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.569+0000 7f69a9600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69a4105090 0x7f69a419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:54.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.569+0000 7f69a9600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69a410a6d0 0x7f69a419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:54.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.569+0000 7f69a9600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4114ab0 con 0x7f69a4105090 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.569+0000 7f69a9600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4114930 con 0x7f69a41046d0 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.569+0000 7f69a9600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4114c30 con 0x7f69a410a6d0 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 0x7f69a419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 0x7f69a419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:45668/0 (socket says 172.21.15.79:45668) 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2a00700 1 -- 172.21.15.79:0/4168513382 learned_addr learned my addr 172.21.15.79:0/4168513382 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a3400700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69a410a6d0 0x7f69a419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2000700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69a4105090 0x7f69a419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2a00700 1 -- 172.21.15.79:0/4168513382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69a410a6d0 msgr2=0x7f69a419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2a00700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69a410a6d0 0x7f69a419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2a00700 1 -- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69a4105090 msgr2=0x7f69a419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2a00700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69a4105090 0x7f69a419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2a00700 1 -- 172.21.15.79:0/4168513382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a419fee0 con 0x7f69a41046d0 2024-06-03T02:13:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.570+0000 7f69a2a00700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 0x7f69a419af20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f699400e9c0 tx=0x7f699400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.571+0000 7f699b400700 1 -- 172.21.15.79:0/4168513382 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699400cbd0 con 0x7f69a41046d0 2024-06-03T02:13:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.571+0000 7f699b400700 1 -- 172.21.15.79:0/4168513382 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f699400cd70 con 0x7f69a41046d0 2024-06-03T02:13:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.571+0000 7f699b400700 1 -- 172.21.15.79:0/4168513382 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994010430 con 0x7f69a41046d0 2024-06-03T02:13:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.571+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a41a01d0 con 0x7f69a41046d0 2024-06-03T02:13:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.571+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a41a7a50 con 0x7f69a41046d0 2024-06-03T02:13:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.572+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a4069180 con 0x7f69a41046d0 2024-06-03T02:13:54.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.574+0000 7f699b400700 1 -- 172.21.15.79:0/4168513382 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6994004750 con 0x7f69a41046d0 2024-06-03T02:13:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.575+0000 7f699b400700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f69880a2ca0 0x7f69880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.576+0000 7f699b400700 1 -- 172.21.15.79:0/4168513382 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f69940cee00 con 0x7f69a41046d0 2024-06-03T02:13:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.576+0000 7f69a2000700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f69880a2ca0 0x7f69880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.577+0000 7f69a2000700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f69880a2ca0 0x7f69880a5160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f699000b2e0 tx=0x7f699001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.580+0000 7f699b400700 1 -- 172.21.15.79:0/4168513382 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f699409c160 con 0x7f69a41046d0 2024-06-03T02:13:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:54 smithi135 ceph-mon[36653]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:54 smithi123 ceph-mon[36640]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:54.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.861+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69a402dca0 con 0x7f69a41046d0 2024-06-03T02:13:54.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.862+0000 7f699b400700 1 -- 172.21.15.79:0/4168513382 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7f699409ba60 con 0x7f69a41046d0 2024-06-03T02:13:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:13:54.863 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-03T02:13:54.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f69880a2ca0 msgr2=0x7f69880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:54.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f69880a2ca0 0x7f69880a5160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f699000b2e0 tx=0x7f699001a040 comp rx=0 tx=0).stop 2024-06-03T02:13:54.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 msgr2=0x7f69a419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:54.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 0x7f69a419af20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f699400e9c0 tx=0x7f699400ecd0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 shutdown_connections 2024-06-03T02:13:54.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69a410a6d0 0x7f69a419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69a41046d0 0x7f69a419af20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f69880a2ca0 0x7f69880a5160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 --2- 172.21.15.79:0/4168513382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69a4105090 0x7f69a419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:54.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 >> 172.21.15.79:0/4168513382 conn(0x7f69a41001c0 msgr2=0x7f69a4100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:54.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 shutdown_connections 2024-06-03T02:13:54.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:54.865+0000 7f69a9600700 1 -- 172.21.15.79:0/4168513382 wait complete. 2024-06-03T02:13:54.867 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 13 2024-06-03T02:13:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:55 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:55 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:55 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:55 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:55 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4168513382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:55 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:55 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:55 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:55 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:55 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4168513382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:55 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:55 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:55 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:55 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:55 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4168513382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:13:56.342 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:56 smithi079 ceph-mon[31819]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:56.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-03T02:13:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:56 smithi135 ceph-mon[36653]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:56.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:56 smithi123 ceph-mon[36640]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:56.814 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:13:59.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.130+0000 7f83acc00700 1 -- 172.21.15.79:0/4060398800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a806aa00 msgr2=0x7f83a806ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:59.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.130+0000 7f83acc00700 1 --2- 172.21.15.79:0/4060398800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a806aa00 0x7f83a806ae80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f83940098f0 tx=0x7f8394009c00 comp rx=0 tx=0).stop 2024-06-03T02:13:59.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.130+0000 7f83acc00700 1 -- 172.21.15.79:0/4060398800 shutdown_connections 2024-06-03T02:13:59.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.130+0000 7f83acc00700 1 --2- 172.21.15.79:0/4060398800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a806b3c0 0x7f83a810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.130+0000 7f83acc00700 1 --2- 172.21.15.79:0/4060398800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83a81084f0 0x7f83a81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.130+0000 7f83acc00700 1 --2- 172.21.15.79:0/4060398800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a806aa00 0x7f83a806ae80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.130+0000 7f83acc00700 1 -- 172.21.15.79:0/4060398800 >> 172.21.15.79:0/4060398800 conn(0x7f83a8078300 msgr2=0x7f83a8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:59.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.130+0000 7f83acc00700 1 -- 172.21.15.79:0/4060398800 shutdown_connections 2024-06-03T02:13:59.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.131+0000 7f83acc00700 1 -- 172.21.15.79:0/4060398800 wait complete. 2024-06-03T02:13:59.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83acc00700 1 Processor -- start 2024-06-03T02:13:59.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83acc00700 1 -- start start 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83acc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a806aa00 0x7f83a819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83acc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a806b3c0 0x7f83a819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83acc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83a81084f0 0x7f83a819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83acc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81127c0 con 0x7f83a806b3c0 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83acc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8112640 con 0x7f83a806aa00 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83acc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8112940 con 0x7f83a81084f0 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83a5400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a806b3c0 0x7f83a819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83a5400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a806b3c0 0x7f83a819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38664/0 (socket says 172.21.15.79:38664) 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.132+0000 7f83a5400700 1 -- 172.21.15.79:0/2534559963 learned_addr learned my addr 172.21.15.79:0/2534559963 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f83a6800700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83a81084f0 0x7f83a819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:59.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f83a5e00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a806aa00 0x7f83a819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:59.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f83a6800700 1 -- 172.21.15.79:0/2534559963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a806aa00 msgr2=0x7f83a819af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:59.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f83a6800700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a806aa00 0x7f83a819af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f83a6800700 1 -- 172.21.15.79:0/2534559963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a806b3c0 msgr2=0x7f83a819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:59.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f83a6800700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a806b3c0 0x7f83a819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f83a6800700 1 -- 172.21.15.79:0/2534559963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a819ff20 con 0x7f83a81084f0 2024-06-03T02:13:59.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f83a5e00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a806aa00 0x7f83a819af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:13:59.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f83a6800700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83a81084f0 0x7f83a819f7e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83a000efc0 tx=0x7f83a000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:59.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.133+0000 7f839ea00700 1 -- 172.21.15.79:0/2534559963 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00092e0 con 0x7f83a81084f0 2024-06-03T02:13:59.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.134+0000 7f839ea00700 1 -- 172.21.15.79:0/2534559963 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83a0004bb0 con 0x7f83a81084f0 2024-06-03T02:13:59.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.134+0000 7f839ea00700 1 -- 172.21.15.79:0/2534559963 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0004280 con 0x7f83a81084f0 2024-06-03T02:13:59.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.134+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a81a01b0 con 0x7f83a81084f0 2024-06-03T02:13:59.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.134+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a81a7a90 con 0x7f83a81084f0 2024-06-03T02:13:59.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.135+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a8069180 con 0x7f83a81084f0 2024-06-03T02:13:59.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.137+0000 7f839ea00700 1 -- 172.21.15.79:0/2534559963 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f83a001b030 con 0x7f83a81084f0 2024-06-03T02:13:59.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.138+0000 7f839ea00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f838c0a2c20 0x7f838c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:13:59.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.138+0000 7f839ea00700 1 -- 172.21.15.79:0/2534559963 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f83a00ce790 con 0x7f83a81084f0 2024-06-03T02:13:59.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.138+0000 7f83a5e00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f838c0a2c20 0x7f838c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:13:59.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.139+0000 7f83a5e00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f838c0a2c20 0x7f838c0a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8398005fd0 tx=0x7f8398009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:13:59.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.142+0000 7f839ea00700 1 -- 172.21.15.79:0/2534559963 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f83a0022020 con 0x7f83a81084f0 2024-06-03T02:13:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:13:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:58 smithi123 ceph-mon[36640]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:59.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:59.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:59.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:59.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:13:59.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:59.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:58 smithi135 ceph-mon[36653]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:59.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:59.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:59.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:13:59.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:13:59.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:13:59.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:58 smithi079 ceph-mon[31819]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:13:59.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.369+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f83a81a7c60 con 0x7f83a81084f0 2024-06-03T02:13:59.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.370+0000 7f839ea00700 1 -- 172.21.15.79:0/2534559963 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f83a0010040 con 0x7f83a81084f0 2024-06-03T02:13:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:13:59.369 INFO:teuthology.orchestra.run.smithi079.stdout:[ 2024-06-03T02:13:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:13:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 1, 2024-06-03T02:13:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": ".mgr" 2024-06-03T02:13:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:13:59.369 INFO:teuthology.orchestra.run.smithi079.stdout:] 2024-06-03T02:13:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.372+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f838c0a2c20 msgr2=0x7f838c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.372+0000 7f83acc00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f838c0a2c20 0x7f838c0a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8398005fd0 tx=0x7f8398009310 comp rx=0 tx=0).stop 2024-06-03T02:13:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.372+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83a81084f0 msgr2=0x7f83a819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:13:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.372+0000 7f83acc00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83a81084f0 0x7f83a819f7e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83a000efc0 tx=0x7f83a000c3a0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.373+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 shutdown_connections 2024-06-03T02:13:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.373+0000 7f83acc00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83a81084f0 0x7f83a819f7e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.373+0000 7f83acc00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a806aa00 0x7f83a819af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.373+0000 7f83acc00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f838c0a2c20 0x7f838c0a50e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.373+0000 7f83acc00700 1 --2- 172.21.15.79:0/2534559963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a806b3c0 0x7f83a819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:13:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.373+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 >> 172.21.15.79:0/2534559963 conn(0x7f83a8078300 msgr2=0x7f83a810d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:13:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.373+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 shutdown_connections 2024-06-03T02:13:59.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:13:59.373+0000 7f83acc00700 1 -- 172.21.15.79:0/2534559963 wait complete. 2024-06-03T02:14:00.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:13:59 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2534559963' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-03T02:14:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:13:59 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2534559963' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-03T02:14:00.229 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-03T02:14:00.229 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-03T02:14:00.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:13:59 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2534559963' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-03T02:14:00.664 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:01.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:00 smithi079 ceph-mon[31819]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:01.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:00 smithi135 ceph-mon[36653]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:01.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:00 smithi123 ceph-mon[36640]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:03.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.028+0000 7f0047e00700 1 -- 172.21.15.79:0/2564821444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 msgr2=0x7f0048104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:03.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.028+0000 7f0047e00700 1 --2- 172.21.15.79:0/2564821444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 0x7f0048104a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f00340099c0 tx=0x7f0034009cd0 comp rx=0 tx=0).stop 2024-06-03T02:14:03.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.028+0000 7f0047e00700 1 -- 172.21.15.79:0/2564821444 shutdown_connections 2024-06-03T02:14:03.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.028+0000 7f0047e00700 1 --2- 172.21.15.79:0/2564821444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0048104f70 0x7f0048111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:03.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.028+0000 7f0047e00700 1 --2- 172.21.15.79:0/2564821444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f004810a660 0x7f004810aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:03.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.028+0000 7f0047e00700 1 --2- 172.21.15.79:0/2564821444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 0x7f0048104a30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:03.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.028+0000 7f0047e00700 1 -- 172.21.15.79:0/2564821444 >> 172.21.15.79:0/2564821444 conn(0x7f00481000c0 msgr2=0x7f00481024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:03.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.028+0000 7f0047e00700 1 -- 172.21.15.79:0/2564821444 shutdown_connections 2024-06-03T02:14:03.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.028+0000 7f0047e00700 1 -- 172.21.15.79:0/2564821444 wait complete. 2024-06-03T02:14:03.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.029+0000 7f0047e00700 1 Processor -- start 2024-06-03T02:14:03.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.029+0000 7f0047e00700 1 -- start start 2024-06-03T02:14:03.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0047e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 0x7f00481a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:03.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0047e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0048104f70 0x7f00481a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0047e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004810a660 0x7f004819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0047e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0048114960 con 0x7f00481045b0 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0047e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00481147e0 con 0x7f004810a660 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0047e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0048114ae0 con 0x7f0048104f70 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 0x7f00481a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 0x7f00481a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38676/0 (socket says 172.21.15.79:38676) 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046a00700 1 -- 172.21.15.79:0/70262313 learned_addr learned my addr 172.21.15.79:0/70262313 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046000700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0048104f70 0x7f00481a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0047400700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004810a660 0x7f004819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046a00700 1 -- 172.21.15.79:0/70262313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0048104f70 msgr2=0x7f00481a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046a00700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0048104f70 0x7f00481a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046a00700 1 -- 172.21.15.79:0/70262313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004810a660 msgr2=0x7f004819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:03.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046a00700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004810a660 0x7f004819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:03.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046a00700 1 -- 172.21.15.79:0/70262313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004819b260 con 0x7f00481045b0 2024-06-03T02:14:03.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0046000700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0048104f70 0x7f00481a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:14:03.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.030+0000 7f0047400700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004810a660 0x7f004819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:14:03.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.031+0000 7f0046a00700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 0x7f00481a0940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f003800e9c0 tx=0x7f003800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:03.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.031+0000 7f003f400700 1 -- 172.21.15.79:0/70262313 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003800cbd0 con 0x7f00481045b0 2024-06-03T02:14:03.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.031+0000 7f003f400700 1 -- 172.21.15.79:0/70262313 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f003800ee60 con 0x7f00481045b0 2024-06-03T02:14:03.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.031+0000 7f003f400700 1 -- 172.21.15.79:0/70262313 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038005630 con 0x7f00481045b0 2024-06-03T02:14:03.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.031+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004819b550 con 0x7f00481045b0 2024-06-03T02:14:03.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.031+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00481a7810 con 0x7f00481045b0 2024-06-03T02:14:03.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.032+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0048069180 con 0x7f00481045b0 2024-06-03T02:14:03.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.034+0000 7f003f400700 1 -- 172.21.15.79:0/70262313 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0038004750 con 0x7f00481045b0 2024-06-03T02:14:03.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.036+0000 7f003f400700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f002c0a2ca0 0x7f002c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:03.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.036+0000 7f003f400700 1 -- 172.21.15.79:0/70262313 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f00380ceff0 con 0x7f00481045b0 2024-06-03T02:14:03.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.036+0000 7f0046000700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f002c0a2ca0 0x7f002c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:03.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.037+0000 7f0046000700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f002c0a2ca0 0x7f002c0a5160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0048109c60 tx=0x7f0034005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.038+0000 7f003f400700 1 -- 172.21.15.79:0/70262313 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0038053850 con 0x7f00481045b0 2024-06-03T02:14:03.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:03.264+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f004802dca0 con 0x7f00481045b0 2024-06-03T02:14:03.264 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:03 smithi079 ceph-mon[31819]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:03.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:03 smithi123 ceph-mon[36640]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:03 smithi135 ceph-mon[36653]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:04.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.219+0000 7f003f400700 1 -- 172.21.15.79:0/70262313 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f003809c390 con 0x7f00481045b0 2024-06-03T02:14:04.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f002c0a2ca0 msgr2=0x7f002c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:04.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f002c0a2ca0 0x7f002c0a5160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0048109c60 tx=0x7f0034005fb0 comp rx=0 tx=0).stop 2024-06-03T02:14:04.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 msgr2=0x7f00481a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:04.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 0x7f00481a0940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f003800e9c0 tx=0x7f003800ecd0 comp rx=0 tx=0).stop 2024-06-03T02:14:04.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 shutdown_connections 2024-06-03T02:14:04.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004810a660 0x7f004819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:04.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f002c0a2ca0 0x7f002c0a5160 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:04.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00481045b0 0x7f00481a0940 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f003800e9c0 tx=0x7f003800ecd0 comp rx=0 tx=0).stop 2024-06-03T02:14:04.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 --2- 172.21.15.79:0/70262313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0048104f70 0x7f00481a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:04.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.225+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 >> 172.21.15.79:0/70262313 conn(0x7f00481000c0 msgr2=0x7f0048101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:04.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.227+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 shutdown_connections 2024-06-03T02:14:04.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:04.227+0000 7f0047e00700 1 -- 172.21.15.79:0/70262313 wait complete. 2024-06-03T02:14:04.232 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' created 2024-06-03T02:14:04.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:04 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/70262313' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T02:14:04.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:04 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/70262313' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T02:14:04.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:04 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/70262313' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T02:14:05.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-03T02:14:05.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:05 smithi079 ceph-mon[31819]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:05.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:05 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/70262313' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-03T02:14:05.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:05 smithi079 ceph-mon[31819]: osdmap e84: 12 total, 12 up, 12 in 2024-06-03T02:14:05.593 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:05 smithi135 ceph-mon[36653]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:05 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/70262313' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-03T02:14:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:05 smithi135 ceph-mon[36653]: osdmap e84: 12 total, 12 up, 12 in 2024-06-03T02:14:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:05 smithi123 ceph-mon[36640]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:05 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/70262313' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-03T02:14:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:05 smithi123 ceph-mon[36640]: osdmap e84: 12 total, 12 up, 12 in 2024-06-03T02:14:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:06 smithi079 ceph-mon[31819]: osdmap e85: 12 total, 12 up, 12 in 2024-06-03T02:14:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:06 smithi079 ceph-mon[31819]: pgmap v322: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:06 smithi135 ceph-mon[36653]: osdmap e85: 12 total, 12 up, 12 in 2024-06-03T02:14:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:06 smithi135 ceph-mon[36653]: pgmap v322: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:06.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:06 smithi123 ceph-mon[36640]: osdmap e85: 12 total, 12 up, 12 in 2024-06-03T02:14:06.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:06 smithi123 ceph-mon[36640]: pgmap v322: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:07.383 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:07 smithi079 ceph-mon[31819]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T02:14:07.383 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:07 smithi079 ceph-mon[31819]: osdmap e86: 12 total, 12 up, 12 in 2024-06-03T02:14:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:07 smithi135 ceph-mon[36653]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T02:14:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:07 smithi135 ceph-mon[36653]: osdmap e86: 12 total, 12 up, 12 in 2024-06-03T02:14:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:07 smithi123 ceph-mon[36640]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T02:14:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:07 smithi123 ceph-mon[36640]: osdmap e86: 12 total, 12 up, 12 in 2024-06-03T02:14:07.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.874+0000 7f2a32c00700 1 -- 172.21.15.79:0/3070974963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a2c102f00 msgr2=0x7f2a2c10f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:07.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.874+0000 7f2a32c00700 1 --2- 172.21.15.79:0/3070974963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a2c102f00 0x7f2a2c10f390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400b0d0 tx=0x7f2a2400b3e0 comp rx=0 tx=0).stop 2024-06-03T02:14:07.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.874+0000 7f2a32c00700 1 -- 172.21.15.79:0/3070974963 shutdown_connections 2024-06-03T02:14:07.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.874+0000 7f2a32c00700 1 --2- 172.21.15.79:0/3070974963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a2c102f00 0x7f2a2c10f390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:07.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.874+0000 7f2a32c00700 1 --2- 172.21.15.79:0/3070974963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a2c1085c0 0x7f2a2c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:07.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.874+0000 7f2a32c00700 1 --2- 172.21.15.79:0/3070974963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c102540 0x7f2a2c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:07.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.874+0000 7f2a32c00700 1 -- 172.21.15.79:0/3070974963 >> 172.21.15.79:0/3070974963 conn(0x7f2a2c0795e0 msgr2=0x7f2a2c0799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:07.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.874+0000 7f2a32c00700 1 -- 172.21.15.79:0/3070974963 shutdown_connections 2024-06-03T02:14:07.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.874+0000 7f2a32c00700 1 -- 172.21.15.79:0/3070974963 wait complete. 2024-06-03T02:14:07.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.875+0000 7f2a32c00700 1 Processor -- start 2024-06-03T02:14:07.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.875+0000 7f2a32c00700 1 -- start start 2024-06-03T02:14:07.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.875+0000 7f2a32c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a2c102540 0x7f2a2c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:07.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.875+0000 7f2a32c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a2c102f00 0x7f2a2c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:07.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.875+0000 7f2a32c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c1085c0 0x7f2a2c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:07.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.875+0000 7f2a32c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c112810 con 0x7f2a2c102540 2024-06-03T02:14:07.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.875+0000 7f2a32c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c112690 con 0x7f2a2c1085c0 2024-06-03T02:14:07.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.875+0000 7f2a32c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c112990 con 0x7f2a2c102f00 2024-06-03T02:14:07.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a23400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a2c102f00 0x7f2a2c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:07.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a2be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a2c102540 0x7f2a2c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:07.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a23400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a2c102f00 0x7f2a2c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:34064/0 (socket says 172.21.15.79:34064) 2024-06-03T02:14:07.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a23400700 1 -- 172.21.15.79:0/2273834511 learned_addr learned my addr 172.21.15.79:0/2273834511 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:07.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a30c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c1085c0 0x7f2a2c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:07.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a30c00700 1 -- 172.21.15.79:0/2273834511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a2c102f00 msgr2=0x7f2a2c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:07.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a30c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a2c102f00 0x7f2a2c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:07.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a30c00700 1 -- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a2c102540 msgr2=0x7f2a2c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:07.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a30c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a2c102540 0x7f2a2c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:07.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a30c00700 1 -- 172.21.15.79:0/2273834511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c19fdc0 con 0x7f2a2c1085c0 2024-06-03T02:14:07.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a23400700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a2c102f00 0x7f2a2c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:14:07.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a2be00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a2c102540 0x7f2a2c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:14:07.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.876+0000 7f2a30c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c1085c0 0x7f2a2c19f680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400b770 tx=0x7f2a24007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:07.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.877+0000 7f2a29600700 1 -- 172.21.15.79:0/2273834511 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24021020 con 0x7f2a2c1085c0 2024-06-03T02:14:07.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.877+0000 7f2a29600700 1 -- 172.21.15.79:0/2273834511 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a24004c50 con 0x7f2a2c1085c0 2024-06-03T02:14:07.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.877+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2c1a0050 con 0x7f2a2c1085c0 2024-06-03T02:14:07.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.877+0000 7f2a29600700 1 -- 172.21.15.79:0/2273834511 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a240052b0 con 0x7f2a2c1085c0 2024-06-03T02:14:07.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.877+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2c1a7930 con 0x7f2a2c1085c0 2024-06-03T02:14:07.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.878+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2c103c30 con 0x7f2a2c1085c0 2024-06-03T02:14:07.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.880+0000 7f2a29600700 1 -- 172.21.15.79:0/2273834511 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2a24024020 con 0x7f2a2c1085c0 2024-06-03T02:14:07.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.881+0000 7f2a29600700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a100a2c20 0x7f2a100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:07.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.881+0000 7f2a29600700 1 -- 172.21.15.79:0/2273834511 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7497+0+0 (secure 0 0 0) 0x7f2a2400f020 con 0x7f2a2c1085c0 2024-06-03T02:14:07.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.881+0000 7f2a2be00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a100a2c20 0x7f2a100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:07.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.882+0000 7f2a2be00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a100a2c20 0x7f2a100a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c011490 tx=0x7f2a1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:07.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:07.885+0000 7f2a29600700 1 -- 172.21.15.79:0/2273834511 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a2409cef0 con 0x7f2a2c1085c0 2024-06-03T02:14:08.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.109+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f2a2c1a7b00 con 0x7f2a2c1085c0 2024-06-03T02:14:08.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.708+0000 7f2a29600700 1 -- 172.21.15.79:0/2273834511 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v87) ==== 171+0+0 (secure 0 0 0) 0x7f2a2401de20 con 0x7f2a2c1085c0 2024-06-03T02:14:08.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.713+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a100a2c20 msgr2=0x7f2a100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:08.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.713+0000 7f2a32c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a100a2c20 0x7f2a100a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c011490 tx=0x7f2a1c009260 comp rx=0 tx=0).stop 2024-06-03T02:14:08.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.713+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c1085c0 msgr2=0x7f2a2c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:08.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.713+0000 7f2a32c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c1085c0 0x7f2a2c19f680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400b770 tx=0x7f2a24007a20 comp rx=0 tx=0).stop 2024-06-03T02:14:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.714+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 shutdown_connections 2024-06-03T02:14:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.714+0000 7f2a32c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c1085c0 0x7f2a2c19f680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.714+0000 7f2a32c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2a100a2c20 0x7f2a100a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.714+0000 7f2a32c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a2c102540 0x7f2a2c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.714+0000 7f2a32c00700 1 --2- 172.21.15.79:0/2273834511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a2c102f00 0x7f2a2c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.714+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 >> 172.21.15.79:0/2273834511 conn(0x7f2a2c0795e0 msgr2=0x7f2a2c0746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.715+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 shutdown_connections 2024-06-03T02:14:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:08.715+0000 7f2a32c00700 1 -- 172.21.15.79:0/2273834511 wait complete. 2024-06-03T02:14:08.718 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' created 2024-06-03T02:14:09.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:08 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2273834511' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T02:14:09.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:08 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T02:14:09.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:08 smithi079 ceph-mon[31819]: pgmap v324: 33 pgs: 6 unknown, 27 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:09.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:08 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2273834511' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T02:14:09.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:08 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T02:14:09.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:08 smithi135 ceph-mon[36653]: pgmap v324: 33 pgs: 6 unknown, 27 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:09.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:08 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2273834511' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T02:14:09.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:08 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T02:14:09.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:08 smithi123 ceph-mon[36640]: pgmap v324: 33 pgs: 6 unknown, 27 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:09.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-03T02:14:09.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:09 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-03T02:14:09.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:09 smithi079 ceph-mon[31819]: osdmap e87: 12 total, 12 up, 12 in 2024-06-03T02:14:10.093 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:10.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:09 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-03T02:14:10.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:09 smithi135 ceph-mon[36653]: osdmap e87: 12 total, 12 up, 12 in 2024-06-03T02:14:10.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:09 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-03T02:14:10.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:09 smithi123 ceph-mon[36640]: osdmap e87: 12 total, 12 up, 12 in 2024-06-03T02:14:10.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:10 smithi079 ceph-mon[31819]: osdmap e88: 12 total, 12 up, 12 in 2024-06-03T02:14:10.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:10 smithi079 ceph-mon[31819]: pgmap v327: 97 pgs: 19 creating+peering, 33 active+clean, 45 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:11.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:10 smithi135 ceph-mon[36653]: osdmap e88: 12 total, 12 up, 12 in 2024-06-03T02:14:11.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:10 smithi135 ceph-mon[36653]: pgmap v327: 97 pgs: 19 creating+peering, 33 active+clean, 45 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:11.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:10 smithi123 ceph-mon[36640]: osdmap e88: 12 total, 12 up, 12 in 2024-06-03T02:14:11.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:10 smithi123 ceph-mon[36640]: pgmap v327: 97 pgs: 19 creating+peering, 33 active+clean, 45 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:12.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:11 smithi079 ceph-mon[31819]: osdmap e89: 12 total, 12 up, 12 in 2024-06-03T02:14:12.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:11 smithi135 ceph-mon[36653]: osdmap e89: 12 total, 12 up, 12 in 2024-06-03T02:14:12.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:11 smithi123 ceph-mon[36640]: osdmap e89: 12 total, 12 up, 12 in 2024-06-03T02:14:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.266+0000 7f902d600700 1 -- 172.21.15.79:0/4110431907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9028102ee0 msgr2=0x7f902810f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.266+0000 7f902d600700 1 --2- 172.21.15.79:0/4110431907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9028102ee0 0x7f902810f390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9020009fc0 tx=0x7f902000b330 comp rx=0 tx=0).stop 2024-06-03T02:14:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.266+0000 7f902d600700 1 -- 172.21.15.79:0/4110431907 shutdown_connections 2024-06-03T02:14:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.266+0000 7f902d600700 1 --2- 172.21.15.79:0/4110431907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9028102ee0 0x7f902810f390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.266+0000 7f902d600700 1 --2- 172.21.15.79:0/4110431907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90281085c0 0x7f90281089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.266+0000 7f902d600700 1 --2- 172.21.15.79:0/4110431907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9028102520 0x7f90281029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.266+0000 7f902d600700 1 -- 172.21.15.79:0/4110431907 >> 172.21.15.79:0/4110431907 conn(0x7f9028079680 msgr2=0x7f9028079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.267+0000 7f902d600700 1 -- 172.21.15.79:0/4110431907 shutdown_connections 2024-06-03T02:14:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.267+0000 7f902d600700 1 -- 172.21.15.79:0/4110431907 wait complete. 2024-06-03T02:14:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.268+0000 7f902d600700 1 Processor -- start 2024-06-03T02:14:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.268+0000 7f902d600700 1 -- start start 2024-06-03T02:14:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.268+0000 7f902d600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9028102520 0x7f902819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.268+0000 7f902d600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9028102ee0 0x7f902819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.268+0000 7f902d600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90281085c0 0x7f9028198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.268+0000 7f902d600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9028112850 con 0x7f9028102ee0 2024-06-03T02:14:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.268+0000 7f902d600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90281126d0 con 0x7f9028102520 2024-06-03T02:14:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.268+0000 7f902d600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90281129d0 con 0x7f90281085c0 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f901fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9028102ee0 0x7f902819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f9027400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90281085c0 0x7f9028198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f9026a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9028102520 0x7f902819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f901fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9028102ee0 0x7f902819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58556/0 (socket says 172.21.15.79:58556) 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f901fe00700 1 -- 172.21.15.79:0/295969019 learned_addr learned my addr 172.21.15.79:0/295969019 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f9026a00700 1 -- 172.21.15.79:0/295969019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90281085c0 msgr2=0x7f9028198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f9026a00700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90281085c0 0x7f9028198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f9026a00700 1 -- 172.21.15.79:0/295969019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9028102ee0 msgr2=0x7f902819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f9026a00700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9028102ee0 0x7f902819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f9026a00700 1 -- 172.21.15.79:0/295969019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90281991b0 con 0x7f9028102520 2024-06-03T02:14:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.269+0000 7f9027400700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90281085c0 0x7f9028198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:14:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.270+0000 7f9026a00700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9028102520 0x7f902819e840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f901800eb90 tx=0x7f901800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.270+0000 7f901f400700 1 -- 172.21.15.79:0/295969019 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901800cc90 con 0x7f9028102520 2024-06-03T02:14:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.270+0000 7f901f400700 1 -- 172.21.15.79:0/295969019 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f901800ce30 con 0x7f9028102520 2024-06-03T02:14:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.270+0000 7f901f400700 1 -- 172.21.15.79:0/295969019 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018010430 con 0x7f9028102520 2024-06-03T02:14:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.270+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90281994a0 con 0x7f9028102520 2024-06-03T02:14:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.270+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90281a5710 con 0x7f9028102520 2024-06-03T02:14:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.271+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9028069180 con 0x7f9028102520 2024-06-03T02:14:12.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.273+0000 7f901f400700 1 -- 172.21.15.79:0/295969019 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9018003680 con 0x7f9028102520 2024-06-03T02:14:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.274+0000 7f901f400700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f900c0a2ca0 0x7f900c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.274+0000 7f901f400700 1 -- 172.21.15.79:0/295969019 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7879+0+0 (secure 0 0 0) 0x7f90180cf1a0 con 0x7f9028102520 2024-06-03T02:14:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.275+0000 7f901fe00700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f900c0a2ca0 0x7f900c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.275+0000 7f901fe00700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f900c0a2ca0 0x7f900c0a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9028079ba0 tx=0x7f9010009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:12.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.277+0000 7f901f400700 1 -- 172.21.15.79:0/295969019 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f901809c210 con 0x7f9028102520 2024-06-03T02:14:12.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.548+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f902802dca0 con 0x7f9028102520 2024-06-03T02:14:12.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.768+0000 7f901f400700 1 -- 172.21.15.79:0/295969019 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f901809bb10 con 0x7f9028102520 2024-06-03T02:14:12.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f900c0a2ca0 msgr2=0x7f900c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:12.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f900c0a2ca0 0x7f900c0a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9028079ba0 tx=0x7f9010009260 comp rx=0 tx=0).stop 2024-06-03T02:14:12.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9028102520 msgr2=0x7f902819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:12.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9028102520 0x7f902819e840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f901800eb90 tx=0x7f901800eea0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 shutdown_connections 2024-06-03T02:14:12.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90281085c0 0x7f9028198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9028102520 0x7f902819e840 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f900c0a2ca0 0x7f900c0a5160 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 --2- 172.21.15.79:0/295969019 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9028102ee0 0x7f902819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:12.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.775+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 >> 172.21.15.79:0/295969019 conn(0x7f9028079680 msgr2=0x7f9028100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:12.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.777+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 shutdown_connections 2024-06-03T02:14:12.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:12.777+0000 7f902d600700 1 -- 172.21.15.79:0/295969019 wait complete. 2024-06-03T02:14:12.785 INFO:teuthology.orchestra.run.smithi079.stderr:new fs with metadata pool 2 and data pool 3 2024-06-03T02:14:12.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:12 smithi079 ceph-mon[31819]: pgmap v329: 97 pgs: 19 creating+peering, 48 active+clean, 30 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:12.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:12 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/295969019' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T02:14:12.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:12 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T02:14:12.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:12 smithi079 ceph-mon[31819]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-03T02:14:13.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:12 smithi135 ceph-mon[36653]: pgmap v329: 97 pgs: 19 creating+peering, 48 active+clean, 30 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:13.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:12 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/295969019' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T02:14:13.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:12 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T02:14:13.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:12 smithi135 ceph-mon[36653]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-03T02:14:13.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:12 smithi123 ceph-mon[36640]: pgmap v329: 97 pgs: 19 creating+peering, 48 active+clean, 30 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:13.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:12 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/295969019' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T02:14:13.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:12 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T02:14:13.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:12 smithi123 ceph-mon[36640]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-03T02:14:13.754 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-03T02:14:13.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-03T02:14:14.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:13 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-03T02:14:14.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:13 smithi079 ceph-mon[31819]: osdmap e90: 12 total, 12 up, 12 in 2024-06-03T02:14:14.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:13 smithi079 ceph-mon[31819]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-03T02:14:14.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:13 smithi079 ceph-mon[31819]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-03T02:14:14.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:13 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-03T02:14:14.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:13 smithi135 ceph-mon[36653]: osdmap e90: 12 total, 12 up, 12 in 2024-06-03T02:14:14.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:13 smithi135 ceph-mon[36653]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-03T02:14:14.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:13 smithi135 ceph-mon[36653]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-03T02:14:14.187 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:14.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:13 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-03T02:14:14.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:13 smithi123 ceph-mon[36640]: osdmap e90: 12 total, 12 up, 12 in 2024-06-03T02:14:14.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:13 smithi123 ceph-mon[36640]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-03T02:14:14.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:13 smithi123 ceph-mon[36640]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-03T02:14:14.722 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:14:14 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b[67499]: 2024-06-03T02:14:14.234+0000 7f47dc5e0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-03T02:14:14.964 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:14 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] up:active 2024-06-03T02:14:14.964 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:14 smithi079 ceph-mon[31819]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:14:14.964 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:14 smithi079 ceph-mon[31819]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:14:14.964 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:14 smithi079 ceph-mon[31819]: pgmap v331: 97 pgs: 8 creating+peering, 89 active+clean; 578 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 564 B/s wr, 1 op/s 2024-06-03T02:14:15.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:14 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] up:active 2024-06-03T02:14:15.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:14 smithi135 ceph-mon[36653]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:14:15.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:14 smithi135 ceph-mon[36653]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:14:15.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:14 smithi135 ceph-mon[36653]: pgmap v331: 97 pgs: 8 creating+peering, 89 active+clean; 578 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 564 B/s wr, 1 op/s 2024-06-03T02:14:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:14 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] up:active 2024-06-03T02:14:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:14 smithi123 ceph-mon[36640]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:14:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:14 smithi123 ceph-mon[36640]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:14:15.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:14 smithi123 ceph-mon[36640]: pgmap v331: 97 pgs: 8 creating+peering, 89 active+clean; 578 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 564 B/s wr, 1 op/s 2024-06-03T02:14:15.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:15 smithi079 ceph-mon[31819]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-03T02:14:15.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:15 smithi079 ceph-mon[31819]: Cluster is now healthy 2024-06-03T02:14:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:15 smithi135 ceph-mon[36653]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-03T02:14:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:15 smithi135 ceph-mon[36653]: Cluster is now healthy 2024-06-03T02:14:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:15 smithi123 ceph-mon[36640]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-03T02:14:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:15 smithi123 ceph-mon[36640]: Cluster is now healthy 2024-06-03T02:14:16.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.326+0000 7f4e4ae00700 1 -- 172.21.15.79:0/3314296632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e44102fc0 msgr2=0x7f4e44107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:16.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.326+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/3314296632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e44102fc0 0x7f4e44107010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e300099c0 tx=0x7f4e30009cd0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.326+0000 7f4e4ae00700 1 -- 172.21.15.79:0/3314296632 shutdown_connections 2024-06-03T02:14:16.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.326+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/3314296632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e441076f0 0x7f4e4410f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.326+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/3314296632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e44102610 0x7f4e441029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.326+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/3314296632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e44102fc0 0x7f4e44107010 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.326+0000 7f4e4ae00700 1 -- 172.21.15.79:0/3314296632 >> 172.21.15.79:0/3314296632 conn(0x7f4e440fdea0 msgr2=0x7f4e441002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:16.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.326+0000 7f4e4ae00700 1 -- 172.21.15.79:0/3314296632 shutdown_connections 2024-06-03T02:14:16.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.326+0000 7f4e4ae00700 1 -- 172.21.15.79:0/3314296632 wait complete. 2024-06-03T02:14:16.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.327+0000 7f4e4ae00700 1 Processor -- start 2024-06-03T02:14:16.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.327+0000 7f4e4ae00700 1 -- start start 2024-06-03T02:14:16.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e4ae00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e44102610 0x7f4e4419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:16.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e4ae00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e44102fc0 0x7f4e4419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:16.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e4ae00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e441076f0 0x7f4e4419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:16.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e4ae00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4e44112700 con 0x7f4e44102610 2024-06-03T02:14:16.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e4ae00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e44112580 con 0x7f4e441076f0 2024-06-03T02:14:16.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e4ae00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e44112880 con 0x7f4e44102fc0 2024-06-03T02:14:16.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e49a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e44102610 0x7f4e4419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:16.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e49a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e44102610 0x7f4e4419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51972/0 (socket says 172.21.15.79:51972) 2024-06-03T02:14:16.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e49a00700 1 -- 172.21.15.79:0/296768581 learned_addr learned my addr 172.21.15.79:0/296768581 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:16.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e4a400700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e441076f0 0x7f4e4419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:16.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.328+0000 7f4e49000700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e44102fc0 0x7f4e4419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:16.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e4a400700 1 -- 172.21.15.79:0/296768581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e44102fc0 msgr2=0x7f4e4419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:16.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e4a400700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e44102fc0 0x7f4e4419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e4a400700 1 -- 172.21.15.79:0/296768581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e44102610 msgr2=0x7f4e4419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:16.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e4a400700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e44102610 0x7f4e4419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e4a400700 1 -- 172.21.15.79:0/296768581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4419fdb0 con 0x7f4e441076f0 2024-06-03T02:14:16.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e4a400700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e441076f0 0x7f4e4419f670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00efc0 tx=0x7f4e3c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:16.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e42000700 1 -- 172.21.15.79:0/296768581 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0092e0 con 0x7f4e441076f0 2024-06-03T02:14:16.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e42000700 1 -- 172.21.15.79:0/296768581 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e3c004bb0 con 0x7f4e441076f0 2024-06-03T02:14:16.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e441a0040 con 0x7f4e441076f0 2024-06-03T02:14:16.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.329+0000 7f4e42000700 1 -- 172.21.15.79:0/296768581 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c004280 con 0x7f4e441076f0 2024-06-03T02:14:16.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.330+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e441a7920 con 0x7f4e441076f0 2024-06-03T02:14:16.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.331+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e44069180 con 0x7f4e441076f0 2024-06-03T02:14:16.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.333+0000 7f4e42000700 1 -- 172.21.15.79:0/296768581 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4e3c01b030 con 0x7f4e441076f0 2024-06-03T02:14:16.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.334+0000 7f4e42000700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e2c0a2c20 0x7f4e2c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:16.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.334+0000 7f4e42000700 1 -- 172.21.15.79:0/296768581 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f4e3c0ce9f0 con 0x7f4e441076f0 2024-06-03T02:14:16.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.334+0000 7f4e49a00700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e2c0a2c20 0x7f4e2c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:16.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.335+0000 7f4e49a00700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e2c0a2c20 0x7f4e2c0a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009d20 tx=0x7f4e38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:16.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.338+0000 7f4e42000700 1 -- 172.21.15.79:0/296768581 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e3c022020 con 0x7f4e441076f0 2024-06-03T02:14:16.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.571+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f4e441a7af0 con 0x7f4e441076f0 2024-06-03T02:14:16.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.820+0000 7f4e42000700 1 -- 172.21.15.79:0/296768581 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v91) ==== 157+0+0 (secure 0 0 0) 0x7f4e3c010040 con 0x7f4e441076f0 2024-06-03T02:14:16.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.824+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e2c0a2c20 msgr2=0x7f4e2c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:16.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.824+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e2c0a2c20 0x7f4e2c0a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009d20 tx=0x7f4e38009260 comp rx=0 tx=0).stop 2024-06-03T02:14:16.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e441076f0 msgr2=0x7f4e4419f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:16.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e441076f0 0x7f4e4419f670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00efc0 tx=0x7f4e3c00c3a0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.825 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:16 smithi079 ceph-mon[31819]: pgmap v332: 97 pgs: 97 active+clean; 578 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 478 B/s wr, 0 op/s 2024-06-03T02:14:16.825 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:16 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/296768581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T02:14:16.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:16 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T02:14:16.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 shutdown_connections 2024-06-03T02:14:16.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e441076f0 0x7f4e4419f670 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e2c0a2c20 0x7f4e2c0a50e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e44102610 0x7f4e4419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 --2- 172.21.15.79:0/296768581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e44102fc0 0x7f4e4419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 >> 172.21.15.79:0/296768581 conn(0x7f4e440fdea0 msgr2=0x7f4e44100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 shutdown_connections 2024-06-03T02:14:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:16.825+0000 7f4e4ae00700 1 -- 172.21.15.79:0/296768581 wait complete. 2024-06-03T02:14:17.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:16 smithi135 ceph-mon[36653]: pgmap v332: 97 pgs: 97 active+clean; 578 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 478 B/s wr, 0 op/s 2024-06-03T02:14:17.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:16 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/296768581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T02:14:17.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:16 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T02:14:17.124 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:16 smithi123 ceph-mon[36640]: pgmap v332: 97 pgs: 97 active+clean; 578 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 478 B/s wr, 0 op/s 2024-06-03T02:14:17.124 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:16 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/296768581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T02:14:17.124 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:16 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T02:14:17.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-03T02:14:17.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:17 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-03T02:14:17.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:17 smithi079 ceph-mon[31819]: osdmap e91: 12 total, 12 up, 12 in 2024-06-03T02:14:18.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:17 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-03T02:14:18.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:17 smithi135 ceph-mon[36653]: osdmap e91: 12 total, 12 up, 12 in 2024-06-03T02:14:18.099 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:18.136 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:17 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-03T02:14:18.136 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:17 smithi123 ceph-mon[36640]: osdmap e91: 12 total, 12 up, 12 in 2024-06-03T02:14:19.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:18 smithi079 ceph-mon[31819]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-03T02:14:19.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:18 smithi135 ceph-mon[36653]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-03T02:14:19.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:18 smithi123 ceph-mon[36640]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-03T02:14:20.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.332+0000 7f1603800700 1 -- 172.21.15.79:0/2997051089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15fc108ac0 msgr2=0x7f15fc108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:20.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.332+0000 7f1603800700 1 --2- 172.21.15.79:0/2997051089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15fc108ac0 0x7f15fc108f40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009990 tx=0x7f15e8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:14:20.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.332+0000 7f1603800700 1 -- 172.21.15.79:0/2997051089 shutdown_connections 2024-06-03T02:14:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.332+0000 7f1603800700 1 --2- 172.21.15.79:0/2997051089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15fc109480 0x7f15fc115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.332+0000 7f1603800700 1 --2- 172.21.15.79:0/2997051089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15fc10eac0 0x7f15fc10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.332+0000 7f1603800700 1 --2- 172.21.15.79:0/2997051089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15fc108ac0 0x7f15fc108f40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.332+0000 7f1603800700 1 -- 172.21.15.79:0/2997051089 >> 172.21.15.79:0/2997051089 conn(0x7f15fc075c60 msgr2=0x7f15fc078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.332+0000 7f1603800700 1 -- 172.21.15.79:0/2997051089 shutdown_connections 2024-06-03T02:14:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.332+0000 7f1603800700 1 -- 172.21.15.79:0/2997051089 wait complete. 2024-06-03T02:14:20.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.333+0000 7f1603800700 1 Processor -- start 2024-06-03T02:14:20.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.333+0000 7f1603800700 1 -- start start 2024-06-03T02:14:20.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.333+0000 7f1603800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15fc108ac0 0x7f15fc19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:20.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.333+0000 7f1603800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15fc109480 0x7f15fc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:20.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.333+0000 7f1603800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15fc10eac0 0x7f15fc1a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.333+0000 7f1603800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc118ea0 con 0x7f15fc108ac0 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.333+0000 7f1603800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc118d20 con 0x7f15fc10eac0 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.333+0000 7f1603800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc119020 con 0x7f15fc109480 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1601800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15fc10eac0 0x7f15fc1a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1600e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15fc108ac0 0x7f15fc19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1601800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15fc10eac0 0x7f15fc1a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36218/0 (socket says 172.21.15.79:36218) 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1601800700 1 -- 172.21.15.79:0/2588122113 learned_addr learned my addr 172.21.15.79:0/2588122113 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f15fbe00700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15fc109480 0x7f15fc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1600e00700 1 -- 172.21.15.79:0/2588122113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15fc109480 msgr2=0x7f15fc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1600e00700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15fc109480 0x7f15fc19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1600e00700 1 -- 172.21.15.79:0/2588122113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15fc10eac0 msgr2=0x7f15fc1a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1600e00700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15fc10eac0 0x7f15fc1a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:20.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1600e00700 1 -- 172.21.15.79:0/2588122113 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc1a41a0 con 0x7f15fc108ac0 2024-06-03T02:14:20.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.334+0000 7f1601800700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15fc10eac0 0x7f15fc1a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:14:20.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.335+0000 7f1600e00700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15fc108ac0 0x7f15fc19f190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f15f000e9f0 tx=0x7f15f000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:20.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.335+0000 7f15f9600700 1 -- 172.21.15.79:0/2588122113 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f000c9c0 con 0x7f15fc108ac0 2024-06-03T02:14:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.335+0000 7f15f9600700 1 -- 172.21.15.79:0/2588122113 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f000cb60 con 0x7f15fc108ac0 2024-06-03T02:14:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.335+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc1a4490 con 0x7f15fc108ac0 2024-06-03T02:14:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.335+0000 7f15f9600700 1 -- 172.21.15.79:0/2588122113 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0018680 con 0x7f15fc108ac0 2024-06-03T02:14:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.335+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc1abd10 con 0x7f15fc108ac0 2024-06-03T02:14:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.336+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc050a00 con 0x7f15fc108ac0 2024-06-03T02:14:20.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.339+0000 7f15f9600700 1 -- 172.21.15.79:0/2588122113 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f15f0010430 con 0x7f15fc108ac0 2024-06-03T02:14:20.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.341+0000 7f15f9600700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f15e40a2cc0 0x7f15e40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:20.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.341+0000 7f15f9600700 1 -- 172.21.15.79:0/2588122113 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f15f00cf320 con 0x7f15fc108ac0 2024-06-03T02:14:20.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.341+0000 7f15fbe00700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f15e40a2cc0 0x7f15e40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:20.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.342+0000 7f15fbe00700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f15e40a2cc0 0x7f15e40a5180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f15fc10dec0 tx=0x7f15e800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:20.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.345+0000 7f15f9600700 1 -- 172.21.15.79:0/2588122113 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f15f009c1f0 con 0x7f15fc108ac0 2024-06-03T02:14:20.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:20.574+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f15fc1abee0 con 0x7f15fc108ac0 2024-06-03T02:14:22.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:21 smithi079 ceph-mon[31819]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T02:14:22.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:21 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2588122113' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T02:14:22.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:21 smithi135 ceph-mon[36653]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T02:14:22.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:21 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2588122113' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T02:14:22.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.218+0000 7f15f9600700 1 -- 172.21.15.79:0/2588122113 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v93) ==== 228+0+0 (secure 0 0 0) 0x7f15f009baf0 con 0x7f15fc108ac0 2024-06-03T02:14:22.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:21 smithi123 ceph-mon[36640]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T02:14:22.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:21 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2588122113' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T02:14:22.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.225+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f15e40a2cc0 msgr2=0x7f15e40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:22.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.225+0000 7f1603800700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f15e40a2cc0 0x7f15e40a5180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f15fc10dec0 tx=0x7f15e800b260 comp rx=0 tx=0).stop 2024-06-03T02:14:22.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.225+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15fc108ac0 msgr2=0x7f15fc19f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:22.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.225+0000 7f1603800700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15fc108ac0 0x7f15fc19f190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f15f000e9f0 tx=0x7f15f000ed00 comp rx=0 tx=0).stop 2024-06-03T02:14:22.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.227+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 shutdown_connections 2024-06-03T02:14:22.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.227+0000 7f1603800700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15fc10eac0 0x7f15fc1a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:22.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.227+0000 7f1603800700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f15e40a2cc0 0x7f15e40a5180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:22.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.227+0000 7f1603800700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15fc108ac0 0x7f15fc19f190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:22.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.227+0000 7f1603800700 1 --2- 172.21.15.79:0/2588122113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15fc109480 0x7f15fc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:22.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.227+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 >> 172.21.15.79:0/2588122113 conn(0x7f15fc075c60 msgr2=0x7f15fc0760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:22.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.228+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 shutdown_connections 2024-06-03T02:14:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:22.229+0000 7f1603800700 1 -- 172.21.15.79:0/2588122113 wait complete. 2024-06-03T02:14:22.237 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data_ec' created 2024-06-03T02:14:22.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:22 smithi079 ceph-mon[31819]: osdmap e92: 12 total, 12 up, 12 in 2024-06-03T02:14:22.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:22 smithi079 ceph-mon[31819]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-03T02:14:22.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:22 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2588122113' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-03T02:14:22.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:22 smithi079 ceph-mon[31819]: osdmap e93: 12 total, 12 up, 12 in 2024-06-03T02:14:23.005 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-03T02:14:23.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:22 smithi135 ceph-mon[36653]: osdmap e92: 12 total, 12 up, 12 in 2024-06-03T02:14:23.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:22 smithi135 ceph-mon[36653]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-03T02:14:23.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:22 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2588122113' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-03T02:14:23.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:22 smithi135 ceph-mon[36653]: osdmap e93: 12 total, 12 up, 12 in 2024-06-03T02:14:23.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:22 smithi123 ceph-mon[36640]: osdmap e92: 12 total, 12 up, 12 in 2024-06-03T02:14:23.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:22 smithi123 ceph-mon[36640]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-03T02:14:23.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:22 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2588122113' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-03T02:14:23.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:22 smithi123 ceph-mon[36640]: osdmap e93: 12 total, 12 up, 12 in 2024-06-03T02:14:23.453 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:24 smithi079 ceph-mon[31819]: osdmap e94: 12 total, 12 up, 12 in 2024-06-03T02:14:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:24 smithi135 ceph-mon[36653]: osdmap e94: 12 total, 12 up, 12 in 2024-06-03T02:14:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:24 smithi123 ceph-mon[36640]: osdmap e94: 12 total, 12 up, 12 in 2024-06-03T02:14:25.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:25 smithi079 ceph-mon[31819]: pgmap v340: 129 pgs: 3 creating+peering, 25 unknown, 101 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:25.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:25 smithi079 ceph-mon[31819]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T02:14:25.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:25 smithi079 ceph-mon[31819]: osdmap e95: 12 total, 12 up, 12 in 2024-06-03T02:14:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:25 smithi135 ceph-mon[36653]: pgmap v340: 129 pgs: 3 creating+peering, 25 unknown, 101 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:25 smithi135 ceph-mon[36653]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T02:14:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:25 smithi135 ceph-mon[36653]: osdmap e95: 12 total, 12 up, 12 in 2024-06-03T02:14:25.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.625+0000 7f5bcea00700 1 -- 172.21.15.79:0/1010922472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc8108500 msgr2=0x7f5bc81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:25.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.625+0000 7f5bcea00700 1 --2- 172.21.15.79:0/1010922472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc8108500 0x7f5bc81088e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009990 tx=0x7f5bb8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:14:25.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.626+0000 7f5bcea00700 1 -- 172.21.15.79:0/1010922472 shutdown_connections 2024-06-03T02:14:25.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.626+0000 7f5bcea00700 1 --2- 172.21.15.79:0/1010922472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc806ab30 0x7f5bc810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:25.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.626+0000 7f5bcea00700 1 --2- 172.21.15.79:0/1010922472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc8108500 0x7f5bc81088e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:25.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.626+0000 7f5bcea00700 1 --2- 172.21.15.79:0/1010922472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bc806a170 0x7f5bc806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:25.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.626+0000 7f5bcea00700 1 -- 172.21.15.79:0/1010922472 >> 172.21.15.79:0/1010922472 conn(0x7f5bc8078340 msgr2=0x7f5bc8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:25.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.626+0000 7f5bcea00700 1 -- 172.21.15.79:0/1010922472 shutdown_connections 2024-06-03T02:14:25.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.626+0000 7f5bcea00700 1 -- 172.21.15.79:0/1010922472 wait complete. 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.627+0000 7f5bcea00700 1 Processor -- start 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.627+0000 7f5bcea00700 1 -- start start 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.627+0000 7f5bcea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc806a170 0x7f5bc819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.627+0000 7f5bcea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc806ab30 0x7f5bc819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.627+0000 7f5bcea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bc8108500 0x7f5bc819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bcea00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81127d0 con 0x7f5bc806ab30 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bcea00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8112650 con 0x7f5bc8108500 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bcea00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8112950 con 0x7f5bc806a170 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bc7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc806a170 0x7f5bc819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bc7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc806a170 0x7f5bc819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:49902/0 (socket says 172.21.15.79:49902) 2024-06-03T02:14:25.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bc7e00700 1 -- 172.21.15.79:0/2892424827 learned_addr learned my addr 172.21.15.79:0/2892424827 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:25.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bc7400700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc806ab30 0x7f5bc819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:25.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bcca00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bc8108500 0x7f5bc819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:25.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bc7e00700 1 -- 172.21.15.79:0/2892424827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bc8108500 msgr2=0x7f5bc819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bc7e00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bc8108500 0x7f5bc819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bc7e00700 1 -- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc806ab30 msgr2=0x7f5bc819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bc7e00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc806ab30 0x7f5bc819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.628+0000 7f5bc7e00700 1 -- 172.21.15.79:0/2892424827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc819fee0 con 0x7f5bc806a170 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.629+0000 7f5bc7e00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc806a170 0x7f5bc819aed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000c00 tx=0x7f5bb800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.629+0000 7f5bc4c00700 1 -- 172.21.15.79:0/2892424827 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8027070 con 0x7f5bc806a170 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.629+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc81a0170 con 0x7f5bc806a170 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.629+0000 7f5bc4c00700 1 -- 172.21.15.79:0/2892424827 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb80056d0 con 0x7f5bc806a170 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.629+0000 7f5bc4c00700 1 -- 172.21.15.79:0/2892424827 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8022e10 con 0x7f5bc806a170 2024-06-03T02:14:25.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.629+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81a7a50 con 0x7f5bc806a170 2024-06-03T02:14:25.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.630+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc8103bc0 con 0x7f5bc806a170 2024-06-03T02:14:25.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.632+0000 7f5bc4c00700 1 -- 172.21.15.79:0/2892424827 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5bb802a070 con 0x7f5bc806a170 2024-06-03T02:14:25.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.633+0000 7f5bc4c00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bac0a2c20 0x7f5bac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:25.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.634+0000 7f5bc4c00700 1 -- 172.21.15.79:0/2892424827 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8770+0+0 (secure 0 0 0) 0x7f5bb80cfed0 con 0x7f5bc806a170 2024-06-03T02:14:25.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.634+0000 7f5bc7400700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bac0a2c20 0x7f5bac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:25.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.635+0000 7f5bc7400700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bac0a2c20 0x7f5bac0a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8103420 tx=0x7f5bb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:25.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.638+0000 7f5bc4c00700 1 -- 172.21.15.79:0/2892424827 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5bb8013070 con 0x7f5bc806a170 2024-06-03T02:14:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:25 smithi123 ceph-mon[36640]: pgmap v340: 129 pgs: 3 creating+peering, 25 unknown, 101 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:25 smithi123 ceph-mon[36640]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T02:14:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:25 smithi123 ceph-mon[36640]: osdmap e95: 12 total, 12 up, 12 in 2024-06-03T02:14:25.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:25.868+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f5bc81a7c20 con 0x7f5bc806a170 2024-06-03T02:14:26.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.248+0000 7f5bc4c00700 1 -- 172.21.15.79:0/2892424827 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v96) ==== 169+0+0 (secure 0 0 0) 0x7f5bb809cc40 con 0x7f5bc806a170 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.251+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bac0a2c20 msgr2=0x7f5bac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.251+0000 7f5bcea00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bac0a2c20 0x7f5bac0a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8103420 tx=0x7f5bb4009260 comp rx=0 tx=0).stop 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.251+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc806a170 msgr2=0x7f5bc819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.251+0000 7f5bcea00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc806a170 0x7f5bc819aed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000c00 tx=0x7f5bb800bab0 comp rx=0 tx=0).stop 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.252+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 shutdown_connections 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.252+0000 7f5bcea00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bc8108500 0x7f5bc819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.252+0000 7f5bcea00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc806a170 0x7f5bc819aed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.252+0000 7f5bcea00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5bac0a2c20 0x7f5bac0a50e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.252+0000 7f5bcea00700 1 --2- 172.21.15.79:0/2892424827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bc806ab30 0x7f5bc819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.252+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 >> 172.21.15.79:0/2892424827 conn(0x7f5bc8078340 msgr2=0x7f5bc810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:26.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.252+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 shutdown_connections 2024-06-03T02:14:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:26.252+0000 7f5bcea00700 1 -- 172.21.15.79:0/2892424827 wait complete. 2024-06-03T02:14:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:set pool 4 allow_ec_overwrites to true 2024-06-03T02:14:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:26 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2892424827' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T02:14:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:26 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T02:14:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:26 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2892424827' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T02:14:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:26 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T02:14:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:26 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2892424827' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T02:14:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:26 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T02:14:27.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-03T02:14:27.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:27 smithi079 ceph-mon[31819]: pgmap v342: 129 pgs: 3 creating+peering, 23 unknown, 103 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:27.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:27 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-03T02:14:27.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:27 smithi079 ceph-mon[31819]: osdmap e96: 12 total, 12 up, 12 in 2024-06-03T02:14:27.516 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:27.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:27 smithi135 ceph-mon[36653]: pgmap v342: 129 pgs: 3 creating+peering, 23 unknown, 103 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:27.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:27 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-03T02:14:27.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:27 smithi135 ceph-mon[36653]: osdmap e96: 12 total, 12 up, 12 in 2024-06-03T02:14:27.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:27 smithi123 ceph-mon[36640]: pgmap v342: 129 pgs: 3 creating+peering, 23 unknown, 103 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:27.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:27 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-03T02:14:27.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:27 smithi123 ceph-mon[36640]: osdmap e96: 12 total, 12 up, 12 in 2024-06-03T02:14:28.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:28 smithi079 ceph-mon[31819]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:28 smithi135 ceph-mon[36653]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:28 smithi123 ceph-mon[36640]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.680+0000 7fa803000700 1 -- 172.21.15.79:0/648277478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7fc06b3c0 msgr2=0x7fa7fc10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.680+0000 7fa803000700 1 --2- 172.21.15.79:0/648277478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7fc06b3c0 0x7fa7fc10f310 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009f90 tx=0x7fa7f400b330 comp rx=0 tx=0).stop 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.680+0000 7fa803000700 1 -- 172.21.15.79:0/648277478 shutdown_connections 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.680+0000 7fa803000700 1 --2- 172.21.15.79:0/648277478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7fc06b3c0 0x7fa7fc10f310 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.680+0000 7fa803000700 1 --2- 172.21.15.79:0/648277478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7fc1084f0 0x7fa7fc1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.680+0000 7fa803000700 1 --2- 172.21.15.79:0/648277478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc06aa00 0x7fa7fc06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.680+0000 7fa803000700 1 -- 172.21.15.79:0/648277478 >> 172.21.15.79:0/648277478 conn(0x7fa7fc078300 msgr2=0x7fa7fc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.680+0000 7fa803000700 1 -- 172.21.15.79:0/648277478 shutdown_connections 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.680+0000 7fa803000700 1 -- 172.21.15.79:0/648277478 wait complete. 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.681+0000 7fa803000700 1 Processor -- start 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.681+0000 7fa803000700 1 -- start start 2024-06-03T02:14:29.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.681+0000 7fa803000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7fc06aa00 0x7fa7fc19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.681+0000 7fa803000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7fc06b3c0 0x7fa7fc19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.681+0000 7fa803000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1084f0 0x7fa7fc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.681+0000 7fa803000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1127a0 con 0x7fa7fc06b3c0 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.681+0000 7fa803000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc112620 con 0x7fa7fc1084f0 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.681+0000 7fa803000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc112920 con 0x7fa7fc06aa00 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7fc06aa00 0x7fa7fc19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7fc06aa00 0x7fa7fc19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:49916/0 (socket says 172.21.15.79:49916) 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fbe00700 1 -- 172.21.15.79:0/2795795503 learned_addr learned my addr 172.21.15.79:0/2795795503 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7fc06b3c0 0x7fa7fc19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa801000700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1084f0 0x7fa7fc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fbe00700 1 -- 172.21.15.79:0/2795795503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1084f0 msgr2=0x7fa7fc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fbe00700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1084f0 0x7fa7fc19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fbe00700 1 -- 172.21.15.79:0/2795795503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7fc06b3c0 msgr2=0x7fa7fc19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:29.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fbe00700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7fc06b3c0 0x7fa7fc19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fbe00700 1 -- 172.21.15.79:0/2795795503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc19fe10 con 0x7fa7fc06aa00 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa801000700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1084f0 0x7fa7fc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.682+0000 7fa7fbe00700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7fc06aa00 0x7fa7fc19ae00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00ec10 tx=0x7fa7ec00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.683+0000 7fa7f8c00700 1 -- 172.21.15.79:0/2795795503 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00cc90 con 0x7fa7fc06aa00 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.683+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc1a0100 con 0x7fa7fc06aa00 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.683+0000 7fa7f8c00700 1 -- 172.21.15.79:0/2795795503 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec00ce30 con 0x7fa7fc06aa00 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.683+0000 7fa7f8c00700 1 -- 172.21.15.79:0/2795795503 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec010690 con 0x7fa7fc06aa00 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.683+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc1a7980 con 0x7fa7fc06aa00 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.684+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc069180 con 0x7fa7fc06aa00 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.687+0000 7fa7f8c00700 1 -- 172.21.15.79:0/2795795503 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa7ec003680 con 0x7fa7fc06aa00 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.688+0000 7fa7f8c00700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7e00a2c20 0x7fa7e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.688+0000 7fa7f8c00700 1 -- 172.21.15.79:0/2795795503 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8770+0+0 (secure 0 0 0) 0x7fa7ec0cf5f0 con 0x7fa7fc06aa00 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.688+0000 7fa7fb400700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7e00a2c20 0x7fa7e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:29.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.689+0000 7fa7fb400700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7e00a2c20 0x7fa7e00a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc1033e0 tx=0x7fa7e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:29.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.690+0000 7fa7f8c00700 1 -- 172.21.15.79:0/2795795503 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa7ec01f340 con 0x7fa7fc06aa00 2024-06-03T02:14:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:29.966+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fa7fc19c160 con 0x7fa7fc06aa00 2024-06-03T02:14:30.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.026+0000 7fa7f8c00700 1 -- 172.21.15.79:0/2795795503 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fa7ec09c3c0 con 0x7fa7fc06aa00 2024-06-03T02:14:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7e00a2c20 msgr2=0x7fa7e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7e00a2c20 0x7fa7e00a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc1033e0 tx=0x7fa7e8009260 comp rx=0 tx=0).stop 2024-06-03T02:14:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7fc06aa00 msgr2=0x7fa7fc19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7fc06aa00 0x7fa7fc19ae00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00ec10 tx=0x7fa7ec00ef20 comp rx=0 tx=0).stop 2024-06-03T02:14:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 shutdown_connections 2024-06-03T02:14:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1084f0 0x7fa7fc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7fc06aa00 0x7fa7fc19ae00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7e00a2c20 0x7fa7e00a50e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 --2- 172.21.15.79:0/2795795503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7fc06b3c0 0x7fa7fc19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.031+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 >> 172.21.15.79:0/2795795503 conn(0x7fa7fc078300 msgr2=0x7fa7fc0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.032+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 shutdown_connections 2024-06-03T02:14:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:30.032+0000 7fa803000700 1 -- 172.21.15.79:0/2795795503 wait complete. 2024-06-03T02:14:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:added data pool 4 to fsmap 2024-06-03T02:14:30.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:30 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2795795503' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T02:14:30.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:30 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T02:14:30.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:30 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2795795503' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T02:14:30.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:30 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T02:14:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:30 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2795795503' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T02:14:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:30 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T02:14:30.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:14:31.301 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:31.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:31 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-03T02:14:31.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:31 smithi079 ceph-mon[31819]: osdmap e97: 12 total, 12 up, 12 in 2024-06-03T02:14:31.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:31 smithi079 ceph-mon[31819]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:14:31.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:31 smithi079 ceph-mon[31819]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:31.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:31 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-03T02:14:31.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:31 smithi135 ceph-mon[36653]: osdmap e97: 12 total, 12 up, 12 in 2024-06-03T02:14:31.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:31 smithi135 ceph-mon[36653]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:14:31.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:31 smithi135 ceph-mon[36653]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:31.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:31 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-03T02:14:31.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:31 smithi123 ceph-mon[36640]: osdmap e97: 12 total, 12 up, 12 in 2024-06-03T02:14:31.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:31 smithi123 ceph-mon[36640]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:14:31.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:31 smithi123 ceph-mon[36640]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:32.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:32 smithi079 ceph-mon[31819]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T02:14:32.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:32 smithi079 ceph-mon[31819]: Cluster is now healthy 2024-06-03T02:14:32.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:32 smithi135 ceph-mon[36653]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T02:14:32.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:32 smithi135 ceph-mon[36653]: Cluster is now healthy 2024-06-03T02:14:32.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:32 smithi123 ceph-mon[36640]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T02:14:32.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:32 smithi123 ceph-mon[36640]: Cluster is now healthy 2024-06-03T02:14:33.178 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:33 smithi079 ceph-mon[31819]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:33.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:33 smithi135 ceph-mon[36653]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:33.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.441+0000 7f1845a00700 1 -- 172.21.15.79:0/2902025058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f184010d8a0 msgr2=0x7f184010dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:33.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.441+0000 7f1845a00700 1 --2- 172.21.15.79:0/2902025058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f184010d8a0 0x7f184010dc80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1830009990 tx=0x7f1830009ca0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.441+0000 7f1845a00700 1 -- 172.21.15.79:0/2902025058 shutdown_connections 2024-06-03T02:14:33.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.441+0000 7f1845a00700 1 --2- 172.21.15.79:0/2902025058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1840109390 0x7f1840115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.441+0000 7f1845a00700 1 --2- 172.21.15.79:0/2902025058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f184010d8a0 0x7f184010dc80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.441+0000 7f1845a00700 1 --2- 172.21.15.79:0/2902025058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18401089d0 0x7f1840108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.441+0000 7f1845a00700 1 -- 172.21.15.79:0/2902025058 >> 172.21.15.79:0/2902025058 conn(0x7f1840078220 msgr2=0x7f18401072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:33.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.441+0000 7f1845a00700 1 -- 172.21.15.79:0/2902025058 shutdown_connections 2024-06-03T02:14:33.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.441+0000 7f1845a00700 1 -- 172.21.15.79:0/2902025058 wait complete. 2024-06-03T02:14:33.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.442+0000 7f1845a00700 1 Processor -- start 2024-06-03T02:14:33.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.442+0000 7f1845a00700 1 -- start start 2024-06-03T02:14:33.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1845a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18401089d0 0x7f184019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:33.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1845a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1840109390 0x7f184019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:33.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1845a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184010d8a0 0x7f18401a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:33.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1845a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1840118cc0 con 0x7f1840109390 2024-06-03T02:14:33.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1845a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1840118b40 con 0x7f184010d8a0 2024-06-03T02:14:33.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1845a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1840118e40 con 0x7f18401089d0 2024-06-03T02:14:33.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1837400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1840109390 0x7f184019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:33.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1837400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1840109390 0x7f184019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:48474/0 (socket says 172.21.15.79:48474) 2024-06-03T02:14:33.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1837400700 1 -- 172.21.15.79:0/2030936931 learned_addr learned my addr 172.21.15.79:0/2030936931 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:33.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f183fe00700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18401089d0 0x7f184019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:33.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.443+0000 7f1845000700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184010d8a0 0x7f18401a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f1837400700 1 -- 172.21.15.79:0/2030936931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18401089d0 msgr2=0x7f184019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f1837400700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18401089d0 0x7f184019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f1837400700 1 -- 172.21.15.79:0/2030936931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184010d8a0 msgr2=0x7f18401a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f1837400700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184010d8a0 0x7f18401a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f1837400700 1 -- 172.21.15.79:0/2030936931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18401a4160 con 0x7f1840109390 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f1837400700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1840109390 0x7f184019f690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f182800e9a0 tx=0x7f182800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f183d600700 1 -- 172.21.15.79:0/2030936931 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182800c9c0 con 0x7f1840109390 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18401a4450 con 0x7f1840109390 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f183d600700 1 -- 172.21.15.79:0/2030936931 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f182800cb60 con 0x7f1840109390 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f183d600700 1 -- 172.21.15.79:0/2030936931 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828018680 con 0x7f1840109390 2024-06-03T02:14:33.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.444+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18401abcd0 con 0x7f1840109390 2024-06-03T02:14:33.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.445+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1840069180 con 0x7f1840109390 2024-06-03T02:14:33.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.448+0000 7f183d600700 1 -- 172.21.15.79:0/2030936931 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f182800cd10 con 0x7f1840109390 2024-06-03T02:14:33.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.449+0000 7f183d600700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18240a2c20 0x7f18240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:33.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.449+0000 7f183d600700 1 -- 172.21.15.79:0/2030936931 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8802+0+0 (secure 0 0 0) 0x7f18280cf660 con 0x7f1840109390 2024-06-03T02:14:33.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.449+0000 7f183fe00700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18240a2c20 0x7f18240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:33.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.450+0000 7f183fe00700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18240a2c20 0x7f18240a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f18300098e0 tx=0x7f1830005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:33.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.452+0000 7f183d600700 1 -- 172.21.15.79:0/2030936931 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f182809c320 con 0x7f1840109390 2024-06-03T02:14:33.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:33 smithi123 ceph-mon[36640]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:33.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.726+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f184002dca0 con 0x7f1840109390 2024-06-03T02:14:33.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.728+0000 7f183d600700 1 -- 172.21.15.79:0/2030936931 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7f1828086090 con 0x7f1840109390 2024-06-03T02:14:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:14:33.729 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:14:29.969980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24539,"qdb_cluster":[24539]},"id":1}]} 2024-06-03T02:14:33.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.731+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18240a2c20 msgr2=0x7f18240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:33.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.731+0000 7f1845a00700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18240a2c20 0x7f18240a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f18300098e0 tx=0x7f1830005960 comp rx=0 tx=0).stop 2024-06-03T02:14:33.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.731+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1840109390 msgr2=0x7f184019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:33.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.731+0000 7f1845a00700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1840109390 0x7f184019f690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f182800e9a0 tx=0x7f182800ecb0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.732+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 shutdown_connections 2024-06-03T02:14:33.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.732+0000 7f1845a00700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184010d8a0 0x7f18401a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.732+0000 7f1845a00700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18401089d0 0x7f184019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.732+0000 7f1845a00700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18240a2c20 0x7f18240a50e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.732+0000 7f1845a00700 1 --2- 172.21.15.79:0/2030936931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1840109390 0x7f184019f690 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:33.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.732+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 >> 172.21.15.79:0/2030936931 conn(0x7f1840078220 msgr2=0x7f1840078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:33.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.732+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 shutdown_connections 2024-06-03T02:14:33.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:33.732+0000 7f1845a00700 1 -- 172.21.15.79:0/2030936931 wait complete. 2024-06-03T02:14:33.733 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 17 2024-06-03T02:14:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:34 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2030936931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:14:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:34 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2030936931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:14:34.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:34 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2030936931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:14:34.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:14:35.002 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:35.323 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:35 smithi079 ceph-mon[31819]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:35.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:35 smithi135 ceph-mon[36653]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:35 smithi123 ceph-mon[36640]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:37.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.233+0000 7f7c96c00700 1 -- 172.21.15.79:0/2767005090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c900746c0 msgr2=0x7f7c90074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.233+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2767005090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c900746c0 0x7f7c90074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009990 tx=0x7f7c80009ca0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.233+0000 7f7c96c00700 1 -- 172.21.15.79:0/2767005090 shutdown_connections 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.233+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2767005090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c90113660 0x7f7c90115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.233+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2767005090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c900746c0 0x7f7c90074aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.233+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2767005090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c90074fe0 0x7f7c90113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.233+0000 7f7c96c00700 1 -- 172.21.15.79:0/2767005090 >> 172.21.15.79:0/2767005090 conn(0x7f7c900795f0 msgr2=0x7f7c90079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.233+0000 7f7c96c00700 1 -- 172.21.15.79:0/2767005090 shutdown_connections 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.234+0000 7f7c96c00700 1 -- 172.21.15.79:0/2767005090 wait complete. 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c96c00700 1 Processor -- start 2024-06-03T02:14:37.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c96c00700 1 -- start start 2024-06-03T02:14:37.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c96c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c900746c0 0x7f7c9019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:37.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c96c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c90074fe0 0x7f7c9019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:37.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c96c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c90113660 0x7f7c901a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:37.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c96c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90118df0 con 0x7f7c90074fe0 2024-06-03T02:14:37.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c96c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90118c70 con 0x7f7c90113660 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c96c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90118f70 con 0x7f7c900746c0 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c8fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c900746c0 0x7f7c9019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c8fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c900746c0 0x7f7c9019f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36832/0 (socket says 172.21.15.79:36832) 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c8fe00700 1 -- 172.21.15.79:0/2675698784 learned_addr learned my addr 172.21.15.79:0/2675698784 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c87400700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c90074fe0 0x7f7c9019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.235+0000 7f7c94c00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c90113660 0x7f7c901a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.236+0000 7f7c8fe00700 1 -- 172.21.15.79:0/2675698784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c90113660 msgr2=0x7f7c901a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.236+0000 7f7c8fe00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c90113660 0x7f7c901a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.236+0000 7f7c8fe00700 1 -- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c90074fe0 msgr2=0x7f7c9019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.236+0000 7f7c8fe00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c90074fe0 0x7f7c9019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.236+0000 7f7c8fe00700 1 -- 172.21.15.79:0/2675698784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c901a4290 con 0x7f7c900746c0 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.236+0000 7f7c87400700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c90074fe0 0x7f7c9019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:14:37.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.236+0000 7f7c8fe00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c900746c0 0x7f7c9019f280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005910 tx=0x7f7c8000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:37.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.237+0000 7f7c8d600700 1 -- 172.21.15.79:0/2675698784 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80027070 con 0x7f7c900746c0 2024-06-03T02:14:37.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.237+0000 7f7c8d600700 1 -- 172.21.15.79:0/2675698784 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c80004650 con 0x7f7c900746c0 2024-06-03T02:14:37.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.237+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c901a4520 con 0x7f7c900746c0 2024-06-03T02:14:37.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.237+0000 7f7c8d600700 1 -- 172.21.15.79:0/2675698784 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8000f460 con 0x7f7c900746c0 2024-06-03T02:14:37.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.237+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c901abe00 con 0x7f7c900746c0 2024-06-03T02:14:37.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.238+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c90069180 con 0x7f7c900746c0 2024-06-03T02:14:37.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.240+0000 7f7c8d600700 1 -- 172.21.15.79:0/2675698784 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7c8002a070 con 0x7f7c900746c0 2024-06-03T02:14:37.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.241+0000 7f7c8d600700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c740a2ca0 0x7f7c740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:37.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.241+0000 7f7c8d600700 1 -- 172.21.15.79:0/2675698784 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8802+0+0 (secure 0 0 0) 0x7f7c800d0120 con 0x7f7c900746c0 2024-06-03T02:14:37.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.242+0000 7f7c87400700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c740a2ca0 0x7f7c740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:37.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.242+0000 7f7c87400700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c740a2ca0 0x7f7c740a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c9010ffd0 tx=0x7f7c7800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:37.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.246+0000 7f7c8d600700 1 -- 172.21.15.79:0/2675698784 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c8005e850 con 0x7f7c900746c0 2024-06-03T02:14:37.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.469+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c901abfd0 con 0x7f7c900746c0 2024-06-03T02:14:37.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:37 smithi079 ceph-mon[31819]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:37.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.470+0000 7f7c8d600700 1 -- 172.21.15.79:0/2675698784 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24134 (secure 0 0 0) 0x7f7c80013070 con 0x7f7c900746c0 2024-06-03T02:14:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:14:37.471 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":97,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:14:29.969754+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:14:37.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:37 smithi123 ceph-mon[36640]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:37.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c740a2ca0 msgr2=0x7f7c740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c740a2ca0 0x7f7c740a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c9010ffd0 tx=0x7f7c7800b040 comp rx=0 tx=0).stop 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c900746c0 msgr2=0x7f7c9019f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c900746c0 0x7f7c9019f280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005910 tx=0x7f7c8000bab0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 shutdown_connections 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c90113660 0x7f7c901a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c900746c0 0x7f7c9019f280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c740a2ca0 0x7f7c740a5160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 --2- 172.21.15.79:0/2675698784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c90074fe0 0x7f7c9019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 >> 172.21.15.79:0/2675698784 conn(0x7f7c900795f0 msgr2=0x7f7c90112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 shutdown_connections 2024-06-03T02:14:37.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:37.474+0000 7f7c96c00700 1 -- 172.21.15.79:0/2675698784 wait complete. 2024-06-03T02:14:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:37 smithi135 ceph-mon[36653]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:38.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:14:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:38 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2675698784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:14:38.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:38 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2675698784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:14:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:38 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2675698784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:14:38.756 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:39 smithi079 ceph-mon[31819]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:39 smithi135 ceph-mon[36653]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:39.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:39 smithi123 ceph-mon[36640]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:40.315 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:40 smithi079 ceph-mon[31819]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:40.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:40 smithi135 ceph-mon[36653]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:40.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:40 smithi123 ceph-mon[36640]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:40.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.855+0000 7f2260800700 1 -- 172.21.15.79:0/590737232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2258108ac0 msgr2=0x7f2258108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:40.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.855+0000 7f2260800700 1 --2- 172.21.15.79:0/590737232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2258108ac0 0x7f2258108f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f22480098f0 tx=0x7f2248009c00 comp rx=0 tx=0).stop 2024-06-03T02:14:40.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.855+0000 7f2260800700 1 -- 172.21.15.79:0/590737232 shutdown_connections 2024-06-03T02:14:40.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.855+0000 7f2260800700 1 --2- 172.21.15.79:0/590737232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2258109480 0x7f2258115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:40.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.855+0000 7f2260800700 1 --2- 172.21.15.79:0/590737232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f225810eac0 0x7f225810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:40.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.855+0000 7f2260800700 1 --2- 172.21.15.79:0/590737232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2258108ac0 0x7f2258108f40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:40.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.855+0000 7f2260800700 1 -- 172.21.15.79:0/590737232 >> 172.21.15.79:0/590737232 conn(0x7f2258075c60 msgr2=0x7f22581072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:40.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.855+0000 7f2260800700 1 -- 172.21.15.79:0/590737232 shutdown_connections 2024-06-03T02:14:40.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.855+0000 7f2260800700 1 -- 172.21.15.79:0/590737232 wait complete. 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.856+0000 7f2260800700 1 Processor -- start 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.856+0000 7f2260800700 1 -- start start 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f2260800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2258108ac0 0x7f225819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f2260800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2258109480 0x7f225819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f2260800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225810eac0 0x7f22581a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f2260800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2258118e50 con 0x7f2258108ac0 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f2260800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2258118cd0 con 0x7f225810eac0 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f2260800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2258118fd0 con 0x7f2258109480 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2258108ac0 0x7f225819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:40.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2258109480 0x7f225819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225e800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225810eac0 0x7f22581a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2258108ac0 0x7f225819f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:55268/0 (socket says 172.21.15.79:55268) 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2258109480 0x7f225819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36858/0 (socket says 172.21.15.79:36858) 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225de00700 1 -- 172.21.15.79:0/3006006872 learned_addr learned my addr 172.21.15.79:0/3006006872 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225e800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225810eac0 0x7f22581a3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58142/0 (socket says 172.21.15.79:58142) 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225e800700 1 -- 172.21.15.79:0/3006006872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2258109480 msgr2=0x7f225819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225e800700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2258109480 0x7f225819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225e800700 1 -- 172.21.15.79:0/3006006872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2258108ac0 msgr2=0x7f225819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225e800700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2258108ac0 0x7f225819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.857+0000 7f225e800700 1 -- 172.21.15.79:0/3006006872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22581a42d0 con 0x7f225810eac0 2024-06-03T02:14:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.858+0000 7f225e800700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225810eac0 0x7f22581a3b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f225400efc0 tx=0x7f225400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:40.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.858+0000 7f2252a00700 1 -- 172.21.15.79:0/3006006872 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540092e0 con 0x7f225810eac0 2024-06-03T02:14:40.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.858+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22581a4560 con 0x7f225810eac0 2024-06-03T02:14:40.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.858+0000 7f2252a00700 1 -- 172.21.15.79:0/3006006872 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2254004bb0 con 0x7f225810eac0 2024-06-03T02:14:40.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.858+0000 7f2252a00700 1 -- 172.21.15.79:0/3006006872 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254004280 con 0x7f225810eac0 2024-06-03T02:14:40.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.858+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22581abe40 con 0x7f225810eac0 2024-06-03T02:14:40.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.859+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2258069180 con 0x7f225810eac0 2024-06-03T02:14:40.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.861+0000 7f2252a00700 1 -- 172.21.15.79:0/3006006872 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f225401b030 con 0x7f225810eac0 2024-06-03T02:14:40.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.863+0000 7f2252a00700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22400a2ca0 0x7f22400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:40.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.863+0000 7f2252a00700 1 -- 172.21.15.79:0/3006006872 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8802+0+0 (secure 0 0 0) 0x7f22540ced10 con 0x7f225810eac0 2024-06-03T02:14:40.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.863+0000 7f225de00700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22400a2ca0 0x7f22400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:40.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.864+0000 7f225de00700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22400a2ca0 0x7f22400a5160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f224c005950 tx=0x7f224c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:40.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:40.867+0000 7f2252a00700 1 -- 172.21.15.79:0/3006006872 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f22540d4020 con 0x7f225810eac0 2024-06-03T02:14:41.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.090+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f225802dca0 con 0x7f225810eac0 2024-06-03T02:14:41.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.092+0000 7f2252a00700 1 -- 172.21.15.79:0/3006006872 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24134 (secure 0 0 0) 0x7f2254022020 con 0x7f225810eac0 2024-06-03T02:14:41.093 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:14:41.093 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":97,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:14:29.969754+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:14:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.095+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22400a2ca0 msgr2=0x7f22400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.095+0000 7f2260800700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22400a2ca0 0x7f22400a5160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f224c005950 tx=0x7f224c009310 comp rx=0 tx=0).stop 2024-06-03T02:14:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.095+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225810eac0 msgr2=0x7f22581a3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.095+0000 7f2260800700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225810eac0 0x7f22581a3b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f225400efc0 tx=0x7f225400c3a0 comp rx=0 tx=0).stop 2024-06-03T02:14:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.096+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 shutdown_connections 2024-06-03T02:14:41.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.096+0000 7f2260800700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225810eac0 0x7f22581a3b90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:41.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.096+0000 7f2260800700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22400a2ca0 0x7f22400a5160 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:41.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.096+0000 7f2260800700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2258108ac0 0x7f225819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:41.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.096+0000 7f2260800700 1 --2- 172.21.15.79:0/3006006872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2258109480 0x7f225819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:41.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.096+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 >> 172.21.15.79:0/3006006872 conn(0x7f2258075c60 msgr2=0x7f22580761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:41.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.096+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 shutdown_connections 2024-06-03T02:14:41.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:41.096+0000 7f2260800700 1 -- 172.21.15.79:0/3006006872 wait complete. 2024-06-03T02:14:41.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:41 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3006006872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:14:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:41 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3006006872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:14:41.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:41 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3006006872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:14:41.943 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:14:42.393 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:42 smithi079 ceph-mon[31819]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:42 smithi135 ceph-mon[36653]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:42 smithi123 ceph-mon[36640]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:44.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.546+0000 7f5a59a00700 1 -- 172.21.15.79:0/689092913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a5406aa00 msgr2=0x7f5a5406ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:44.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.546+0000 7f5a59a00700 1 --2- 172.21.15.79:0/689092913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a5406aa00 0x7f5a5406ae60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009990 tx=0x7f5a40009ca0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.547+0000 7f5a59a00700 1 -- 172.21.15.79:0/689092913 shutdown_connections 2024-06-03T02:14:44.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.547+0000 7f5a59a00700 1 --2- 172.21.15.79:0/689092913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a5406b3a0 0x7f5a5410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.547+0000 7f5a59a00700 1 --2- 172.21.15.79:0/689092913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a541084d0 0x7f5a541088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.547+0000 7f5a59a00700 1 --2- 172.21.15.79:0/689092913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a5406aa00 0x7f5a5406ae60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.547+0000 7f5a59a00700 1 -- 172.21.15.79:0/689092913 >> 172.21.15.79:0/689092913 conn(0x7f5a54078330 msgr2=0x7f5a54078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:44.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.547+0000 7f5a59a00700 1 -- 172.21.15.79:0/689092913 shutdown_connections 2024-06-03T02:14:44.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.547+0000 7f5a59a00700 1 -- 172.21.15.79:0/689092913 wait complete. 2024-06-03T02:14:44.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.548+0000 7f5a59a00700 1 Processor -- start 2024-06-03T02:14:44.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.548+0000 7f5a59a00700 1 -- start start 2024-06-03T02:14:44.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.548+0000 7f5a59a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a5406aa00 0x7f5a541a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:44.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.548+0000 7f5a59a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a5406b3a0 0x7f5a541a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.548+0000 7f5a59a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a541084d0 0x7f5a5419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.548+0000 7f5a59a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54112780 con 0x7f5a5406aa00 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.548+0000 7f5a59a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54112600 con 0x7f5a541084d0 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.548+0000 7f5a59a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54112900 con 0x7f5a5406b3a0 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a5406aa00 0x7f5a541a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a5406b3a0 0x7f5a541a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a5406aa00 0x7f5a541a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:55282/0 (socket says 172.21.15.79:55282) 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a5406b3a0 0x7f5a541a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36866/0 (socket says 172.21.15.79:36866) 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52a00700 1 -- 172.21.15.79:0/1657110902 learned_addr learned my addr 172.21.15.79:0/1657110902 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:44.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a53400700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a541084d0 0x7f5a5419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:44.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52000700 1 -- 172.21.15.79:0/1657110902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a541084d0 msgr2=0x7f5a5419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:44.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52000700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a541084d0 0x7f5a5419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52000700 1 -- 172.21.15.79:0/1657110902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a5406aa00 msgr2=0x7f5a541a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:44.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52000700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a5406aa00 0x7f5a541a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.549+0000 7f5a52000700 1 -- 172.21.15.79:0/1657110902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5419b200 con 0x7f5a5406b3a0 2024-06-03T02:14:44.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.550+0000 7f5a52000700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a5406b3a0 0x7f5a541a0e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000b820 tx=0x7f5a4000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:44.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.550+0000 7f5a4b400700 1 -- 172.21.15.79:0/1657110902 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40003890 con 0x7f5a5406b3a0 2024-06-03T02:14:44.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.550+0000 7f5a4b400700 1 -- 172.21.15.79:0/1657110902 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a40004830 con 0x7f5a5406b3a0 2024-06-03T02:14:44.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.550+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5419b490 con 0x7f5a5406b3a0 2024-06-03T02:14:44.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.550+0000 7f5a4b400700 1 -- 172.21.15.79:0/1657110902 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40005510 con 0x7f5a5406b3a0 2024-06-03T02:14:44.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.550+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a541a77b0 con 0x7f5a5406b3a0 2024-06-03T02:14:44.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.551+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a54103b80 con 0x7f5a5406b3a0 2024-06-03T02:14:44.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.553+0000 7f5a4b400700 1 -- 172.21.15.79:0/1657110902 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5a40029060 con 0x7f5a5406b3a0 2024-06-03T02:14:44.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.555+0000 7f5a4b400700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a380a2c20 0x7f5a380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:44.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.555+0000 7f5a4b400700 1 -- 172.21.15.79:0/1657110902 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8802+0+0 (secure 0 0 0) 0x7f5a400d01a0 con 0x7f5a5406b3a0 2024-06-03T02:14:44.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.555+0000 7f5a52a00700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a380a2c20 0x7f5a380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:44.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.556+0000 7f5a52a00700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a380a2c20 0x7f5a380a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a5419bbe0 tx=0x7f5a44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:44.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.557+0000 7f5a4b400700 1 -- 172.21.15.79:0/1657110902 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a4005d850 con 0x7f5a5406b3a0 2024-06-03T02:14:44.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.828+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a54069180 con 0x7f5a5406b3a0 2024-06-03T02:14:44.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.830+0000 7f5a4b400700 1 -- 172.21.15.79:0/1657110902 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7f5a40013070 con 0x7f5a5406b3a0 2024-06-03T02:14:44.829 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:14:44.829 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:14:29.969980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24539,"qdb_cluster":[24539]},"id":1}]} 2024-06-03T02:14:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.833+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a380a2c20 msgr2=0x7f5a380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.833+0000 7f5a59a00700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a380a2c20 0x7f5a380a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a5419bbe0 tx=0x7f5a44009260 comp rx=0 tx=0).stop 2024-06-03T02:14:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.833+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a5406b3a0 msgr2=0x7f5a541a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.833+0000 7f5a59a00700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a5406b3a0 0x7f5a541a0e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000b820 tx=0x7f5a4000bab0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.834+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 shutdown_connections 2024-06-03T02:14:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.834+0000 7f5a59a00700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a541084d0 0x7f5a5419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.834+0000 7f5a59a00700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a380a2c20 0x7f5a380a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.834+0000 7f5a59a00700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a5406aa00 0x7f5a541a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.834+0000 7f5a59a00700 1 --2- 172.21.15.79:0/1657110902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a5406b3a0 0x7f5a541a0e20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:44.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.834+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 >> 172.21.15.79:0/1657110902 conn(0x7f5a54078330 msgr2=0x7f5a5410d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:44.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.834+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 shutdown_connections 2024-06-03T02:14:44.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:44.834+0000 7f5a59a00700 1 -- 172.21.15.79:0/1657110902 wait complete. 2024-06-03T02:14:44.834 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 17 2024-06-03T02:14:45.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:45 smithi123 ceph-mon[36640]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:45.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:45 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1657110902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:14:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:45 smithi079 ceph-mon[31819]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:45.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:45 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1657110902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:14:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:45 smithi135 ceph-mon[36653]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:45 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1657110902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:14:45.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:14:46.135 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:47.482 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:47 smithi079 ceph-mon[31819]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:47.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:47 smithi135 ceph-mon[36653]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:47.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:47 smithi123 ceph-mon[36640]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:48.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.315+0000 7ff5ab000700 1 -- 172.21.15.79:0/1019932796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5a410a650 msgr2=0x7ff5a410aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:48.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.315+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1019932796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5a410a650 0x7ff5a410aa30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff598009990 tx=0x7ff598009ca0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.315+0000 7ff5ab000700 1 -- 172.21.15.79:0/1019932796 shutdown_connections 2024-06-03T02:14:48.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.315+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1019932796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a4104f80 0x7ff5a4111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.315+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1019932796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5a410a650 0x7ff5a410aa30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.315+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1019932796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a41045c0 0x7ff5a4104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.315+0000 7ff5ab000700 1 -- 172.21.15.79:0/1019932796 >> 172.21.15.79:0/1019932796 conn(0x7ff5a41000b0 msgr2=0x7ff5a41024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:48.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.315+0000 7ff5ab000700 1 -- 172.21.15.79:0/1019932796 shutdown_connections 2024-06-03T02:14:48.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.315+0000 7ff5ab000700 1 -- 172.21.15.79:0/1019932796 wait complete. 2024-06-03T02:14:48.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.316+0000 7ff5ab000700 1 Processor -- start 2024-06-03T02:14:48.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.316+0000 7ff5ab000700 1 -- start start 2024-06-03T02:14:48.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.316+0000 7ff5ab000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a41045c0 0x7ff5a41a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.316+0000 7ff5ab000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a4104f80 0x7ff5a41a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.316+0000 7ff5ab000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5a410a650 0x7ff5a419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.316+0000 7ff5ab000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41148f0 con 0x7ff5a410a650 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.316+0000 7ff5ab000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4114770 con 0x7ff5a41045c0 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.316+0000 7ff5ab000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4114a70 con 0x7ff5a4104f80 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a41045c0 0x7ff5a41a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5aa600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5a410a650 0x7ff5a419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a41045c0 0x7ff5a41a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47858/0 (socket says 172.21.15.79:47858) 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9c00700 1 -- 172.21.15.79:0/1877729933 learned_addr learned my addr 172.21.15.79:0/1877729933 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9200700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a4104f80 0x7ff5a41a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9c00700 1 -- 172.21.15.79:0/1877729933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a4104f80 msgr2=0x7ff5a41a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:48.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9c00700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a4104f80 0x7ff5a41a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9c00700 1 -- 172.21.15.79:0/1877729933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5a410a650 msgr2=0x7ff5a419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:48.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9c00700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5a410a650 0x7ff5a419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9c00700 1 -- 172.21.15.79:0/1877729933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a419b3f0 con 0x7ff5a41045c0 2024-06-03T02:14:48.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.317+0000 7ff5a9c00700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a41045c0 0x7ff5a41a0a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff598000c00 tx=0x7ff59800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:48.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.318+0000 7ff5a2000700 1 -- 172.21.15.79:0/1877729933 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598027070 con 0x7ff5a41045c0 2024-06-03T02:14:48.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.318+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a419b680 con 0x7ff5a41045c0 2024-06-03T02:14:48.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.318+0000 7ff5a2000700 1 -- 172.21.15.79:0/1877729933 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff598003d70 con 0x7ff5a41045c0 2024-06-03T02:14:48.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.318+0000 7ff5a2000700 1 -- 172.21.15.79:0/1877729933 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59800f460 con 0x7ff5a41045c0 2024-06-03T02:14:48.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.318+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a41a79c0 con 0x7ff5a41045c0 2024-06-03T02:14:48.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.319+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a410cb50 con 0x7ff5a41045c0 2024-06-03T02:14:48.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.321+0000 7ff5a2000700 1 -- 172.21.15.79:0/1877729933 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff59802a070 con 0x7ff5a41045c0 2024-06-03T02:14:48.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.322+0000 7ff5a2000700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff58c0a2c20 0x7ff58c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:48.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.322+0000 7ff5a2000700 1 -- 172.21.15.79:0/1877729933 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8802+0+0 (secure 0 0 0) 0x7ff5980d0070 con 0x7ff5a41045c0 2024-06-03T02:14:48.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.323+0000 7ff5a9200700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff58c0a2c20 0x7ff58c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:48.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.323+0000 7ff5a9200700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff58c0a2c20 0x7ff58c0a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff5a419c260 tx=0x7ff590009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:48.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.327+0000 7ff5a2000700 1 -- 172.21.15.79:0/1877729933 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff59805e850 con 0x7ff5a41045c0 2024-06-03T02:14:48.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.550+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5a4069180 con 0x7ff5a41045c0 2024-06-03T02:14:48.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.552+0000 7ff5a2000700 1 -- 172.21.15.79:0/1877729933 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24134 (secure 0 0 0) 0x7ff598013070 con 0x7ff5a41045c0 2024-06-03T02:14:48.553 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:14:48.553 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":97,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:14:29.969754+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:14:48.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.555+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff58c0a2c20 msgr2=0x7ff58c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:48.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.555+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff58c0a2c20 0x7ff58c0a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff5a419c260 tx=0x7ff590009260 comp rx=0 tx=0).stop 2024-06-03T02:14:48.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.555+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a41045c0 msgr2=0x7ff5a41a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:48.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.555+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a41045c0 0x7ff5a41a0a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff598000c00 tx=0x7ff59800bab0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.556+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 shutdown_connections 2024-06-03T02:14:48.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.556+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff58c0a2c20 0x7ff58c0a50e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.556+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5a410a650 0x7ff5a419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.556+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a41045c0 0x7ff5a41a0a40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.556+0000 7ff5ab000700 1 --2- 172.21.15.79:0/1877729933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a4104f80 0x7ff5a41a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:48.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.556+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 >> 172.21.15.79:0/1877729933 conn(0x7ff5a41000b0 msgr2=0x7ff5a4101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:48.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.556+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 shutdown_connections 2024-06-03T02:14:48.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:48.556+0000 7ff5ab000700 1 -- 172.21.15.79:0/1877729933 wait complete. 2024-06-03T02:14:49.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-03T02:14:12.550646+0000', 'modified': '2024-06-03T02:14:29.969980+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.123:6839/1178082108', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 1178082108}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 1178082108}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24539, 'qdb_cluster': [24539]} 2024-06-03T02:14:49.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-03T02:14:49.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-03T02:14:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:49 smithi079 ceph-mon[31819]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:49 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1877729933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:14:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:49 smithi135 ceph-mon[36653]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:49 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1877729933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:14:49.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:49 smithi123 ceph-mon[36640]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:49.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:49 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1877729933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:14:49.791 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:51.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:51 smithi079 ceph-mon[31819]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:51 smithi135 ceph-mon[36653]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:51.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:51 smithi123 ceph-mon[36640]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:51.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.903+0000 7f6cabc00700 1 -- 172.21.15.79:0/3117599887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca4102520 msgr2=0x7f6ca41029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:51.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.903+0000 7f6cabc00700 1 --2- 172.21.15.79:0/3117599887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca4102520 0x7f6ca41029a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009990 tx=0x7f6c90009ca0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.903+0000 7f6cabc00700 1 -- 172.21.15.79:0/3117599887 shutdown_connections 2024-06-03T02:14:51.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.903+0000 7f6cabc00700 1 --2- 172.21.15.79:0/3117599887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ca4102ee0 0x7f6ca410f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.903+0000 7f6cabc00700 1 --2- 172.21.15.79:0/3117599887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ca41085c0 0x7f6ca41089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.903+0000 7f6cabc00700 1 --2- 172.21.15.79:0/3117599887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca4102520 0x7f6ca41029a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.903+0000 7f6cabc00700 1 -- 172.21.15.79:0/3117599887 >> 172.21.15.79:0/3117599887 conn(0x7f6ca4079680 msgr2=0x7f6ca4079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:51.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.903+0000 7f6cabc00700 1 -- 172.21.15.79:0/3117599887 shutdown_connections 2024-06-03T02:14:51.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.903+0000 7f6cabc00700 1 -- 172.21.15.79:0/3117599887 wait complete. 2024-06-03T02:14:51.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.904+0000 7f6cabc00700 1 Processor -- start 2024-06-03T02:14:51.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.904+0000 7f6cabc00700 1 -- start start 2024-06-03T02:14:51.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.904+0000 7f6cabc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ca4102520 0x7f6ca419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.904+0000 7f6cabc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ca4102ee0 0x7f6ca419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.904+0000 7f6cabc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca41085c0 0x7f6ca4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.904+0000 7f6cabc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4112850 con 0x7f6ca4102520 2024-06-03T02:14:51.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.904+0000 7f6cabc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41126d0 con 0x7f6ca4102ee0 2024-06-03T02:14:51.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.904+0000 7f6cabc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41129d0 con 0x7f6ca41085c0 2024-06-03T02:14:51.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca9c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca41085c0 0x7f6ca4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ca4102ee0 0x7f6ca419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ca4102520 0x7f6ca419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ca4102ee0 0x7f6ca419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47882/0 (socket says 172.21.15.79:47882) 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca9c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca41085c0 0x7f6ca4198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:39622/0 (socket says 172.21.15.79:39622) 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca3e00700 1 -- 172.21.15.79:0/1246879552 learned_addr learned my addr 172.21.15.79:0/1246879552 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ca4102520 0x7f6ca419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37690/0 (socket says 172.21.15.79:37690) 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca3e00700 1 -- 172.21.15.79:0/1246879552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca41085c0 msgr2=0x7f6ca4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca41085c0 0x7f6ca4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca3e00700 1 -- 172.21.15.79:0/1246879552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ca4102520 msgr2=0x7f6ca419e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ca4102520 0x7f6ca419e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca3e00700 1 -- 172.21.15.79:0/1246879552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca4199270 con 0x7f6ca4102ee0 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.905+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ca4102ee0 0x7f6ca419ee40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c90005910 tx=0x7f6c900047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.906+0000 7f6ca1600700 1 -- 172.21.15.79:0/1246879552 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90027070 con 0x7f6ca4102ee0 2024-06-03T02:14:51.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.906+0000 7f6cabc00700 1 -- 172.21.15.79:0/1246879552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca4199500 con 0x7f6ca4102ee0 2024-06-03T02:14:51.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.906+0000 7f6ca1600700 1 -- 172.21.15.79:0/1246879552 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c9000fde0 con 0x7f6ca4102ee0 2024-06-03T02:14:51.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.906+0000 7f6ca1600700 1 -- 172.21.15.79:0/1246879552 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c900173f0 con 0x7f6ca4102ee0 2024-06-03T02:14:51.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.906+0000 7f6cabc00700 1 -- 172.21.15.79:0/1246879552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca40746c0 con 0x7f6ca4102ee0 2024-06-03T02:14:51.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.909+0000 7f6ca1600700 1 -- 172.21.15.79:0/1246879552 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6c9002a070 con 0x7f6ca4102ee0 2024-06-03T02:14:51.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.910+0000 7f6cb6664b80 1 Processor -- start 2024-06-03T02:14:51.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.910+0000 7f6ca1600700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6c8c0a2ca0 0x7f6c8c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.910+0000 7f6cb6664b80 1 -- start start 2024-06-03T02:14:51.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.910+0000 7f6ca1600700 1 -- 172.21.15.79:0/1246879552 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8802+0+0 (secure 0 0 0) 0x7f6c900cfef0 con 0x7f6ca4102ee0 2024-06-03T02:14:51.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6ca9c00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6c8c0a2ca0 0x7f6c8c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d8460 0x564b885d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d21f0 0x564b885d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564b884e09c0 con 0x564b885d7b40 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564b88523ab0 con 0x564b885d8460 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564b884de8a0 con 0x564b885d21f0 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885d7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37698/0 (socket says 172.21.15.79:37698) 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6ca9200700 1 -- 172.21.15.79:0/652678223 learned_addr learned my addr 172.21.15.79:0/652678223 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6ca9c00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6c8c0a2ca0 0x7f6c8c0a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00f590 tx=0x7f6c9c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.911+0000 7f6ca9c00700 1 --2- 172.21.15.79:0/652678223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d21f0 0x564b885d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.912+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/652678223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d8460 0x564b885d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.912+0000 7f6ca9200700 1 -- 172.21.15.79:0/652678223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d21f0 msgr2=0x564b885d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:14:51.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.912+0000 7f6ca9200700 1 --2- 172.21.15.79:0/652678223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d21f0 0x564b885d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.912+0000 7f6ca9200700 1 -- 172.21.15.79:0/652678223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d8460 msgr2=0x564b885d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:14:51.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.912+0000 7f6ca9200700 1 --2- 172.21.15.79:0/652678223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d8460 0x564b885d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.912+0000 7f6ca9200700 1 -- 172.21.15.79:0/652678223 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b884cbf40 con 0x564b885d7b40 2024-06-03T02:14:51.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.913+0000 7f6ca9200700 1 --2- 172.21.15.79:0/652678223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885d7f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c98010dc0 tx=0x7f6c98009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a809d16f9919114 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:51.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.913+0000 7f6c96000700 1 -- 172.21.15.79:0/652678223 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9801e030 con 0x564b885d7b40 2024-06-03T02:14:51.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.913+0000 7f6c96000700 1 -- 172.21.15.79:0/652678223 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c98009d90 con 0x564b885d7b40 2024-06-03T02:14:51.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.913+0000 7f6c96000700 1 -- 172.21.15.79:0/652678223 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9801a400 con 0x564b885d7b40 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.914+0000 7f6cb6664b80 1 -- 172.21.15.79:0/652678223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 msgr2=0x564b885d7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.914+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/652678223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885d7f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c98010dc0 tx=0x7f6c98009260 comp rx=0 tx=0).stop 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.914+0000 7f6cb6664b80 1 -- 172.21.15.79:0/652678223 shutdown_connections 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.914+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/652678223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d21f0 0x564b885d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.914+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/652678223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885d7f20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.914+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/652678223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d8460 0x564b885d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.914+0000 7f6cb6664b80 1 -- 172.21.15.79:0/652678223 >> 172.21.15.79:0/652678223 conn(0x564b884fe120 msgr2=0x564b885ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.914+0000 7f6cb6664b80 1 -- 172.21.15.79:0/652678223 shutdown_connections 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.914+0000 7f6cb6664b80 1 -- 172.21.15.79:0/652678223 wait complete. 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.915+0000 7f6cb6664b80 1 -- start start 2024-06-03T02:14:51.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.915+0000 7f6cb6664b80 1 Processor -- start 2024-06-03T02:14:51.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885db4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564b884bff10 con 0x564b885d7b40 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564b8851fa70 con 0x564b885d21f0 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564b885dee40 con 0x564b885d8460 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6cb6664b80 10 client.0 _ll_register_callbacks cb 0x564b884b9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885db4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885de1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47910/0 (socket says 172.21.15.79:47910) 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca9200700 1 -- 172.21.15.79:0/1548892448 learned_addr learned my addr 172.21.15.79:0/1548892448 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca9c00700 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca9200700 1 -- 172.21.15.79:0/1548892448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 msgr2=0x564b885db9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca9200700 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885db9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca9200700 1 -- 172.21.15.79:0/1548892448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 msgr2=0x564b885db4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca9200700 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885db4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.916+0000 7f6ca9200700 1 -- 172.21.15.79:0/1548892448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b88472880 con 0x564b885d21f0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.917+0000 7f6ca9200700 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885de1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c98004980 tx=0x7f6c980049b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a9009bdc2abe83 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.917+0000 7f6c8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.917+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1548892448 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98024640 con 0x564b885d21f0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.917+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1548892448 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8802+0+0 (secure 0 0 0) 0x7f6c980247e0 con 0x564b885d21f0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.917+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1548892448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c98009d90 con 0x564b885d21f0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.917+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1548892448 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f6c98005ce0 con 0x564b885d21f0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.917+0000 7f6cb6664b80 10 client.24635 fetch_fsmap learned FSMap version 17 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.917+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1548892448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x564b885e95a0 con 0x564b885d21f0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.918+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1548892448 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c98009d90 con 0x564b885d21f0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.918+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1548892448 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f6c9802b030 con 0x564b885d21f0 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.918+0000 7f6cb6664b80 10 client.24635 fetch_fsmap finished waiting for FSMap version 17 2024-06-03T02:14:51.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.918+0000 7f6cb6664b80 10 client.24635 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T02:14:51.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.918+0000 7f6cb6664b80 20 client.24635 populate_metadata read hostname 'smithi079' 2024-06-03T02:14:51.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.918+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564b885eb8e0 0x564b885ecd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.919+0000 7f6cb6664b80 4 client.24635 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:14:51.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.919+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1548892448 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b88530fb0 con 0x564b885eb8e0 2024-06-03T02:14:51.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.919+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564b885eb8e0 0x564b885ecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.919+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564b885eb8e0 0x564b885ecd50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade03bd901741745 server_cookie=2aa3ccb3a519b56c in_seq=0 out_seq=0 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.919+0000 7f6c8aa00700 10 client.24635 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.920+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1548892448 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564b88530fb0 con 0x564b885eb8e0 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.920+0000 7f6c8aa00700 10 client.24635 handle_command_reply: tid=0 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.920+0000 7f6cb6664b80 1 client.24635 shutdown 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1548892448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 msgr2=0x564b885de1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885de1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c98004980 tx=0x7f6c980049b0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1548892448 shutdown_connections 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885db9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885de1b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564b885eb8e0 0x564b885ecd50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1548892448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885db4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1548892448 >> 172.21.15.79:0/1548892448 conn(0x564b884fe120 msgr2=0x564b885c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:51.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6c8aa00700 0 client.24635 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:14:51.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1548892448 shutdown_connections 2024-06-03T02:14:51.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1548892448 wait complete. 2024-06-03T02:14:51.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.921+0000 7f6cb6664b80 20 client.24635 trim_cache size 0 max 16384 2024-06-03T02:14:51.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.944+0000 7f6cb6664b80 1 Processor -- start 2024-06-03T02:14:51.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.944+0000 7f6cb6664b80 1 -- start start 2024-06-03T02:14:51.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.944+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.944+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.944+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.944+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564b885eb090 con 0x564b885d7b40 2024-06-03T02:14:51.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.944+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564b885eb760 con 0x564b885d21f0 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.944+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564b885ebe30 con 0x564b885d8460 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37718/0 (socket says 172.21.15.79:37718) 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca3e00700 1 -- 172.21.15.79:0/3662067591 learned_addr learned my addr 172.21.15.79:0/3662067591 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca9c00700 1 --2- 172.21.15.79:0/3662067591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca9200700 1 --2- 172.21.15.79:0/3662067591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca3e00700 1 -- 172.21.15.79:0/3662067591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 msgr2=0x564b885ea9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/3662067591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885ea9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca3e00700 1 -- 172.21.15.79:0/3662067591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 msgr2=0x564b885d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/3662067591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.945+0000 7f6ca3e00700 1 -- 172.21.15.79:0/3662067591 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b885d7640 con 0x564b885d7b40 2024-06-03T02:14:51.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.946+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/3662067591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885cd150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6c900cd750 tx=0x7f6c90087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83695370e227025b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.946+0000 7f6c95600700 1 -- 172.21.15.79:0/3662067591 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c900a08f0 con 0x564b885d7b40 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.946+0000 7f6c95600700 1 -- 172.21.15.79:0/3662067591 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c900a5120 con 0x564b885d7b40 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.946+0000 7f6c95600700 1 -- 172.21.15.79:0/3662067591 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9009c7a0 con 0x564b885d7b40 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 -- 172.21.15.79:0/3662067591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 msgr2=0x564b885cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/3662067591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885cd150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6c900cd750 tx=0x7f6c90087040 comp rx=0 tx=0).stop 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 -- 172.21.15.79:0/3662067591 shutdown_connections 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/3662067591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/3662067591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/3662067591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885cd150 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 -- 172.21.15.79:0/3662067591 >> 172.21.15.79:0/3662067591 conn(0x564b884fe120 msgr2=0x564b885cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 -- 172.21.15.79:0/3662067591 shutdown_connections 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 -- 172.21.15.79:0/3662067591 wait complete. 2024-06-03T02:14:51.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.947+0000 7f6cb6664b80 1 -- start start 2024-06-03T02:14:51.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6cb6664b80 1 Processor -- start 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885f6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6cb6664b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564b885ecbd0 con 0x564b885d7b40 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564b885ec500 con 0x564b885d21f0 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6cb6664b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564b885ed2a0 con 0x564b885d8460 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6cb6664b80 10 client.0 _ll_register_callbacks cb 0x564b884b9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37730/0 (socket says 172.21.15.79:37730) 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6ca9c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6ca3e00700 1 -- 172.21.15.79:0/1472928159 learned_addr learned my addr 172.21.15.79:0/1472928159 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.948+0000 7f6ca9200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885f6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47932/0 (socket says 172.21.15.79:47932) 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6ca9200700 1 -- 172.21.15.79:0/1472928159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 msgr2=0x564b885c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6ca9200700 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6ca9200700 1 -- 172.21.15.79:0/1472928159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 msgr2=0x564b885c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6ca9200700 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6ca9200700 1 -- 172.21.15.79:0/1472928159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b885c6f70 con 0x564b885d21f0 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6ca9200700 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885f6990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c9801a930 tx=0x7f6c9801a6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cee480aebbf2591 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6c8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1472928159 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9801e020 con 0x564b885d21f0 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1472928159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c900a5120 con 0x564b885d21f0 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.949+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1472928159 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8802+0+0 (secure 0 0 0) 0x7f6c98033480 con 0x564b885d21f0 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1472928159 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f6c98041050 con 0x564b885d21f0 2024-06-03T02:14:51.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6cb6664b80 10 client.24641 fetch_fsmap learned FSMap version 17 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1472928159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x564b886571d0 con 0x564b885d21f0 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1472928159 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c900a5120 con 0x564b885d21f0 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1472928159 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f6c9802b030 con 0x564b885d21f0 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6cb6664b80 10 client.24641 fetch_fsmap finished waiting for FSMap version 17 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6cb6664b80 10 client.24641 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6cb6664b80 20 client.24641 populate_metadata read hostname 'smithi079' 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564b8865bd10 0x564b8865aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6cb6664b80 4 client.24641 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.950+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1472928159 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "status"}) -- 0x564b8852b530 con 0x564b8865bd10 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564b8865bd10 0x564b8865aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6ca3e00700 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564b8865bd10 0x564b8865aa20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd80e6b65d69368 server_cookie=d7a1544462c4d6f9 in_seq=0 out_seq=0 2024-06-03T02:14:51.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6c8aa00700 10 client.24641 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:14:51.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6c8aa00700 1 -- 172.21.15.79:0/1472928159 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564b8852b530 con 0x564b8865bd10 2024-06-03T02:14:51.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6c8aa00700 10 client.24641 handle_command_reply: tid=0 2024-06-03T02:14:51.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6cb6664b80 1 client.24641 shutdown 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1472928159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 msgr2=0x564b885f6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885f6990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c9801a930 tx=0x7f6c9801a6b0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1472928159 shutdown_connections 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b885d8460 0x564b885c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b885d21f0 0x564b885f6990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564b8865bd10 0x564b8865aa20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.951+0000 7f6cb6664b80 1 --2- 172.21.15.79:0/1472928159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564b885d7b40 0x564b885c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.952+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1472928159 >> 172.21.15.79:0/1472928159 conn(0x564b884fe120 msgr2=0x564b885d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.952+0000 7f6c8aa00700 0 client.24641 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.952+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1472928159 shutdown_connections 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.952+0000 7f6cb6664b80 1 -- 172.21.15.79:0/1472928159 wait complete. 2024-06-03T02:14:51.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.952+0000 7f6cb6664b80 20 client.24641 trim_cache size 0 max 16384 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24539, 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 39.187818837000002, 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 17, 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 97, 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 96, 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 62.899364480999999 2024-06-03T02:14:51.956 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:14:51.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 -- 172.21.15.79:0/1246879552 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6c8c0a2ca0 msgr2=0x7f6c8c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:51.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6c8c0a2ca0 0x7f6c8c0a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00f590 tx=0x7f6c9c00b040 comp rx=0 tx=0).stop 2024-06-03T02:14:51.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 -- 172.21.15.79:0/1246879552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ca4102ee0 msgr2=0x7f6ca419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:51.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ca4102ee0 0x7f6ca419ee40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c90005910 tx=0x7f6c900047b0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 -- 172.21.15.79:0/1246879552 shutdown_connections 2024-06-03T02:14:51.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca41085c0 0x7f6ca4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6c8c0a2ca0 0x7f6c8c0a5160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ca4102520 0x7f6ca419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 --2- 172.21.15.79:0/1246879552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ca4102ee0 0x7f6ca419ee40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:51.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 -- 172.21.15.79:0/1246879552 >> 172.21.15.79:0/1246879552 conn(0x7f6ca4079680 msgr2=0x7f6ca4101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:51.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 -- 172.21.15.79:0/1246879552 shutdown_connections 2024-06-03T02:14:51.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:51.953+0000 7f6cabc00700 1 -- 172.21.15.79:0/1246879552 wait complete. 2024-06-03T02:14:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-03T02:14:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-06-03T02:14:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-06-03T02:14:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-06-03T02:14:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-06-03T02:14:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-06-03T02:14:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-03T02:14:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-03T02:14:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-06-03T02:14:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-06-03T02:14:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-03T02:14:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-06-03T02:14:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-06-03T02:14:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-06-03T02:14:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-06-03T02:14:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-06-03T02:14:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-03T02:14:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-03T02:14:52.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-06-03T02:14:52.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-06-03T02:14:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-03T02:14:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-06-03T02:14:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-06-03T02:14:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-06-03T02:14:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-06-03T02:14:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-06-03T02:14:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-03T02:14:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-03T02:14:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-06-03T02:14:52.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-06-03T02:14:52.869 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-03T02:14:52.870 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-03T02:14:52.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-03T02:14:53.320 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:14:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-03T02:14:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-06-03T02:14:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-06-03T02:14:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-06-03T02:14:53.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-06-03T02:14:53.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-06-03T02:14:53.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-03T02:14:53.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-03T02:14:53.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-06-03T02:14:53.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-06-03T02:14:53.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:53 smithi079 ceph-mon[31819]: osdmap e98: 12 total, 12 up, 12 in 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-06-03T02:14:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:53 smithi135 ceph-mon[36653]: osdmap e98: 12 total, 12 up, 12 in 2024-06-03T02:14:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:14:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-03T02:14:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-06-03T02:14:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-06-03T02:14:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-06-03T02:14:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-06-03T02:14:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-06-03T02:14:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-03T02:14:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-03T02:14:53.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-06-03T02:14:53.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-06-03T02:14:53.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:53 smithi123 ceph-mon[36640]: osdmap e98: 12 total, 12 up, 12 in 2024-06-03T02:14:54.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:54 smithi079 ceph-mon[31819]: osdmap e99: 12 total, 12 up, 12 in 2024-06-03T02:14:54.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:54 smithi079 ceph-mon[31819]: pgmap v360: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:14:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:54 smithi135 ceph-mon[36653]: osdmap e99: 12 total, 12 up, 12 in 2024-06-03T02:14:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:54 smithi135 ceph-mon[36653]: pgmap v360: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:14:54.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:54 smithi123 ceph-mon[36640]: osdmap e99: 12 total, 12 up, 12 in 2024-06-03T02:14:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:54 smithi123 ceph-mon[36640]: pgmap v360: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:14:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.533+0000 7fb841000700 1 -- 172.21.15.79:0/3249795408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb83c10a6b0 msgr2=0x7fb83c10aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.533+0000 7fb841000700 1 --2- 172.21.15.79:0/3249795408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb83c10a6b0 0x7fb83c10aa90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009990 tx=0x7fb82c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.533+0000 7fb841000700 1 -- 172.21.15.79:0/3249795408 shutdown_connections 2024-06-03T02:14:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.533+0000 7fb841000700 1 --2- 172.21.15.79:0/3249795408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb83c105070 0x7fb83c111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.533+0000 7fb841000700 1 --2- 172.21.15.79:0/3249795408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb83c10a6b0 0x7fb83c10aa90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.533+0000 7fb841000700 1 --2- 172.21.15.79:0/3249795408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c1046b0 0x7fb83c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.533+0000 7fb841000700 1 -- 172.21.15.79:0/3249795408 >> 172.21.15.79:0/3249795408 conn(0x7fb83c1001a0 msgr2=0x7fb83c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.533+0000 7fb841000700 1 -- 172.21.15.79:0/3249795408 shutdown_connections 2024-06-03T02:14:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.533+0000 7fb841000700 1 -- 172.21.15.79:0/3249795408 wait complete. 2024-06-03T02:14:55.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.534+0000 7fb841000700 1 Processor -- start 2024-06-03T02:14:55.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb841000700 1 -- start start 2024-06-03T02:14:55.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb841000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb83c1046b0 0x7fb83c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb841000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c105070 0x7fb83c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb841000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb83c10a6b0 0x7fb83c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb841000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c114a00 con 0x7fb83c10a6b0 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb841000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c114880 con 0x7fb83c105070 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb841000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c114b80 con 0x7fb83c1046b0 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb83a800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb83c10a6b0 0x7fb83c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb839e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb83c1046b0 0x7fb83c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.535+0000 7fb83a800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb83c10a6b0 0x7fb83c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33298/0 (socket says 172.21.15.79:33298) 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.536+0000 7fb83a800700 1 -- 172.21.15.79:0/164522224 learned_addr learned my addr 172.21.15.79:0/164522224 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:14:55.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.536+0000 7fb839400700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c105070 0x7fb83c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.536+0000 7fb839e00700 1 -- 172.21.15.79:0/164522224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c105070 msgr2=0x7fb83c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.536+0000 7fb839e00700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c105070 0x7fb83c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.536+0000 7fb839e00700 1 -- 172.21.15.79:0/164522224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb83c10a6b0 msgr2=0x7fb83c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.536+0000 7fb839e00700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb83c10a6b0 0x7fb83c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.536+0000 7fb839e00700 1 -- 172.21.15.79:0/164522224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c19fe40 con 0x7fb83c1046b0 2024-06-03T02:14:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.536+0000 7fb839e00700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb83c1046b0 0x7fb83c19ae30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0059d0 tx=0x7fb82c00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.537+0000 7fb832a00700 1 -- 172.21.15.79:0/164522224 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c027070 con 0x7fb83c1046b0 2024-06-03T02:14:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.537+0000 7fb832a00700 1 -- 172.21.15.79:0/164522224 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82c003d70 con 0x7fb83c1046b0 2024-06-03T02:14:55.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.537+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83c1a00d0 con 0x7fb83c1046b0 2024-06-03T02:14:55.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.537+0000 7fb832a00700 1 -- 172.21.15.79:0/164522224 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c022e30 con 0x7fb83c1046b0 2024-06-03T02:14:55.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.537+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83c1a79b0 con 0x7fb83c1046b0 2024-06-03T02:14:55.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.538+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb83c069180 con 0x7fb83c1046b0 2024-06-03T02:14:55.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.540+0000 7fb832a00700 1 -- 172.21.15.79:0/164522224 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb82c02a070 con 0x7fb83c1046b0 2024-06-03T02:14:55.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.541+0000 7fb832a00700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb8200a2c20 0x7fb8200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:14:55.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.541+0000 7fb832a00700 1 -- 172.21.15.79:0/164522224 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7fb82c0d0600 con 0x7fb83c1046b0 2024-06-03T02:14:55.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.542+0000 7fb839400700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb8200a2c20 0x7fb8200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:14:55.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.542+0000 7fb839400700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb8200a2c20 0x7fb8200a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb83c109c50 tx=0x7fb828009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:14:55.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.544+0000 7fb832a00700 1 -- 172.21.15.79:0/164522224 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb82c013070 con 0x7fb83c1046b0 2024-06-03T02:14:55.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.814+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb83c02dca0 con 0x7fb83c1046b0 2024-06-03T02:14:55.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.815+0000 7fb832a00700 1 -- 172.21.15.79:0/164522224 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fb82c09d230 con 0x7fb83c1046b0 2024-06-03T02:14:55.815 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-03T02:14:55.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb8200a2c20 msgr2=0x7fb8200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:55.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb8200a2c20 0x7fb8200a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb83c109c50 tx=0x7fb828009260 comp rx=0 tx=0).stop 2024-06-03T02:14:55.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb83c1046b0 msgr2=0x7fb83c19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:14:55.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb83c1046b0 0x7fb83c19ae30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0059d0 tx=0x7fb82c00bab0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 shutdown_connections 2024-06-03T02:14:55.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb8200a2c20 0x7fb8200a50e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb83c10a6b0 0x7fb83c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb83c1046b0 0x7fb83c19ae30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 --2- 172.21.15.79:0/164522224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c105070 0x7fb83c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:14:55.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 >> 172.21.15.79:0/164522224 conn(0x7fb83c1001a0 msgr2=0x7fb83c1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:14:55.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 shutdown_connections 2024-06-03T02:14:55.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:55.818+0000 7fb841000700 1 -- 172.21.15.79:0/164522224 wait complete. 2024-06-03T02:14:56.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:55 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/164522224' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T02:14:56.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:55 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/164522224' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T02:14:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:55 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/164522224' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T02:14:56.655 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-03T02:14:56.655 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-03T02:14:56.655 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2024-06-03T02:14:56.655 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-03T02:14:56.655 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-03T02:14:56.655 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-03T02:14:56.655 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-03T02:14:56.655 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-03T02:14:56.655 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-03T02:14:56.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:14:56.656 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2024-06-03T02:14:56.710 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-03T02:14:56.710 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-03T02:14:56.710 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2024-06-03T02:14:56.710 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-06-03T02:14:56.710 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2024-06-03T02:14:56.710 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-06-03T02:14:56.710 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T02:14:56.710 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f0 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f1 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 41969sec preferred_lft 41969sec 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::ec4:7aff:fe8f:ce87/64 scope link 2024-06-03T02:14:56.711 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-06-03T02:14:56.713 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-03T02:14:56.713 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:14:56.713 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:14:56.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add name ceph-brx type bridge 2024-06-03T02:14:56.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr flush dev ceph-brx 2024-06-03T02:14:56.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set ceph-brx up 2024-06-03T02:14:56.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-03T02:14:56.713 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:14:56.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:14:57.140 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:56 smithi079 ceph-mon[31819]: pgmap v361: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:14:57.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:14:57.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:14:57.199 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-03T02:14:57.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:56 smithi123 ceph-mon[36640]: pgmap v361: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:14:57.263 INFO:teuthology.orchestra.run.smithi079.stdout:1 2024-06-03T02:14:57.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:14:57.267 DEBUG:teuthology.orchestra.run.smithi079:> route 2024-06-03T02:14:57.324 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2024-06-03T02:14:57.324 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-03T02:14:57.324 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-03T02:14:57.324 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-03T02:14:57.324 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-03T02:14:57.325 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:14:57.325 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:14:57.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-03T02:14:57.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-03T02:14:57.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-03T02:14:57.325 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:14:57.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:56 smithi135 ceph-mon[36653]: pgmap v361: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:14:57.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:14:58.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:14:58.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:14:58.111 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-06-03T02:14:58.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:14:58.155 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2024-06-03T02:14:58.247 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:14:58.247 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:14:58.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T02:14:58.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-03T02:14:58.247 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:14:58.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:14:58.359 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:58 smithi079 ceph-mon[31819]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:14:58.359 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:14:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:14:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:14:58.411 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-03T02:14:58.411 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:14:58.411 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:14:58.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-03T02:14:58.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-03T02:14:58.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-03T02:14:58.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-03T02:14:58.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-03T02:14:58.411 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:14:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:14:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:58 smithi135 ceph-mon[36653]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:14:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:14:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:14:58.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:14:58.679 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:14:58.679 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:14:58.679 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2024-06-03T02:14:58.679 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.0 master ceph-brx 2024-06-03T02:14:58.679 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:14:58.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:58 smithi123 ceph-mon[36640]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:14:58.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:14:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:14:58.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:14:58.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:14:58.861 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-03T02:14:58.861 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T02:14:58.861 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-03T02:14:58.898 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-03T02:14:58.899 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T02:14:58.899 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-03T02:14:58.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2024-06-03T02:14:59.092 DEBUG:teuthology.orchestra.run.smithi079:> cat /proc/self/mounts | awk '{print $2}' 2024-06-03T02:14:59.129 INFO:teuthology.orchestra.run.smithi079.stdout:/proc 2024-06-03T02:14:59.129 INFO:teuthology.orchestra.run.smithi079.stdout:/sys 2024-06-03T02:14:59.129 INFO:teuthology.orchestra.run.smithi079.stdout:/dev 2024-06-03T02:14:59.129 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/security 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/shm 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/pts 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/run 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/cgroup 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/pstore 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/bpf 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/ 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/selinux 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/hugepages 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/mqueue 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/debug 2024-06-03T02:14:59.130 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/tracing 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/fuse/connections 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/config 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-sysctl.service 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/nfs/rpc_pipefs 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/run/user/1000 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/1ea6ec915164f245ddb3df05634f9d6f9a9c79502f33aaa3f540bf6024aa9544/merged 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/4e600d2783c181eee4075b878752b4b14217dce8c31f75708c0e282da9756ef8/merged 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/run/user/0 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/1c9cea24a38e19eecda0a5413e089ebf41d4b65450bf7c592b16c64bfb9ee1ce/merged 2024-06-03T02:14:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/21166499dd8746897f126033c21120aff54f7e6ae6f1443aaa07ac204778d007/merged 2024-06-03T02:14:59.132 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/1ec742493c09d25e4d7c52255c9b47a48fdcdc02d14621b8a236aa827ddc1728/merged 2024-06-03T02:14:59.132 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/e52bdb640a349c3aa400a620ed1a6dafb7e722f87ff840286ece4961b917f7a4/merged 2024-06-03T02:14:59.132 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/b9dd7f362ec4fc3ef0ea19b63868e4a5d971462c4055f8e47bbcf66f315b2bb3/merged 2024-06-03T02:14:59.132 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/ce98d459e0c4ccab3682c7089f1afae5a2aeb0bd3a18366bf35e56efde1f40b7/merged 2024-06-03T02:14:59.132 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/165525a93cfcc72ab8901c2d34ad7fba829a3fe3450b6ddb06c471fc530a18ca/merged 2024-06-03T02:14:59.132 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/723bd6c6c92a1200415f5ebeffecf34b9ad64f1ff4a0b9887104a54eccf0931c/merged 2024-06-03T02:14:59.132 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns 2024-06-03T02:14:59.132 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T02:14:59.132 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T02:14:59.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:14:59.133 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2024-06-03T02:14:59.213 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-03T02:14:59.214 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-03T02:14:59.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2024-06-03T02:14:59.323 DEBUG:teuthology.orchestra.run.smithi079:> cat /proc/self/mounts | awk '{print $2}' 2024-06-03T02:14:59.362 INFO:teuthology.orchestra.run.smithi079.stdout:/proc 2024-06-03T02:14:59.362 INFO:teuthology.orchestra.run.smithi079.stdout:/sys 2024-06-03T02:14:59.362 INFO:teuthology.orchestra.run.smithi079.stdout:/dev 2024-06-03T02:14:59.362 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/security 2024-06-03T02:14:59.362 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/shm 2024-06-03T02:14:59.362 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/pts 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/run 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/cgroup 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/pstore 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/bpf 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/ 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/selinux 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/hugepages 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/mqueue 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/debug 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/tracing 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/fuse/connections 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/config 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-sysctl.service 2024-06-03T02:14:59.363 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/nfs/rpc_pipefs 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/run/user/1000 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/1ea6ec915164f245ddb3df05634f9d6f9a9c79502f33aaa3f540bf6024aa9544/merged 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/4e600d2783c181eee4075b878752b4b14217dce8c31f75708c0e282da9756ef8/merged 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/run/user/0 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/1c9cea24a38e19eecda0a5413e089ebf41d4b65450bf7c592b16c64bfb9ee1ce/merged 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/21166499dd8746897f126033c21120aff54f7e6ae6f1443aaa07ac204778d007/merged 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/1ec742493c09d25e4d7c52255c9b47a48fdcdc02d14621b8a236aa827ddc1728/merged 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/e52bdb640a349c3aa400a620ed1a6dafb7e722f87ff840286ece4961b917f7a4/merged 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/b9dd7f362ec4fc3ef0ea19b63868e4a5d971462c4055f8e47bbcf66f315b2bb3/merged 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/ce98d459e0c4ccab3682c7089f1afae5a2aeb0bd3a18366bf35e56efde1f40b7/merged 2024-06-03T02:14:59.364 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/165525a93cfcc72ab8901c2d34ad7fba829a3fe3450b6ddb06c471fc530a18ca/merged 2024-06-03T02:14:59.365 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/723bd6c6c92a1200415f5ebeffecf34b9ad64f1ff4a0b9887104a54eccf0931c/merged 2024-06-03T02:14:59.365 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns 2024-06-03T02:14:59.365 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T02:14:59.365 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T02:14:59.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:14:59.366 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2024-06-03T02:14:59.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:2024-06-03T02:14:59.493+0000 7f076fe0e480 -1 init, newargv = 0x56035884b950 newargc=15 2024-06-03T02:14:59.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:ceph-fuse[78420]: starting ceph client 2024-06-03T02:14:59.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:ceph-fuse[78420]: starting fuse 2024-06-03T02:15:00.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2024-06-03T02:15:00.630 DEBUG:teuthology.orchestra.run.smithi079:> cat /proc/self/mounts | awk '{print $2}' 2024-06-03T02:15:00.673 INFO:teuthology.orchestra.run.smithi079.stdout:/proc 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/sys 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/dev 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/security 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/shm 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/pts 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/run 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/cgroup 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/pstore 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/bpf 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/ 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/selinux 2024-06-03T02:15:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/hugepages 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/mqueue 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/debug 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/tracing 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/fuse/connections 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/config 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-sysctl.service 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/nfs/rpc_pipefs 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/run/user/1000 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/1ea6ec915164f245ddb3df05634f9d6f9a9c79502f33aaa3f540bf6024aa9544/merged 2024-06-03T02:15:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/4e600d2783c181eee4075b878752b4b14217dce8c31f75708c0e282da9756ef8/merged 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/run/user/0 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/1c9cea24a38e19eecda0a5413e089ebf41d4b65450bf7c592b16c64bfb9ee1ce/merged 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/21166499dd8746897f126033c21120aff54f7e6ae6f1443aaa07ac204778d007/merged 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/1ec742493c09d25e4d7c52255c9b47a48fdcdc02d14621b8a236aa827ddc1728/merged 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/e52bdb640a349c3aa400a620ed1a6dafb7e722f87ff840286ece4961b917f7a4/merged 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/b9dd7f362ec4fc3ef0ea19b63868e4a5d971462c4055f8e47bbcf66f315b2bb3/merged 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/ce98d459e0c4ccab3682c7089f1afae5a2aeb0bd3a18366bf35e56efde1f40b7/merged 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/165525a93cfcc72ab8901c2d34ad7fba829a3fe3450b6ddb06c471fc530a18ca/merged 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/containers/storage/overlay/723bd6c6c92a1200415f5ebeffecf34b9ad64f1ff4a0b9887104a54eccf0931c/merged 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T02:15:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-03T02:15:00.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:15:00.677 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2024-06-03T02:15:00.759 INFO:teuthology.orchestra.run.smithi079.stdout:119 2024-06-03T02:15:00.759 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo stdin-killer -- python3 -c ' 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> import glob 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> import re 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> import os 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> import subprocess 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> def _find_admin_socket(client_name): 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> files = glob.glob(asok_path) 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> 2024-06-03T02:15:00.760 DEBUG:teuthology.orchestra.run.smithi079:> # Given a non-glob path, it better be there 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> if "*" not in asok_path: 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> assert(len(files) == 1) 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> return files[0] 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> for f in files: 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> contents = proc_f.read() 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> if mountpoint in contents: 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> return f 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> print(_find_admin_socket("client.admin")) 2024-06-03T02:15:00.761 DEBUG:teuthology.orchestra.run.smithi079:> ' 2024-06-03T02:15:00.958 INFO:teuthology.orchestra.run.smithi079.stdout:/var/run/ceph/ceph-client.admin.78420.asok 2024-06-03T02:15:00.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:15:00.976 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78420.asok 2024-06-03T02:15:00.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:15:00.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78420.asok status 2024-06-03T02:15:01.185 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:15:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "metadata": { 2024-06-03T02:15:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-03T02:15:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "entity_id": "admin", 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": "smithi079", 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "pid": "78420", 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "root": "/" 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_count": 0, 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_pinned_count": 0, 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24650, 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "inst": { 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "name": { 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "client", 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 24650 2024-06-03T02:15:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:0", 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 872397606 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:0", 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 872397606 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:15:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "inst_str": "client.24650 172.21.15.79:0/872397606", 2024-06-03T02:15:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "addr_str": "172.21.15.79:0/872397606", 2024-06-03T02:15:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "inode_count": 1, 2024-06-03T02:15:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_epoch": 17, 2024-06-03T02:15:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch": 99, 2024-06-03T02:15:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch_barrier": 0, 2024-06-03T02:15:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklisted": false, 2024-06-03T02:15:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs" 2024-06-03T02:15:01.188 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:15:01.198 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.198 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.198 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.198 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:00 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.199 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:00 smithi079 ceph-mon[31819]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:01.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:15:01.200 DEBUG:teuthology.orchestra.run.smithi079:> stat --file-system '--printf=%T 2024-06-03T02:15:01.200 DEBUG:teuthology.orchestra.run.smithi079:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-03T02:15:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:00 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:00 smithi123 ceph-mon[36640]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:01.246 INFO:teuthology.orchestra.run.smithi079.stdout:fuseblk 2024-06-03T02:15:01.246 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-03T02:15:01.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:15:01.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-03T02:15:01.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:00 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:01.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:00 smithi135 ceph-mon[36653]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:01.359 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-03T02:15:01.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T02:15:01.495 INFO:teuthology.orchestra.run.smithi079.stdout:# file: . 2024-06-03T02:15:01.496 INFO:teuthology.orchestra.run.smithi079.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-03T02:15:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:15:01.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:15:01.505 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:15:01.505 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T02:15:01.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T02:15:02.008 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:02.214 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:02.215 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:02.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:02.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:02.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:02.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:15:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:15:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:02 smithi123 ceph-mon[36640]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:03.223 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:15:03.224 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:15:03.224 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:03.224 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:02 smithi079 ceph-mon[31819]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:15:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:15:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:15:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:02 smithi135 ceph-mon[36653]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:04.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.356+0000 7f47e6400700 1 -- 172.21.15.79:0/1271518303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f47e010a680 msgr2=0x7f47e010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:04.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.356+0000 7f47e6400700 1 --2- 172.21.15.79:0/1271518303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f47e010a680 0x7f47e010aa60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009990 tx=0x7f47d0009ca0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.356+0000 7f47e6400700 1 -- 172.21.15.79:0/1271518303 shutdown_connections 2024-06-03T02:15:04.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.356+0000 7f47e6400700 1 --2- 172.21.15.79:0/1271518303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47e0105040 0x7f47e0111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.356+0000 7f47e6400700 1 --2- 172.21.15.79:0/1271518303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f47e010a680 0x7f47e010aa60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.356+0000 7f47e6400700 1 --2- 172.21.15.79:0/1271518303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e0104680 0x7f47e0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.356+0000 7f47e6400700 1 -- 172.21.15.79:0/1271518303 >> 172.21.15.79:0/1271518303 conn(0x7f47e0100170 msgr2=0x7f47e0102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:04.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.357+0000 7f47e6400700 1 -- 172.21.15.79:0/1271518303 shutdown_connections 2024-06-03T02:15:04.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.357+0000 7f47e6400700 1 -- 172.21.15.79:0/1271518303 wait complete. 2024-06-03T02:15:04.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.358+0000 7f47e6400700 1 Processor -- start 2024-06-03T02:15:04.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.358+0000 7f47e6400700 1 -- start start 2024-06-03T02:15:04.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.358+0000 7f47e6400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e0104680 0x7f47e019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:04.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.358+0000 7f47e6400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47e0105040 0x7f47e019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:04.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.358+0000 7f47e6400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f47e010a680 0x7f47e019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:04.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.358+0000 7f47e6400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f47e0114a20 con 0x7f47e010a680 2024-06-03T02:15:04.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.358+0000 7f47e6400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47e01148a0 con 0x7f47e0104680 2024-06-03T02:15:04.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.358+0000 7f47e6400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47e0114ba0 con 0x7f47e0105040 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47df400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e0104680 0x7f47e019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47dea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47e0105040 0x7f47e019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47df400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e0104680 0x7f47e019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:57612/0 (socket says 172.21.15.79:57612) 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47df400700 1 -- 172.21.15.79:0/2656802941 learned_addr learned my addr 172.21.15.79:0/2656802941 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47dfe00700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f47e010a680 0x7f47e019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47dea00700 1 -- 172.21.15.79:0/2656802941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e0104680 msgr2=0x7f47e019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47dea00700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e0104680 0x7f47e019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47dea00700 1 -- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f47e010a680 msgr2=0x7f47e019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47dea00700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f47e010a680 0x7f47e019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47dea00700 1 -- 172.21.15.79:0/2656802941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e019fe80 con 0x7f47e0105040 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.359+0000 7f47df400700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e0104680 0x7f47e019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.360+0000 7f47dea00700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47e0105040 0x7f47e019b3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00e9f0 tx=0x7f47cc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:04.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.360+0000 7f47d7e00700 1 -- 172.21.15.79:0/2656802941 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00c9c0 con 0x7f47e0105040 2024-06-03T02:15:04.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.360+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e01a0170 con 0x7f47e0105040 2024-06-03T02:15:04.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.360+0000 7f47d7e00700 1 -- 172.21.15.79:0/2656802941 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47cc00cb60 con 0x7f47e0105040 2024-06-03T02:15:04.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.360+0000 7f47d7e00700 1 -- 172.21.15.79:0/2656802941 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc018680 con 0x7f47e0105040 2024-06-03T02:15:04.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.360+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e01a79f0 con 0x7f47e0105040 2024-06-03T02:15:04.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.361+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e0069180 con 0x7f47e0105040 2024-06-03T02:15:04.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.364+0000 7f47d7e00700 1 -- 172.21.15.79:0/2656802941 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f47cc010430 con 0x7f47e0105040 2024-06-03T02:15:04.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.365+0000 7f47d7e00700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f47c40a2c20 0x7f47c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:04.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.365+0000 7f47d7e00700 1 -- 172.21.15.79:0/2656802941 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f47cc0cf510 con 0x7f47e0105040 2024-06-03T02:15:04.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.365+0000 7f47df400700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f47c40a2c20 0x7f47c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:04.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.366+0000 7f47df400700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f47c40a2c20 0x7f47c40a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f47d0006010 tx=0x7f47d000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:04.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.370+0000 7f47d7e00700 1 -- 172.21.15.79:0/2656802941 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f47cc09c0b0 con 0x7f47e0105040 2024-06-03T02:15:04.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.606+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47e002dca0 con 0x7f47e0105040 2024-06-03T02:15:04.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.607+0000 7f47d7e00700 1 -- 172.21.15.79:0/2656802941 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v99) ==== 81+0+584 (secure 0 0 0) 0x7f47cc09b9b0 con 0x7f47e0105040 2024-06-03T02:15:04.607 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6801/54480245 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:04.608 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6800/54480245 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:04.608 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3960540784 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:04.608 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3085681251 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:04.608 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6800/1845575623 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:04.608 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/37254677 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:04.608 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6801/1845575623 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:04.608 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/603700731 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:04.608 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/161718363 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:04.608 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3870624303 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:04.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f47c40a2c20 msgr2=0x7f47c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:04.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f47c40a2c20 0x7f47c40a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f47d0006010 tx=0x7f47d000b260 comp rx=0 tx=0).stop 2024-06-03T02:15:04.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47e0105040 msgr2=0x7f47e019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:04.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47e0105040 0x7f47e019b3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00e9f0 tx=0x7f47cc00ed00 comp rx=0 tx=0).stop 2024-06-03T02:15:04.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 shutdown_connections 2024-06-03T02:15:04.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f47c40a2c20 0x7f47c40a50e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f47e010a680 0x7f47e019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e0104680 0x7f47e019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 --2- 172.21.15.79:0/2656802941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47e0105040 0x7f47e019b3b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:04.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 >> 172.21.15.79:0/2656802941 conn(0x7f47e0100170 msgr2=0x7f47e0100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:04.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 shutdown_connections 2024-06-03T02:15:04.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:04.610+0000 7f47e6400700 1 -- 172.21.15.79:0/2656802941 wait complete. 2024-06-03T02:15:04.612 INFO:teuthology.orchestra.run.smithi079.stderr:listed 10 entries 2024-06-03T02:15:05.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:05 smithi079 ceph-mon[31819]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 375 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:05.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:05 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2656802941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T02:15:05.484 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:15:05.485 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T02:15:05.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T02:15:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:05 smithi135 ceph-mon[36653]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 375 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:05 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2656802941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T02:15:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:05 smithi123 ceph-mon[36640]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 375 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:05 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2656802941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T02:15:05.996 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:07.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:07 smithi079 ceph-mon[31819]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:07 smithi135 ceph-mon[36653]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:07 smithi123 ceph-mon[36640]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:08.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.203+0000 7f4475e00700 1 -- 172.21.15.79:0/2371181210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f447010a6d0 msgr2=0x7f447010aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:08.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.203+0000 7f4475e00700 1 --2- 172.21.15.79:0/2371181210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f447010a6d0 0x7f447010aab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f44600098f0 tx=0x7f4460009c00 comp rx=0 tx=0).stop 2024-06-03T02:15:08.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.204+0000 7f4475e00700 1 -- 172.21.15.79:0/2371181210 shutdown_connections 2024-06-03T02:15:08.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.204+0000 7f4475e00700 1 --2- 172.21.15.79:0/2371181210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4470105090 0x7f44701115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:08.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.204+0000 7f4475e00700 1 --2- 172.21.15.79:0/2371181210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f447010a6d0 0x7f447010aab0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:08.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.204+0000 7f4475e00700 1 --2- 172.21.15.79:0/2371181210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44701046d0 0x7f4470104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:08.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.204+0000 7f4475e00700 1 -- 172.21.15.79:0/2371181210 >> 172.21.15.79:0/2371181210 conn(0x7f44701001c0 msgr2=0x7f44701025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:08.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.204+0000 7f4475e00700 1 -- 172.21.15.79:0/2371181210 shutdown_connections 2024-06-03T02:15:08.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.204+0000 7f4475e00700 1 -- 172.21.15.79:0/2371181210 wait complete. 2024-06-03T02:15:08.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.205+0000 7f4475e00700 1 Processor -- start 2024-06-03T02:15:08.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.205+0000 7f4475e00700 1 -- start start 2024-06-03T02:15:08.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f4475e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44701046d0 0x7f447019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:08.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f4475e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4470105090 0x7f447019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:08.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f4475e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f447010a6d0 0x7f447019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:08.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f4475e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4470114a00 con 0x7f447010a6d0 2024-06-03T02:15:08.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f4475e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4470114880 con 0x7f4470105090 2024-06-03T02:15:08.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f4475e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4470114b80 con 0x7f44701046d0 2024-06-03T02:15:08.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f446ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44701046d0 0x7f447019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:08.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f446f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f447010a6d0 0x7f447019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:08.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f446ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44701046d0 0x7f447019af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55754/0 (socket says 172.21.15.79:55754) 2024-06-03T02:15:08.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f446ea00700 1 -- 172.21.15.79:0/844405148 learned_addr learned my addr 172.21.15.79:0/844405148 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:08.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.206+0000 7f446e000700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4470105090 0x7f447019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:08.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.207+0000 7f446ea00700 1 -- 172.21.15.79:0/844405148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4470105090 msgr2=0x7f447019b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:08.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.207+0000 7f446ea00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4470105090 0x7f447019b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:08.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.207+0000 7f446ea00700 1 -- 172.21.15.79:0/844405148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f447010a6d0 msgr2=0x7f447019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:08.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.207+0000 7f446ea00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f447010a6d0 0x7f447019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:08.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.207+0000 7f446ea00700 1 -- 172.21.15.79:0/844405148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447019ff10 con 0x7f44701046d0 2024-06-03T02:15:08.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.207+0000 7f446ea00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44701046d0 0x7f447019af50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4460009230 tx=0x7f446000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:08.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.207+0000 7f4474a00700 1 -- 172.21.15.79:0/844405148 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460005180 con 0x7f44701046d0 2024-06-03T02:15:08.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.207+0000 7f4474a00700 1 -- 172.21.15.79:0/844405148 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446000b560 con 0x7f44701046d0 2024-06-03T02:15:08.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.207+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44701a01a0 con 0x7f44701046d0 2024-06-03T02:15:08.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.208+0000 7f4474a00700 1 -- 172.21.15.79:0/844405148 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460003a70 con 0x7f44701046d0 2024-06-03T02:15:08.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.208+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44701a7a80 con 0x7f44701046d0 2024-06-03T02:15:08.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.209+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4470069180 con 0x7f44701046d0 2024-06-03T02:15:08.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.211+0000 7f4474a00700 1 -- 172.21.15.79:0/844405148 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4460029060 con 0x7f44701046d0 2024-06-03T02:15:08.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.212+0000 7f4474a00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44540a2ca0 0x7f44540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:08.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.212+0000 7f4474a00700 1 -- 172.21.15.79:0/844405148 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f44600cfc30 con 0x7f44701046d0 2024-06-03T02:15:08.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.212+0000 7f446e000700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44540a2ca0 0x7f44540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:08.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.213+0000 7f446e000700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44540a2ca0 0x7f44540a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f445c009750 tx=0x7f445c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:08.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.217+0000 7f4474a00700 1 -- 172.21.15.79:0/844405148 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4460013030 con 0x7f44701046d0 2024-06-03T02:15:08.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.456+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f447002dca0 con 0x7f44701046d0 2024-06-03T02:15:08.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.457+0000 7f4474a00700 1 -- 172.21.15.79:0/844405148 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v99) ==== 81+0+584 (secure 0 0 0) 0x7f446009c860 con 0x7f44701046d0 2024-06-03T02:15:08.457 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6801/54480245 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:08.457 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6800/54480245 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:08.458 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3960540784 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:08.458 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3085681251 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:08.458 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6800/1845575623 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:08.458 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/37254677 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:08.458 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6801/1845575623 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:08.458 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/603700731 2024-06-04T02:05:16.591415+0000 2024-06-03T02:15:08.458 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/161718363 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:08.458 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3870624303 2024-06-04T02:05:52.008734+0000 2024-06-03T02:15:08.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44540a2ca0 msgr2=0x7f44540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:08.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44540a2ca0 0x7f44540a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f445c009750 tx=0x7f445c009310 comp rx=0 tx=0).stop 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44701046d0 msgr2=0x7f447019af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44701046d0 0x7f447019af50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4460009230 tx=0x7f446000fa70 comp rx=0 tx=0).stop 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 shutdown_connections 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44540a2ca0 0x7f44540a5160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f447010a6d0 0x7f447019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44701046d0 0x7f447019af50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 --2- 172.21.15.79:0/844405148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4470105090 0x7f447019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 >> 172.21.15.79:0/844405148 conn(0x7f44701001c0 msgr2=0x7f4470100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.461+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 shutdown_connections 2024-06-03T02:15:08.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:08.462+0000 7f4475e00700 1 -- 172.21.15.79:0/844405148 wait complete. 2024-06-03T02:15:08.463 INFO:teuthology.orchestra.run.smithi079.stderr:listed 10 entries 2024-06-03T02:15:09.216 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi079.front.sepia.ceph.com... 2024-06-03T02:15:09.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:15:09.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-03T02:15:09.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:ceph-fuse[78420]: fuse finished with error 0 and tester_r 0 2024-06-03T02:15:09.302 INFO:teuthology.orchestra.run:waiting for 300 2024-06-03T02:15:09.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:09 smithi079 ceph-mon[31819]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:09.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:09 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/844405148' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T02:15:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:09 smithi135 ceph-mon[36653]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:09 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/844405148' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T02:15:09.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:09 smithi123 ceph-mon[36640]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T02:15:09.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:09 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/844405148' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T02:15:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:10 smithi079 ceph-mon[31819]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T02:15:10.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:10 smithi135 ceph-mon[36653]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T02:15:10.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:10 smithi123 ceph-mon[36640]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T02:15:13.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:13 smithi079 ceph-mon[31819]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T02:15:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:13 smithi135 ceph-mon[36653]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T02:15:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:13 smithi123 ceph-mon[36640]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T02:15:15.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:15 smithi079 ceph-mon[31819]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T02:15:15.404 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2024-06-03T02:15:15.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:15:15.404 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-03T02:15:15.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:15:15.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:15 smithi135 ceph-mon[36653]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T02:15:15.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:15 smithi123 ceph-mon[36640]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T02:15:15.889 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:16.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:16 smithi079 ceph-mon[31819]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:16 smithi135 ceph-mon[36653]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:16.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:16 smithi123 ceph-mon[36640]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.355+0000 7fa107400700 1 -- 172.21.15.79:0/851200354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa100108500 msgr2=0x7fa1001088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.355+0000 7fa107400700 1 --2- 172.21.15.79:0/851200354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa100108500 0x7fa1001088e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009990 tx=0x7fa0e8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.355+0000 7fa107400700 1 -- 172.21.15.79:0/851200354 shutdown_connections 2024-06-03T02:15:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.355+0000 7fa107400700 1 --2- 172.21.15.79:0/851200354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa10006ab30 0x7fa10010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.355+0000 7fa107400700 1 --2- 172.21.15.79:0/851200354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa100108500 0x7fa1001088e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.355+0000 7fa107400700 1 --2- 172.21.15.79:0/851200354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10006a170 0x7fa10006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.355+0000 7fa107400700 1 -- 172.21.15.79:0/851200354 >> 172.21.15.79:0/851200354 conn(0x7fa100078340 msgr2=0x7fa100078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.355+0000 7fa107400700 1 -- 172.21.15.79:0/851200354 shutdown_connections 2024-06-03T02:15:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.355+0000 7fa107400700 1 -- 172.21.15.79:0/851200354 wait complete. 2024-06-03T02:15:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.356+0000 7fa107400700 1 Processor -- start 2024-06-03T02:15:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.356+0000 7fa107400700 1 -- start start 2024-06-03T02:15:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa107400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa10006a170 0x7fa10019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa107400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10006ab30 0x7fa10019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa107400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa100108500 0x7fa10019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa107400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa1001127d0 con 0x7fa10006a170 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa107400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa100112650 con 0x7fa10006ab30 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa107400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa100112950 con 0x7fa100108500 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa104a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa10006a170 0x7fa10019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa104a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa10006a170 0x7fa10019aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41874/0 (socket says 172.21.15.79:41874) 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa104a00700 1 -- 172.21.15.79:0/339778804 learned_addr learned my addr 172.21.15.79:0/339778804 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa0f7e00700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10006ab30 0x7fa10019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa105400700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa100108500 0x7fa10019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa104a00700 1 -- 172.21.15.79:0/339778804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa100108500 msgr2=0x7fa10019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa104a00700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa100108500 0x7fa10019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa104a00700 1 -- 172.21.15.79:0/339778804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10006ab30 msgr2=0x7fa10019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa104a00700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10006ab30 0x7fa10019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.357+0000 7fa104a00700 1 -- 172.21.15.79:0/339778804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10019fee0 con 0x7fa10006a170 2024-06-03T02:15:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.358+0000 7fa104a00700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa10006a170 0x7fa10019aed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8000c00 tx=0x7fa0e800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.358+0000 7fa0fe000700 1 -- 172.21.15.79:0/339778804 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8027070 con 0x7fa10006a170 2024-06-03T02:15:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.358+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1001a0170 con 0x7fa10006a170 2024-06-03T02:15:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.358+0000 7fa0fe000700 1 -- 172.21.15.79:0/339778804 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e80056d0 con 0x7fa10006a170 2024-06-03T02:15:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.358+0000 7fa0fe000700 1 -- 172.21.15.79:0/339778804 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8022e10 con 0x7fa10006a170 2024-06-03T02:15:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.358+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1001a7a50 con 0x7fa10006a170 2024-06-03T02:15:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.360+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa100103bc0 con 0x7fa10006a170 2024-06-03T02:15:18.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.362+0000 7fa0fe000700 1 -- 172.21.15.79:0/339778804 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa0e802a070 con 0x7fa10006a170 2024-06-03T02:15:18.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.363+0000 7fa0fe000700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa0e00a2c20 0x7fa0e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:18.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.363+0000 7fa0fe000700 1 -- 172.21.15.79:0/339778804 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7fa0e80cfff0 con 0x7fa10006a170 2024-06-03T02:15:18.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.363+0000 7fa0f7e00700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa0e00a2c20 0x7fa0e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:18.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.364+0000 7fa0f7e00700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa0e00a2c20 0x7fa0e00a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009680 tx=0x7fa0f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:18.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.368+0000 7fa0fe000700 1 -- 172.21.15.79:0/339778804 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa0e8013070 con 0x7fa10006a170 2024-06-03T02:15:18.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.593+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa10019c230 con 0x7fa10006a170 2024-06-03T02:15:18.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.597+0000 7fa0fe000700 1 -- 172.21.15.79:0/339778804 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24661 (secure 0 0 0) 0x7fa0e809cc40 con 0x7fa10006a170 2024-06-03T02:15:18.598 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:15:18.599 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":99,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:14:53.251002+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:15:18.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa0e00a2c20 msgr2=0x7fa0e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:18.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa0e00a2c20 0x7fa0e00a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009680 tx=0x7fa0f0009260 comp rx=0 tx=0).stop 2024-06-03T02:15:18.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa10006a170 msgr2=0x7fa10019aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:18.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa10006a170 0x7fa10019aed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8000c00 tx=0x7fa0e800bab0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 shutdown_connections 2024-06-03T02:15:18.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa100108500 0x7fa10019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa0e00a2c20 0x7fa0e00a50e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa10006a170 0x7fa10019aed0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 --2- 172.21.15.79:0/339778804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10006ab30 0x7fa10019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:18.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 >> 172.21.15.79:0/339778804 conn(0x7fa100078340 msgr2=0x7fa10010d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:18.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 shutdown_connections 2024-06-03T02:15:18.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:18.601+0000 7fa107400700 1 -- 172.21.15.79:0/339778804 wait complete. 2024-06-03T02:15:19.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:15:19.557 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:19 smithi079 ceph-mon[31819]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:19.557 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:19 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/339778804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:15:19.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:19 smithi135 ceph-mon[36653]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:19.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:19 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/339778804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:15:19.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:19 smithi123 ceph-mon[36640]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:19.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:19 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/339778804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:15:19.922 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:21 smithi079 ceph-mon[31819]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:21 smithi135 ceph-mon[36653]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:21.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:21 smithi123 ceph-mon[36640]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.196+0000 7f7da5000700 1 -- 172.21.15.79:0/3315268297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da01045c0 msgr2=0x7f7da0104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.196+0000 7f7da5000700 1 --2- 172.21.15.79:0/3315268297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da01045c0 0x7f7da0104a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009990 tx=0x7f7d8c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.196+0000 7f7da5000700 1 -- 172.21.15.79:0/3315268297 shutdown_connections 2024-06-03T02:15:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.196+0000 7f7da5000700 1 --2- 172.21.15.79:0/3315268297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7da0104f80 0x7f7da0111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.196+0000 7f7da5000700 1 --2- 172.21.15.79:0/3315268297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da010a650 0x7f7da010aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.196+0000 7f7da5000700 1 --2- 172.21.15.79:0/3315268297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da01045c0 0x7f7da0104a40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.196+0000 7f7da5000700 1 -- 172.21.15.79:0/3315268297 >> 172.21.15.79:0/3315268297 conn(0x7f7da01000b0 msgr2=0x7f7da01024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.196+0000 7f7da5000700 1 -- 172.21.15.79:0/3315268297 shutdown_connections 2024-06-03T02:15:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.196+0000 7f7da5000700 1 -- 172.21.15.79:0/3315268297 wait complete. 2024-06-03T02:15:22.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.197+0000 7f7da5000700 1 Processor -- start 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7da5000700 1 -- start start 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7da5000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7da01045c0 0x7f7da01a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7da5000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0104f80 0x7f7da01a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7da5000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da010a650 0x7f7da019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7da5000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0114960 con 0x7f7da010a650 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7da5000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01147e0 con 0x7f7da0104f80 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7da5000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0114ae0 con 0x7f7da01045c0 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7d9fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da010a650 0x7f7da019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7d9f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7da01045c0 0x7f7da01a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7d9fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da010a650 0x7f7da019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41888/0 (socket says 172.21.15.79:41888) 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7d9fe00700 1 -- 172.21.15.79:0/3640506250 learned_addr learned my addr 172.21.15.79:0/3640506250 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7d9f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7da01045c0 0x7f7da01a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42794/0 (socket says 172.21.15.79:42794) 2024-06-03T02:15:22.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.198+0000 7f7d9ea00700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0104f80 0x7f7da01a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:22.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.199+0000 7f7d9ea00700 1 -- 172.21.15.79:0/3640506250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7da01045c0 msgr2=0x7f7da01a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:22.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.199+0000 7f7d9ea00700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7da01045c0 0x7f7da01a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.199+0000 7f7d9ea00700 1 -- 172.21.15.79:0/3640506250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da010a650 msgr2=0x7f7da019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.199+0000 7f7d9ea00700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da010a650 0x7f7da019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.199+0000 7f7d9ea00700 1 -- 172.21.15.79:0/3640506250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da019b260 con 0x7f7da0104f80 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.199+0000 7f7d9f400700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7da01045c0 0x7f7da01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.199+0000 7f7d9ea00700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0104f80 0x7f7da01a0e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0098e0 tx=0x7f7d8c00b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.200+0000 7f7d97e00700 1 -- 172.21.15.79:0/3640506250 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c027070 con 0x7f7da0104f80 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.200+0000 7f7d97e00700 1 -- 172.21.15.79:0/3640506250 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d8c003b60 con 0x7f7da0104f80 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.200+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da019b4f0 con 0x7f7da0104f80 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.200+0000 7f7d97e00700 1 -- 172.21.15.79:0/3640506250 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c022d50 con 0x7f7da0104f80 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.200+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da01a7810 con 0x7f7da0104f80 2024-06-03T02:15:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.201+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da0105d00 con 0x7f7da0104f80 2024-06-03T02:15:22.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.203+0000 7f7d97e00700 1 -- 172.21.15.79:0/3640506250 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7d8c02a070 con 0x7f7da0104f80 2024-06-03T02:15:22.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.204+0000 7f7d97e00700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d840a2c20 0x7f7d840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:22.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.204+0000 7f7d97e00700 1 -- 172.21.15.79:0/3640506250 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f7d8c0d0ac0 con 0x7f7da0104f80 2024-06-03T02:15:22.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.205+0000 7f7d9f400700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d840a2c20 0x7f7d840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:22.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.205+0000 7f7d9f400700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d840a2c20 0x7f7d840a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7da0105700 tx=0x7f7d90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:22.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.207+0000 7f7d97e00700 1 -- 172.21.15.79:0/3640506250 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7d8c031020 con 0x7f7da0104f80 2024-06-03T02:15:22.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:22 smithi079 ceph-mon[31819]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:22.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.435+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7da0069180 con 0x7f7da0104f80 2024-06-03T02:15:22.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.437+0000 7f7d97e00700 1 -- 172.21.15.79:0/3640506250 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24661 (secure 0 0 0) 0x7f7da0069180 con 0x7f7da0104f80 2024-06-03T02:15:22.438 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:15:22.438 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":99,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:14:53.251002+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.440+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d840a2c20 msgr2=0x7f7d840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.440+0000 7f7da5000700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d840a2c20 0x7f7d840a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7da0105700 tx=0x7f7d90009260 comp rx=0 tx=0).stop 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.440+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0104f80 msgr2=0x7f7da01a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.440+0000 7f7da5000700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0104f80 0x7f7da01a0e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0098e0 tx=0x7f7d8c00b8a0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.441+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 shutdown_connections 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.441+0000 7f7da5000700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d840a2c20 0x7f7d840a50e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.441+0000 7f7da5000700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da010a650 0x7f7da019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.441+0000 7f7da5000700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7da01045c0 0x7f7da01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.441+0000 7f7da5000700 1 --2- 172.21.15.79:0/3640506250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7da0104f80 0x7f7da01a0e80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.441+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 >> 172.21.15.79:0/3640506250 conn(0x7f7da01000b0 msgr2=0x7f7da01016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.441+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 shutdown_connections 2024-06-03T02:15:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:22.441+0000 7f7da5000700 1 -- 172.21.15.79:0/3640506250 wait complete. 2024-06-03T02:15:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:22 smithi135 ceph-mon[36653]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:22.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:22 smithi123 ceph-mon[36640]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:23.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-03T02:15:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:23 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3640506250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:15:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:23 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3640506250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:15:23.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:23 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3640506250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:15:23.790 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:24 smithi079 ceph-mon[31819]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:24 smithi135 ceph-mon[36653]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:24 smithi123 ceph-mon[36640]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T02:15:26.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.037+0000 7f6bda800700 1 -- 172.21.15.79:0/1745555594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bd4104690 msgr2=0x7f6bd4104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:26.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.037+0000 7f6bda800700 1 --2- 172.21.15.79:0/1745555594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bd4104690 0x7f6bd4104b10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0098f0 tx=0x7f6bbc009c00 comp rx=0 tx=0).stop 2024-06-03T02:15:26.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.037+0000 7f6bda800700 1 -- 172.21.15.79:0/1745555594 shutdown_connections 2024-06-03T02:15:26.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.037+0000 7f6bda800700 1 --2- 172.21.15.79:0/1745555594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd4105050 0x7f6bd4111560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:26.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.037+0000 7f6bda800700 1 --2- 172.21.15.79:0/1745555594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd410a690 0x7f6bd410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:26.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.037+0000 7f6bda800700 1 --2- 172.21.15.79:0/1745555594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bd4104690 0x7f6bd4104b10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:26.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.037+0000 7f6bda800700 1 -- 172.21.15.79:0/1745555594 >> 172.21.15.79:0/1745555594 conn(0x7f6bd41001a0 msgr2=0x7f6bd41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:26.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.037+0000 7f6bda800700 1 -- 172.21.15.79:0/1745555594 shutdown_connections 2024-06-03T02:15:26.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.038+0000 7f6bda800700 1 -- 172.21.15.79:0/1745555594 wait complete. 2024-06-03T02:15:26.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.039+0000 7f6bda800700 1 Processor -- start 2024-06-03T02:15:26.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.039+0000 7f6bda800700 1 -- start start 2024-06-03T02:15:26.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.039+0000 7f6bda800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd4104690 0x7f6bd419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:26.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.039+0000 7f6bda800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bd4105050 0x7f6bd419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:26.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.039+0000 7f6bda800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd410a690 0x7f6bd419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:26.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.039+0000 7f6bda800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd41149c0 con 0x7f6bd410a690 2024-06-03T02:15:26.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.039+0000 7f6bda800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd4114840 con 0x7f6bd4104690 2024-06-03T02:15:26.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.039+0000 7f6bda800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd4114b40 con 0x7f6bd4105050 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bd3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd4104690 0x7f6bd419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bcbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bd4105050 0x7f6bd419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bd3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd410a690 0x7f6bd419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bd3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd4104690 0x7f6bd419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53542/0 (socket says 172.21.15.79:53542) 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bd3400700 1 -- 172.21.15.79:0/1512469420 learned_addr learned my addr 172.21.15.79:0/1512469420 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bcbe00700 1 -- 172.21.15.79:0/1512469420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd4104690 msgr2=0x7f6bd419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bcbe00700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd4104690 0x7f6bd419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bcbe00700 1 -- 172.21.15.79:0/1512469420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd410a690 msgr2=0x7f6bd419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bcbe00700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd410a690 0x7f6bd419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bcbe00700 1 -- 172.21.15.79:0/1512469420 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd419fe40 con 0x7f6bd4105050 2024-06-03T02:15:26.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bd3400700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd4104690 0x7f6bd419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:15:26.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.040+0000 7f6bcbe00700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bd4105050 0x7f6bd419b370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00c010 tx=0x7f6bbc00fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:26.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.041+0000 7f6bd0c00700 1 -- 172.21.15.79:0/1512469420 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc005180 con 0x7f6bd4105050 2024-06-03T02:15:26.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.041+0000 7f6bd0c00700 1 -- 172.21.15.79:0/1512469420 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bbc00b560 con 0x7f6bd4105050 2024-06-03T02:15:26.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.041+0000 7f6bd0c00700 1 -- 172.21.15.79:0/1512469420 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc003a80 con 0x7f6bd4105050 2024-06-03T02:15:26.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.041+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd41a00d0 con 0x7f6bd4105050 2024-06-03T02:15:26.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.041+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd41a79b0 con 0x7f6bd4105050 2024-06-03T02:15:26.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.042+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd4105dd0 con 0x7f6bd4105050 2024-06-03T02:15:26.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.044+0000 7f6bd0c00700 1 -- 172.21.15.79:0/1512469420 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6bbc00b710 con 0x7f6bd4105050 2024-06-03T02:15:26.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.045+0000 7f6bd0c00700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bb80a2c20 0x7f6bb80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:26.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.046+0000 7f6bd0c00700 1 -- 172.21.15.79:0/1512469420 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f6bbc0d0660 con 0x7f6bd4105050 2024-06-03T02:15:26.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.046+0000 7f6bd3400700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bb80a2c20 0x7f6bb80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:26.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.046+0000 7f6bd3400700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bb80a2c20 0x7f6bb80a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009750 tx=0x7f6bc4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:26.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.048+0000 7f6bd0c00700 1 -- 172.21.15.79:0/1512469420 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6bbc013030 con 0x7f6bd4105050 2024-06-03T02:15:26.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:26.320+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6bd419c190 con 0x7f6bd4105050 2024-06-03T02:15:27.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.247+0000 7f6bd0c00700 1 -- 172.21.15.79:0/1512469420 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f6bbc09d290 con 0x7f6bd4105050 2024-06-03T02:15:27.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bb80a2c20 msgr2=0x7f6bb80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bb80a2c20 0x7f6bb80a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009750 tx=0x7f6bc4009310 comp rx=0 tx=0).stop 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bd4105050 msgr2=0x7f6bd419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bd4105050 0x7f6bd419b370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00c010 tx=0x7f6bbc00fa70 comp rx=0 tx=0).stop 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 shutdown_connections 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bb80a2c20 0x7f6bb80a50e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd410a690 0x7f6bd419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd4104690 0x7f6bd419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 --2- 172.21.15.79:0/1512469420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bd4105050 0x7f6bd419b370 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.253+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 >> 172.21.15.79:0/1512469420 conn(0x7f6bd41001a0 msgr2=0x7f6bd41007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:27.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.254+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 shutdown_connections 2024-06-03T02:15:27.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:27.254+0000 7f6bda800700 1 -- 172.21.15.79:0/1512469420 wait complete. 2024-06-03T02:15:27.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:27 smithi079 ceph-mon[31819]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:27.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:27 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1512469420' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T02:15:27.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:27 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T02:15:27.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:27 smithi135 ceph-mon[36653]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:27.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:27 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1512469420' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T02:15:27.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:27 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T02:15:27.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:27 smithi123 ceph-mon[36640]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:27.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:27 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1512469420' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T02:15:27.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:27 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T02:15:28.208 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-06-03T02:15:28.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-06-03T02:15:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:28 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-03T02:15:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:28 smithi079 ceph-mon[31819]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:15:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:28 smithi079 ceph-mon[31819]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:28 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-03T02:15:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:28 smithi135 ceph-mon[36653]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:15:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:28 smithi135 ceph-mon[36653]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:28.659 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:28 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-03T02:15:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:28 smithi123 ceph-mon[36640]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:15:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:28 smithi123 ceph-mon[36640]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:30.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.962+0000 7f5a7c200700 1 -- 172.21.15.79:0/586760065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a741084e0 msgr2=0x7f5a741088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:30.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.962+0000 7f5a7c200700 1 --2- 172.21.15.79:0/586760065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a741084e0 0x7f5a741088c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a680099c0 tx=0x7f5a68009cd0 comp rx=0 tx=0).stop 2024-06-03T02:15:30.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.962+0000 7f5a7c200700 1 -- 172.21.15.79:0/586760065 shutdown_connections 2024-06-03T02:15:30.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.962+0000 7f5a7c200700 1 --2- 172.21.15.79:0/586760065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a7406ab10 0x7f5a7410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:30.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.962+0000 7f5a7c200700 1 --2- 172.21.15.79:0/586760065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a741084e0 0x7f5a741088c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:30.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.962+0000 7f5a7c200700 1 --2- 172.21.15.79:0/586760065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7406a170 0x7f5a7406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:30.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.962+0000 7f5a7c200700 1 -- 172.21.15.79:0/586760065 >> 172.21.15.79:0/586760065 conn(0x7f5a740782c0 msgr2=0x7f5a740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:30.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.962+0000 7f5a7c200700 1 -- 172.21.15.79:0/586760065 shutdown_connections 2024-06-03T02:15:30.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.962+0000 7f5a7c200700 1 -- 172.21.15.79:0/586760065 wait complete. 2024-06-03T02:15:30.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.963+0000 7f5a7c200700 1 Processor -- start 2024-06-03T02:15:30.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.963+0000 7f5a7c200700 1 -- start start 2024-06-03T02:15:30.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.963+0000 7f5a7c200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7406a170 0x7f5a74101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:30.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.963+0000 7f5a7c200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a7406ab10 0x7f5a741019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:30.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.963+0000 7f5a7c200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a741084e0 0x7f5a74101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:30.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a7c200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74112790 con 0x7f5a7406ab10 2024-06-03T02:15:30.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a7c200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74112610 con 0x7f5a7406a170 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a7c200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74112910 con 0x7f5a741084e0 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a79800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7406a170 0x7f5a74101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a78e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a7406ab10 0x7f5a741019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a79800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7406a170 0x7f5a74101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53554/0 (socket says 172.21.15.79:53554) 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a79800700 1 -- 172.21.15.79:0/2501938150 learned_addr learned my addr 172.21.15.79:0/2501938150 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a7a200700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a741084e0 0x7f5a74101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a78e00700 1 -- 172.21.15.79:0/2501938150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a741084e0 msgr2=0x7f5a74101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a78e00700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a741084e0 0x7f5a74101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a78e00700 1 -- 172.21.15.79:0/2501938150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7406a170 msgr2=0x7f5a74101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a78e00700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7406a170 0x7f5a74101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a78e00700 1 -- 172.21.15.79:0/2501938150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a741a7160 con 0x7f5a7406ab10 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.964+0000 7f5a79800700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7406a170 0x7f5a74101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:15:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.965+0000 7f5a78e00700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a7406ab10 0x7f5a741019a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000e9f0 tx=0x7f5a6000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:30.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.965+0000 7f5a72000700 1 -- 172.21.15.79:0/2501938150 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000c9c0 con 0x7f5a7406ab10 2024-06-03T02:15:30.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.965+0000 7f5a72000700 1 -- 172.21.15.79:0/2501938150 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a6000cb60 con 0x7f5a7406ab10 2024-06-03T02:15:30.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.965+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a741a7450 con 0x7f5a7406ab10 2024-06-03T02:15:30.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.965+0000 7f5a72000700 1 -- 172.21.15.79:0/2501938150 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60018680 con 0x7f5a7406ab10 2024-06-03T02:15:30.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.965+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a741a7990 con 0x7f5a7406ab10 2024-06-03T02:15:30.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.966+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a74069180 con 0x7f5a7406ab10 2024-06-03T02:15:30.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.968+0000 7f5a72000700 1 -- 172.21.15.79:0/2501938150 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5a60010430 con 0x7f5a7406ab10 2024-06-03T02:15:30.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.969+0000 7f5a72000700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:30.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.970+0000 7f5a72000700 1 -- 172.21.15.79:0/2501938150 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f5a600cf510 con 0x7f5a7406ab10 2024-06-03T02:15:30.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.970+0000 7f5a79800700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:30.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.970+0000 7f5a79800700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009990 tx=0x7f5a68005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:30.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:30.974+0000 7f5a72000700 1 -- 172.21.15.79:0/2501938150 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a6009c0b0 con 0x7f5a7406ab10 2024-06-03T02:15:31.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:31.252+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f5a7402dca0 con 0x7f5a7406ab10 2024-06-03T02:15:31.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:31 smithi079 ceph-mon[31819]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:31.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:31 smithi135 ceph-mon[36653]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:31.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:31 smithi123 ceph-mon[36640]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:32.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.256+0000 7f5a72000700 1 -- 172.21.15.79:0/2501938150 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f5a6009b9b0 con 0x7f5a7406ab10 2024-06-03T02:15:32.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a5c0a2c20 msgr2=0x7f5a5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:32.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009990 tx=0x7f5a68005ab0 comp rx=0 tx=0).stop 2024-06-03T02:15:32.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a7406ab10 msgr2=0x7f5a741019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:32.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a7406ab10 0x7f5a741019a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000e9f0 tx=0x7f5a6000ed00 comp rx=0 tx=0).stop 2024-06-03T02:15:32.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 shutdown_connections 2024-06-03T02:15:32.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a741084e0 0x7f5a74101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:32.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7406a170 0x7f5a74101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:32.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:32.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 --2- 172.21.15.79:0/2501938150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a7406ab10 0x7f5a741019a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:32.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.261+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 >> 172.21.15.79:0/2501938150 conn(0x7f5a740782c0 msgr2=0x7f5a740ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:32.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.262+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 shutdown_connections 2024-06-03T02:15:32.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:32.262+0000 7f5a7c200700 1 -- 172.21.15.79:0/2501938150 wait complete. 2024-06-03T02:15:32.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:32 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2501938150' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T02:15:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:32 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2501938150' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T02:15:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:32 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2501938150' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T02:15:33.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-03T02:15:33.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:33 smithi079 ceph-mon[31819]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:33 smithi079 ceph-mon[31819]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T02:15:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:33 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2501938150' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-03T02:15:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:33 smithi079 ceph-mon[31819]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T02:15:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:33 smithi079 ceph-mon[31819]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:15:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:33 smithi079 ceph-mon[31819]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-06-03T02:15:33.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:33 smithi079 ceph-mon[31819]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-06-03T02:15:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:33 smithi135 ceph-mon[36653]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:33 smithi135 ceph-mon[36653]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T02:15:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:33 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2501938150' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-03T02:15:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:33 smithi135 ceph-mon[36653]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T02:15:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:33 smithi135 ceph-mon[36653]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:15:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:33 smithi135 ceph-mon[36653]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-06-03T02:15:33.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:33 smithi135 ceph-mon[36653]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-06-03T02:15:33.624 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:33.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:33 smithi123 ceph-mon[36640]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:15:33.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:33 smithi123 ceph-mon[36640]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T02:15:33.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:33 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2501938150' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-03T02:15:33.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:33 smithi123 ceph-mon[36640]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T02:15:33.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:33 smithi123 ceph-mon[36640]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T02:15:33.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:33 smithi123 ceph-mon[36640]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-06-03T02:15:33.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:33 smithi123 ceph-mon[36640]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-06-03T02:15:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:34 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] up:active 2024-06-03T02:15:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:34 smithi079 ceph-mon[31819]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-03T02:15:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:34 smithi079 ceph-mon[31819]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-06-03T02:15:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:34 smithi079 ceph-mon[31819]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-06-03T02:15:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:34 smithi079 ceph-mon[31819]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-03T02:15:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:34 smithi079 ceph-mon[31819]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-06-03T02:15:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:34 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] up:active 2024-06-03T02:15:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:34 smithi135 ceph-mon[36653]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-03T02:15:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:34 smithi135 ceph-mon[36653]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-06-03T02:15:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:34 smithi135 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-06-03T02:15:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:34 smithi135 ceph-mon[36653]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-03T02:15:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:34 smithi135 ceph-mon[36653]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-06-03T02:15:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:34 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] up:active 2024-06-03T02:15:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:34 smithi123 ceph-mon[36640]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-03T02:15:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:34 smithi123 ceph-mon[36640]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-06-03T02:15:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:34 smithi123 ceph-mon[36640]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-06-03T02:15:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:34 smithi123 ceph-mon[36640]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-03T02:15:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:34 smithi123 ceph-mon[36640]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-06-03T02:15:35.414 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:35 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] up:active 2024-06-03T02:15:35.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:35 smithi079 ceph-mon[31819]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-03T02:15:35.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:35 smithi079 ceph-mon[31819]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-06-03T02:15:35.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:35 smithi079 ceph-mon[31819]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-06-03T02:15:35.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:35 smithi079 ceph-mon[31819]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-06-03T02:15:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:35 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] up:active 2024-06-03T02:15:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:35 smithi135 ceph-mon[36653]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-03T02:15:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:35 smithi135 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-06-03T02:15:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:35 smithi135 ceph-mon[36653]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-06-03T02:15:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:35 smithi135 ceph-mon[36653]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-06-03T02:15:35.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:35 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] up:active 2024-06-03T02:15:35.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:35 smithi123 ceph-mon[36640]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-03T02:15:35.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:35 smithi123 ceph-mon[36640]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-06-03T02:15:35.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:35 smithi123 ceph-mon[36640]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-06-03T02:15:35.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:35 smithi123 ceph-mon[36640]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-06-03T02:15:35.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.930+0000 7f0abf000700 1 -- 172.21.15.79:0/2500446942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8107840 msgr2=0x7f0ab810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:35.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.930+0000 7f0abf000700 1 --2- 172.21.15.79:0/2500446942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8107840 0x7f0ab810f3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000b0d0 tx=0x7f0ab000b3e0 comp rx=0 tx=0).stop 2024-06-03T02:15:35.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.931+0000 7f0abf000700 1 -- 172.21.15.79:0/2500446942 shutdown_connections 2024-06-03T02:15:35.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.931+0000 7f0abf000700 1 --2- 172.21.15.79:0/2500446942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8107840 0x7f0ab810f3c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:35.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.931+0000 7f0abf000700 1 --2- 172.21.15.79:0/2500446942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8102740 0x7f0ab8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:35.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.931+0000 7f0abf000700 1 --2- 172.21.15.79:0/2500446942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ab81030f0 0x7f0ab8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:35.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.931+0000 7f0abf000700 1 -- 172.21.15.79:0/2500446942 >> 172.21.15.79:0/2500446942 conn(0x7f0ab80795f0 msgr2=0x7f0ab8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:35.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.931+0000 7f0abf000700 1 -- 172.21.15.79:0/2500446942 shutdown_connections 2024-06-03T02:15:35.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.931+0000 7f0abf000700 1 -- 172.21.15.79:0/2500446942 wait complete. 2024-06-03T02:15:35.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.932+0000 7f0abf000700 1 Processor -- start 2024-06-03T02:15:35.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.932+0000 7f0abf000700 1 -- start start 2024-06-03T02:15:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.932+0000 7f0abf000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8102740 0x7f0ab819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.932+0000 7f0abf000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ab81030f0 0x7f0ab819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.932+0000 7f0abf000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8107840 0x7f0ab819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0abf000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab81127b0 con 0x7f0ab8102740 2024-06-03T02:15:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0abf000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8112630 con 0x7f0ab8107840 2024-06-03T02:15:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0abf000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8112930 con 0x7f0ab81030f0 2024-06-03T02:15:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0ab7e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8102740 0x7f0ab819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0ab7e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8102740 0x7f0ab819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58376/0 (socket says 172.21.15.79:58376) 2024-06-03T02:15:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0ab7e00700 1 -- 172.21.15.79:0/2093108504 learned_addr learned my addr 172.21.15.79:0/2093108504 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0ab7400700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ab81030f0 0x7f0ab819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0abd000700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8107840 0x7f0ab819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0ab7e00700 1 -- 172.21.15.79:0/2093108504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ab81030f0 msgr2=0x7f0ab819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0ab7e00700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ab81030f0 0x7f0ab819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0ab7e00700 1 -- 172.21.15.79:0/2093108504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8107840 msgr2=0x7f0ab819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0ab7e00700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8107840 0x7f0ab819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0ab7e00700 1 -- 172.21.15.79:0/2093108504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab819fd50 con 0x7f0ab8102740 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.933+0000 7f0abd000700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8107840 0x7f0ab819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.934+0000 7f0ab7e00700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8102740 0x7f0ab819ad40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800e9a0 tx=0x7f0aa800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.934+0000 7f0ab4c00700 1 -- 172.21.15.79:0/2093108504 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa800cbd0 con 0x7f0ab8102740 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.934+0000 7f0ab4c00700 1 -- 172.21.15.79:0/2093108504 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aa800cd70 con 0x7f0ab8102740 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.934+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab81a0040 con 0x7f0ab8102740 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.934+0000 7f0ab4c00700 1 -- 172.21.15.79:0/2093108504 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8018750 con 0x7f0ab8102740 2024-06-03T02:15:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.934+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab81a78c0 con 0x7f0ab8102740 2024-06-03T02:15:35.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.935+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab8069180 con 0x7f0ab8102740 2024-06-03T02:15:35.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.937+0000 7f0ab4c00700 1 -- 172.21.15.79:0/2093108504 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0aa8010430 con 0x7f0ab8102740 2024-06-03T02:15:35.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.939+0000 7f0ab4c00700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a9c0a2c20 0x7f0a9c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:35.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.939+0000 7f0ab4c00700 1 -- 172.21.15.79:0/2093108504 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f0aa80cfb70 con 0x7f0ab8102740 2024-06-03T02:15:35.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.939+0000 7f0ab7400700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a9c0a2c20 0x7f0a9c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:35.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.940+0000 7f0ab7400700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a9c0a2c20 0x7f0a9c0a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009680 tx=0x7f0aa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:35.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:35.942+0000 7f0ab4c00700 1 -- 172.21.15.79:0/2093108504 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0aa809c710 con 0x7f0ab8102740 2024-06-03T02:15:36.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.219+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0ab81a7a90 con 0x7f0ab8102740 2024-06-03T02:15:36.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.305+0000 7f0ab4c00700 1 -- 172.21.15.79:0/2093108504 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f0aa809c010 con 0x7f0ab8102740 2024-06-03T02:15:36.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a9c0a2c20 msgr2=0x7f0a9c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:36.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a9c0a2c20 0x7f0a9c0a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009680 tx=0x7f0aa4009260 comp rx=0 tx=0).stop 2024-06-03T02:15:36.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8102740 msgr2=0x7f0ab819ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:36.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8102740 0x7f0ab819ad40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800e9a0 tx=0x7f0aa800ecb0 comp rx=0 tx=0).stop 2024-06-03T02:15:36.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 shutdown_connections 2024-06-03T02:15:36.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8107840 0x7f0ab819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:36.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a9c0a2c20 0x7f0a9c0a50e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:36.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ab8102740 0x7f0ab819ad40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:36.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 --2- 172.21.15.79:0/2093108504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ab81030f0 0x7f0ab819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:36.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 >> 172.21.15.79:0/2093108504 conn(0x7f0ab80795f0 msgr2=0x7f0ab81006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:36.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 shutdown_connections 2024-06-03T02:15:36.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:36.309+0000 7f0abf000700 1 -- 172.21.15.79:0/2093108504 wait complete. 2024-06-03T02:15:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:36 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] up:active 2024-06-03T02:15:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:36 smithi079 ceph-mon[31819]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-03T02:15:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:36 smithi079 ceph-mon[31819]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T02:15:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:36 smithi079 ceph-mon[31819]: Cluster is now healthy 2024-06-03T02:15:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:36 smithi079 ceph-mon[31819]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-06-03T02:15:36.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:36 smithi079 ceph-mon[31819]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-03T02:15:36.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:36 smithi079 ceph-mon[31819]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-06-03T02:15:36.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:36 smithi079 ceph-mon[31819]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-06-03T02:15:36.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:36 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2093108504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T02:15:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:36 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] up:active 2024-06-03T02:15:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:36 smithi135 ceph-mon[36653]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-03T02:15:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:36 smithi135 ceph-mon[36653]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T02:15:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:36 smithi135 ceph-mon[36653]: Cluster is now healthy 2024-06-03T02:15:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:36 smithi135 ceph-mon[36653]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-06-03T02:15:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:36 smithi135 ceph-mon[36653]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-03T02:15:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:36 smithi135 ceph-mon[36653]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-06-03T02:15:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:36 smithi135 ceph-mon[36653]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-06-03T02:15:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:36 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2093108504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T02:15:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:36 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] up:active 2024-06-03T02:15:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:36 smithi123 ceph-mon[36640]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-03T02:15:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:36 smithi123 ceph-mon[36640]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T02:15:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:36 smithi123 ceph-mon[36640]: Cluster is now healthy 2024-06-03T02:15:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:36 smithi123 ceph-mon[36640]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-06-03T02:15:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:36 smithi123 ceph-mon[36640]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-03T02:15:36.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:36 smithi123 ceph-mon[36640]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-06-03T02:15:36.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:36 smithi123 ceph-mon[36640]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-06-03T02:15:36.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:36 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2093108504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T02:15:37.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-03T02:15:37.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:37 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2093108504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-03T02:15:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:37 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] up:active 2024-06-03T02:15:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:37 smithi079 ceph-mon[31819]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T02:15:37.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:37 smithi079 ceph-mon[31819]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T02:15:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:37 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2093108504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-03T02:15:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:37 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] up:active 2024-06-03T02:15:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:37 smithi135 ceph-mon[36653]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T02:15:37.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:37 smithi135 ceph-mon[36653]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T02:15:37.691 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:37.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:37 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2093108504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-03T02:15:37.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:37 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] up:active 2024-06-03T02:15:37.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:37 smithi123 ceph-mon[36640]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T02:15:37.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:37 smithi123 ceph-mon[36640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T02:15:38.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:38 smithi079 ceph-mon[31819]: pgmap v382: 129 pgs: 129 active+clean; 586 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-06-03T02:15:38.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:38 smithi135 ceph-mon[36653]: pgmap v382: 129 pgs: 129 active+clean; 586 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-06-03T02:15:38.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:38 smithi123 ceph-mon[36640]: pgmap v382: 129 pgs: 129 active+clean; 586 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-06-03T02:15:39.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.739+0000 7f390c800700 1 -- 172.21.15.79:0/1913045792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f390410eac0 msgr2=0x7f390410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:39.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.739+0000 7f390c800700 1 --2- 172.21.15.79:0/1913045792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f390410eac0 0x7f390410eea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009990 tx=0x7f38f8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:15:39.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.739+0000 7f390c800700 1 -- 172.21.15.79:0/1913045792 shutdown_connections 2024-06-03T02:15:39.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.739+0000 7f390c800700 1 --2- 172.21.15.79:0/1913045792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3904109480 0x7f3904115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:39.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.739+0000 7f390c800700 1 --2- 172.21.15.79:0/1913045792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f390410eac0 0x7f390410eea0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:39.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.739+0000 7f390c800700 1 --2- 172.21.15.79:0/1913045792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3904108ac0 0x7f3904108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:39.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.739+0000 7f390c800700 1 -- 172.21.15.79:0/1913045792 >> 172.21.15.79:0/1913045792 conn(0x7f3904075c60 msgr2=0x7f39041072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:39.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.739+0000 7f390c800700 1 -- 172.21.15.79:0/1913045792 shutdown_connections 2024-06-03T02:15:39.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.739+0000 7f390c800700 1 -- 172.21.15.79:0/1913045792 wait complete. 2024-06-03T02:15:39.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.740+0000 7f390c800700 1 Processor -- start 2024-06-03T02:15:39.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.740+0000 7f390c800700 1 -- start start 2024-06-03T02:15:39.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f390c800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3904108ac0 0x7f390419f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:39.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f390c800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3904109480 0x7f390419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:39.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f390c800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f390410eac0 0x7f39041a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:39.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f390c800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3904118e40 con 0x7f390410eac0 2024-06-03T02:15:39.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f390c800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3904118cc0 con 0x7f3904108ac0 2024-06-03T02:15:39.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f390c800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3904118fc0 con 0x7f3904109480 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f390a800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f390410eac0 0x7f39041a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f3909e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3904108ac0 0x7f390419f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f390a800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f390410eac0 0x7f39041a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58398/0 (socket says 172.21.15.79:58398) 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f3909e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3904108ac0 0x7f390419f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53398/0 (socket says 172.21.15.79:53398) 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f390a800700 1 -- 172.21.15.79:0/1656131998 learned_addr learned my addr 172.21.15.79:0/1656131998 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.741+0000 7f3909400700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3904109480 0x7f390419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.742+0000 7f3909e00700 1 -- 172.21.15.79:0/1656131998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3904109480 msgr2=0x7f390419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.742+0000 7f3909e00700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3904109480 0x7f390419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.742+0000 7f3909e00700 1 -- 172.21.15.79:0/1656131998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f390410eac0 msgr2=0x7f39041a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.742+0000 7f3909e00700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f390410eac0 0x7f39041a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:39.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.742+0000 7f3909e00700 1 -- 172.21.15.79:0/1656131998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39041a4290 con 0x7f3904108ac0 2024-06-03T02:15:39.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.742+0000 7f3909e00700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3904108ac0 0x7f390419f2d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f38f80098e0 tx=0x7f38f800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:39.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.742+0000 7f38fea00700 1 -- 172.21.15.79:0/1656131998 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8027070 con 0x7f3904108ac0 2024-06-03T02:15:39.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.742+0000 7f38fea00700 1 -- 172.21.15.79:0/1656131998 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f8003b60 con 0x7f3904108ac0 2024-06-03T02:15:39.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.742+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39041a4520 con 0x7f3904108ac0 2024-06-03T02:15:39.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.743+0000 7f38fea00700 1 -- 172.21.15.79:0/1656131998 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8022d50 con 0x7f3904108ac0 2024-06-03T02:15:39.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.743+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39041abe00 con 0x7f3904108ac0 2024-06-03T02:15:39.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.744+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390410a200 con 0x7f3904108ac0 2024-06-03T02:15:39.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.746+0000 7f38fea00700 1 -- 172.21.15.79:0/1656131998 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f38f802a070 con 0x7f3904108ac0 2024-06-03T02:15:39.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.747+0000 7f38fea00700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f38ec0a2c20 0x7f38ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:39.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.747+0000 7f38fea00700 1 -- 172.21.15.79:0/1656131998 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f38f80d0ac0 con 0x7f3904108ac0 2024-06-03T02:15:39.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.747+0000 7f3909400700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f38ec0a2c20 0x7f38ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:39.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.748+0000 7f3909400700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f38ec0a2c20 0x7f38ec0a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3904077b60 tx=0x7f38f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:39.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:39.749+0000 7f38fea00700 1 -- 172.21.15.79:0/1656131998 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38f8031020 con 0x7f3904108ac0 2024-06-03T02:15:40.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.026+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f390402dca0 con 0x7f3904108ac0 2024-06-03T02:15:40.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.085+0000 7f38fea00700 1 -- 172.21.15.79:0/1656131998 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f38f8013070 con 0x7f3904108ac0 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f38ec0a2c20 msgr2=0x7f38ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f38ec0a2c20 0x7f38ec0a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3904077b60 tx=0x7f38f4009260 comp rx=0 tx=0).stop 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3904108ac0 msgr2=0x7f390419f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3904108ac0 0x7f390419f2d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f38f80098e0 tx=0x7f38f800b8a0 comp rx=0 tx=0).stop 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 shutdown_connections 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f38ec0a2c20 0x7f38ec0a50e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f390410eac0 0x7f39041a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3904108ac0 0x7f390419f2d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 --2- 172.21.15.79:0/1656131998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3904109480 0x7f390419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 >> 172.21.15.79:0/1656131998 conn(0x7f3904075c60 msgr2=0x7f3904076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 shutdown_connections 2024-06-03T02:15:40.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:40.088+0000 7f390c800700 1 -- 172.21.15.79:0/1656131998 wait complete. 2024-06-03T02:15:40.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:40 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1656131998' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T02:15:40.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:40 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T02:15:40.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:40 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1656131998' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T02:15:40.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:40 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T02:15:40.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:40 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1656131998' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T02:15:40.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:40 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T02:15:41.013 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-06-03T02:15:41.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-03T02:15:41.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:41 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-03T02:15:41.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:41 smithi135 ceph-mon[36653]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T02:15:41.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:41 smithi135 ceph-mon[36653]: pgmap v383: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-06-03T02:15:41.460 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:41.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:41 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-03T02:15:41.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:41 smithi123 ceph-mon[36640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T02:15:41.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:41 smithi123 ceph-mon[36640]: pgmap v383: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-06-03T02:15:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:41 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-03T02:15:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:41 smithi079 ceph-mon[31819]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T02:15:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:41 smithi079 ceph-mon[31819]: pgmap v383: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-06-03T02:15:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:42 smithi079 ceph-mon[31819]: pgmap v384: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-03T02:15:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:42 smithi135 ceph-mon[36653]: pgmap v384: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-03T02:15:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:42 smithi123 ceph-mon[36640]: pgmap v384: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-03T02:15:43.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.678+0000 7f7c0c800700 1 -- 172.21.15.79:0/3368741698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c0410ee90 msgr2=0x7f7c04111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:43.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.678+0000 7f7c0c800700 1 --2- 172.21.15.79:0/3368741698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c0410ee90 0x7f7c04111280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b0a0 tx=0x7f7c0000b3b0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.679+0000 7f7c0c800700 1 -- 172.21.15.79:0/3368741698 shutdown_connections 2024-06-03T02:15:43.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.679+0000 7f7c0c800700 1 --2- 172.21.15.79:0/3368741698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c0410ee90 0x7f7c04111280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.679+0000 7f7c0c800700 1 --2- 172.21.15.79:0/3368741698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c040746c0 0x7f7c04074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.679+0000 7f7c0c800700 1 --2- 172.21.15.79:0/3368741698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c04074fe0 0x7f7c0410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.679+0000 7f7c0c800700 1 -- 172.21.15.79:0/3368741698 >> 172.21.15.79:0/3368741698 conn(0x7f7c040795f0 msgr2=0x7f7c04079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.679+0000 7f7c0c800700 1 -- 172.21.15.79:0/3368741698 shutdown_connections 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.679+0000 7f7c0c800700 1 -- 172.21.15.79:0/3368741698 wait complete. 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.680+0000 7f7c0c800700 1 Processor -- start 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.680+0000 7f7c0c800700 1 -- start start 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.680+0000 7f7c0c800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c040746c0 0x7f7c0419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.680+0000 7f7c0c800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c04074fe0 0x7f7c0419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c0c800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0410ee90 0x7f7c0419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c0c800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04114a10 con 0x7f7c040746c0 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c0c800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04114890 con 0x7f7c04074fe0 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c0c800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04114b90 con 0x7f7c0410ee90 2024-06-03T02:15:43.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c09400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c04074fe0 0x7f7c0419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c09400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c04074fe0 0x7f7c0419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53418/0 (socket says 172.21.15.79:53418) 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c09400700 1 -- 172.21.15.79:0/41764631 learned_addr learned my addr 172.21.15.79:0/41764631 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c09e00700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c040746c0 0x7f7c0419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c0a800700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0410ee90 0x7f7c0419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c09400700 1 -- 172.21.15.79:0/41764631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0410ee90 msgr2=0x7f7c0419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c09400700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0410ee90 0x7f7c0419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c09400700 1 -- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c040746c0 msgr2=0x7f7c0419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c09400700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c040746c0 0x7f7c0419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.681+0000 7f7c09400700 1 -- 172.21.15.79:0/41764631 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0419fe20 con 0x7f7c04074fe0 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.682+0000 7f7c09400700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c04074fe0 0x7f7c0419b3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400e9c0 tx=0x7f7bf400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:43.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.682+0000 7f7bfea00700 1 -- 172.21.15.79:0/41764631 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400cbd0 con 0x7f7c04074fe0 2024-06-03T02:15:43.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.682+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c041a0110 con 0x7f7c04074fe0 2024-06-03T02:15:43.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.682+0000 7f7bfea00700 1 -- 172.21.15.79:0/41764631 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf400ee60 con 0x7f7c04074fe0 2024-06-03T02:15:43.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.682+0000 7f7bfea00700 1 -- 172.21.15.79:0/41764631 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4005630 con 0x7f7c04074fe0 2024-06-03T02:15:43.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.682+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c041a7990 con 0x7f7c04074fe0 2024-06-03T02:15:43.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.683+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c04069180 con 0x7f7c04074fe0 2024-06-03T02:15:43.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.686+0000 7f7bfea00700 1 -- 172.21.15.79:0/41764631 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7bf4044b30 con 0x7f7c04074fe0 2024-06-03T02:15:43.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.687+0000 7f7bfea00700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7bec0a2c20 0x7f7bec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:43.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.687+0000 7f7bfea00700 1 -- 172.21.15.79:0/41764631 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f7bf40cf600 con 0x7f7c04074fe0 2024-06-03T02:15:43.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.687+0000 7f7c09e00700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7bec0a2c20 0x7f7bec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:43.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.688+0000 7f7c09e00700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7bec0a2c20 0x7f7bec0a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009c60 tx=0x7f7bf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:43.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.689+0000 7f7bfea00700 1 -- 172.21.15.79:0/41764631 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7bf4044e20 con 0x7f7c04074fe0 2024-06-03T02:15:43.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.946+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7c040ffd10 con 0x7f7bec0a2c20 2024-06-03T02:15:43.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.988+0000 7f7bfea00700 1 -- 172.21.15.79:0/41764631 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c040ffd10 con 0x7f7bec0a2c20 2024-06-03T02:15:43.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7bec0a2c20 msgr2=0x7f7bec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:43.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7bec0a2c20 0x7f7bec0a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009c60 tx=0x7f7bf8009260 comp rx=0 tx=0).stop 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c04074fe0 msgr2=0x7f7c0419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c04074fe0 0x7f7c0419b3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400e9c0 tx=0x7f7bf400ecd0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 shutdown_connections 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0410ee90 0x7f7c0419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7bec0a2c20 0x7f7bec0a50e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c040746c0 0x7f7c0419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 --2- 172.21.15.79:0/41764631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c04074fe0 0x7f7c0419b3a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.991+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 >> 172.21.15.79:0/41764631 conn(0x7f7c040795f0 msgr2=0x7f7c040ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.992+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 shutdown_connections 2024-06-03T02:15:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:43.992+0000 7f7c0c800700 1 -- 172.21.15.79:0/41764631 wait complete. 2024-06-03T02:15:44.301 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:15:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:15:43.957+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:15:44.302 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:15:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:15:43.957+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:15:44.302 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:15:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:15:43.957+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:15:44.302 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:15:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:15:43.957+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:15:44.302 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:15:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:15:43.957+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:15:44.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:44 smithi079 ceph-mon[31819]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:44.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:44 smithi079 ceph-mon[31819]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-06-03T02:15:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:44 smithi135 ceph-mon[36653]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:44 smithi135 ceph-mon[36653]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-06-03T02:15:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:44 smithi123 ceph-mon[36640]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:44 smithi123 ceph-mon[36640]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-06-03T02:15:44.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-03T02:15:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:46.551 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:46 smithi079 ceph-mon[31819]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-03T02:15:46.551 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:46 smithi079 ceph-mon[31819]: pgmap v386: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-06-03T02:15:46.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:46 smithi135 ceph-mon[36653]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-03T02:15:46.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:46 smithi135 ceph-mon[36653]: pgmap v386: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-06-03T02:15:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:46 smithi123 ceph-mon[36640]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-03T02:15:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:46 smithi123 ceph-mon[36640]: pgmap v386: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-06-03T02:15:47.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.455+0000 7f3fd1e00700 1 -- 172.21.15.79:0/4122476776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fcc107840 msgr2=0x7f3fcc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:47.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.455+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/4122476776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fcc107840 0x7f3fcc10f3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009fc0 tx=0x7f3fc400b330 comp rx=0 tx=0).stop 2024-06-03T02:15:47.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.455+0000 7f3fd1e00700 1 -- 172.21.15.79:0/4122476776 shutdown_connections 2024-06-03T02:15:47.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.455+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/4122476776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fcc107840 0x7f3fcc10f3c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.455+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/4122476776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fcc102740 0x7f3fcc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.455+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/4122476776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc1030f0 0x7f3fcc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.455+0000 7f3fd1e00700 1 -- 172.21.15.79:0/4122476776 >> 172.21.15.79:0/4122476776 conn(0x7f3fcc0795f0 msgr2=0x7f3fcc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.456+0000 7f3fd1e00700 1 -- 172.21.15.79:0/4122476776 shutdown_connections 2024-06-03T02:15:47.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.456+0000 7f3fd1e00700 1 -- 172.21.15.79:0/4122476776 wait complete. 2024-06-03T02:15:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.457+0000 7f3fd1e00700 1 Processor -- start 2024-06-03T02:15:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.457+0000 7f3fd1e00700 1 -- start start 2024-06-03T02:15:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.457+0000 7f3fd1e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102740 0x7f3fcc19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.457+0000 7f3fd1e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fcc1030f0 0x7f3fcc19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.457+0000 7f3fd1e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fcc107840 0x7f3fcc19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.457+0000 7f3fd1e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc1127b0 con 0x7f3fcc107840 2024-06-03T02:15:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.457+0000 7f3fd1e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc112630 con 0x7f3fcc102740 2024-06-03T02:15:47.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.457+0000 7f3fd1e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc112930 con 0x7f3fcc1030f0 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fc2000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fcc1030f0 0x7f3fcc19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fcaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102740 0x7f3fcc19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fc2000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fcc1030f0 0x7f3fcc19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54182/0 (socket says 172.21.15.79:54182) 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fc2000700 1 -- 172.21.15.79:0/556978403 learned_addr learned my addr 172.21.15.79:0/556978403 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fcb400700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fcc107840 0x7f3fcc19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fc2000700 1 -- 172.21.15.79:0/556978403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102740 msgr2=0x7f3fcc19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fc2000700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102740 0x7f3fcc19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fc2000700 1 -- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fcc107840 msgr2=0x7f3fcc19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fc2000700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fcc107840 0x7f3fcc19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fc2000700 1 -- 172.21.15.79:0/556978403 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc19fd70 con 0x7f3fcc1030f0 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fcb400700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fcc107840 0x7f3fcc19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.458+0000 7f3fc2000700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fcc1030f0 0x7f3fcc19b2a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e9a0 tx=0x7f3fb400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.459+0000 7f3fd0a00700 1 -- 172.21.15.79:0/556978403 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb400cbd0 con 0x7f3fcc1030f0 2024-06-03T02:15:47.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.459+0000 7f3fd0a00700 1 -- 172.21.15.79:0/556978403 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb400cd70 con 0x7f3fcc1030f0 2024-06-03T02:15:47.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.459+0000 7f3fd0a00700 1 -- 172.21.15.79:0/556978403 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4018750 con 0x7f3fcc1030f0 2024-06-03T02:15:47.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.459+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fcc1a0060 con 0x7f3fcc1030f0 2024-06-03T02:15:47.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.459+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fcc1a78e0 con 0x7f3fcc1030f0 2024-06-03T02:15:47.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.460+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fcc069180 con 0x7f3fcc1030f0 2024-06-03T02:15:47.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.462+0000 7f3fd0a00700 1 -- 172.21.15.79:0/556978403 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3fb4010430 con 0x7f3fcc1030f0 2024-06-03T02:15:47.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.463+0000 7f3fd0a00700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fb00a2d30 0x7f3fb00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.464+0000 7f3fd0a00700 1 -- 172.21.15.79:0/556978403 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f3fb40cfc70 con 0x7f3fcc1030f0 2024-06-03T02:15:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.464+0000 7f3fcaa00700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fb00a2d30 0x7f3fb00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:47.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.464+0000 7f3fcaa00700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fb00a2d30 0x7f3fb00a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009750 tx=0x7f3fbc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:47.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.466+0000 7f3fd0a00700 1 -- 172.21.15.79:0/556978403 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3fb409c6b0 con 0x7f3fcc1030f0 2024-06-03T02:15:47.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.720+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3fcc0756d0 con 0x7f3fb00a2d30 2024-06-03T02:15:47.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.754+0000 7f3fd0a00700 1 -- 172.21.15.79:0/556978403 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3fcc0756d0 con 0x7f3fb00a2d30 2024-06-03T02:15:47.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fb00a2d30 msgr2=0x7f3fb00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:47.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fb00a2d30 0x7f3fb00a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009750 tx=0x7f3fbc009310 comp rx=0 tx=0).stop 2024-06-03T02:15:47.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fcc1030f0 msgr2=0x7f3fcc19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:47.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fcc1030f0 0x7f3fcc19b2a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e9a0 tx=0x7f3fb400ecb0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 shutdown_connections 2024-06-03T02:15:47.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3fb00a2d30 0x7f3fb00a51f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fcc107840 0x7f3fcc19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102740 0x7f3fcc19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 --2- 172.21.15.79:0/556978403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fcc1030f0 0x7f3fcc19b2a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:47.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 >> 172.21.15.79:0/556978403 conn(0x7f3fcc0795f0 msgr2=0x7f3fcc1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:47.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.758+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 shutdown_connections 2024-06-03T02:15:47.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:47.759+0000 7f3fd1e00700 1 -- 172.21.15.79:0/556978403 wait complete. 2024-06-03T02:15:48.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:48 smithi079 ceph-mon[31819]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:48.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:48 smithi079 ceph-mon[31819]: pgmap v387: 129 pgs: 129 active+clean; 588 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.1 KiB/s wr, 22 op/s 2024-06-03T02:15:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:48 smithi135 ceph-mon[36653]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:48 smithi135 ceph-mon[36653]: pgmap v387: 129 pgs: 129 active+clean; 588 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.1 KiB/s wr, 22 op/s 2024-06-03T02:15:48.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-03T02:15:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:48 smithi123 ceph-mon[36640]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:48 smithi123 ceph-mon[36640]: pgmap v387: 129 pgs: 129 active+clean; 588 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.1 KiB/s wr, 22 op/s 2024-06-03T02:15:49.165 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:50.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:50 smithi079 ceph-mon[31819]: pgmap v388: 129 pgs: 129 active+clean; 608 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-06-03T02:15:50.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:50 smithi135 ceph-mon[36653]: pgmap v388: 129 pgs: 129 active+clean; 608 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-06-03T02:15:50.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:50 smithi123 ceph-mon[36640]: pgmap v388: 129 pgs: 129 active+clean; 608 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-06-03T02:15:51.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.352+0000 7f1c22800700 1 -- 172.21.15.79:0/1848142245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 msgr2=0x7f1c1c10aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:51.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.352+0000 7f1c22800700 1 --2- 172.21.15.79:0/1848142245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 0x7f1c1c10aa50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c100099c0 tx=0x7f1c10009cd0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.352+0000 7f1c22800700 1 -- 172.21.15.79:0/1848142245 shutdown_connections 2024-06-03T02:15:51.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.352+0000 7f1c22800700 1 --2- 172.21.15.79:0/1848142245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c1c104f80 0x7f1c1c111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.352+0000 7f1c22800700 1 --2- 172.21.15.79:0/1848142245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 0x7f1c1c10aa50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.352+0000 7f1c22800700 1 --2- 172.21.15.79:0/1848142245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c1c1045c0 0x7f1c1c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.352+0000 7f1c22800700 1 -- 172.21.15.79:0/1848142245 >> 172.21.15.79:0/1848142245 conn(0x7f1c1c1000d0 msgr2=0x7f1c1c1024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:51.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.353+0000 7f1c22800700 1 -- 172.21.15.79:0/1848142245 shutdown_connections 2024-06-03T02:15:51.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.353+0000 7f1c22800700 1 -- 172.21.15.79:0/1848142245 wait complete. 2024-06-03T02:15:51.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.354+0000 7f1c22800700 1 Processor -- start 2024-06-03T02:15:51.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.354+0000 7f1c22800700 1 -- start start 2024-06-03T02:15:51.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.354+0000 7f1c22800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c1c1045c0 0x7f1c1c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:51.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.354+0000 7f1c22800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c1c104f80 0x7f1c1c1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:51.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.354+0000 7f1c22800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 0x7f1c1c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:51.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.354+0000 7f1c22800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1148e0 con 0x7f1c1c1045c0 2024-06-03T02:15:51.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.354+0000 7f1c22800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c114760 con 0x7f1c1c10a670 2024-06-03T02:15:51.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.354+0000 7f1c22800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c114a60 con 0x7f1c1c104f80 2024-06-03T02:15:51.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 0x7f1c1c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:51.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c1c1045c0 0x7f1c1c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:51.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 0x7f1c1c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47232/0 (socket says 172.21.15.79:47232) 2024-06-03T02:15:51.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21e00700 1 -- 172.21.15.79:0/1969247930 learned_addr learned my addr 172.21.15.79:0/1969247930 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:51.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c20a00700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c1c104f80 0x7f1c1c1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21e00700 1 -- 172.21.15.79:0/1969247930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c1c104f80 msgr2=0x7f1c1c1a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21e00700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c1c104f80 0x7f1c1c1a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21e00700 1 -- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c1c1045c0 msgr2=0x7f1c1c1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21e00700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c1c1045c0 0x7f1c1c1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21e00700 1 -- 172.21.15.79:0/1969247930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c19b400 con 0x7f1c1c10a670 2024-06-03T02:15:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.355+0000 7f1c21400700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c1c1045c0 0x7f1c1c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:15:51.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.356+0000 7f1c21e00700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 0x7f1c1c19ab20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ef30 tx=0x7f1c1400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.356+0000 7f1c1a000700 1 -- 172.21.15.79:0/1969247930 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140092e0 con 0x7f1c1c10a670 2024-06-03T02:15:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.356+0000 7f1c1a000700 1 -- 172.21.15.79:0/1969247930 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c14004bb0 con 0x7f1c1c10a670 2024-06-03T02:15:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.356+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c19b690 con 0x7f1c1c10a670 2024-06-03T02:15:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.356+0000 7f1c1a000700 1 -- 172.21.15.79:0/1969247930 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14004280 con 0x7f1c1c10a670 2024-06-03T02:15:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.356+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1a79d0 con 0x7f1c1c10a670 2024-06-03T02:15:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.357+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1c10cb00 con 0x7f1c1c10a670 2024-06-03T02:15:51.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.359+0000 7f1c1a000700 1 -- 172.21.15.79:0/1969247930 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1c1401b030 con 0x7f1c1c10a670 2024-06-03T02:15:51.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.361+0000 7f1c1a000700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1c040a2d30 0x7f1c040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:51.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.361+0000 7f1c1a000700 1 -- 172.21.15.79:0/1969247930 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9092+0+0 (secure 0 0 0) 0x7f1c140cef50 con 0x7f1c1c10a670 2024-06-03T02:15:51.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.361+0000 7f1c21400700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1c040a2d30 0x7f1c040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:51.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.362+0000 7f1c21400700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1c040a2d30 0x7f1c040a51f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000c00 tx=0x7f1c10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:51.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.365+0000 7f1c1a000700 1 -- 172.21.15.79:0/1969247930 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1c14022020 con 0x7f1c1c10a670 2024-06-03T02:15:51.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.629+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1c1c19bfe0 con 0x7f1c040a2d30 2024-06-03T02:15:51.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.639+0000 7f1c1a000700 1 -- 172.21.15.79:0/1969247930 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1c1c19bfe0 con 0x7f1c040a2d30 2024-06-03T02:15:51.639 INFO:teuthology.orchestra.run.smithi079.stdout:/volumes/qa/sv_0/37d21a4a-14ff-4be9-898b-e228cf1a5092 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.641+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1c040a2d30 msgr2=0x7f1c040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.641+0000 7f1c22800700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1c040a2d30 0x7f1c040a51f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000c00 tx=0x7f1c10005fb0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 msgr2=0x7f1c1c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 0x7f1c1c19ab20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400ef30 tx=0x7f1c1400c3a0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 shutdown_connections 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1c10a670 0x7f1c1c19ab20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1c040a2d30 0x7f1c040a51f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c1c1045c0 0x7f1c1c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 --2- 172.21.15.79:0/1969247930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c1c104f80 0x7f1c1c1a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 >> 172.21.15.79:0/1969247930 conn(0x7f1c1c1000d0 msgr2=0x7f1c1c1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 shutdown_connections 2024-06-03T02:15:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:51.642+0000 7f1c22800700 1 -- 172.21.15.79:0/1969247930 wait complete. 2024-06-03T02:15:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:52 smithi079 ceph-mon[31819]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:52 smithi079 ceph-mon[31819]: pgmap v389: 129 pgs: 129 active+clean; 608 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-06-03T02:15:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-03T02:15:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-06-03T02:15:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-06-03T02:15:52.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-03T02:15:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:52 smithi135 ceph-mon[36653]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:52 smithi135 ceph-mon[36653]: pgmap v389: 129 pgs: 129 active+clean; 608 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-06-03T02:15:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-03T02:15:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-06-03T02:15:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-06-03T02:15:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:52 smithi123 ceph-mon[36640]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:52 smithi123 ceph-mon[36640]: pgmap v389: 129 pgs: 129 active+clean; 608 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-06-03T02:15:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-03T02:15:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-06-03T02:15:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-06-03T02:15:52.889 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:54.498 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-03T02:15:54.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-06-03T02:15:54.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:54 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-06-03T02:15:54.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:54 smithi079 ceph-mon[31819]: osdmap e100: 12 total, 12 up, 12 in 2024-06-03T02:15:54.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:54 smithi079 ceph-mon[31819]: pgmap v391: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-06-03T02:15:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-03T02:15:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-06-03T02:15:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:54 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-06-03T02:15:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:54 smithi135 ceph-mon[36653]: osdmap e100: 12 total, 12 up, 12 in 2024-06-03T02:15:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:54 smithi135 ceph-mon[36653]: pgmap v391: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-06-03T02:15:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-03T02:15:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-06-03T02:15:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:54 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-06-03T02:15:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:54 smithi123 ceph-mon[36640]: osdmap e100: 12 total, 12 up, 12 in 2024-06-03T02:15:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:54 smithi123 ceph-mon[36640]: pgmap v391: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-06-03T02:15:54.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.996+0000 7faa0e000700 1 -- 172.21.15.79:0/301568197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa08105070 msgr2=0x7faa08111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:54.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.996+0000 7faa0e000700 1 --2- 172.21.15.79:0/301568197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa08105070 0x7faa08111580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa00009990 tx=0x7faa00009ca0 comp rx=0 tx=0).stop 2024-06-03T02:15:54.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.997+0000 7faa0e000700 1 -- 172.21.15.79:0/301568197 shutdown_connections 2024-06-03T02:15:54.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.997+0000 7faa0e000700 1 --2- 172.21.15.79:0/301568197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa08105070 0x7faa08111580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:54.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.997+0000 7faa0e000700 1 --2- 172.21.15.79:0/301568197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0810a6b0 0x7faa0810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:54.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.997+0000 7faa0e000700 1 --2- 172.21.15.79:0/301568197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa081046b0 0x7faa08104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:54.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.997+0000 7faa0e000700 1 -- 172.21.15.79:0/301568197 >> 172.21.15.79:0/301568197 conn(0x7faa08100160 msgr2=0x7faa08102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:54.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.997+0000 7faa0e000700 1 -- 172.21.15.79:0/301568197 shutdown_connections 2024-06-03T02:15:54.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.997+0000 7faa0e000700 1 -- 172.21.15.79:0/301568197 wait complete. 2024-06-03T02:15:54.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.998+0000 7faa0e000700 1 Processor -- start 2024-06-03T02:15:55.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.998+0000 7faa0e000700 1 -- start start 2024-06-03T02:15:55.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.998+0000 7faa0e000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa081046b0 0x7faa0819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.998+0000 7faa0e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa08105070 0x7faa0819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.998+0000 7faa0e000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0810a6b0 0x7faa0819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.998+0000 7faa0e000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faa08114a40 con 0x7faa08105070 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.998+0000 7faa0e000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa081148c0 con 0x7faa0810a6b0 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.998+0000 7faa0e000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa08114bc0 con 0x7faa081046b0 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7faa07400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa081046b0 0x7faa0819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7faa07400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa081046b0 0x7faa0819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:49974/0 (socket says 172.21.15.79:49974) 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7faa07400700 1 -- 172.21.15.79:0/4085276107 learned_addr learned my addr 172.21.15.79:0/4085276107 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7fa9fea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa08105070 0x7faa0819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7faa07e00700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0810a6b0 0x7faa0819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7fa9fea00700 1 -- 172.21.15.79:0/4085276107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa081046b0 msgr2=0x7faa0819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7fa9fea00700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa081046b0 0x7faa0819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7fa9fea00700 1 -- 172.21.15.79:0/4085276107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0810a6b0 msgr2=0x7faa0819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7fa9fea00700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0810a6b0 0x7faa0819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7fa9fea00700 1 -- 172.21.15.79:0/4085276107 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0819fe20 con 0x7faa08105070 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:54.999+0000 7faa07400700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa081046b0 0x7faa0819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.000+0000 7fa9fea00700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa08105070 0x7faa0819b350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000e9f0 tx=0x7fa9f000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.000+0000 7faa04c00700 1 -- 172.21.15.79:0/4085276107 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000c9c0 con 0x7faa08105070 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.000+0000 7faa04c00700 1 -- 172.21.15.79:0/4085276107 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f000cb60 con 0x7faa08105070 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.000+0000 7faa04c00700 1 -- 172.21.15.79:0/4085276107 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0018680 con 0x7faa08105070 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.000+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa081a0110 con 0x7faa08105070 2024-06-03T02:15:55.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.000+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa081a7990 con 0x7faa08105070 2024-06-03T02:15:55.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.001+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa08069180 con 0x7faa08105070 2024-06-03T02:15:55.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.003+0000 7faa04c00700 1 -- 172.21.15.79:0/4085276107 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa9f0010430 con 0x7faa08105070 2024-06-03T02:15:55.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.005+0000 7faa04c00700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9ec0a2d30 0x7fa9ec0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:55.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.005+0000 7faa04c00700 1 -- 172.21.15.79:0/4085276107 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa9f00cf660 con 0x7faa08105070 2024-06-03T02:15:55.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.005+0000 7faa07400700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9ec0a2d30 0x7fa9ec0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:55.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.006+0000 7faa07400700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9ec0a2d30 0x7fa9ec0a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009c10 tx=0x7fa9f800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:55.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.010+0000 7faa04c00700 1 -- 172.21.15.79:0/4085276107 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa9f009c050 con 0x7faa08105070 2024-06-03T02:15:55.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.267+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faa080628d0 con 0x7fa9ec0a2d30 2024-06-03T02:15:55.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.303+0000 7faa04c00700 1 -- 172.21.15.79:0/4085276107 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa080628d0 con 0x7fa9ec0a2d30 2024-06-03T02:15:55.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9ec0a2d30 msgr2=0x7fa9ec0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9ec0a2d30 0x7fa9ec0a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009c10 tx=0x7fa9f800b040 comp rx=0 tx=0).stop 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa08105070 msgr2=0x7faa0819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa08105070 0x7faa0819b350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000e9f0 tx=0x7fa9f000ed00 comp rx=0 tx=0).stop 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 shutdown_connections 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa0810a6b0 0x7faa0819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa081046b0 0x7faa0819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9ec0a2d30 0x7fa9ec0a51f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 --2- 172.21.15.79:0/4085276107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa08105070 0x7faa0819b350 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 >> 172.21.15.79:0/4085276107 conn(0x7faa08100160 msgr2=0x7faa08100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 shutdown_connections 2024-06-03T02:15:55.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:55.306+0000 7faa0e000700 1 -- 172.21.15.79:0/4085276107 wait complete. 2024-06-03T02:15:55.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:55 smithi079 ceph-mon[31819]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T02:15:55.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:55 smithi079 ceph-mon[31819]: osdmap e101: 12 total, 12 up, 12 in 2024-06-03T02:15:55.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:55 smithi135 ceph-mon[36653]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T02:15:55.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:55 smithi135 ceph-mon[36653]: osdmap e101: 12 total, 12 up, 12 in 2024-06-03T02:15:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:55 smithi123 ceph-mon[36640]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T02:15:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:55 smithi123 ceph-mon[36640]: osdmap e101: 12 total, 12 up, 12 in 2024-06-03T02:15:56.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-03T02:15:56.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:56 smithi079 ceph-mon[31819]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:56.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:56 smithi079 ceph-mon[31819]: pgmap v393: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-06-03T02:15:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:56 smithi135 ceph-mon[36653]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:56 smithi135 ceph-mon[36653]: pgmap v393: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-06-03T02:15:56.595 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:15:56.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:56 smithi123 ceph-mon[36640]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:56.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:56 smithi123 ceph-mon[36640]: pgmap v393: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-06-03T02:15:58.429 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:58 smithi079 ceph-mon[31819]: pgmap v394: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-03T02:15:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:58 smithi135 ceph-mon[36653]: pgmap v394: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-03T02:15:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.715+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2033754911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa4074fe0 msgr2=0x7fbaa410e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.715+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2033754911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa4074fe0 0x7fbaa410e950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba90009990 tx=0x7fba90009ca0 comp rx=0 tx=0).stop 2024-06-03T02:15:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.716+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2033754911 shutdown_connections 2024-06-03T02:15:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.716+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2033754911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa410ee90 0x7fbaa4111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.716+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2033754911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaa40746c0 0x7fbaa4074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.716+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2033754911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa4074fe0 0x7fbaa410e950 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.716+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2033754911 >> 172.21.15.79:0/2033754911 conn(0x7fbaa40795f0 msgr2=0x7fbaa4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.716+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2033754911 shutdown_connections 2024-06-03T02:15:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.716+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2033754911 wait complete. 2024-06-03T02:15:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.717+0000 7fbaaaa00700 1 Processor -- start 2024-06-03T02:15:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.717+0000 7fbaaaa00700 1 -- start start 2024-06-03T02:15:58.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.717+0000 7fbaaaa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa40746c0 0x7fbaa419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:58.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.717+0000 7fbaaaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4074fe0 0x7fbaa419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:58.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.717+0000 7fbaaaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaa410ee90 0x7fbaa419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:58.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.717+0000 7fbaaaa00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41149a0 con 0x7fbaa40746c0 2024-06-03T02:15:58.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.717+0000 7fbaaaa00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa4114820 con 0x7fbaa410ee90 2024-06-03T02:15:58.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.717+0000 7fbaaaa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa4114b20 con 0x7fbaa4074fe0 2024-06-03T02:15:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa40746c0 0x7fbaa419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa40746c0 0x7fbaa419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46776/0 (socket says 172.21.15.79:46776) 2024-06-03T02:15:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa3e00700 1 -- 172.21.15.79:0/2003621737 learned_addr learned my addr 172.21.15.79:0/2003621737 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:15:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa3400700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4074fe0 0x7fbaa419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa8a00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaa410ee90 0x7fbaa419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa3e00700 1 -- 172.21.15.79:0/2003621737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4074fe0 msgr2=0x7fbaa419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa3e00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4074fe0 0x7fbaa419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa3e00700 1 -- 172.21.15.79:0/2003621737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaa410ee90 msgr2=0x7fbaa419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa3e00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaa410ee90 0x7fbaa419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa3e00700 1 -- 172.21.15.79:0/2003621737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa419fdf0 con 0x7fbaa40746c0 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.718+0000 7fbaa8a00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaa410ee90 0x7fbaa419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.719+0000 7fbaa3e00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa40746c0 0x7fbaa419ae30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fba9400e9a0 tx=0x7fba9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.719+0000 7fbaa0c00700 1 -- 172.21.15.79:0/2003621737 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9400ca10 con 0x7fbaa40746c0 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.719+0000 7fbaa0c00700 1 -- 172.21.15.79:0/2003621737 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9400cbb0 con 0x7fbaa40746c0 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.719+0000 7fbaa0c00700 1 -- 172.21.15.79:0/2003621737 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94018680 con 0x7fbaa40746c0 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.719+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa41a00e0 con 0x7fbaa40746c0 2024-06-03T02:15:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.719+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa41a7960 con 0x7fbaa40746c0 2024-06-03T02:15:58.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.720+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa4069180 con 0x7fbaa40746c0 2024-06-03T02:15:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.722+0000 7fbaa0c00700 1 -- 172.21.15.79:0/2003621737 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fba94010430 con 0x7fbaa40746c0 2024-06-03T02:15:58.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:58 smithi123 ceph-mon[36640]: pgmap v394: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-03T02:15:58.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.723+0000 7fbaa0c00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fba880a2cb0 0x7fba880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:15:58.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.724+0000 7fbaa0c00700 1 -- 172.21.15.79:0/2003621737 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fba940cf820 con 0x7fbaa40746c0 2024-06-03T02:15:58.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.724+0000 7fbaa3400700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fba880a2cb0 0x7fba880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:15:58.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.725+0000 7fbaa3400700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fba880a2cb0 0x7fba880a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba90006010 tx=0x7fba90005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:15:58.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.728+0000 7fbaa0c00700 1 -- 172.21.15.79:0/2003621737 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba9409c210 con 0x7fbaa40746c0 2024-06-03T02:15:58.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:58.992+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fbaa40ffd50 con 0x7fba880a2cb0 2024-06-03T02:15:59.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.003+0000 7fbaa0c00700 1 -- 172.21.15.79:0/2003621737 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbaa40ffd50 con 0x7fba880a2cb0 2024-06-03T02:15:59.003 INFO:teuthology.orchestra.run.smithi079.stdout:/volumes/qa/sv_1/384b230e-7fb1-414a-9276-a11ddd986ce8 2024-06-03T02:15:59.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fba880a2cb0 msgr2=0x7fba880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:59.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fba880a2cb0 0x7fba880a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba90006010 tx=0x7fba90005960 comp rx=0 tx=0).stop 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa40746c0 msgr2=0x7fbaa419ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa40746c0 0x7fbaa419ae30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fba9400e9a0 tx=0x7fba9400ecb0 comp rx=0 tx=0).stop 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 shutdown_connections 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaa410ee90 0x7fbaa419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fba880a2cb0 0x7fba880a5170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbaa40746c0 0x7fbaa419ae30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 --2- 172.21.15.79:0/2003621737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4074fe0 0x7fbaa419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 >> 172.21.15.79:0/2003621737 conn(0x7fbaa40795f0 msgr2=0x7fbaa40ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.006+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 shutdown_connections 2024-06-03T02:15:59.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:15:59.007+0000 7fbaaaa00700 1 -- 172.21.15.79:0/2003621737 wait complete. 2024-06-03T02:15:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:15:59 smithi079 ceph-mon[31819]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:59.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:15:59 smithi135 ceph-mon[36653]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:59.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:15:59 smithi123 ceph-mon[36640]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:15:59.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:16:00.248 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:00.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:00 smithi079 ceph-mon[31819]: pgmap v395: 129 pgs: 129 active+clean; 631 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-06-03T02:16:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:00 smithi135 ceph-mon[36653]: pgmap v395: 129 pgs: 129 active+clean; 631 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-06-03T02:16:00.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:00 smithi123 ceph-mon[36640]: pgmap v395: 129 pgs: 129 active+clean; 631 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-06-03T02:16:01.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:01 smithi079 ceph-mon[31819]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T02:16:01.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:01 smithi079 ceph-mon[31819]: Cluster is now healthy 2024-06-03T02:16:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:01 smithi135 ceph-mon[36653]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T02:16:01.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:01 smithi135 ceph-mon[36653]: Cluster is now healthy 2024-06-03T02:16:01.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:01 smithi123 ceph-mon[36640]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T02:16:01.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:01 smithi123 ceph-mon[36640]: Cluster is now healthy 2024-06-03T02:16:02.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.418+0000 7fc935200700 1 -- 172.21.15.79:0/1185564669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9301085f0 msgr2=0x7fc9301089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:02.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.418+0000 7fc935200700 1 --2- 172.21.15.79:0/1185564669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9301085f0 0x7fc9301089d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098f0 tx=0x7fc920009c00 comp rx=0 tx=0).stop 2024-06-03T02:16:02.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.418+0000 7fc935200700 1 -- 172.21.15.79:0/1185564669 shutdown_connections 2024-06-03T02:16:02.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.418+0000 7fc935200700 1 --2- 172.21.15.79:0/1185564669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc930102ea0 0x7fc93010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.418+0000 7fc935200700 1 --2- 172.21.15.79:0/1185564669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9301085f0 0x7fc9301089d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.418+0000 7fc935200700 1 --2- 172.21.15.79:0/1185564669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9301024e0 0x7fc930102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.418+0000 7fc935200700 1 -- 172.21.15.79:0/1185564669 >> 172.21.15.79:0/1185564669 conn(0x7fc930079640 msgr2=0x7fc930079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:02.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.418+0000 7fc935200700 1 -- 172.21.15.79:0/1185564669 shutdown_connections 2024-06-03T02:16:02.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.418+0000 7fc935200700 1 -- 172.21.15.79:0/1185564669 wait complete. 2024-06-03T02:16:02.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.419+0000 7fc935200700 1 Processor -- start 2024-06-03T02:16:02.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.419+0000 7fc935200700 1 -- start start 2024-06-03T02:16:02.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc935200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9301024e0 0x7fc93019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:02.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc935200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc930102ea0 0x7fc93019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:02.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc935200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9301085f0 0x7fc93019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:02.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc935200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc930112800 con 0x7fc930102ea0 2024-06-03T02:16:02.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc935200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc930112680 con 0x7fc9301085f0 2024-06-03T02:16:02.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc935200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc930112980 con 0x7fc9301024e0 2024-06-03T02:16:02.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc92d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc930102ea0 0x7fc93019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:02.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc92d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc930102ea0 0x7fc93019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46804/0 (socket says 172.21.15.79:46804) 2024-06-03T02:16:02.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc92de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9301024e0 0x7fc93019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:02.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc92d400700 1 -- 172.21.15.79:0/2558903464 learned_addr learned my addr 172.21.15.79:0/2558903464 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:02.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.420+0000 7fc92e800700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9301085f0 0x7fc93019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:02.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.421+0000 7fc92e800700 1 -- 172.21.15.79:0/2558903464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9301024e0 msgr2=0x7fc93019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:02.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.421+0000 7fc92e800700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9301024e0 0x7fc93019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.421+0000 7fc92e800700 1 -- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc930102ea0 msgr2=0x7fc93019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:02.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.421+0000 7fc92e800700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc930102ea0 0x7fc93019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.421+0000 7fc92e800700 1 -- 172.21.15.79:0/2558903464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93019ff20 con 0x7fc9301085f0 2024-06-03T02:16:02.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.421+0000 7fc92d400700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc930102ea0 0x7fc93019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:02.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.421+0000 7fc92e800700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9301085f0 0x7fc93019f7e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc928007f10 tx=0x7fc92800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:02.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.422+0000 7fc926a00700 1 -- 172.21.15.79:0/2558903464 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928004db0 con 0x7fc9301085f0 2024-06-03T02:16:02.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.422+0000 7fc926a00700 1 -- 172.21.15.79:0/2558903464 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc928009e20 con 0x7fc9301085f0 2024-06-03T02:16:02.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.422+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9301a0210 con 0x7fc9301085f0 2024-06-03T02:16:02.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.422+0000 7fc926a00700 1 -- 172.21.15.79:0/2558903464 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928015400 con 0x7fc9301085f0 2024-06-03T02:16:02.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.422+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9301a7a90 con 0x7fc9301085f0 2024-06-03T02:16:02.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.423+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc910005330 con 0x7fc9301085f0 2024-06-03T02:16:02.424 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:02 smithi079 ceph-mon[31819]: pgmap v396: 129 pgs: 129 active+clean; 631 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-06-03T02:16:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.426+0000 7fc926a00700 1 -- 172.21.15.79:0/2558903464 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc92800db20 con 0x7fc9301085f0 2024-06-03T02:16:02.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.428+0000 7fc926a00700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc9140a2d30 0x7fc9140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:02.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.428+0000 7fc926a00700 1 -- 172.21.15.79:0/2558903464 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc9280cec40 con 0x7fc9301085f0 2024-06-03T02:16:02.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.428+0000 7fc92de00700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc9140a2d30 0x7fc9140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:02.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.429+0000 7fc92de00700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc9140a2d30 0x7fc9140a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc920009810 tx=0x7fc920019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.433+0000 7fc926a00700 1 -- 172.21.15.79:0/2558903464 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc928010070 con 0x7fc9301085f0 2024-06-03T02:16:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:02 smithi135 ceph-mon[36653]: pgmap v396: 129 pgs: 129 active+clean; 631 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-06-03T02:16:02.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.701+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9100058f0 con 0x7fc9301085f0 2024-06-03T02:16:02.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.702+0000 7fc926a00700 1 -- 172.21.15.79:0/2558903464 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7fc92809b630 con 0x7fc9301085f0 2024-06-03T02:16:02.703 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:16:02.703 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:15:40.029293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24490,24482,24500,24515]},"id":1}]} 2024-06-03T02:16:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc9140a2d30 msgr2=0x7fc9140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc9140a2d30 0x7fc9140a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc920009810 tx=0x7fc920019040 comp rx=0 tx=0).stop 2024-06-03T02:16:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9301085f0 msgr2=0x7fc93019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9301085f0 0x7fc93019f7e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc928007f10 tx=0x7fc92800d3b0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 shutdown_connections 2024-06-03T02:16:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9301085f0 0x7fc93019f7e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9301024e0 0x7fc93019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc9140a2d30 0x7fc9140a51f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 --2- 172.21.15.79:0/2558903464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc930102ea0 0x7fc93019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.705+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 >> 172.21.15.79:0/2558903464 conn(0x7fc930079640 msgr2=0x7fc930100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.706+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 shutdown_connections 2024-06-03T02:16:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:02.706+0000 7fc935200700 1 -- 172.21.15.79:0/2558903464 wait complete. 2024-06-03T02:16:02.707 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 29 2024-06-03T02:16:02.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:02 smithi123 ceph-mon[36640]: pgmap v396: 129 pgs: 129 active+clean; 631 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-06-03T02:16:03.394 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:16:03.394 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:03 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2558903464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:16:03.394 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:03.394 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:03.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:16:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:16:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:03 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2558903464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:16:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:03.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:16:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:03 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2558903464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:16:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:03.898 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:05.267 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.267 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.267 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.267 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.267 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:05 smithi079 ceph-mon[31819]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-06-03T02:16:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:05 smithi123 ceph-mon[36640]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-06-03T02:16:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:05 smithi135 ceph-mon[36653]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-06-03T02:16:06.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.283+0000 7f7670800700 1 -- 172.21.15.79:0/536551113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76681030f0 msgr2=0x7f7668107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.283+0000 7f7670800700 1 --2- 172.21.15.79:0/536551113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76681030f0 0x7f7668107110 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f76580099c0 tx=0x7f7658009cd0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.283+0000 7f7670800700 1 -- 172.21.15.79:0/536551113 shutdown_connections 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.283+0000 7f7670800700 1 --2- 172.21.15.79:0/536551113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7668107840 0x7f766810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.283+0000 7f7670800700 1 --2- 172.21.15.79:0/536551113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7668102740 0x7f7668102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.283+0000 7f7670800700 1 --2- 172.21.15.79:0/536551113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76681030f0 0x7f7668107110 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.283+0000 7f7670800700 1 -- 172.21.15.79:0/536551113 >> 172.21.15.79:0/536551113 conn(0x7f76680795f0 msgr2=0x7f7668079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.283+0000 7f7670800700 1 -- 172.21.15.79:0/536551113 shutdown_connections 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.283+0000 7f7670800700 1 -- 172.21.15.79:0/536551113 wait complete. 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.284+0000 7f7670800700 1 Processor -- start 2024-06-03T02:16:06.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.284+0000 7f7670800700 1 -- start start 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f7670800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7668102740 0x7f766819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f7670800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76681030f0 0x7f766819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f7670800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7668107840 0x7f766819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f7670800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f76681127c0 con 0x7f7668102740 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f7670800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7668112640 con 0x7f76681030f0 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f7670800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7668112940 con 0x7f7668107840 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7668102740 0x7f766819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76681030f0 0x7f766819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76681030f0 0x7f766819b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41008/0 (socket says 172.21.15.79:41008) 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7668107840 0x7f766819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766d400700 1 -- 172.21.15.79:0/3987583702 learned_addr learned my addr 172.21.15.79:0/3987583702 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766d400700 1 -- 172.21.15.79:0/3987583702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7668107840 msgr2=0x7f766819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:06.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766d400700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7668107840 0x7f766819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766d400700 1 -- 172.21.15.79:0/3987583702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7668102740 msgr2=0x7f766819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766d400700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7668102740 0x7f766819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.285+0000 7f766d400700 1 -- 172.21.15.79:0/3987583702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766819fd90 con 0x7f76681030f0 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.286+0000 7f766d400700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76681030f0 0x7f766819b2c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7658006010 tx=0x7f7658003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.286+0000 7f7662a00700 1 -- 172.21.15.79:0/3987583702 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658003e50 con 0x7f76681030f0 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.286+0000 7f7662a00700 1 -- 172.21.15.79:0/3987583702 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76580055a0 con 0x7f76681030f0 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.286+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76681a0020 con 0x7f76681030f0 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.286+0000 7f7662a00700 1 -- 172.21.15.79:0/3987583702 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765800fe30 con 0x7f76681030f0 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.286+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76681a7900 con 0x7f76681030f0 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.287+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7668069180 con 0x7f76681030f0 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.289+0000 7f7662a00700 1 -- 172.21.15.79:0/3987583702 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f765800f4f0 con 0x7f76681030f0 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.290+0000 7f7662a00700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f76500a2ed0 0x7f76500a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:06.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.290+0000 7f7662a00700 1 -- 172.21.15.79:0/3987583702 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f76580d0440 con 0x7f76681030f0 2024-06-03T02:16:06.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.290+0000 7f766de00700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f76500a2ed0 0x7f76500a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:06.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.291+0000 7f766de00700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f76500a2ed0 0x7f76500a5390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f765c009c60 tx=0x7f765c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:06.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.295+0000 7f7662a00700 1 -- 172.21.15.79:0/3987583702 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7658013070 con 0x7f76681030f0 2024-06-03T02:16:06.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.525+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76681a7ad0 con 0x7f76681030f0 2024-06-03T02:16:06.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.528+0000 7f7662a00700 1 -- 172.21.15.79:0/3987583702 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+24805 (secure 0 0 0) 0x7f765809cec0 con 0x7f76681030f0 2024-06-03T02:16:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:16:06.529 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":101,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:15:54.257434+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:16:06.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f76500a2ed0 msgr2=0x7f76500a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:06.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f76500a2ed0 0x7f76500a5390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f765c009c60 tx=0x7f765c009260 comp rx=0 tx=0).stop 2024-06-03T02:16:06.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76681030f0 msgr2=0x7f766819b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:06.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76681030f0 0x7f766819b2c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7658006010 tx=0x7f7658003a90 comp rx=0 tx=0).stop 2024-06-03T02:16:06.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 shutdown_connections 2024-06-03T02:16:06.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7668107840 0x7f766819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f76500a2ed0 0x7f76500a5390 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7668102740 0x7f766819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 --2- 172.21.15.79:0/3987583702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76681030f0 0x7f766819b2c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:06.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 >> 172.21.15.79:0/3987583702 conn(0x7f76680795f0 msgr2=0x7f76681004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:06.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 shutdown_connections 2024-06-03T02:16:06.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:06.531+0000 7f7670800700 1 -- 172.21.15.79:0/3987583702 wait complete. 2024-06-03T02:16:06.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:06 smithi079 ceph-mon[31819]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-06-03T02:16:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:06 smithi135 ceph-mon[36653]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-06-03T02:16:06.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:06 smithi123 ceph-mon[36640]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-06-03T02:16:07.488 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:07 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3987583702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:16:07.537 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-03T02:16:07.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T02:16:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:07 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3987583702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:16:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:07 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3987583702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:16:07.967 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:08.485 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:08 smithi079 ceph-mon[31819]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-03T02:16:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:08 smithi135 ceph-mon[36653]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-03T02:16:08.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:08 smithi123 ceph-mon[36640]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-03T02:16:10.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.098+0000 7f16fe600700 1 -- 172.21.15.79:0/4089614927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 msgr2=0x7f16f8074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:10.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.098+0000 7f16fe600700 1 --2- 172.21.15.79:0/4089614927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 0x7f16f8074b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009990 tx=0x7f16ec009ca0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.099+0000 7f16fe600700 1 -- 172.21.15.79:0/4089614927 shutdown_connections 2024-06-03T02:16:10.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.099+0000 7f16fe600700 1 --2- 172.21.15.79:0/4089614927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16f810ec80 0x7f16f8111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.099+0000 7f16fe600700 1 --2- 172.21.15.79:0/4089614927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 0x7f16f8074b90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.099+0000 7f16fe600700 1 --2- 172.21.15.79:0/4089614927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f80750d0 0x7f16f810e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.099+0000 7f16fe600700 1 -- 172.21.15.79:0/4089614927 >> 172.21.15.79:0/4089614927 conn(0x7f16f80fdf10 msgr2=0x7f16f8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:10.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.099+0000 7f16fe600700 1 -- 172.21.15.79:0/4089614927 shutdown_connections 2024-06-03T02:16:10.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.099+0000 7f16fe600700 1 -- 172.21.15.79:0/4089614927 wait complete. 2024-06-03T02:16:10.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.100+0000 7f16fe600700 1 Processor -- start 2024-06-03T02:16:10.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.101+0000 7f16fe600700 1 -- start start 2024-06-03T02:16:10.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.101+0000 7f16fe600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 0x7f16f819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:10.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.101+0000 7f16fe600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16f80750d0 0x7f16f819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:10.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.101+0000 7f16fe600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f810ec80 0x7f16f819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:10.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.101+0000 7f16fe600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16f8114900 con 0x7f16f80750d0 2024-06-03T02:16:10.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.101+0000 7f16fe600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16f8114780 con 0x7f16f80747b0 2024-06-03T02:16:10.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.101+0000 7f16fe600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f16f8114a80 con 0x7f16f810ec80 2024-06-03T02:16:10.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.101+0000 7f16f7e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16f80750d0 0x7f16f819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:10.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fd200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 0x7f16f819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:10.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fd200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 0x7f16f819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41020/0 (socket says 172.21.15.79:41020) 2024-06-03T02:16:10.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fd200700 1 -- 172.21.15.79:0/601757208 learned_addr learned my addr 172.21.15.79:0/601757208 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:10.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fdc00700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f810ec80 0x7f16f819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:10.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fd200700 1 -- 172.21.15.79:0/601757208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f810ec80 msgr2=0x7f16f819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:10.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fd200700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f810ec80 0x7f16f819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fd200700 1 -- 172.21.15.79:0/601757208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16f80750d0 msgr2=0x7f16f819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:10.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fd200700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16f80750d0 0x7f16f819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fd200700 1 -- 172.21.15.79:0/601757208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f819fd70 con 0x7f16f80747b0 2024-06-03T02:16:10.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.102+0000 7f16fd200700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 0x7f16f819adb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16ec005910 tx=0x7f16ec00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:10.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.103+0000 7f16f5600700 1 -- 172.21.15.79:0/601757208 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec027070 con 0x7f16f80747b0 2024-06-03T02:16:10.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.103+0000 7f16f5600700 1 -- 172.21.15.79:0/601757208 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ec004650 con 0x7f16f80747b0 2024-06-03T02:16:10.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.103+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f81a0000 con 0x7f16f80747b0 2024-06-03T02:16:10.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.103+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f81a78e0 con 0x7f16f80747b0 2024-06-03T02:16:10.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.103+0000 7f16f5600700 1 -- 172.21.15.79:0/601757208 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00f460 con 0x7f16f80747b0 2024-06-03T02:16:10.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.104+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f8069180 con 0x7f16f80747b0 2024-06-03T02:16:10.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.106+0000 7f16f5600700 1 -- 172.21.15.79:0/601757208 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f16ec02a070 con 0x7f16f80747b0 2024-06-03T02:16:10.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.108+0000 7f16f5600700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e00a2d30 0x7f16e00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.108+0000 7f16f5600700 1 -- 172.21.15.79:0/601757208 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f16ec0d0850 con 0x7f16f80747b0 2024-06-03T02:16:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.109+0000 7f16f7e00700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e00a2d30 0x7f16e00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.109+0000 7f16f7e00700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e00a2d30 0x7f16e00a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f16f810ff30 tx=0x7f16e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.110+0000 7f16f5600700 1 -- 172.21.15.79:0/601757208 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f16ec031070 con 0x7f16f80747b0 2024-06-03T02:16:10.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:10.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:10.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:10.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:10.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:10.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:10.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.350+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16f819c0c0 con 0x7f16e00a2d30 2024-06-03T02:16:10.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.365+0000 7f16f5600700 1 -- 172.21.15.79:0/601757208 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340544 (secure 0 0 0) 0x7f16f819c0c0 con 0x7f16e00a2d30 2024-06-03T02:16:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.372+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e00a2d30 msgr2=0x7f16e00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.372+0000 7f16fe600700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e00a2d30 0x7f16e00a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f16f810ff30 tx=0x7f16e4009260 comp rx=0 tx=0).stop 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.372+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 msgr2=0x7f16f819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.372+0000 7f16fe600700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 0x7f16f819adb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16ec005910 tx=0x7f16ec00bab0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.373+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 shutdown_connections 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.373+0000 7f16fe600700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f810ec80 0x7f16f819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.373+0000 7f16fe600700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f80747b0 0x7f16f819adb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.373+0000 7f16fe600700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e00a2d30 0x7f16e00a51f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.373+0000 7f16fe600700 1 --2- 172.21.15.79:0/601757208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16f80750d0 0x7f16f819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:10.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.373+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 >> 172.21.15.79:0/601757208 conn(0x7f16f80fdf10 msgr2=0x7f16f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:10.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.373+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 shutdown_connections 2024-06-03T02:16:10.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:10.373+0000 7f16fe600700 1 -- 172.21.15.79:0/601757208 wait complete. 2024-06-03T02:16:10.376 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-06-03T02:16:11.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:10 smithi079 ceph-mon[31819]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-06-03T02:16:11.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:16:11.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:16:11.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:10 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:11.156 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-06-03T02:16:10.207876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660408,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":484,"num_read_kb":365,"num_write":434,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":355,"ondisk_log_size":355,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360848,"kb_used_data":15824,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580976,"statfs":{"total":1151940427776,"available":1151570919424,"internally_reserved":0,"allocated":16203776,"data_stored":8886110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31871,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":97,"num_write":19,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005732"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272883+0000","last_change":"2024-06-03T02:14:09.768431+0000","last_active":"2024-06-03T02:15:54.272883+0000","last_peered":"2024-06-03T02:15:54.272883+0000","last_clean":"2024-06-03T02:15:54.272883+0000","last_became_active":"2024-06-03T02:14:09.768059+0000","last_became_peered":"2024-06-03T02:14:09.768059+0000","last_unstale":"2024-06-03T02:15:54.272883+0000","last_undegraded":"2024-06-03T02:15:54.272883+0000","last_fullsized":"2024-06-03T02:15:54.272883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:12.265896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342398+0000","last_change":"2024-06-03T02:14:09.771842+0000","last_active":"2024-06-03T02:15:54.342398+0000","last_peered":"2024-06-03T02:15:54.342398+0000","last_clean":"2024-06-03T02:15:54.342398+0000","last_became_active":"2024-06-03T02:14:09.770961+0000","last_became_peered":"2024-06-03T02:14:09.770961+0000","last_unstale":"2024-06-03T02:15:54.342398+0000","last_undegraded":"2024-06-03T02:15:54.342398+0000","last_fullsized":"2024-06-03T02:15:54.342398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:49:02.297128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342266+0000","last_change":"2024-06-03T02:14:09.758407+0000","last_active":"2024-06-03T02:15:54.342266+0000","last_peered":"2024-06-03T02:15:54.342266+0000","last_clean":"2024-06-03T02:15:54.342266+0000","last_became_active":"2024-06-03T02:14:09.758045+0000","last_became_peered":"2024-06-03T02:14:09.758045+0000","last_unstale":"2024-06-03T02:15:54.342266+0000","last_undegraded":"2024-06-03T02:15:54.342266+0000","last_fullsized":"2024-06-03T02:15:54.342266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:01:08.166971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341515+0000","last_change":"2024-06-03T02:14:09.747658+0000","last_active":"2024-06-03T02:15:54.341515+0000","last_peered":"2024-06-03T02:15:54.341515+0000","last_clean":"2024-06-03T02:15:54.341515+0000","last_became_active":"2024-06-03T02:14:09.747321+0000","last_became_peered":"2024-06-03T02:14:09.747321+0000","last_unstale":"2024-06-03T02:15:54.341515+0000","last_undegraded":"2024-06-03T02:15:54.341515+0000","last_fullsized":"2024-06-03T02:15:54.341515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:08.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270618+0000","last_change":"2024-06-03T02:14:09.758161+0000","last_active":"2024-06-03T02:15:54.270618+0000","last_peered":"2024-06-03T02:15:54.270618+0000","last_clean":"2024-06-03T02:15:54.270618+0000","last_became_active":"2024-06-03T02:14:09.757800+0000","last_became_peered":"2024-06-03T02:14:09.757800+0000","last_unstale":"2024-06-03T02:15:54.270618+0000","last_undegraded":"2024-06-03T02:15:54.270618+0000","last_fullsized":"2024-06-03T02:15:54.270618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:47.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263499+0000","last_change":"2024-06-03T02:14:09.741818+0000","last_active":"2024-06-03T02:15:54.263499+0000","last_peered":"2024-06-03T02:15:54.263499+0000","last_clean":"2024-06-03T02:15:54.263499+0000","last_became_active":"2024-06-03T02:14:09.741219+0000","last_became_peered":"2024-06-03T02:14:09.741219+0000","last_unstale":"2024-06-03T02:15:54.263499+0000","last_undegraded":"2024-06-03T02:15:54.263499+0000","last_fullsized":"2024-06-03T02:15:54.263499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:16:30.585245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340981+0000","last_change":"2024-06-03T02:14:09.744863+0000","last_active":"2024-06-03T02:15:54.340981+0000","last_peered":"2024-06-03T02:15:54.340981+0000","last_clean":"2024-06-03T02:15:54.340981+0000","last_became_active":"2024-06-03T02:14:09.744514+0000","last_became_peered":"2024-06-03T02:14:09.744514+0000","last_unstale":"2024-06-03T02:15:54.340981+0000","last_undegraded":"2024-06-03T02:15:54.340981+0000","last_fullsized":"2024-06-03T02:15:54.340981+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:51:42.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266789+0000","last_change":"2024-06-03T02:14:09.766897+0000","last_active":"2024-06-03T02:15:54.266789+0000","last_peered":"2024-06-03T02:15:54.266789+0000","last_clean":"2024-06-03T02:15:54.266789+0000","last_became_active":"2024-06-03T02:14:09.765461+0000","last_became_peered":"2024-06-03T02:14:09.765461+0000","last_unstale":"2024-06-03T02:15:54.266789+0000","last_undegraded":"2024-06-03T02:15:54.266789+0000","last_fullsized":"2024-06-03T02:15:54.266789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:39:28.641517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341477+0000","last_change":"2024-06-03T02:14:09.746451+0000","last_active":"2024-06-03T02:15:54.341477+0000","last_peered":"2024-06-03T02:15:54.341477+0000","last_clean":"2024-06-03T02:15:54.341477+0000","last_became_active":"2024-06-03T02:14:09.746117+0000","last_became_peered":"2024-06-03T02:14:09.746117+0000","last_unstale":"2024-06-03T02:15:54.341477+0000","last_undegraded":"2024-06-03T02:15:54.341477+0000","last_fullsized":"2024-06-03T02:15:54.341477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:42.599993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.264014+0000","last_change":"2024-06-03T02:14:09.750680+0000","last_active":"2024-06-03T02:15:54.264014+0000","last_peered":"2024-06-03T02:15:54.264014+0000","last_clean":"2024-06-03T02:15:54.264014+0000","last_became_active":"2024-06-03T02:14:09.746138+0000","last_became_peered":"2024-06-03T02:14:09.746138+0000","last_unstale":"2024-06-03T02:15:54.264014+0000","last_undegraded":"2024-06-03T02:15:54.264014+0000","last_fullsized":"2024-06-03T02:15:54.264014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:04:28.188323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341516+0000","last_change":"2024-06-03T02:14:09.743467+0000","last_active":"2024-06-03T02:15:54.341516+0000","last_peered":"2024-06-03T02:15:54.341516+0000","last_clean":"2024-06-03T02:15:54.341516+0000","last_became_active":"2024-06-03T02:14:09.743152+0000","last_became_peered":"2024-06-03T02:14:09.743152+0000","last_unstale":"2024-06-03T02:15:54.341516+0000","last_undegraded":"2024-06-03T02:15:54.341516+0000","last_fullsized":"2024-06-03T02:15:54.341516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:27.336630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341391+0000","last_change":"2024-06-03T02:14:09.747142+0000","last_active":"2024-06-03T02:15:54.341391+0000","last_peered":"2024-06-03T02:15:54.341391+0000","last_clean":"2024-06-03T02:15:54.341391+0000","last_became_active":"2024-06-03T02:14:09.746694+0000","last_became_peered":"2024-06-03T02:14:09.746694+0000","last_unstale":"2024-06-03T02:15:54.341391+0000","last_undegraded":"2024-06-03T02:15:54.341391+0000","last_fullsized":"2024-06-03T02:15:54.341391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:04.507667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342866+0000","last_change":"2024-06-03T02:14:09.759906+0000","last_active":"2024-06-03T02:15:54.342866+0000","last_peered":"2024-06-03T02:15:54.342866+0000","last_clean":"2024-06-03T02:15:54.342866+0000","last_became_active":"2024-06-03T02:14:09.755450+0000","last_became_peered":"2024-06-03T02:14:09.755450+0000","last_unstale":"2024-06-03T02:15:54.342866+0000","last_undegraded":"2024-06-03T02:15:54.342866+0000","last_fullsized":"2024-06-03T02:15:54.342866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:23.057956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341599+0000","last_change":"2024-06-03T02:14:09.769388+0000","last_active":"2024-06-03T02:15:54.341599+0000","last_peered":"2024-06-03T02:15:54.341599+0000","last_clean":"2024-06-03T02:15:54.341599+0000","last_became_active":"2024-06-03T02:14:09.769029+0000","last_became_peered":"2024-06-03T02:14:09.769029+0000","last_unstale":"2024-06-03T02:15:54.341599+0000","last_undegraded":"2024-06-03T02:15:54.341599+0000","last_fullsized":"2024-06-03T02:15:54.341599+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:24.751039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341616+0000","last_change":"2024-06-03T02:14:09.768584+0000","last_active":"2024-06-03T02:15:54.341616+0000","last_peered":"2024-06-03T02:15:54.341616+0000","last_clean":"2024-06-03T02:15:54.341616+0000","last_became_active":"2024-06-03T02:14:09.765737+0000","last_became_peered":"2024-06-03T02:14:09.765737+0000","last_unstale":"2024-06-03T02:15:54.341616+0000","last_undegraded":"2024-06-03T02:15:54.341616+0000","last_fullsized":"2024-06-03T02:15:54.341616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:36:18.807762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339814+0000","last_change":"2024-06-03T02:14:09.757828+0000","last_active":"2024-06-03T02:15:54.339814+0000","last_peered":"2024-06-03T02:15:54.339814+0000","last_clean":"2024-06-03T02:15:54.339814+0000","last_became_active":"2024-06-03T02:14:09.757040+0000","last_became_peered":"2024-06-03T02:14:09.757040+0000","last_unstale":"2024-06-03T02:15:54.339814+0000","last_undegraded":"2024-06-03T02:15:54.339814+0000","last_fullsized":"2024-06-03T02:15:54.339814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:26:28.694022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272608+0000","last_change":"2024-06-03T02:14:09.768263+0000","last_active":"2024-06-03T02:15:54.272608+0000","last_peered":"2024-06-03T02:15:54.272608+0000","last_clean":"2024-06-03T02:15:54.272608+0000","last_became_active":"2024-06-03T02:14:09.767988+0000","last_became_peered":"2024-06-03T02:14:09.767988+0000","last_unstale":"2024-06-03T02:15:54.272608+0000","last_undegraded":"2024-06-03T02:15:54.272608+0000","last_fullsized":"2024-06-03T02:15:54.272608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:28:37.432659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.265565+0000","last_change":"2024-06-03T02:14:09.747118+0000","last_active":"2024-06-03T02:15:54.265565+0000","last_peered":"2024-06-03T02:15:54.265565+0000","last_clean":"2024-06-03T02:15:54.265565+0000","last_became_active":"2024-06-03T02:14:09.746580+0000","last_became_peered":"2024-06-03T02:14:09.746580+0000","last_unstale":"2024-06-03T02:15:54.265565+0000","last_undegraded":"2024-06-03T02:15:54.265565+0000","last_fullsized":"2024-06-03T02:15:54.265565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:33:04.607285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272733+0000","last_change":"2024-06-03T02:14:09.755755+0000","last_active":"2024-06-03T02:15:54.272733+0000","last_peered":"2024-06-03T02:15:54.272733+0000","last_clean":"2024-06-03T02:15:54.272733+0000","last_became_active":"2024-06-03T02:14:09.754503+0000","last_became_peered":"2024-06-03T02:14:09.754503+0000","last_unstale":"2024-06-03T02:15:54.272733+0000","last_undegraded":"2024-06-03T02:15:54.272733+0000","last_fullsized":"2024-06-03T02:15:54.272733+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:30:58.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271140+0000","last_change":"2024-06-03T02:14:23.260021+0000","last_active":"2024-06-03T02:15:54.271140+0000","last_peered":"2024-06-03T02:15:54.271140+0000","last_clean":"2024-06-03T02:15:54.271140+0000","last_became_active":"2024-06-03T02:14:23.259587+0000","last_became_peered":"2024-06-03T02:14:23.259587+0000","last_unstale":"2024-06-03T02:15:54.271140+0000","last_undegraded":"2024-06-03T02:15:54.271140+0000","last_fullsized":"2024-06-03T02:15:54.271140+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:09:02.403936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'13","reported_seq":88,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:04.386397+0000","last_change":"2024-06-03T02:14:05.239623+0000","last_active":"2024-06-03T02:16:04.386397+0000","last_peered":"2024-06-03T02:16:04.386397+0000","last_clean":"2024-06-03T02:16:04.386397+0000","last_became_active":"2024-06-03T02:14:05.239112+0000","last_became_peered":"2024-06-03T02:14:05.239112+0000","last_unstale":"2024-06-03T02:16:04.386397+0000","last_undegraded":"2024-06-03T02:16:04.386397+0000","last_fullsized":"2024-06-03T02:16:04.386397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:12:14.020068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267640+0000","last_change":"2024-06-03T02:14:09.764972+0000","last_active":"2024-06-03T02:15:54.267640+0000","last_peered":"2024-06-03T02:15:54.267640+0000","last_clean":"2024-06-03T02:15:54.267640+0000","last_became_active":"2024-06-03T02:14:09.764435+0000","last_became_peered":"2024-06-03T02:14:09.764435+0000","last_unstale":"2024-06-03T02:15:54.267640+0000","last_undegraded":"2024-06-03T02:15:54.267640+0000","last_fullsized":"2024-06-03T02:15:54.267640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:46:58.941554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339903+0000","last_change":"2024-06-03T02:14:23.272076+0000","last_active":"2024-06-03T02:15:54.339903+0000","last_peered":"2024-06-03T02:15:54.339903+0000","last_clean":"2024-06-03T02:15:54.339903+0000","last_became_active":"2024-06-03T02:14:23.271578+0000","last_became_peered":"2024-06-03T02:14:23.271578+0000","last_unstale":"2024-06-03T02:15:54.339903+0000","last_undegraded":"2024-06-03T02:15:54.339903+0000","last_fullsized":"2024-06-03T02:15:54.339903+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:17:59.000291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"99'2","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268287+0000","last_change":"2024-06-03T02:14:05.251952+0000","last_active":"2024-06-03T02:15:54.268287+0000","last_peered":"2024-06-03T02:15:54.268287+0000","last_clean":"2024-06-03T02:15:54.268287+0000","last_became_active":"2024-06-03T02:14:05.251152+0000","last_became_peered":"2024-06-03T02:14:05.251152+0000","last_unstale":"2024-06-03T02:15:54.268287+0000","last_undegraded":"2024-06-03T02:15:54.268287+0000","last_fullsized":"2024-06-03T02:15:54.268287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:24:32.802236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342512+0000","last_change":"2024-06-03T02:14:09.764230+0000","last_active":"2024-06-03T02:15:54.342512+0000","last_peered":"2024-06-03T02:15:54.342512+0000","last_clean":"2024-06-03T02:15:54.342512+0000","last_became_active":"2024-06-03T02:14:09.761868+0000","last_became_peered":"2024-06-03T02:14:09.761868+0000","last_unstale":"2024-06-03T02:15:54.342512+0000","last_undegraded":"2024-06-03T02:15:54.342512+0000","last_fullsized":"2024-06-03T02:15:54.342512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:39:23.828546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340345+0000","last_change":"2024-06-03T02:14:23.272169+0000","last_active":"2024-06-03T02:15:54.340345+0000","last_peered":"2024-06-03T02:15:54.340345+0000","last_clean":"2024-06-03T02:15:54.340345+0000","last_became_active":"2024-06-03T02:14:23.271636+0000","last_became_peered":"2024-06-03T02:14:23.271636+0000","last_unstale":"2024-06-03T02:15:54.340345+0000","last_undegraded":"2024-06-03T02:15:54.340345+0000","last_fullsized":"2024-06-03T02:15:54.340345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:51:25.478028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"99'12","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270004+0000","last_change":"2024-06-03T02:14:05.238590+0000","last_active":"2024-06-03T02:15:54.270004+0000","last_peered":"2024-06-03T02:15:54.270004+0000","last_clean":"2024-06-03T02:15:54.270004+0000","last_became_active":"2024-06-03T02:14:05.238128+0000","last_became_peered":"2024-06-03T02:14:05.238128+0000","last_unstale":"2024-06-03T02:15:54.270004+0000","last_undegraded":"2024-06-03T02:15:54.270004+0000","last_fullsized":"2024-06-03T02:15:54.270004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:21.298255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340609+0000","last_change":"2024-06-03T02:14:09.754489+0000","last_active":"2024-06-03T02:15:54.340609+0000","last_peered":"2024-06-03T02:15:54.340609+0000","last_clean":"2024-06-03T02:15:54.340609+0000","last_became_active":"2024-06-03T02:14:09.754155+0000","last_became_peered":"2024-06-03T02:14:09.754155+0000","last_unstale":"2024-06-03T02:15:54.340609+0000","last_undegraded":"2024-06-03T02:15:54.340609+0000","last_fullsized":"2024-06-03T02:15:54.340609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:20:43.841271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340482+0000","last_change":"2024-06-03T02:14:23.261232+0000","last_active":"2024-06-03T02:15:54.340482+0000","last_peered":"2024-06-03T02:15:54.340482+0000","last_clean":"2024-06-03T02:15:54.340482+0000","last_became_active":"2024-06-03T02:14:23.260772+0000","last_became_peered":"2024-06-03T02:14:23.260772+0000","last_unstale":"2024-06-03T02:15:54.340482+0000","last_undegraded":"2024-06-03T02:15:54.340482+0000","last_fullsized":"2024-06-03T02:15:54.340482+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:52.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"99'5","reported_seq":23,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272188+0000","last_change":"2024-06-03T02:14:53.274084+0000","last_active":"2024-06-03T02:15:54.272188+0000","last_peered":"2024-06-03T02:15:54.272188+0000","last_clean":"2024-06-03T02:15:54.272188+0000","last_became_active":"2024-06-03T02:14:53.265253+0000","last_became_peered":"2024-06-03T02:14:53.265253+0000","last_unstale":"2024-06-03T02:15:54.272188+0000","last_undegraded":"2024-06-03T02:15:54.272188+0000","last_fullsized":"2024-06-03T02:15:54.272188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:25:56.189308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339882+0000","last_change":"2024-06-03T02:14:09.743450+0000","last_active":"2024-06-03T02:15:54.339882+0000","last_peered":"2024-06-03T02:15:54.339882+0000","last_clean":"2024-06-03T02:15:54.339882+0000","last_became_active":"2024-06-03T02:14:09.743071+0000","last_became_peered":"2024-06-03T02:14:09.743071+0000","last_unstale":"2024-06-03T02:15:54.339882+0000","last_undegraded":"2024-06-03T02:15:54.339882+0000","last_fullsized":"2024-06-03T02:15:54.339882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:43:44.278633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":35,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342978+0000","last_change":"2024-06-03T02:15:54.342978+0000","last_active":"2024-06-03T02:15:54.342978+0000","last_peered":"2024-06-03T02:15:54.342978+0000","last_clean":"2024-06-03T02:15:54.342978+0000","last_became_active":"2024-06-03T02:15:54.342477+0000","last_became_peered":"2024-06-03T02:15:54.342477+0000","last_unstale":"2024-06-03T02:15:54.342978+0000","last_undegraded":"2024-06-03T02:15:54.342978+0000","last_fullsized":"2024-06-03T02:15:54.342978+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:47.980513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'6","reported_seq":84,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:05.382064+0000","last_change":"2024-06-03T02:14:05.237464+0000","last_active":"2024-06-03T02:16:05.382064+0000","last_peered":"2024-06-03T02:16:05.382064+0000","last_clean":"2024-06-03T02:16:05.382064+0000","last_became_active":"2024-06-03T02:14:05.237067+0000","last_became_peered":"2024-06-03T02:14:05.237067+0000","last_unstale":"2024-06-03T02:16:05.382064+0000","last_undegraded":"2024-06-03T02:16:05.382064+0000","last_fullsized":"2024-06-03T02:16:05.382064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:06:38.850484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270471+0000","last_change":"2024-06-03T02:14:09.748741+0000","last_active":"2024-06-03T02:15:54.270471+0000","last_peered":"2024-06-03T02:15:54.270471+0000","last_clean":"2024-06-03T02:15:54.270471+0000","last_became_active":"2024-06-03T02:14:09.748402+0000","last_became_peered":"2024-06-03T02:14:09.748402+0000","last_unstale":"2024-06-03T02:15:54.270471+0000","last_undegraded":"2024-06-03T02:15:54.270471+0000","last_fullsized":"2024-06-03T02:15:54.270471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:03:25.664415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271782+0000","last_change":"2024-06-03T02:14:23.260397+0000","last_active":"2024-06-03T02:15:54.271782+0000","last_peered":"2024-06-03T02:15:54.271782+0000","last_clean":"2024-06-03T02:15:54.271782+0000","last_became_active":"2024-06-03T02:14:23.259886+0000","last_became_peered":"2024-06-03T02:14:23.259886+0000","last_unstale":"2024-06-03T02:15:54.271782+0000","last_undegraded":"2024-06-03T02:15:54.271782+0000","last_fullsized":"2024-06-03T02:15:54.271782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:46:18.562706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"99'7","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263469+0000","last_change":"2024-06-03T02:14:05.241536+0000","last_active":"2024-06-03T02:15:54.263469+0000","last_peered":"2024-06-03T02:15:54.263469+0000","last_clean":"2024-06-03T02:15:54.263469+0000","last_became_active":"2024-06-03T02:14:05.241203+0000","last_became_peered":"2024-06-03T02:14:05.241203+0000","last_unstale":"2024-06-03T02:15:54.263469+0000","last_undegraded":"2024-06-03T02:15:54.263469+0000","last_fullsized":"2024-06-03T02:15:54.263469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:47.052563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340387+0000","last_change":"2024-06-03T02:14:53.475893+0000","last_active":"2024-06-03T02:15:54.340387+0000","last_peered":"2024-06-03T02:15:54.340387+0000","last_clean":"2024-06-03T02:15:54.340387+0000","last_became_active":"2024-06-03T02:14:53.475494+0000","last_became_peered":"2024-06-03T02:14:53.475494+0000","last_unstale":"2024-06-03T02:15:54.340387+0000","last_undegraded":"2024-06-03T02:15:54.340387+0000","last_fullsized":"2024-06-03T02:15:54.340387+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:27.113842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.344417+0000","last_change":"2024-06-03T02:15:54.344417+0000","last_active":"2024-06-03T02:15:54.344417+0000","last_peered":"2024-06-03T02:15:54.344417+0000","last_clean":"2024-06-03T02:15:54.344417+0000","last_became_active":"2024-06-03T02:15:54.343904+0000","last_became_peered":"2024-06-03T02:15:54.343904+0000","last_unstale":"2024-06-03T02:15:54.344417+0000","last_undegraded":"2024-06-03T02:15:54.344417+0000","last_fullsized":"2024-06-03T02:15:54.344417+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:42:18.121696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"99'3","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.343756+0000","last_change":"2024-06-03T02:14:05.237743+0000","last_active":"2024-06-03T02:15:54.343756+0000","last_peered":"2024-06-03T02:15:54.343756+0000","last_clean":"2024-06-03T02:15:54.343756+0000","last_became_active":"2024-06-03T02:14:05.236944+0000","last_became_peered":"2024-06-03T02:14:05.236944+0000","last_unstale":"2024-06-03T02:15:54.343756+0000","last_undegraded":"2024-06-03T02:15:54.343756+0000","last_fullsized":"2024-06-03T02:15:54.343756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:14:09.850862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266443+0000","last_change":"2024-06-03T02:14:09.761694+0000","last_active":"2024-06-03T02:15:54.266443+0000","last_peered":"2024-06-03T02:15:54.266443+0000","last_clean":"2024-06-03T02:15:54.266443+0000","last_became_active":"2024-06-03T02:14:09.761361+0000","last_became_peered":"2024-06-03T02:14:09.761361+0000","last_unstale":"2024-06-03T02:15:54.266443+0000","last_undegraded":"2024-06-03T02:15:54.266443+0000","last_fullsized":"2024-06-03T02:15:54.266443+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:19:17.803088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342247+0000","last_change":"2024-06-03T02:14:23.258634+0000","last_active":"2024-06-03T02:15:54.342247+0000","last_peered":"2024-06-03T02:15:54.342247+0000","last_clean":"2024-06-03T02:15:54.342247+0000","last_became_active":"2024-06-03T02:14:23.257799+0000","last_became_peered":"2024-06-03T02:14:23.257799+0000","last_unstale":"2024-06-03T02:15:54.342247+0000","last_undegraded":"2024-06-03T02:15:54.342247+0000","last_fullsized":"2024-06-03T02:15:54.342247+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:38:33.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"99'3","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339831+0000","last_change":"2024-06-03T02:14:05.236790+0000","last_active":"2024-06-03T02:15:54.339831+0000","last_peered":"2024-06-03T02:15:54.339831+0000","last_clean":"2024-06-03T02:15:54.339831+0000","last_became_active":"2024-06-03T02:14:05.236419+0000","last_became_peered":"2024-06-03T02:14:05.236419+0000","last_unstale":"2024-06-03T02:15:54.339831+0000","last_undegraded":"2024-06-03T02:15:54.339831+0000","last_fullsized":"2024-06-03T02:15:54.339831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:44:00.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.264958+0000","last_change":"2024-06-03T02:14:09.744680+0000","last_active":"2024-06-03T02:15:54.264958+0000","last_peered":"2024-06-03T02:15:54.264958+0000","last_clean":"2024-06-03T02:15:54.264958+0000","last_became_active":"2024-06-03T02:14:09.744250+0000","last_became_peered":"2024-06-03T02:14:09.744250+0000","last_unstale":"2024-06-03T02:15:54.264958+0000","last_undegraded":"2024-06-03T02:15:54.264958+0000","last_fullsized":"2024-06-03T02:15:54.264958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:58.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267464+0000","last_change":"2024-06-03T02:14:23.259325+0000","last_active":"2024-06-03T02:15:54.267464+0000","last_peered":"2024-06-03T02:15:54.267464+0000","last_clean":"2024-06-03T02:15:54.267464+0000","last_became_active":"2024-06-03T02:14:23.258925+0000","last_became_peered":"2024-06-03T02:14:23.258925+0000","last_unstale":"2024-06-03T02:15:54.267464+0000","last_undegraded":"2024-06-03T02:15:54.267464+0000","last_fullsized":"2024-06-03T02:15:54.267464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:32.038564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":114,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:08.373467+0000","last_change":"2024-06-03T02:14:05.242690+0000","last_active":"2024-06-03T02:16:08.373467+0000","last_peered":"2024-06-03T02:16:08.373467+0000","last_clean":"2024-06-03T02:16:08.373467+0000","last_became_active":"2024-06-03T02:14:05.242320+0000","last_became_peered":"2024-06-03T02:14:05.242320+0000","last_unstale":"2024-06-03T02:16:08.373467+0000","last_undegraded":"2024-06-03T02:16:08.373467+0000","last_fullsized":"2024-06-03T02:16:08.373467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:02.793186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341190+0000","last_change":"2024-06-03T02:14:09.747146+0000","last_active":"2024-06-03T02:15:54.341190+0000","last_peered":"2024-06-03T02:15:54.341190+0000","last_clean":"2024-06-03T02:15:54.341190+0000","last_became_active":"2024-06-03T02:14:09.746803+0000","last_became_peered":"2024-06-03T02:14:09.746803+0000","last_unstale":"2024-06-03T02:15:54.341190+0000","last_undegraded":"2024-06-03T02:15:54.341190+0000","last_fullsized":"2024-06-03T02:15:54.341190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:04:36.262223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.264835+0000","last_change":"2024-06-03T02:14:23.270328+0000","last_active":"2024-06-03T02:15:54.264835+0000","last_peered":"2024-06-03T02:15:54.264835+0000","last_clean":"2024-06-03T02:15:54.264835+0000","last_became_active":"2024-06-03T02:14:23.264189+0000","last_became_peered":"2024-06-03T02:14:23.264189+0000","last_unstale":"2024-06-03T02:15:54.264835+0000","last_undegraded":"2024-06-03T02:15:54.264835+0000","last_fullsized":"2024-06-03T02:15:54.264835+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:36.248884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"99'4","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340089+0000","last_change":"2024-06-03T02:14:05.238898+0000","last_active":"2024-06-03T02:15:54.340089+0000","last_peered":"2024-06-03T02:15:54.340089+0000","last_clean":"2024-06-03T02:15:54.340089+0000","last_became_active":"2024-06-03T02:14:05.238552+0000","last_became_peered":"2024-06-03T02:14:05.238552+0000","last_unstale":"2024-06-03T02:15:54.340089+0000","last_undegraded":"2024-06-03T02:15:54.340089+0000","last_fullsized":"2024-06-03T02:15:54.340089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:29:33.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342202+0000","last_change":"2024-06-03T02:14:09.769973+0000","last_active":"2024-06-03T02:15:54.342202+0000","last_peered":"2024-06-03T02:15:54.342202+0000","last_clean":"2024-06-03T02:15:54.342202+0000","last_became_active":"2024-06-03T02:14:09.768984+0000","last_became_peered":"2024-06-03T02:14:09.768984+0000","last_unstale":"2024-06-03T02:15:54.342202+0000","last_undegraded":"2024-06-03T02:15:54.342202+0000","last_fullsized":"2024-06-03T02:15:54.342202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:48:33.720441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340563+0000","last_change":"2024-06-03T02:14:23.265947+0000","last_active":"2024-06-03T02:15:54.340563+0000","last_peered":"2024-06-03T02:15:54.340563+0000","last_clean":"2024-06-03T02:15:54.340563+0000","last_became_active":"2024-06-03T02:14:23.265412+0000","last_became_peered":"2024-06-03T02:14:23.265412+0000","last_unstale":"2024-06-03T02:15:54.340563+0000","last_undegraded":"2024-06-03T02:15:54.340563+0000","last_fullsized":"2024-06-03T02:15:54.340563+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:35:08.664884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"99'5","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339833+0000","last_change":"2024-06-03T02:14:05.252751+0000","last_active":"2024-06-03T02:15:54.339833+0000","last_peered":"2024-06-03T02:15:54.339833+0000","last_clean":"2024-06-03T02:15:54.339833+0000","last_became_active":"2024-06-03T02:14:05.252267+0000","last_became_peered":"2024-06-03T02:14:05.252267+0000","last_unstale":"2024-06-03T02:15:54.339833+0000","last_undegraded":"2024-06-03T02:15:54.339833+0000","last_fullsized":"2024-06-03T02:15:54.339833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:58.091711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340723+0000","last_change":"2024-06-03T02:14:09.750757+0000","last_active":"2024-06-03T02:15:54.340723+0000","last_peered":"2024-06-03T02:15:54.340723+0000","last_clean":"2024-06-03T02:15:54.340723+0000","last_became_active":"2024-06-03T02:14:09.750414+0000","last_became_peered":"2024-06-03T02:14:09.750414+0000","last_unstale":"2024-06-03T02:15:54.340723+0000","last_undegraded":"2024-06-03T02:15:54.340723+0000","last_fullsized":"2024-06-03T02:15:54.340723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:29:55.435408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.264974+0000","last_change":"2024-06-03T02:14:23.263440+0000","last_active":"2024-06-03T02:15:54.264974+0000","last_peered":"2024-06-03T02:15:54.264974+0000","last_clean":"2024-06-03T02:15:54.264974+0000","last_became_active":"2024-06-03T02:14:23.254019+0000","last_became_peered":"2024-06-03T02:14:23.254019+0000","last_unstale":"2024-06-03T02:15:54.264974+0000","last_undegraded":"2024-06-03T02:15:54.264974+0000","last_fullsized":"2024-06-03T02:15:54.264974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:48:58.745990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'35","reported_seq":154,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:08.373492+0000","last_change":"2024-06-03T02:14:05.239353+0000","last_active":"2024-06-03T02:16:08.373492+0000","last_peered":"2024-06-03T02:16:08.373492+0000","last_clean":"2024-06-03T02:16:08.373492+0000","last_became_active":"2024-06-03T02:14:05.238976+0000","last_became_peered":"2024-06-03T02:14:05.238976+0000","last_unstale":"2024-06-03T02:16:08.373492+0000","last_undegraded":"2024-06-03T02:16:08.373492+0000","last_fullsized":"2024-06-03T02:16:08.373492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:51.598533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62214,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":97,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270245+0000","last_change":"2024-06-03T02:14:09.746546+0000","last_active":"2024-06-03T02:15:54.270245+0000","last_peered":"2024-06-03T02:15:54.270245+0000","last_clean":"2024-06-03T02:15:54.270245+0000","last_became_active":"2024-06-03T02:14:09.746049+0000","last_became_peered":"2024-06-03T02:14:09.746049+0000","last_unstale":"2024-06-03T02:15:54.270245+0000","last_undegraded":"2024-06-03T02:15:54.270245+0000","last_fullsized":"2024-06-03T02:15:54.270245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:34.667133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340487+0000","last_change":"2024-06-03T02:14:23.257296+0000","last_active":"2024-06-03T02:15:54.340487+0000","last_peered":"2024-06-03T02:15:54.340487+0000","last_clean":"2024-06-03T02:15:54.340487+0000","last_became_active":"2024-06-03T02:14:23.256858+0000","last_became_peered":"2024-06-03T02:14:23.256858+0000","last_unstale":"2024-06-03T02:15:54.340487+0000","last_undegraded":"2024-06-03T02:15:54.340487+0000","last_fullsized":"2024-06-03T02:15:54.340487+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:00.464902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"99'9","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271601+0000","last_change":"2024-06-03T02:14:05.245163+0000","last_active":"2024-06-03T02:15:54.271601+0000","last_peered":"2024-06-03T02:15:54.271601+0000","last_clean":"2024-06-03T02:15:54.271601+0000","last_became_active":"2024-06-03T02:14:05.244813+0000","last_became_peered":"2024-06-03T02:14:05.244813+0000","last_unstale":"2024-06-03T02:15:54.271601+0000","last_undegraded":"2024-06-03T02:15:54.271601+0000","last_fullsized":"2024-06-03T02:15:54.271601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:46:47.017859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.344473+0000","last_change":"2024-06-03T02:14:09.757200+0000","last_active":"2024-06-03T02:15:54.344473+0000","last_peered":"2024-06-03T02:15:54.344473+0000","last_clean":"2024-06-03T02:15:54.344473+0000","last_became_active":"2024-06-03T02:14:09.756380+0000","last_became_peered":"2024-06-03T02:14:09.756380+0000","last_unstale":"2024-06-03T02:15:54.344473+0000","last_undegraded":"2024-06-03T02:15:54.344473+0000","last_fullsized":"2024-06-03T02:15:54.344473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:56.045338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272345+0000","last_change":"2024-06-03T02:14:23.258441+0000","last_active":"2024-06-03T02:15:54.272345+0000","last_peered":"2024-06-03T02:15:54.272345+0000","last_clean":"2024-06-03T02:15:54.272345+0000","last_became_active":"2024-06-03T02:14:23.258015+0000","last_became_peered":"2024-06-03T02:14:23.258015+0000","last_unstale":"2024-06-03T02:15:54.272345+0000","last_undegraded":"2024-06-03T02:15:54.272345+0000","last_fullsized":"2024-06-03T02:15:54.272345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:03.123511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"99'5","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340103+0000","last_change":"2024-06-03T02:14:05.240456+0000","last_active":"2024-06-03T02:15:54.340103+0000","last_peered":"2024-06-03T02:15:54.340103+0000","last_clean":"2024-06-03T02:15:54.340103+0000","last_became_active":"2024-06-03T02:14:05.240078+0000","last_became_peered":"2024-06-03T02:14:05.240078+0000","last_unstale":"2024-06-03T02:15:54.340103+0000","last_undegraded":"2024-06-03T02:15:54.340103+0000","last_fullsized":"2024-06-03T02:15:54.340103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:00:26.491773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340396+0000","last_change":"2024-06-03T02:14:09.750876+0000","last_active":"2024-06-03T02:15:54.340396+0000","last_peered":"2024-06-03T02:15:54.340396+0000","last_clean":"2024-06-03T02:15:54.340396+0000","last_became_active":"2024-06-03T02:14:09.750547+0000","last_became_peered":"2024-06-03T02:14:09.750547+0000","last_unstale":"2024-06-03T02:15:54.340396+0000","last_undegraded":"2024-06-03T02:15:54.340396+0000","last_fullsized":"2024-06-03T02:15:54.340396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:08:41.136225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268624+0000","last_change":"2024-06-03T02:14:23.276172+0000","last_active":"2024-06-03T02:15:54.268624+0000","last_peered":"2024-06-03T02:15:54.268624+0000","last_clean":"2024-06-03T02:15:54.268624+0000","last_became_active":"2024-06-03T02:14:23.275410+0000","last_became_peered":"2024-06-03T02:14:23.275410+0000","last_unstale":"2024-06-03T02:15:54.268624+0000","last_undegraded":"2024-06-03T02:15:54.268624+0000","last_fullsized":"2024-06-03T02:15:54.268624+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:16:30.859453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'6","reported_seq":83,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:04.380725+0000","last_change":"2024-06-03T02:14:05.237263+0000","last_active":"2024-06-03T02:16:04.380725+0000","last_peered":"2024-06-03T02:16:04.380725+0000","last_clean":"2024-06-03T02:16:04.380725+0000","last_became_active":"2024-06-03T02:14:05.236890+0000","last_became_peered":"2024-06-03T02:14:05.236890+0000","last_unstale":"2024-06-03T02:16:04.380725+0000","last_undegraded":"2024-06-03T02:16:04.380725+0000","last_fullsized":"2024-06-03T02:16:04.380725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:42.292545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341259+0000","last_change":"2024-06-03T02:14:09.746260+0000","last_active":"2024-06-03T02:15:54.341259+0000","last_peered":"2024-06-03T02:15:54.341259+0000","last_clean":"2024-06-03T02:15:54.341259+0000","last_became_active":"2024-06-03T02:14:09.745940+0000","last_became_peered":"2024-06-03T02:14:09.745940+0000","last_unstale":"2024-06-03T02:15:54.341259+0000","last_undegraded":"2024-06-03T02:15:54.341259+0000","last_fullsized":"2024-06-03T02:15:54.341259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:08:55.851464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345328+0000","last_change":"2024-06-03T02:14:23.265440+0000","last_active":"2024-06-03T02:15:54.345328+0000","last_peered":"2024-06-03T02:15:54.345328+0000","last_clean":"2024-06-03T02:15:54.345328+0000","last_became_active":"2024-06-03T02:14:23.265028+0000","last_became_peered":"2024-06-03T02:14:23.265028+0000","last_unstale":"2024-06-03T02:15:54.345328+0000","last_undegraded":"2024-06-03T02:15:54.345328+0000","last_fullsized":"2024-06-03T02:15:54.345328+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:56.237500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"99'17","reported_seq":78,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.269940+0000","last_change":"2024-06-03T02:14:53.271821+0000","last_active":"2024-06-03T02:15:54.269940+0000","last_peered":"2024-06-03T02:15:54.269940+0000","last_clean":"2024-06-03T02:15:54.269940+0000","last_became_active":"2024-06-03T02:14:53.263047+0000","last_became_peered":"2024-06-03T02:14:53.263047+0000","last_unstale":"2024-06-03T02:15:54.269940+0000","last_undegraded":"2024-06-03T02:15:54.269940+0000","last_fullsized":"2024-06-03T02:15:54.269940+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:02:25.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340413+0000","last_change":"2024-06-03T02:14:09.754819+0000","last_active":"2024-06-03T02:15:54.340413+0000","last_peered":"2024-06-03T02:15:54.340413+0000","last_clean":"2024-06-03T02:15:54.340413+0000","last_became_active":"2024-06-03T02:14:09.753995+0000","last_became_peered":"2024-06-03T02:14:09.753995+0000","last_unstale":"2024-06-03T02:15:54.340413+0000","last_undegraded":"2024-06-03T02:15:54.340413+0000","last_fullsized":"2024-06-03T02:15:54.340413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:19:51.123774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340453+0000","last_change":"2024-06-03T02:14:23.265003+0000","last_active":"2024-06-03T02:15:54.340453+0000","last_peered":"2024-06-03T02:15:54.340453+0000","last_clean":"2024-06-03T02:15:54.340453+0000","last_became_active":"2024-06-03T02:14:23.264529+0000","last_became_peered":"2024-06-03T02:14:23.264529+0000","last_unstale":"2024-06-03T02:15:54.340453+0000","last_undegraded":"2024-06-03T02:15:54.340453+0000","last_fullsized":"2024-06-03T02:15:54.340453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:50:19.838274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270555+0000","last_change":"2024-06-03T02:14:05.236901+0000","last_active":"2024-06-03T02:15:54.270555+0000","last_peered":"2024-06-03T02:15:54.270555+0000","last_clean":"2024-06-03T02:15:54.270555+0000","last_became_active":"2024-06-03T02:14:05.236460+0000","last_became_peered":"2024-06-03T02:14:05.236460+0000","last_unstale":"2024-06-03T02:15:54.270555+0000","last_undegraded":"2024-06-03T02:15:54.270555+0000","last_fullsized":"2024-06-03T02:15:54.270555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:59.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272038+0000","last_change":"2024-06-03T02:14:09.743135+0000","last_active":"2024-06-03T02:15:54.272038+0000","last_peered":"2024-06-03T02:15:54.272038+0000","last_clean":"2024-06-03T02:15:54.272038+0000","last_became_active":"2024-06-03T02:14:09.742506+0000","last_became_peered":"2024-06-03T02:14:09.742506+0000","last_unstale":"2024-06-03T02:15:54.272038+0000","last_undegraded":"2024-06-03T02:15:54.272038+0000","last_fullsized":"2024-06-03T02:15:54.272038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:51.413537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270340+0000","last_change":"2024-06-03T02:14:23.255512+0000","last_active":"2024-06-03T02:15:54.270340+0000","last_peered":"2024-06-03T02:15:54.270340+0000","last_clean":"2024-06-03T02:15:54.270340+0000","last_became_active":"2024-06-03T02:14:23.255103+0000","last_became_peered":"2024-06-03T02:14:23.255103+0000","last_unstale":"2024-06-03T02:15:54.270340+0000","last_undegraded":"2024-06-03T02:15:54.270340+0000","last_fullsized":"2024-06-03T02:15:54.270340+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:12:05.295848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"99'1","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266404+0000","last_change":"2024-06-03T02:14:05.248728+0000","last_active":"2024-06-03T02:15:54.266404+0000","last_peered":"2024-06-03T02:15:54.266404+0000","last_clean":"2024-06-03T02:15:54.266404+0000","last_became_active":"2024-06-03T02:14:05.248144+0000","last_became_peered":"2024-06-03T02:14:05.248144+0000","last_unstale":"2024-06-03T02:15:54.266404+0000","last_undegraded":"2024-06-03T02:15:54.266404+0000","last_fullsized":"2024-06-03T02:15:54.266404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:53:27.814443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263703+0000","last_change":"2024-06-03T02:14:09.752029+0000","last_active":"2024-06-03T02:15:54.263703+0000","last_peered":"2024-06-03T02:15:54.263703+0000","last_clean":"2024-06-03T02:15:54.263703+0000","last_became_active":"2024-06-03T02:14:09.751638+0000","last_became_peered":"2024-06-03T02:14:09.751638+0000","last_unstale":"2024-06-03T02:15:54.263703+0000","last_undegraded":"2024-06-03T02:15:54.263703+0000","last_fullsized":"2024-06-03T02:15:54.263703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:51.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.346477+0000","last_change":"2024-06-03T02:14:23.250406+0000","last_active":"2024-06-03T02:15:54.346477+0000","last_peered":"2024-06-03T02:15:54.346477+0000","last_clean":"2024-06-03T02:15:54.346477+0000","last_became_active":"2024-06-03T02:14:23.249946+0000","last_became_peered":"2024-06-03T02:14:23.249946+0000","last_unstale":"2024-06-03T02:15:54.346477+0000","last_undegraded":"2024-06-03T02:15:54.346477+0000","last_fullsized":"2024-06-03T02:15:54.346477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:04:47.714532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"99'6","reported_seq":84,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:06.390989+0000","last_change":"2024-06-03T02:14:05.241666+0000","last_active":"2024-06-03T02:16:06.390989+0000","last_peered":"2024-06-03T02:16:06.390989+0000","last_clean":"2024-06-03T02:16:06.390989+0000","last_became_active":"2024-06-03T02:14:05.241299+0000","last_became_peered":"2024-06-03T02:14:05.241299+0000","last_unstale":"2024-06-03T02:16:06.390989+0000","last_undegraded":"2024-06-03T02:16:06.390989+0000","last_fullsized":"2024-06-03T02:16:06.390989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:21:16.792022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263546+0000","last_change":"2024-06-03T02:14:09.768602+0000","last_active":"2024-06-03T02:15:54.263546+0000","last_peered":"2024-06-03T02:15:54.263546+0000","last_clean":"2024-06-03T02:15:54.263546+0000","last_became_active":"2024-06-03T02:14:09.768195+0000","last_became_peered":"2024-06-03T02:14:09.768195+0000","last_unstale":"2024-06-03T02:15:54.263546+0000","last_undegraded":"2024-06-03T02:15:54.263546+0000","last_fullsized":"2024-06-03T02:15:54.263546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:45:42.361966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.346593+0000","last_change":"2024-06-03T02:14:23.252472+0000","last_active":"2024-06-03T02:15:54.346593+0000","last_peered":"2024-06-03T02:15:54.346593+0000","last_clean":"2024-06-03T02:15:54.346593+0000","last_became_active":"2024-06-03T02:14:23.252018+0000","last_became_peered":"2024-06-03T02:14:23.252018+0000","last_unstale":"2024-06-03T02:15:54.346593+0000","last_undegraded":"2024-06-03T02:15:54.346593+0000","last_fullsized":"2024-06-03T02:15:54.346593+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:28.468657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340511+0000","last_change":"2024-06-03T02:14:09.744443+0000","last_active":"2024-06-03T02:15:54.340511+0000","last_peered":"2024-06-03T02:15:54.340511+0000","last_clean":"2024-06-03T02:15:54.340511+0000","last_became_active":"2024-06-03T02:14:09.744100+0000","last_became_peered":"2024-06-03T02:14:09.744100+0000","last_unstale":"2024-06-03T02:15:54.340511+0000","last_undegraded":"2024-06-03T02:15:54.340511+0000","last_fullsized":"2024-06-03T02:15:54.340511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:44.566745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:04.380040+0000","last_change":"2024-06-03T02:14:05.250734+0000","last_active":"2024-06-03T02:16:04.380040+0000","last_peered":"2024-06-03T02:16:04.380040+0000","last_clean":"2024-06-03T02:16:04.380040+0000","last_became_active":"2024-06-03T02:14:05.250300+0000","last_became_peered":"2024-06-03T02:14:05.250300+0000","last_unstale":"2024-06-03T02:16:04.380040+0000","last_undegraded":"2024-06-03T02:16:04.380040+0000","last_fullsized":"2024-06-03T02:16:04.380040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:52.322895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":36,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341425+0000","last_change":"2024-06-03T02:14:53.471397+0000","last_active":"2024-06-03T02:15:54.341425+0000","last_peered":"2024-06-03T02:15:54.341425+0000","last_clean":"2024-06-03T02:15:54.341425+0000","last_became_active":"2024-06-03T02:14:53.470868+0000","last_became_peered":"2024-06-03T02:14:53.470868+0000","last_unstale":"2024-06-03T02:15:54.341425+0000","last_undegraded":"2024-06-03T02:15:54.341425+0000","last_fullsized":"2024-06-03T02:15:54.341425+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:58.726070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268231+0000","last_change":"2024-06-03T02:14:09.765193+0000","last_active":"2024-06-03T02:15:54.268231+0000","last_peered":"2024-06-03T02:15:54.268231+0000","last_clean":"2024-06-03T02:15:54.268231+0000","last_became_active":"2024-06-03T02:14:09.764834+0000","last_became_peered":"2024-06-03T02:14:09.764834+0000","last_unstale":"2024-06-03T02:15:54.268231+0000","last_undegraded":"2024-06-03T02:15:54.268231+0000","last_fullsized":"2024-06-03T02:15:54.268231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:20:37.464286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"99'6","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340449+0000","last_change":"2024-06-03T02:14:05.240118+0000","last_active":"2024-06-03T02:15:54.340449+0000","last_peered":"2024-06-03T02:15:54.340449+0000","last_clean":"2024-06-03T02:15:54.340449+0000","last_became_active":"2024-06-03T02:14:05.239771+0000","last_became_peered":"2024-06-03T02:14:05.239771+0000","last_unstale":"2024-06-03T02:15:54.340449+0000","last_undegraded":"2024-06-03T02:15:54.340449+0000","last_fullsized":"2024-06-03T02:15:54.340449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:08.256222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":370,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340223+0000","last_change":"2024-06-03T02:12:31.842613+0000","last_active":"2024-06-03T02:15:54.340223+0000","last_peered":"2024-06-03T02:15:54.340223+0000","last_clean":"2024-06-03T02:15:54.340223+0000","last_became_active":"2024-06-03T02:12:31.842051+0000","last_became_peered":"2024-06-03T02:12:31.842051+0000","last_unstale":"2024-06-03T02:15:54.340223+0000","last_undegraded":"2024-06-03T02:15:54.340223+0000","last_fullsized":"2024-06-03T02:15:54.340223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_clean_scrub_stamp":"2024-06-03T02:08:38.749702+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:40:29.748789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340282+0000","last_change":"2024-06-03T02:14:23.255849+0000","last_active":"2024-06-03T02:15:54.340282+0000","last_peered":"2024-06-03T02:15:54.340282+0000","last_clean":"2024-06-03T02:15:54.340282+0000","last_became_active":"2024-06-03T02:14:23.255380+0000","last_became_peered":"2024-06-03T02:14:23.255380+0000","last_unstale":"2024-06-03T02:15:54.340282+0000","last_undegraded":"2024-06-03T02:15:54.340282+0000","last_fullsized":"2024-06-03T02:15:54.340282+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:41:16.051326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341820+0000","last_change":"2024-06-03T02:14:53.270423+0000","last_active":"2024-06-03T02:15:54.341820+0000","last_peered":"2024-06-03T02:15:54.341820+0000","last_clean":"2024-06-03T02:15:54.341820+0000","last_became_active":"2024-06-03T02:14:53.269260+0000","last_became_peered":"2024-06-03T02:14:53.269260+0000","last_unstale":"2024-06-03T02:15:54.341820+0000","last_undegraded":"2024-06-03T02:15:54.341820+0000","last_fullsized":"2024-06-03T02:15:54.341820+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:57.392012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"99'4","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341175+0000","last_change":"2024-06-03T02:14:05.235982+0000","last_active":"2024-06-03T02:15:54.341175+0000","last_peered":"2024-06-03T02:15:54.341175+0000","last_clean":"2024-06-03T02:15:54.341175+0000","last_became_active":"2024-06-03T02:14:05.235638+0000","last_became_peered":"2024-06-03T02:14:05.235638+0000","last_unstale":"2024-06-03T02:15:54.341175+0000","last_undegraded":"2024-06-03T02:15:54.341175+0000","last_fullsized":"2024-06-03T02:15:54.341175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:42:07.544005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271797+0000","last_change":"2024-06-03T02:14:23.257790+0000","last_active":"2024-06-03T02:15:54.271797+0000","last_peered":"2024-06-03T02:15:54.271797+0000","last_clean":"2024-06-03T02:15:54.271797+0000","last_became_active":"2024-06-03T02:14:23.257297+0000","last_became_peered":"2024-06-03T02:14:23.257297+0000","last_unstale":"2024-06-03T02:15:54.271797+0000","last_undegraded":"2024-06-03T02:15:54.271797+0000","last_fullsized":"2024-06-03T02:15:54.271797+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:13.715455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"99'4","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272719+0000","last_change":"2024-06-03T02:14:53.268495+0000","last_active":"2024-06-03T02:15:54.272719+0000","last_peered":"2024-06-03T02:15:54.272719+0000","last_clean":"2024-06-03T02:15:54.272719+0000","last_became_active":"2024-06-03T02:14:53.268147+0000","last_became_peered":"2024-06-03T02:14:53.268147+0000","last_unstale":"2024-06-03T02:15:54.272719+0000","last_undegraded":"2024-06-03T02:15:54.272719+0000","last_fullsized":"2024-06-03T02:15:54.272719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:23.076319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345591+0000","last_change":"2024-06-03T02:14:09.758244+0000","last_active":"2024-06-03T02:15:54.345591+0000","last_peered":"2024-06-03T02:15:54.345591+0000","last_clean":"2024-06-03T02:15:54.345591+0000","last_became_active":"2024-06-03T02:14:09.757812+0000","last_became_peered":"2024-06-03T02:14:09.757812+0000","last_unstale":"2024-06-03T02:15:54.345591+0000","last_undegraded":"2024-06-03T02:15:54.345591+0000","last_fullsized":"2024-06-03T02:15:54.345591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:08.243703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266108+0000","last_change":"2024-06-03T02:14:23.259980+0000","last_active":"2024-06-03T02:15:54.266108+0000","last_peered":"2024-06-03T02:15:54.266108+0000","last_clean":"2024-06-03T02:15:54.266108+0000","last_became_active":"2024-06-03T02:14:23.258308+0000","last_became_peered":"2024-06-03T02:14:23.258308+0000","last_unstale":"2024-06-03T02:15:54.266108+0000","last_undegraded":"2024-06-03T02:15:54.266108+0000","last_fullsized":"2024-06-03T02:15:54.266108+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:28.833994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'16","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:01.785970+0000","last_change":"2024-06-03T02:14:05.244855+0000","last_active":"2024-06-03T02:16:01.785970+0000","last_peered":"2024-06-03T02:16:01.785970+0000","last_clean":"2024-06-03T02:16:01.785970+0000","last_became_active":"2024-06-03T02:14:05.244335+0000","last_became_peered":"2024-06-03T02:14:05.244335+0000","last_unstale":"2024-06-03T02:16:01.785970+0000","last_undegraded":"2024-06-03T02:16:01.785970+0000","last_fullsized":"2024-06-03T02:16:01.785970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:53:19.065411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268184+0000","last_change":"2024-06-03T02:14:09.750761+0000","last_active":"2024-06-03T02:15:54.268184+0000","last_peered":"2024-06-03T02:15:54.268184+0000","last_clean":"2024-06-03T02:15:54.268184+0000","last_became_active":"2024-06-03T02:14:09.750336+0000","last_became_peered":"2024-06-03T02:14:09.750336+0000","last_unstale":"2024-06-03T02:15:54.268184+0000","last_undegraded":"2024-06-03T02:15:54.268184+0000","last_fullsized":"2024-06-03T02:15:54.268184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:59:46.651149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270963+0000","last_change":"2024-06-03T02:14:23.243287+0000","last_active":"2024-06-03T02:15:54.270963+0000","last_peered":"2024-06-03T02:15:54.270963+0000","last_clean":"2024-06-03T02:15:54.270963+0000","last_became_active":"2024-06-03T02:14:23.242828+0000","last_became_peered":"2024-06-03T02:14:23.242828+0000","last_unstale":"2024-06-03T02:15:54.270963+0000","last_undegraded":"2024-06-03T02:15:54.270963+0000","last_fullsized":"2024-06-03T02:15:54.270963+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:12.312438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"99'7","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341870+0000","last_change":"2024-06-03T02:14:05.596288+0000","last_active":"2024-06-03T02:15:54.341870+0000","last_peered":"2024-06-03T02:15:54.341870+0000","last_clean":"2024-06-03T02:15:54.341870+0000","last_became_active":"2024-06-03T02:14:05.595868+0000","last_became_peered":"2024-06-03T02:14:05.595868+0000","last_unstale":"2024-06-03T02:15:54.341870+0000","last_undegraded":"2024-06-03T02:15:54.341870+0000","last_fullsized":"2024-06-03T02:15:54.341870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:50.291541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342398+0000","last_change":"2024-06-03T02:14:09.765317+0000","last_active":"2024-06-03T02:15:54.342398+0000","last_peered":"2024-06-03T02:15:54.342398+0000","last_clean":"2024-06-03T02:15:54.342398+0000","last_became_active":"2024-06-03T02:14:09.764983+0000","last_became_peered":"2024-06-03T02:14:09.764983+0000","last_unstale":"2024-06-03T02:15:54.342398+0000","last_undegraded":"2024-06-03T02:15:54.342398+0000","last_fullsized":"2024-06-03T02:15:54.342398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:20.075276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341589+0000","last_change":"2024-06-03T02:14:23.265352+0000","last_active":"2024-06-03T02:15:54.341589+0000","last_peered":"2024-06-03T02:15:54.341589+0000","last_clean":"2024-06-03T02:15:54.341589+0000","last_became_active":"2024-06-03T02:14:23.264932+0000","last_became_peered":"2024-06-03T02:14:23.264932+0000","last_unstale":"2024-06-03T02:15:54.341589+0000","last_undegraded":"2024-06-03T02:15:54.341589+0000","last_fullsized":"2024-06-03T02:15:54.341589+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:43.803170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"99'2","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339930+0000","last_change":"2024-06-03T02:14:05.248563+0000","last_active":"2024-06-03T02:15:54.339930+0000","last_peered":"2024-06-03T02:15:54.339930+0000","last_clean":"2024-06-03T02:15:54.339930+0000","last_became_active":"2024-06-03T02:14:05.248306+0000","last_became_peered":"2024-06-03T02:14:05.248306+0000","last_unstale":"2024-06-03T02:15:54.339930+0000","last_undegraded":"2024-06-03T02:15:54.339930+0000","last_fullsized":"2024-06-03T02:15:54.339930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:23:58.130157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.269836+0000","last_change":"2024-06-03T02:14:09.745948+0000","last_active":"2024-06-03T02:15:54.269836+0000","last_peered":"2024-06-03T02:15:54.269836+0000","last_clean":"2024-06-03T02:15:54.269836+0000","last_became_active":"2024-06-03T02:14:09.745626+0000","last_became_peered":"2024-06-03T02:14:09.745626+0000","last_unstale":"2024-06-03T02:15:54.269836+0000","last_undegraded":"2024-06-03T02:15:54.269836+0000","last_fullsized":"2024-06-03T02:15:54.269836+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:34.742129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342659+0000","last_change":"2024-06-03T02:15:54.341824+0000","last_active":"2024-06-03T02:15:54.342659+0000","last_peered":"2024-06-03T02:15:54.342659+0000","last_clean":"2024-06-03T02:15:54.342659+0000","last_became_active":"2024-06-03T02:15:54.341267+0000","last_became_peered":"2024-06-03T02:15:54.341267+0000","last_unstale":"2024-06-03T02:15:54.342659+0000","last_undegraded":"2024-06-03T02:15:54.342659+0000","last_fullsized":"2024-06-03T02:15:54.342659+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:47.714056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"99'7","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267630+0000","last_change":"2024-06-03T02:14:05.244806+0000","last_active":"2024-06-03T02:15:54.267630+0000","last_peered":"2024-06-03T02:15:54.267630+0000","last_clean":"2024-06-03T02:15:54.267630+0000","last_became_active":"2024-06-03T02:14:05.242719+0000","last_became_peered":"2024-06-03T02:14:05.242719+0000","last_unstale":"2024-06-03T02:15:54.267630+0000","last_undegraded":"2024-06-03T02:15:54.267630+0000","last_fullsized":"2024-06-03T02:15:54.267630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:31.314266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271781+0000","last_change":"2024-06-03T02:14:09.755102+0000","last_active":"2024-06-03T02:15:54.271781+0000","last_peered":"2024-06-03T02:15:54.271781+0000","last_clean":"2024-06-03T02:15:54.271781+0000","last_became_active":"2024-06-03T02:14:09.754748+0000","last_became_peered":"2024-06-03T02:14:09.754748+0000","last_unstale":"2024-06-03T02:15:54.271781+0000","last_undegraded":"2024-06-03T02:15:54.271781+0000","last_fullsized":"2024-06-03T02:15:54.271781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:59.920710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"101'1","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341202+0000","last_change":"2024-06-03T02:14:23.271974+0000","last_active":"2024-06-03T02:15:54.341202+0000","last_peered":"2024-06-03T02:15:54.341202+0000","last_clean":"2024-06-03T02:15:54.341202+0000","last_became_active":"2024-06-03T02:14:23.271470+0000","last_became_peered":"2024-06-03T02:14:23.271470+0000","last_unstale":"2024-06-03T02:15:54.341202+0000","last_undegraded":"2024-06-03T02:15:54.341202+0000","last_fullsized":"2024-06-03T02:15:54.341202+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:03.159351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"99'3","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271407+0000","last_change":"2024-06-03T02:14:05.242261+0000","last_active":"2024-06-03T02:15:54.271407+0000","last_peered":"2024-06-03T02:15:54.271407+0000","last_clean":"2024-06-03T02:15:54.271407+0000","last_became_active":"2024-06-03T02:14:05.241854+0000","last_became_peered":"2024-06-03T02:14:05.241854+0000","last_unstale":"2024-06-03T02:15:54.271407+0000","last_undegraded":"2024-06-03T02:15:54.271407+0000","last_fullsized":"2024-06-03T02:15:54.271407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:21.427703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268011+0000","last_change":"2024-06-03T02:14:09.763591+0000","last_active":"2024-06-03T02:15:54.268011+0000","last_peered":"2024-06-03T02:15:54.268011+0000","last_clean":"2024-06-03T02:15:54.268011+0000","last_became_active":"2024-06-03T02:14:09.763240+0000","last_became_peered":"2024-06-03T02:14:09.763240+0000","last_unstale":"2024-06-03T02:15:54.268011+0000","last_undegraded":"2024-06-03T02:15:54.268011+0000","last_fullsized":"2024-06-03T02:15:54.268011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:40.146898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341547+0000","last_change":"2024-06-03T02:14:23.265011+0000","last_active":"2024-06-03T02:15:54.341547+0000","last_peered":"2024-06-03T02:15:54.341547+0000","last_clean":"2024-06-03T02:15:54.341547+0000","last_became_active":"2024-06-03T02:14:23.263767+0000","last_became_peered":"2024-06-03T02:14:23.263767+0000","last_unstale":"2024-06-03T02:15:54.341547+0000","last_undegraded":"2024-06-03T02:15:54.341547+0000","last_fullsized":"2024-06-03T02:15:54.341547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:34:04.274545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"99'4","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270449+0000","last_change":"2024-06-03T02:14:53.474879+0000","last_active":"2024-06-03T02:15:54.270449+0000","last_peered":"2024-06-03T02:15:54.270449+0000","last_clean":"2024-06-03T02:15:54.270449+0000","last_became_active":"2024-06-03T02:14:53.472209+0000","last_became_peered":"2024-06-03T02:14:53.472209+0000","last_unstale":"2024-06-03T02:15:54.270449+0000","last_undegraded":"2024-06-03T02:15:54.270449+0000","last_fullsized":"2024-06-03T02:15:54.270449+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:36:57.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340200+0000","last_change":"2024-06-03T02:14:09.747756+0000","last_active":"2024-06-03T02:15:54.340200+0000","last_peered":"2024-06-03T02:15:54.340200+0000","last_clean":"2024-06-03T02:15:54.340200+0000","last_became_active":"2024-06-03T02:14:09.747453+0000","last_became_peered":"2024-06-03T02:14:09.747453+0000","last_unstale":"2024-06-03T02:15:54.340200+0000","last_undegraded":"2024-06-03T02:15:54.340200+0000","last_fullsized":"2024-06-03T02:15:54.340200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:19:02.341367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339663+0000","last_change":"2024-06-03T02:14:23.248594+0000","last_active":"2024-06-03T02:15:54.339663+0000","last_peered":"2024-06-03T02:15:54.339663+0000","last_clean":"2024-06-03T02:15:54.339663+0000","last_became_active":"2024-06-03T02:14:23.248080+0000","last_became_peered":"2024-06-03T02:14:23.248080+0000","last_unstale":"2024-06-03T02:15:54.339663+0000","last_undegraded":"2024-06-03T02:15:54.339663+0000","last_fullsized":"2024-06-03T02:15:54.339663+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:24:42.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"99'5","reported_seq":82,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:05.369383+0000","last_change":"2024-06-03T02:14:05.596274+0000","last_active":"2024-06-03T02:16:05.369383+0000","last_peered":"2024-06-03T02:16:05.369383+0000","last_clean":"2024-06-03T02:16:05.369383+0000","last_became_active":"2024-06-03T02:14:05.595678+0000","last_became_peered":"2024-06-03T02:14:05.595678+0000","last_unstale":"2024-06-03T02:16:05.369383+0000","last_undegraded":"2024-06-03T02:16:05.369383+0000","last_fullsized":"2024-06-03T02:16:05.369383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:13:25.436688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341638+0000","last_change":"2024-06-03T02:14:09.744934+0000","last_active":"2024-06-03T02:15:54.341638+0000","last_peered":"2024-06-03T02:15:54.341638+0000","last_clean":"2024-06-03T02:15:54.341638+0000","last_became_active":"2024-06-03T02:14:09.744621+0000","last_became_peered":"2024-06-03T02:14:09.744621+0000","last_unstale":"2024-06-03T02:15:54.341638+0000","last_undegraded":"2024-06-03T02:15:54.341638+0000","last_fullsized":"2024-06-03T02:15:54.341638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:01:51.970160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341780+0000","last_change":"2024-06-03T02:14:23.273123+0000","last_active":"2024-06-03T02:15:54.341780+0000","last_peered":"2024-06-03T02:15:54.341780+0000","last_clean":"2024-06-03T02:15:54.341780+0000","last_became_active":"2024-06-03T02:14:23.272153+0000","last_became_peered":"2024-06-03T02:14:23.272153+0000","last_unstale":"2024-06-03T02:15:54.341780+0000","last_undegraded":"2024-06-03T02:15:54.341780+0000","last_fullsized":"2024-06-03T02:15:54.341780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:01:43.374413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"99'3","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340330+0000","last_change":"2024-06-03T02:14:05.245094+0000","last_active":"2024-06-03T02:15:54.340330+0000","last_peered":"2024-06-03T02:15:54.340330+0000","last_clean":"2024-06-03T02:15:54.340330+0000","last_became_active":"2024-06-03T02:14:05.244678+0000","last_became_peered":"2024-06-03T02:14:05.244678+0000","last_unstale":"2024-06-03T02:15:54.340330+0000","last_undegraded":"2024-06-03T02:15:54.340330+0000","last_fullsized":"2024-06-03T02:15:54.340330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:13:26.546135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340574+0000","last_change":"2024-06-03T02:14:09.745919+0000","last_active":"2024-06-03T02:15:54.340574+0000","last_peered":"2024-06-03T02:15:54.340574+0000","last_clean":"2024-06-03T02:15:54.340574+0000","last_became_active":"2024-06-03T02:14:09.745576+0000","last_became_peered":"2024-06-03T02:14:09.745576+0000","last_unstale":"2024-06-03T02:15:54.340574+0000","last_undegraded":"2024-06-03T02:15:54.340574+0000","last_fullsized":"2024-06-03T02:15:54.340574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:29:31.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339508+0000","last_change":"2024-06-03T02:14:23.244174+0000","last_active":"2024-06-03T02:15:54.339508+0000","last_peered":"2024-06-03T02:15:54.339508+0000","last_clean":"2024-06-03T02:15:54.339508+0000","last_became_active":"2024-06-03T02:14:23.243595+0000","last_became_peered":"2024-06-03T02:14:23.243595+0000","last_unstale":"2024-06-03T02:15:54.339508+0000","last_undegraded":"2024-06-03T02:15:54.339508+0000","last_fullsized":"2024-06-03T02:15:54.339508+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:12:36.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"99'5","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345635+0000","last_change":"2024-06-03T02:14:05.247669+0000","last_active":"2024-06-03T02:15:54.345635+0000","last_peered":"2024-06-03T02:15:54.345635+0000","last_clean":"2024-06-03T02:15:54.345635+0000","last_became_active":"2024-06-03T02:14:05.246685+0000","last_became_peered":"2024-06-03T02:14:05.246685+0000","last_unstale":"2024-06-03T02:15:54.345635+0000","last_undegraded":"2024-06-03T02:15:54.345635+0000","last_fullsized":"2024-06-03T02:15:54.345635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:39.454422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340640+0000","last_change":"2024-06-03T02:14:09.749242+0000","last_active":"2024-06-03T02:15:54.340640+0000","last_peered":"2024-06-03T02:15:54.340640+0000","last_clean":"2024-06-03T02:15:54.340640+0000","last_became_active":"2024-06-03T02:14:09.748370+0000","last_became_peered":"2024-06-03T02:14:09.748370+0000","last_unstale":"2024-06-03T02:15:54.340640+0000","last_undegraded":"2024-06-03T02:15:54.340640+0000","last_fullsized":"2024-06-03T02:15:54.340640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:26.189960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271931+0000","last_change":"2024-06-03T02:14:09.745966+0000","last_active":"2024-06-03T02:15:54.271931+0000","last_peered":"2024-06-03T02:15:54.271931+0000","last_clean":"2024-06-03T02:15:54.271931+0000","last_became_active":"2024-06-03T02:14:09.745590+0000","last_became_peered":"2024-06-03T02:14:09.745590+0000","last_unstale":"2024-06-03T02:15:54.271931+0000","last_undegraded":"2024-06-03T02:15:54.271931+0000","last_fullsized":"2024-06-03T02:15:54.271931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:46.744629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271380+0000","last_change":"2024-06-03T02:14:09.744964+0000","last_active":"2024-06-03T02:15:54.271380+0000","last_peered":"2024-06-03T02:15:54.271380+0000","last_clean":"2024-06-03T02:15:54.271380+0000","last_became_active":"2024-06-03T02:14:09.743877+0000","last_became_peered":"2024-06-03T02:14:09.743877+0000","last_unstale":"2024-06-03T02:15:54.271380+0000","last_undegraded":"2024-06-03T02:15:54.271380+0000","last_fullsized":"2024-06-03T02:15:54.271380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:05:48.020670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271297+0000","last_change":"2024-06-03T02:14:09.761736+0000","last_active":"2024-06-03T02:15:54.271297+0000","last_peered":"2024-06-03T02:15:54.271297+0000","last_clean":"2024-06-03T02:15:54.271297+0000","last_became_active":"2024-06-03T02:14:09.761477+0000","last_became_peered":"2024-06-03T02:14:09.761477+0000","last_unstale":"2024-06-03T02:15:54.271297+0000","last_undegraded":"2024-06-03T02:15:54.271297+0000","last_fullsized":"2024-06-03T02:15:54.271297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:00:04.631330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.343120+0000","last_change":"2024-06-03T02:14:53.266594+0000","last_active":"2024-06-03T02:15:54.343120+0000","last_peered":"2024-06-03T02:15:54.343120+0000","last_clean":"2024-06-03T02:15:54.343120+0000","last_became_active":"2024-06-03T02:14:53.266104+0000","last_became_peered":"2024-06-03T02:14:53.266104+0000","last_unstale":"2024-06-03T02:15:54.343120+0000","last_undegraded":"2024-06-03T02:15:54.343120+0000","last_fullsized":"2024-06-03T02:15:54.343120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:12.018082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345423+0000","last_change":"2024-06-03T02:14:09.772332+0000","last_active":"2024-06-03T02:15:54.345423+0000","last_peered":"2024-06-03T02:15:54.345423+0000","last_clean":"2024-06-03T02:15:54.345423+0000","last_became_active":"2024-06-03T02:14:09.771888+0000","last_became_peered":"2024-06-03T02:14:09.771888+0000","last_unstale":"2024-06-03T02:15:54.345423+0000","last_undegraded":"2024-06-03T02:15:54.345423+0000","last_fullsized":"2024-06-03T02:15:54.345423+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:58:49.414986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.273006+0000","last_change":"2024-06-03T02:14:53.270828+0000","last_active":"2024-06-03T02:15:54.273006+0000","last_peered":"2024-06-03T02:15:54.273006+0000","last_clean":"2024-06-03T02:15:54.273006+0000","last_became_active":"2024-06-03T02:14:53.270365+0000","last_became_peered":"2024-06-03T02:14:53.270365+0000","last_unstale":"2024-06-03T02:15:54.273006+0000","last_undegraded":"2024-06-03T02:15:54.273006+0000","last_fullsized":"2024-06-03T02:15:54.273006+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.051756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272851+0000","last_change":"2024-06-03T02:14:09.747709+0000","last_active":"2024-06-03T02:15:54.272851+0000","last_peered":"2024-06-03T02:15:54.272851+0000","last_clean":"2024-06-03T02:15:54.272851+0000","last_became_active":"2024-06-03T02:14:09.747359+0000","last_became_peered":"2024-06-03T02:14:09.747359+0000","last_unstale":"2024-06-03T02:15:54.272851+0000","last_undegraded":"2024-06-03T02:15:54.272851+0000","last_fullsized":"2024-06-03T02:15:54.272851+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:37:24.873442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266772+0000","last_change":"2024-06-03T02:14:53.473136+0000","last_active":"2024-06-03T02:15:54.266772+0000","last_peered":"2024-06-03T02:15:54.266772+0000","last_clean":"2024-06-03T02:15:54.266772+0000","last_became_active":"2024-06-03T02:14:53.471214+0000","last_became_peered":"2024-06-03T02:14:53.471214+0000","last_unstale":"2024-06-03T02:15:54.266772+0000","last_undegraded":"2024-06-03T02:15:54.266772+0000","last_fullsized":"2024-06-03T02:15:54.266772+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:25.822042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340080+0000","last_change":"2024-06-03T02:14:09.746463+0000","last_active":"2024-06-03T02:15:54.340080+0000","last_peered":"2024-06-03T02:15:54.340080+0000","last_clean":"2024-06-03T02:15:54.340080+0000","last_became_active":"2024-06-03T02:14:09.746115+0000","last_became_peered":"2024-06-03T02:14:09.746115+0000","last_unstale":"2024-06-03T02:15:54.340080+0000","last_undegraded":"2024-06-03T02:15:54.340080+0000","last_fullsized":"2024-06-03T02:15:54.340080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:51:23.050739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267946+0000","last_change":"2024-06-03T02:14:09.755467+0000","last_active":"2024-06-03T02:15:54.267946+0000","last_peered":"2024-06-03T02:15:54.267946+0000","last_clean":"2024-06-03T02:15:54.267946+0000","last_became_active":"2024-06-03T02:14:09.755103+0000","last_became_peered":"2024-06-03T02:14:09.755103+0000","last_unstale":"2024-06-03T02:15:54.267946+0000","last_undegraded":"2024-06-03T02:15:54.267946+0000","last_fullsized":"2024-06-03T02:15:54.267946+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:22.834519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340654+0000","last_change":"2024-06-03T02:14:09.758201+0000","last_active":"2024-06-03T02:15:54.340654+0000","last_peered":"2024-06-03T02:15:54.340654+0000","last_clean":"2024-06-03T02:15:54.340654+0000","last_became_active":"2024-06-03T02:14:09.757819+0000","last_became_peered":"2024-06-03T02:14:09.757819+0000","last_unstale":"2024-06-03T02:15:54.340654+0000","last_undegraded":"2024-06-03T02:15:54.340654+0000","last_fullsized":"2024-06-03T02:15:54.340654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:44.802694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342731+0000","last_change":"2024-06-03T02:14:09.768132+0000","last_active":"2024-06-03T02:15:54.342731+0000","last_peered":"2024-06-03T02:15:54.342731+0000","last_clean":"2024-06-03T02:15:54.342731+0000","last_became_active":"2024-06-03T02:14:09.767642+0000","last_became_peered":"2024-06-03T02:14:09.767642+0000","last_unstale":"2024-06-03T02:15:54.342731+0000","last_undegraded":"2024-06-03T02:15:54.342731+0000","last_fullsized":"2024-06-03T02:15:54.342731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:32:30.455418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272556+0000","last_change":"2024-06-03T02:14:09.745603+0000","last_active":"2024-06-03T02:15:54.272556+0000","last_peered":"2024-06-03T02:15:54.272556+0000","last_clean":"2024-06-03T02:15:54.272556+0000","last_became_active":"2024-06-03T02:14:09.745245+0000","last_became_peered":"2024-06-03T02:14:09.745245+0000","last_unstale":"2024-06-03T02:15:54.272556+0000","last_undegraded":"2024-06-03T02:15:54.272556+0000","last_fullsized":"2024-06-03T02:15:54.272556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:09:14.507101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69806,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":358,"num_read_kb":256,"num_write":251,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":209418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318314,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1179648,"data_stored":578717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37160,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707992,"statfs":{"total":95995035648,"available":95956983808,"internally_reserved":0,"allocated":1867776,"data_stored":1231539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":577953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005113,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":577637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266751,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28764,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716388,"statfs":{"total":95995035648,"available":95965581312,"internally_reserved":0,"allocated":1785856,"data_stored":1175607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430277,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1167360,"data_stored":573847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28232,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716920,"statfs":{"total":95995035648,"available":95966126080,"internally_reserved":0,"allocated":1245184,"data_stored":637203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1191936,"data_stored":574913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1249280,"data_stored":637199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888143,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":577921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691915,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37096,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708056,"statfs":{"total":95995035648,"available":95957049344,"internally_reserved":0,"allocated":1794048,"data_stored":1170409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T02:16:11.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T02:16:11.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:10 smithi135 ceph-mon[36653]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-06-03T02:16:11.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:16:11.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:16:11.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:10 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:10 smithi123 ceph-mon[36640]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-06-03T02:16:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:16:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:16:11.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:10 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:16:11.647 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:12.057 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:11 smithi079 ceph-mon[31819]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:16:12.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:11 smithi135 ceph-mon[36653]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:16:12.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:11 smithi123 ceph-mon[36640]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:16:13.318 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:12 smithi079 ceph-mon[31819]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-06-03T02:16:13.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:12 smithi135 ceph-mon[36653]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-06-03T02:16:13.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:12 smithi123 ceph-mon[36640]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-06-03T02:16:13.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.883+0000 7fe29b200700 1 -- 172.21.15.79:0/3903449106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c102610 msgr2=0x7fe29c1029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:13.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.883+0000 7fe29b200700 1 --2- 172.21.15.79:0/3903449106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c102610 0x7fe29c1029f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0099c0 tx=0x7fe28c009cd0 comp rx=0 tx=0).stop 2024-06-03T02:16:13.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.883+0000 7fe29b200700 1 -- 172.21.15.79:0/3903449106 shutdown_connections 2024-06-03T02:16:13.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.883+0000 7fe29b200700 1 --2- 172.21.15.79:0/3903449106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1076f0 0x7fe29c10f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:13.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.883+0000 7fe29b200700 1 --2- 172.21.15.79:0/3903449106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c102610 0x7fe29c1029f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:13.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.883+0000 7fe29b200700 1 --2- 172.21.15.79:0/3903449106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe29c102fc0 0x7fe29c107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:13.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.883+0000 7fe29b200700 1 -- 172.21.15.79:0/3903449106 >> 172.21.15.79:0/3903449106 conn(0x7fe29c0fdea0 msgr2=0x7fe29c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:13.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.883+0000 7fe29b200700 1 -- 172.21.15.79:0/3903449106 shutdown_connections 2024-06-03T02:16:13.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.883+0000 7fe29b200700 1 -- 172.21.15.79:0/3903449106 wait complete. 2024-06-03T02:16:13.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.885+0000 7fe29b200700 1 Processor -- start 2024-06-03T02:16:13.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.885+0000 7fe29b200700 1 -- start start 2024-06-03T02:16:13.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe29b200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe29c102610 0x7fe29c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:13.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe29b200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c102fc0 0x7fe29c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:13.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe29b200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1076f0 0x7fe29c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:13.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe29b200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c112740 con 0x7fe29c102fc0 2024-06-03T02:16:13.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe29b200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1125c0 con 0x7fe29c1076f0 2024-06-03T02:16:13.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe29b200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1128c0 con 0x7fe29c102610 2024-06-03T02:16:13.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe299e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe29c102610 0x7fe29c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe299e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe29c102610 0x7fe29c19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46650/0 (socket says 172.21.15.79:46650) 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe299e00700 1 -- 172.21.15.79:0/2837597391 learned_addr learned my addr 172.21.15.79:0/2837597391 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe299400700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c102fc0 0x7fe29c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe29a800700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1076f0 0x7fe29c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.886+0000 7fe299e00700 1 -- 172.21.15.79:0/2837597391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1076f0 msgr2=0x7fe29c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe299e00700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1076f0 0x7fe29c19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe299e00700 1 -- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c102fc0 msgr2=0x7fe29c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe299e00700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c102fc0 0x7fe29c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe299e00700 1 -- 172.21.15.79:0/2837597391 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c19fdf0 con 0x7fe29c102610 2024-06-03T02:16:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe299e00700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe29c102610 0x7fe29c19ae30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005910 tx=0x7fe28c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:13.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe292a00700 1 -- 172.21.15.79:0/2837597391 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c027070 con 0x7fe29c102610 2024-06-03T02:16:13.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c1a0080 con 0x7fe29c102610 2024-06-03T02:16:13.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe292a00700 1 -- 172.21.15.79:0/2837597391 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28c003680 con 0x7fe29c102610 2024-06-03T02:16:13.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe292a00700 1 -- 172.21.15.79:0/2837597391 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0173f0 con 0x7fe29c102610 2024-06-03T02:16:13.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.887+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c1a7960 con 0x7fe29c102610 2024-06-03T02:16:13.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.888+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29c069180 con 0x7fe29c102610 2024-06-03T02:16:13.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.891+0000 7fe292a00700 1 -- 172.21.15.79:0/2837597391 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe28c022590 con 0x7fe29c102610 2024-06-03T02:16:13.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.892+0000 7fe292a00700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe2800a2d30 0x7fe2800a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:13.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.892+0000 7fe292a00700 1 -- 172.21.15.79:0/2837597391 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe28c0d03a0 con 0x7fe29c102610 2024-06-03T02:16:13.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.892+0000 7fe299400700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe2800a2d30 0x7fe2800a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:13.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.893+0000 7fe299400700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe2800a2d30 0x7fe2800a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe29c076720 tx=0x7fe288009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:13.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:13.894+0000 7fe292a00700 1 -- 172.21.15.79:0/2837597391 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe28c013070 con 0x7fe29c102610 2024-06-03T02:16:14.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.115+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe29c10e630 con 0x7fe2800a2d30 2024-06-03T02:16:14.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.129+0000 7fe292a00700 1 -- 172.21.15.79:0/2837597391 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340544 (secure 0 0 0) 0x7fe29c10e630 con 0x7fe2800a2d30 2024-06-03T02:16:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe2800a2d30 msgr2=0x7fe2800a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe2800a2d30 0x7fe2800a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe29c076720 tx=0x7fe288009260 comp rx=0 tx=0).stop 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe29c102610 msgr2=0x7fe29c19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe29c102610 0x7fe29c19ae30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005910 tx=0x7fe28c00f690 comp rx=0 tx=0).stop 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 shutdown_connections 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1076f0 0x7fe29c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe29c102610 0x7fe29c19ae30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe2800a2d30 0x7fe2800a51f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 --2- 172.21.15.79:0/2837597391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe29c102fc0 0x7fe29c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 >> 172.21.15.79:0/2837597391 conn(0x7fe29c0fdea0 msgr2=0x7fe29c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 shutdown_connections 2024-06-03T02:16:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:14.136+0000 7fe29b200700 1 -- 172.21.15.79:0/2837597391 wait complete. 2024-06-03T02:16:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-06-03T02:16:14.495 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:14 smithi079 ceph-mon[31819]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:16:14.688 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:14 smithi079 ceph-mon[31819]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-03T02:16:14.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:14 smithi135 ceph-mon[36653]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:16:14.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:14 smithi135 ceph-mon[36653]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-03T02:16:14.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:14 smithi123 ceph-mon[36640]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:16:14.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:14 smithi123 ceph-mon[36640]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-03T02:16:14.955 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-06-03T02:16:12.208511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660408,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":504,"num_read_kb":375,"num_write":434,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":355,"ondisk_log_size":355,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360848,"kb_used_data":15824,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580976,"statfs":{"total":1151940427776,"available":1151570919424,"internally_reserved":0,"allocated":16203776,"data_stored":8886110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13842,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":74,"num_write":13,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005215"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272883+0000","last_change":"2024-06-03T02:14:09.768431+0000","last_active":"2024-06-03T02:15:54.272883+0000","last_peered":"2024-06-03T02:15:54.272883+0000","last_clean":"2024-06-03T02:15:54.272883+0000","last_became_active":"2024-06-03T02:14:09.768059+0000","last_became_peered":"2024-06-03T02:14:09.768059+0000","last_unstale":"2024-06-03T02:15:54.272883+0000","last_undegraded":"2024-06-03T02:15:54.272883+0000","last_fullsized":"2024-06-03T02:15:54.272883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:12.265896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342398+0000","last_change":"2024-06-03T02:14:09.771842+0000","last_active":"2024-06-03T02:15:54.342398+0000","last_peered":"2024-06-03T02:15:54.342398+0000","last_clean":"2024-06-03T02:15:54.342398+0000","last_became_active":"2024-06-03T02:14:09.770961+0000","last_became_peered":"2024-06-03T02:14:09.770961+0000","last_unstale":"2024-06-03T02:15:54.342398+0000","last_undegraded":"2024-06-03T02:15:54.342398+0000","last_fullsized":"2024-06-03T02:15:54.342398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:49:02.297128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342266+0000","last_change":"2024-06-03T02:14:09.758407+0000","last_active":"2024-06-03T02:15:54.342266+0000","last_peered":"2024-06-03T02:15:54.342266+0000","last_clean":"2024-06-03T02:15:54.342266+0000","last_became_active":"2024-06-03T02:14:09.758045+0000","last_became_peered":"2024-06-03T02:14:09.758045+0000","last_unstale":"2024-06-03T02:15:54.342266+0000","last_undegraded":"2024-06-03T02:15:54.342266+0000","last_fullsized":"2024-06-03T02:15:54.342266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:01:08.166971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341562+0000","last_change":"2024-06-03T02:14:09.747658+0000","last_active":"2024-06-03T02:15:54.341562+0000","last_peered":"2024-06-03T02:15:54.341562+0000","last_clean":"2024-06-03T02:15:54.341562+0000","last_became_active":"2024-06-03T02:14:09.747321+0000","last_became_peered":"2024-06-03T02:14:09.747321+0000","last_unstale":"2024-06-03T02:15:54.341562+0000","last_undegraded":"2024-06-03T02:15:54.341562+0000","last_fullsized":"2024-06-03T02:15:54.341562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:08.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270618+0000","last_change":"2024-06-03T02:14:09.758161+0000","last_active":"2024-06-03T02:15:54.270618+0000","last_peered":"2024-06-03T02:15:54.270618+0000","last_clean":"2024-06-03T02:15:54.270618+0000","last_became_active":"2024-06-03T02:14:09.757800+0000","last_became_peered":"2024-06-03T02:14:09.757800+0000","last_unstale":"2024-06-03T02:15:54.270618+0000","last_undegraded":"2024-06-03T02:15:54.270618+0000","last_fullsized":"2024-06-03T02:15:54.270618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:47.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263499+0000","last_change":"2024-06-03T02:14:09.741818+0000","last_active":"2024-06-03T02:15:54.263499+0000","last_peered":"2024-06-03T02:15:54.263499+0000","last_clean":"2024-06-03T02:15:54.263499+0000","last_became_active":"2024-06-03T02:14:09.741219+0000","last_became_peered":"2024-06-03T02:14:09.741219+0000","last_unstale":"2024-06-03T02:15:54.263499+0000","last_undegraded":"2024-06-03T02:15:54.263499+0000","last_fullsized":"2024-06-03T02:15:54.263499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:16:30.585245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340981+0000","last_change":"2024-06-03T02:14:09.744863+0000","last_active":"2024-06-03T02:15:54.340981+0000","last_peered":"2024-06-03T02:15:54.340981+0000","last_clean":"2024-06-03T02:15:54.340981+0000","last_became_active":"2024-06-03T02:14:09.744514+0000","last_became_peered":"2024-06-03T02:14:09.744514+0000","last_unstale":"2024-06-03T02:15:54.340981+0000","last_undegraded":"2024-06-03T02:15:54.340981+0000","last_fullsized":"2024-06-03T02:15:54.340981+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:51:42.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266789+0000","last_change":"2024-06-03T02:14:09.766897+0000","last_active":"2024-06-03T02:15:54.266789+0000","last_peered":"2024-06-03T02:15:54.266789+0000","last_clean":"2024-06-03T02:15:54.266789+0000","last_became_active":"2024-06-03T02:14:09.765461+0000","last_became_peered":"2024-06-03T02:14:09.765461+0000","last_unstale":"2024-06-03T02:15:54.266789+0000","last_undegraded":"2024-06-03T02:15:54.266789+0000","last_fullsized":"2024-06-03T02:15:54.266789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:39:28.641517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341477+0000","last_change":"2024-06-03T02:14:09.746451+0000","last_active":"2024-06-03T02:15:54.341477+0000","last_peered":"2024-06-03T02:15:54.341477+0000","last_clean":"2024-06-03T02:15:54.341477+0000","last_became_active":"2024-06-03T02:14:09.746117+0000","last_became_peered":"2024-06-03T02:14:09.746117+0000","last_unstale":"2024-06-03T02:15:54.341477+0000","last_undegraded":"2024-06-03T02:15:54.341477+0000","last_fullsized":"2024-06-03T02:15:54.341477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:42.599993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.264014+0000","last_change":"2024-06-03T02:14:09.750680+0000","last_active":"2024-06-03T02:15:54.264014+0000","last_peered":"2024-06-03T02:15:54.264014+0000","last_clean":"2024-06-03T02:15:54.264014+0000","last_became_active":"2024-06-03T02:14:09.746138+0000","last_became_peered":"2024-06-03T02:14:09.746138+0000","last_unstale":"2024-06-03T02:15:54.264014+0000","last_undegraded":"2024-06-03T02:15:54.264014+0000","last_fullsized":"2024-06-03T02:15:54.264014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:04:28.188323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341516+0000","last_change":"2024-06-03T02:14:09.743467+0000","last_active":"2024-06-03T02:15:54.341516+0000","last_peered":"2024-06-03T02:15:54.341516+0000","last_clean":"2024-06-03T02:15:54.341516+0000","last_became_active":"2024-06-03T02:14:09.743152+0000","last_became_peered":"2024-06-03T02:14:09.743152+0000","last_unstale":"2024-06-03T02:15:54.341516+0000","last_undegraded":"2024-06-03T02:15:54.341516+0000","last_fullsized":"2024-06-03T02:15:54.341516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:27.336630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341391+0000","last_change":"2024-06-03T02:14:09.747142+0000","last_active":"2024-06-03T02:15:54.341391+0000","last_peered":"2024-06-03T02:15:54.341391+0000","last_clean":"2024-06-03T02:15:54.341391+0000","last_became_active":"2024-06-03T02:14:09.746694+0000","last_became_peered":"2024-06-03T02:14:09.746694+0000","last_unstale":"2024-06-03T02:15:54.341391+0000","last_undegraded":"2024-06-03T02:15:54.341391+0000","last_fullsized":"2024-06-03T02:15:54.341391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:04.507667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342866+0000","last_change":"2024-06-03T02:14:09.759906+0000","last_active":"2024-06-03T02:15:54.342866+0000","last_peered":"2024-06-03T02:15:54.342866+0000","last_clean":"2024-06-03T02:15:54.342866+0000","last_became_active":"2024-06-03T02:14:09.755450+0000","last_became_peered":"2024-06-03T02:14:09.755450+0000","last_unstale":"2024-06-03T02:15:54.342866+0000","last_undegraded":"2024-06-03T02:15:54.342866+0000","last_fullsized":"2024-06-03T02:15:54.342866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:23.057956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341599+0000","last_change":"2024-06-03T02:14:09.769388+0000","last_active":"2024-06-03T02:15:54.341599+0000","last_peered":"2024-06-03T02:15:54.341599+0000","last_clean":"2024-06-03T02:15:54.341599+0000","last_became_active":"2024-06-03T02:14:09.769029+0000","last_became_peered":"2024-06-03T02:14:09.769029+0000","last_unstale":"2024-06-03T02:15:54.341599+0000","last_undegraded":"2024-06-03T02:15:54.341599+0000","last_fullsized":"2024-06-03T02:15:54.341599+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:24.751039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341616+0000","last_change":"2024-06-03T02:14:09.768584+0000","last_active":"2024-06-03T02:15:54.341616+0000","last_peered":"2024-06-03T02:15:54.341616+0000","last_clean":"2024-06-03T02:15:54.341616+0000","last_became_active":"2024-06-03T02:14:09.765737+0000","last_became_peered":"2024-06-03T02:14:09.765737+0000","last_unstale":"2024-06-03T02:15:54.341616+0000","last_undegraded":"2024-06-03T02:15:54.341616+0000","last_fullsized":"2024-06-03T02:15:54.341616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:36:18.807762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339814+0000","last_change":"2024-06-03T02:14:09.757828+0000","last_active":"2024-06-03T02:15:54.339814+0000","last_peered":"2024-06-03T02:15:54.339814+0000","last_clean":"2024-06-03T02:15:54.339814+0000","last_became_active":"2024-06-03T02:14:09.757040+0000","last_became_peered":"2024-06-03T02:14:09.757040+0000","last_unstale":"2024-06-03T02:15:54.339814+0000","last_undegraded":"2024-06-03T02:15:54.339814+0000","last_fullsized":"2024-06-03T02:15:54.339814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:26:28.694022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272608+0000","last_change":"2024-06-03T02:14:09.768263+0000","last_active":"2024-06-03T02:15:54.272608+0000","last_peered":"2024-06-03T02:15:54.272608+0000","last_clean":"2024-06-03T02:15:54.272608+0000","last_became_active":"2024-06-03T02:14:09.767988+0000","last_became_peered":"2024-06-03T02:14:09.767988+0000","last_unstale":"2024-06-03T02:15:54.272608+0000","last_undegraded":"2024-06-03T02:15:54.272608+0000","last_fullsized":"2024-06-03T02:15:54.272608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:28:37.432659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.265565+0000","last_change":"2024-06-03T02:14:09.747118+0000","last_active":"2024-06-03T02:15:54.265565+0000","last_peered":"2024-06-03T02:15:54.265565+0000","last_clean":"2024-06-03T02:15:54.265565+0000","last_became_active":"2024-06-03T02:14:09.746580+0000","last_became_peered":"2024-06-03T02:14:09.746580+0000","last_unstale":"2024-06-03T02:15:54.265565+0000","last_undegraded":"2024-06-03T02:15:54.265565+0000","last_fullsized":"2024-06-03T02:15:54.265565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:33:04.607285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272733+0000","last_change":"2024-06-03T02:14:09.755755+0000","last_active":"2024-06-03T02:15:54.272733+0000","last_peered":"2024-06-03T02:15:54.272733+0000","last_clean":"2024-06-03T02:15:54.272733+0000","last_became_active":"2024-06-03T02:14:09.754503+0000","last_became_peered":"2024-06-03T02:14:09.754503+0000","last_unstale":"2024-06-03T02:15:54.272733+0000","last_undegraded":"2024-06-03T02:15:54.272733+0000","last_fullsized":"2024-06-03T02:15:54.272733+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:30:58.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271140+0000","last_change":"2024-06-03T02:14:23.260021+0000","last_active":"2024-06-03T02:15:54.271140+0000","last_peered":"2024-06-03T02:15:54.271140+0000","last_clean":"2024-06-03T02:15:54.271140+0000","last_became_active":"2024-06-03T02:14:23.259587+0000","last_became_peered":"2024-06-03T02:14:23.259587+0000","last_unstale":"2024-06-03T02:15:54.271140+0000","last_undegraded":"2024-06-03T02:15:54.271140+0000","last_fullsized":"2024-06-03T02:15:54.271140+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:09:02.403936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'13","reported_seq":94,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:10.398448+0000","last_change":"2024-06-03T02:14:05.239623+0000","last_active":"2024-06-03T02:16:10.398448+0000","last_peered":"2024-06-03T02:16:10.398448+0000","last_clean":"2024-06-03T02:16:10.398448+0000","last_became_active":"2024-06-03T02:14:05.239112+0000","last_became_peered":"2024-06-03T02:14:05.239112+0000","last_unstale":"2024-06-03T02:16:10.398448+0000","last_undegraded":"2024-06-03T02:16:10.398448+0000","last_fullsized":"2024-06-03T02:16:10.398448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:12:14.020068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267640+0000","last_change":"2024-06-03T02:14:09.764972+0000","last_active":"2024-06-03T02:15:54.267640+0000","last_peered":"2024-06-03T02:15:54.267640+0000","last_clean":"2024-06-03T02:15:54.267640+0000","last_became_active":"2024-06-03T02:14:09.764435+0000","last_became_peered":"2024-06-03T02:14:09.764435+0000","last_unstale":"2024-06-03T02:15:54.267640+0000","last_undegraded":"2024-06-03T02:15:54.267640+0000","last_fullsized":"2024-06-03T02:15:54.267640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:46:58.941554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339903+0000","last_change":"2024-06-03T02:14:23.272076+0000","last_active":"2024-06-03T02:15:54.339903+0000","last_peered":"2024-06-03T02:15:54.339903+0000","last_clean":"2024-06-03T02:15:54.339903+0000","last_became_active":"2024-06-03T02:14:23.271578+0000","last_became_peered":"2024-06-03T02:14:23.271578+0000","last_unstale":"2024-06-03T02:15:54.339903+0000","last_undegraded":"2024-06-03T02:15:54.339903+0000","last_fullsized":"2024-06-03T02:15:54.339903+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:17:59.000291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"99'2","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268287+0000","last_change":"2024-06-03T02:14:05.251952+0000","last_active":"2024-06-03T02:15:54.268287+0000","last_peered":"2024-06-03T02:15:54.268287+0000","last_clean":"2024-06-03T02:15:54.268287+0000","last_became_active":"2024-06-03T02:14:05.251152+0000","last_became_peered":"2024-06-03T02:14:05.251152+0000","last_unstale":"2024-06-03T02:15:54.268287+0000","last_undegraded":"2024-06-03T02:15:54.268287+0000","last_fullsized":"2024-06-03T02:15:54.268287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:24:32.802236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342512+0000","last_change":"2024-06-03T02:14:09.764230+0000","last_active":"2024-06-03T02:15:54.342512+0000","last_peered":"2024-06-03T02:15:54.342512+0000","last_clean":"2024-06-03T02:15:54.342512+0000","last_became_active":"2024-06-03T02:14:09.761868+0000","last_became_peered":"2024-06-03T02:14:09.761868+0000","last_unstale":"2024-06-03T02:15:54.342512+0000","last_undegraded":"2024-06-03T02:15:54.342512+0000","last_fullsized":"2024-06-03T02:15:54.342512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:39:23.828546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340345+0000","last_change":"2024-06-03T02:14:23.272169+0000","last_active":"2024-06-03T02:15:54.340345+0000","last_peered":"2024-06-03T02:15:54.340345+0000","last_clean":"2024-06-03T02:15:54.340345+0000","last_became_active":"2024-06-03T02:14:23.271636+0000","last_became_peered":"2024-06-03T02:14:23.271636+0000","last_unstale":"2024-06-03T02:15:54.340345+0000","last_undegraded":"2024-06-03T02:15:54.340345+0000","last_fullsized":"2024-06-03T02:15:54.340345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:51:25.478028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"99'12","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270004+0000","last_change":"2024-06-03T02:14:05.238590+0000","last_active":"2024-06-03T02:15:54.270004+0000","last_peered":"2024-06-03T02:15:54.270004+0000","last_clean":"2024-06-03T02:15:54.270004+0000","last_became_active":"2024-06-03T02:14:05.238128+0000","last_became_peered":"2024-06-03T02:14:05.238128+0000","last_unstale":"2024-06-03T02:15:54.270004+0000","last_undegraded":"2024-06-03T02:15:54.270004+0000","last_fullsized":"2024-06-03T02:15:54.270004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:21.298255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340609+0000","last_change":"2024-06-03T02:14:09.754489+0000","last_active":"2024-06-03T02:15:54.340609+0000","last_peered":"2024-06-03T02:15:54.340609+0000","last_clean":"2024-06-03T02:15:54.340609+0000","last_became_active":"2024-06-03T02:14:09.754155+0000","last_became_peered":"2024-06-03T02:14:09.754155+0000","last_unstale":"2024-06-03T02:15:54.340609+0000","last_undegraded":"2024-06-03T02:15:54.340609+0000","last_fullsized":"2024-06-03T02:15:54.340609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:20:43.841271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340482+0000","last_change":"2024-06-03T02:14:23.261232+0000","last_active":"2024-06-03T02:15:54.340482+0000","last_peered":"2024-06-03T02:15:54.340482+0000","last_clean":"2024-06-03T02:15:54.340482+0000","last_became_active":"2024-06-03T02:14:23.260772+0000","last_became_peered":"2024-06-03T02:14:23.260772+0000","last_unstale":"2024-06-03T02:15:54.340482+0000","last_undegraded":"2024-06-03T02:15:54.340482+0000","last_fullsized":"2024-06-03T02:15:54.340482+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:52.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"99'5","reported_seq":23,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272188+0000","last_change":"2024-06-03T02:14:53.274084+0000","last_active":"2024-06-03T02:15:54.272188+0000","last_peered":"2024-06-03T02:15:54.272188+0000","last_clean":"2024-06-03T02:15:54.272188+0000","last_became_active":"2024-06-03T02:14:53.265253+0000","last_became_peered":"2024-06-03T02:14:53.265253+0000","last_unstale":"2024-06-03T02:15:54.272188+0000","last_undegraded":"2024-06-03T02:15:54.272188+0000","last_fullsized":"2024-06-03T02:15:54.272188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:25:56.189308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339882+0000","last_change":"2024-06-03T02:14:09.743450+0000","last_active":"2024-06-03T02:15:54.339882+0000","last_peered":"2024-06-03T02:15:54.339882+0000","last_clean":"2024-06-03T02:15:54.339882+0000","last_became_active":"2024-06-03T02:14:09.743071+0000","last_became_peered":"2024-06-03T02:14:09.743071+0000","last_unstale":"2024-06-03T02:15:54.339882+0000","last_undegraded":"2024-06-03T02:15:54.339882+0000","last_fullsized":"2024-06-03T02:15:54.339882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:43:44.278633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":35,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342978+0000","last_change":"2024-06-03T02:15:54.342978+0000","last_active":"2024-06-03T02:15:54.342978+0000","last_peered":"2024-06-03T02:15:54.342978+0000","last_clean":"2024-06-03T02:15:54.342978+0000","last_became_active":"2024-06-03T02:15:54.342477+0000","last_became_peered":"2024-06-03T02:15:54.342477+0000","last_unstale":"2024-06-03T02:15:54.342978+0000","last_undegraded":"2024-06-03T02:15:54.342978+0000","last_fullsized":"2024-06-03T02:15:54.342978+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:47.980513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'6","reported_seq":89,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:10.391249+0000","last_change":"2024-06-03T02:14:05.237464+0000","last_active":"2024-06-03T02:16:10.391249+0000","last_peered":"2024-06-03T02:16:10.391249+0000","last_clean":"2024-06-03T02:16:10.391249+0000","last_became_active":"2024-06-03T02:14:05.237067+0000","last_became_peered":"2024-06-03T02:14:05.237067+0000","last_unstale":"2024-06-03T02:16:10.391249+0000","last_undegraded":"2024-06-03T02:16:10.391249+0000","last_fullsized":"2024-06-03T02:16:10.391249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:06:38.850484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270471+0000","last_change":"2024-06-03T02:14:09.748741+0000","last_active":"2024-06-03T02:15:54.270471+0000","last_peered":"2024-06-03T02:15:54.270471+0000","last_clean":"2024-06-03T02:15:54.270471+0000","last_became_active":"2024-06-03T02:14:09.748402+0000","last_became_peered":"2024-06-03T02:14:09.748402+0000","last_unstale":"2024-06-03T02:15:54.270471+0000","last_undegraded":"2024-06-03T02:15:54.270471+0000","last_fullsized":"2024-06-03T02:15:54.270471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:03:25.664415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271782+0000","last_change":"2024-06-03T02:14:23.260397+0000","last_active":"2024-06-03T02:15:54.271782+0000","last_peered":"2024-06-03T02:15:54.271782+0000","last_clean":"2024-06-03T02:15:54.271782+0000","last_became_active":"2024-06-03T02:14:23.259886+0000","last_became_peered":"2024-06-03T02:14:23.259886+0000","last_unstale":"2024-06-03T02:15:54.271782+0000","last_undegraded":"2024-06-03T02:15:54.271782+0000","last_fullsized":"2024-06-03T02:15:54.271782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:46:18.562706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"99'7","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263469+0000","last_change":"2024-06-03T02:14:05.241536+0000","last_active":"2024-06-03T02:15:54.263469+0000","last_peered":"2024-06-03T02:15:54.263469+0000","last_clean":"2024-06-03T02:15:54.263469+0000","last_became_active":"2024-06-03T02:14:05.241203+0000","last_became_peered":"2024-06-03T02:14:05.241203+0000","last_unstale":"2024-06-03T02:15:54.263469+0000","last_undegraded":"2024-06-03T02:15:54.263469+0000","last_fullsized":"2024-06-03T02:15:54.263469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:47.052563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340387+0000","last_change":"2024-06-03T02:14:53.475893+0000","last_active":"2024-06-03T02:15:54.340387+0000","last_peered":"2024-06-03T02:15:54.340387+0000","last_clean":"2024-06-03T02:15:54.340387+0000","last_became_active":"2024-06-03T02:14:53.475494+0000","last_became_peered":"2024-06-03T02:14:53.475494+0000","last_unstale":"2024-06-03T02:15:54.340387+0000","last_undegraded":"2024-06-03T02:15:54.340387+0000","last_fullsized":"2024-06-03T02:15:54.340387+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:27.113842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.344417+0000","last_change":"2024-06-03T02:15:54.344417+0000","last_active":"2024-06-03T02:15:54.344417+0000","last_peered":"2024-06-03T02:15:54.344417+0000","last_clean":"2024-06-03T02:15:54.344417+0000","last_became_active":"2024-06-03T02:15:54.343904+0000","last_became_peered":"2024-06-03T02:15:54.343904+0000","last_unstale":"2024-06-03T02:15:54.344417+0000","last_undegraded":"2024-06-03T02:15:54.344417+0000","last_fullsized":"2024-06-03T02:15:54.344417+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:42:18.121696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"99'3","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.343756+0000","last_change":"2024-06-03T02:14:05.237743+0000","last_active":"2024-06-03T02:15:54.343756+0000","last_peered":"2024-06-03T02:15:54.343756+0000","last_clean":"2024-06-03T02:15:54.343756+0000","last_became_active":"2024-06-03T02:14:05.236944+0000","last_became_peered":"2024-06-03T02:14:05.236944+0000","last_unstale":"2024-06-03T02:15:54.343756+0000","last_undegraded":"2024-06-03T02:15:54.343756+0000","last_fullsized":"2024-06-03T02:15:54.343756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:14:09.850862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266443+0000","last_change":"2024-06-03T02:14:09.761694+0000","last_active":"2024-06-03T02:15:54.266443+0000","last_peered":"2024-06-03T02:15:54.266443+0000","last_clean":"2024-06-03T02:15:54.266443+0000","last_became_active":"2024-06-03T02:14:09.761361+0000","last_became_peered":"2024-06-03T02:14:09.761361+0000","last_unstale":"2024-06-03T02:15:54.266443+0000","last_undegraded":"2024-06-03T02:15:54.266443+0000","last_fullsized":"2024-06-03T02:15:54.266443+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:19:17.803088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342247+0000","last_change":"2024-06-03T02:14:23.258634+0000","last_active":"2024-06-03T02:15:54.342247+0000","last_peered":"2024-06-03T02:15:54.342247+0000","last_clean":"2024-06-03T02:15:54.342247+0000","last_became_active":"2024-06-03T02:14:23.257799+0000","last_became_peered":"2024-06-03T02:14:23.257799+0000","last_unstale":"2024-06-03T02:15:54.342247+0000","last_undegraded":"2024-06-03T02:15:54.342247+0000","last_fullsized":"2024-06-03T02:15:54.342247+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:38:33.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"99'3","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339831+0000","last_change":"2024-06-03T02:14:05.236790+0000","last_active":"2024-06-03T02:15:54.339831+0000","last_peered":"2024-06-03T02:15:54.339831+0000","last_clean":"2024-06-03T02:15:54.339831+0000","last_became_active":"2024-06-03T02:14:05.236419+0000","last_became_peered":"2024-06-03T02:14:05.236419+0000","last_unstale":"2024-06-03T02:15:54.339831+0000","last_undegraded":"2024-06-03T02:15:54.339831+0000","last_fullsized":"2024-06-03T02:15:54.339831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:44:00.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.264958+0000","last_change":"2024-06-03T02:14:09.744680+0000","last_active":"2024-06-03T02:15:54.264958+0000","last_peered":"2024-06-03T02:15:54.264958+0000","last_clean":"2024-06-03T02:15:54.264958+0000","last_became_active":"2024-06-03T02:14:09.744250+0000","last_became_peered":"2024-06-03T02:14:09.744250+0000","last_unstale":"2024-06-03T02:15:54.264958+0000","last_undegraded":"2024-06-03T02:15:54.264958+0000","last_fullsized":"2024-06-03T02:15:54.264958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:58.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267464+0000","last_change":"2024-06-03T02:14:23.259325+0000","last_active":"2024-06-03T02:15:54.267464+0000","last_peered":"2024-06-03T02:15:54.267464+0000","last_clean":"2024-06-03T02:15:54.267464+0000","last_became_active":"2024-06-03T02:14:23.258925+0000","last_became_peered":"2024-06-03T02:14:23.258925+0000","last_unstale":"2024-06-03T02:15:54.267464+0000","last_undegraded":"2024-06-03T02:15:54.267464+0000","last_fullsized":"2024-06-03T02:15:54.267464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:32.038564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":114,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:08.373467+0000","last_change":"2024-06-03T02:14:05.242690+0000","last_active":"2024-06-03T02:16:08.373467+0000","last_peered":"2024-06-03T02:16:08.373467+0000","last_clean":"2024-06-03T02:16:08.373467+0000","last_became_active":"2024-06-03T02:14:05.242320+0000","last_became_peered":"2024-06-03T02:14:05.242320+0000","last_unstale":"2024-06-03T02:16:08.373467+0000","last_undegraded":"2024-06-03T02:16:08.373467+0000","last_fullsized":"2024-06-03T02:16:08.373467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:02.793186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341190+0000","last_change":"2024-06-03T02:14:09.747146+0000","last_active":"2024-06-03T02:15:54.341190+0000","last_peered":"2024-06-03T02:15:54.341190+0000","last_clean":"2024-06-03T02:15:54.341190+0000","last_became_active":"2024-06-03T02:14:09.746803+0000","last_became_peered":"2024-06-03T02:14:09.746803+0000","last_unstale":"2024-06-03T02:15:54.341190+0000","last_undegraded":"2024-06-03T02:15:54.341190+0000","last_fullsized":"2024-06-03T02:15:54.341190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:04:36.262223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.264835+0000","last_change":"2024-06-03T02:14:23.270328+0000","last_active":"2024-06-03T02:15:54.264835+0000","last_peered":"2024-06-03T02:15:54.264835+0000","last_clean":"2024-06-03T02:15:54.264835+0000","last_became_active":"2024-06-03T02:14:23.264189+0000","last_became_peered":"2024-06-03T02:14:23.264189+0000","last_unstale":"2024-06-03T02:15:54.264835+0000","last_undegraded":"2024-06-03T02:15:54.264835+0000","last_fullsized":"2024-06-03T02:15:54.264835+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:36.248884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"99'4","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340089+0000","last_change":"2024-06-03T02:14:05.238898+0000","last_active":"2024-06-03T02:15:54.340089+0000","last_peered":"2024-06-03T02:15:54.340089+0000","last_clean":"2024-06-03T02:15:54.340089+0000","last_became_active":"2024-06-03T02:14:05.238552+0000","last_became_peered":"2024-06-03T02:14:05.238552+0000","last_unstale":"2024-06-03T02:15:54.340089+0000","last_undegraded":"2024-06-03T02:15:54.340089+0000","last_fullsized":"2024-06-03T02:15:54.340089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:29:33.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342202+0000","last_change":"2024-06-03T02:14:09.769973+0000","last_active":"2024-06-03T02:15:54.342202+0000","last_peered":"2024-06-03T02:15:54.342202+0000","last_clean":"2024-06-03T02:15:54.342202+0000","last_became_active":"2024-06-03T02:14:09.768984+0000","last_became_peered":"2024-06-03T02:14:09.768984+0000","last_unstale":"2024-06-03T02:15:54.342202+0000","last_undegraded":"2024-06-03T02:15:54.342202+0000","last_fullsized":"2024-06-03T02:15:54.342202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:48:33.720441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340563+0000","last_change":"2024-06-03T02:14:23.265947+0000","last_active":"2024-06-03T02:15:54.340563+0000","last_peered":"2024-06-03T02:15:54.340563+0000","last_clean":"2024-06-03T02:15:54.340563+0000","last_became_active":"2024-06-03T02:14:23.265412+0000","last_became_peered":"2024-06-03T02:14:23.265412+0000","last_unstale":"2024-06-03T02:15:54.340563+0000","last_undegraded":"2024-06-03T02:15:54.340563+0000","last_fullsized":"2024-06-03T02:15:54.340563+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:35:08.664884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"99'5","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339833+0000","last_change":"2024-06-03T02:14:05.252751+0000","last_active":"2024-06-03T02:15:54.339833+0000","last_peered":"2024-06-03T02:15:54.339833+0000","last_clean":"2024-06-03T02:15:54.339833+0000","last_became_active":"2024-06-03T02:14:05.252267+0000","last_became_peered":"2024-06-03T02:14:05.252267+0000","last_unstale":"2024-06-03T02:15:54.339833+0000","last_undegraded":"2024-06-03T02:15:54.339833+0000","last_fullsized":"2024-06-03T02:15:54.339833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:58.091711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340723+0000","last_change":"2024-06-03T02:14:09.750757+0000","last_active":"2024-06-03T02:15:54.340723+0000","last_peered":"2024-06-03T02:15:54.340723+0000","last_clean":"2024-06-03T02:15:54.340723+0000","last_became_active":"2024-06-03T02:14:09.750414+0000","last_became_peered":"2024-06-03T02:14:09.750414+0000","last_unstale":"2024-06-03T02:15:54.340723+0000","last_undegraded":"2024-06-03T02:15:54.340723+0000","last_fullsized":"2024-06-03T02:15:54.340723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:29:55.435408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.264974+0000","last_change":"2024-06-03T02:14:23.263440+0000","last_active":"2024-06-03T02:15:54.264974+0000","last_peered":"2024-06-03T02:15:54.264974+0000","last_clean":"2024-06-03T02:15:54.264974+0000","last_became_active":"2024-06-03T02:14:23.254019+0000","last_became_peered":"2024-06-03T02:14:23.254019+0000","last_unstale":"2024-06-03T02:15:54.264974+0000","last_undegraded":"2024-06-03T02:15:54.264974+0000","last_fullsized":"2024-06-03T02:15:54.264974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:48:58.745990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'35","reported_seq":154,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:08.373492+0000","last_change":"2024-06-03T02:14:05.239353+0000","last_active":"2024-06-03T02:16:08.373492+0000","last_peered":"2024-06-03T02:16:08.373492+0000","last_clean":"2024-06-03T02:16:08.373492+0000","last_became_active":"2024-06-03T02:14:05.238976+0000","last_became_peered":"2024-06-03T02:14:05.238976+0000","last_unstale":"2024-06-03T02:16:08.373492+0000","last_undegraded":"2024-06-03T02:16:08.373492+0000","last_fullsized":"2024-06-03T02:16:08.373492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:51.598533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62214,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":97,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270245+0000","last_change":"2024-06-03T02:14:09.746546+0000","last_active":"2024-06-03T02:15:54.270245+0000","last_peered":"2024-06-03T02:15:54.270245+0000","last_clean":"2024-06-03T02:15:54.270245+0000","last_became_active":"2024-06-03T02:14:09.746049+0000","last_became_peered":"2024-06-03T02:14:09.746049+0000","last_unstale":"2024-06-03T02:15:54.270245+0000","last_undegraded":"2024-06-03T02:15:54.270245+0000","last_fullsized":"2024-06-03T02:15:54.270245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:34.667133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340487+0000","last_change":"2024-06-03T02:14:23.257296+0000","last_active":"2024-06-03T02:15:54.340487+0000","last_peered":"2024-06-03T02:15:54.340487+0000","last_clean":"2024-06-03T02:15:54.340487+0000","last_became_active":"2024-06-03T02:14:23.256858+0000","last_became_peered":"2024-06-03T02:14:23.256858+0000","last_unstale":"2024-06-03T02:15:54.340487+0000","last_undegraded":"2024-06-03T02:15:54.340487+0000","last_fullsized":"2024-06-03T02:15:54.340487+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:00.464902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"99'9","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271601+0000","last_change":"2024-06-03T02:14:05.245163+0000","last_active":"2024-06-03T02:15:54.271601+0000","last_peered":"2024-06-03T02:15:54.271601+0000","last_clean":"2024-06-03T02:15:54.271601+0000","last_became_active":"2024-06-03T02:14:05.244813+0000","last_became_peered":"2024-06-03T02:14:05.244813+0000","last_unstale":"2024-06-03T02:15:54.271601+0000","last_undegraded":"2024-06-03T02:15:54.271601+0000","last_fullsized":"2024-06-03T02:15:54.271601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:46:47.017859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.344473+0000","last_change":"2024-06-03T02:14:09.757200+0000","last_active":"2024-06-03T02:15:54.344473+0000","last_peered":"2024-06-03T02:15:54.344473+0000","last_clean":"2024-06-03T02:15:54.344473+0000","last_became_active":"2024-06-03T02:14:09.756380+0000","last_became_peered":"2024-06-03T02:14:09.756380+0000","last_unstale":"2024-06-03T02:15:54.344473+0000","last_undegraded":"2024-06-03T02:15:54.344473+0000","last_fullsized":"2024-06-03T02:15:54.344473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:56.045338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272345+0000","last_change":"2024-06-03T02:14:23.258441+0000","last_active":"2024-06-03T02:15:54.272345+0000","last_peered":"2024-06-03T02:15:54.272345+0000","last_clean":"2024-06-03T02:15:54.272345+0000","last_became_active":"2024-06-03T02:14:23.258015+0000","last_became_peered":"2024-06-03T02:14:23.258015+0000","last_unstale":"2024-06-03T02:15:54.272345+0000","last_undegraded":"2024-06-03T02:15:54.272345+0000","last_fullsized":"2024-06-03T02:15:54.272345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:03.123511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"99'5","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340103+0000","last_change":"2024-06-03T02:14:05.240456+0000","last_active":"2024-06-03T02:15:54.340103+0000","last_peered":"2024-06-03T02:15:54.340103+0000","last_clean":"2024-06-03T02:15:54.340103+0000","last_became_active":"2024-06-03T02:14:05.240078+0000","last_became_peered":"2024-06-03T02:14:05.240078+0000","last_unstale":"2024-06-03T02:15:54.340103+0000","last_undegraded":"2024-06-03T02:15:54.340103+0000","last_fullsized":"2024-06-03T02:15:54.340103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:00:26.491773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340396+0000","last_change":"2024-06-03T02:14:09.750876+0000","last_active":"2024-06-03T02:15:54.340396+0000","last_peered":"2024-06-03T02:15:54.340396+0000","last_clean":"2024-06-03T02:15:54.340396+0000","last_became_active":"2024-06-03T02:14:09.750547+0000","last_became_peered":"2024-06-03T02:14:09.750547+0000","last_unstale":"2024-06-03T02:15:54.340396+0000","last_undegraded":"2024-06-03T02:15:54.340396+0000","last_fullsized":"2024-06-03T02:15:54.340396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:08:41.136225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268624+0000","last_change":"2024-06-03T02:14:23.276172+0000","last_active":"2024-06-03T02:15:54.268624+0000","last_peered":"2024-06-03T02:15:54.268624+0000","last_clean":"2024-06-03T02:15:54.268624+0000","last_became_active":"2024-06-03T02:14:23.275410+0000","last_became_peered":"2024-06-03T02:14:23.275410+0000","last_unstale":"2024-06-03T02:15:54.268624+0000","last_undegraded":"2024-06-03T02:15:54.268624+0000","last_fullsized":"2024-06-03T02:15:54.268624+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:16:30.859453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'6","reported_seq":88,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:09.389948+0000","last_change":"2024-06-03T02:14:05.237263+0000","last_active":"2024-06-03T02:16:09.389948+0000","last_peered":"2024-06-03T02:16:09.389948+0000","last_clean":"2024-06-03T02:16:09.389948+0000","last_became_active":"2024-06-03T02:14:05.236890+0000","last_became_peered":"2024-06-03T02:14:05.236890+0000","last_unstale":"2024-06-03T02:16:09.389948+0000","last_undegraded":"2024-06-03T02:16:09.389948+0000","last_fullsized":"2024-06-03T02:16:09.389948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:42.292545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341259+0000","last_change":"2024-06-03T02:14:09.746260+0000","last_active":"2024-06-03T02:15:54.341259+0000","last_peered":"2024-06-03T02:15:54.341259+0000","last_clean":"2024-06-03T02:15:54.341259+0000","last_became_active":"2024-06-03T02:14:09.745940+0000","last_became_peered":"2024-06-03T02:14:09.745940+0000","last_unstale":"2024-06-03T02:15:54.341259+0000","last_undegraded":"2024-06-03T02:15:54.341259+0000","last_fullsized":"2024-06-03T02:15:54.341259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:08:55.851464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345328+0000","last_change":"2024-06-03T02:14:23.265440+0000","last_active":"2024-06-03T02:15:54.345328+0000","last_peered":"2024-06-03T02:15:54.345328+0000","last_clean":"2024-06-03T02:15:54.345328+0000","last_became_active":"2024-06-03T02:14:23.265028+0000","last_became_peered":"2024-06-03T02:14:23.265028+0000","last_unstale":"2024-06-03T02:15:54.345328+0000","last_undegraded":"2024-06-03T02:15:54.345328+0000","last_fullsized":"2024-06-03T02:15:54.345328+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:56.237500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"99'17","reported_seq":78,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.269940+0000","last_change":"2024-06-03T02:14:53.271821+0000","last_active":"2024-06-03T02:15:54.269940+0000","last_peered":"2024-06-03T02:15:54.269940+0000","last_clean":"2024-06-03T02:15:54.269940+0000","last_became_active":"2024-06-03T02:14:53.263047+0000","last_became_peered":"2024-06-03T02:14:53.263047+0000","last_unstale":"2024-06-03T02:15:54.269940+0000","last_undegraded":"2024-06-03T02:15:54.269940+0000","last_fullsized":"2024-06-03T02:15:54.269940+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:02:25.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340413+0000","last_change":"2024-06-03T02:14:09.754819+0000","last_active":"2024-06-03T02:15:54.340413+0000","last_peered":"2024-06-03T02:15:54.340413+0000","last_clean":"2024-06-03T02:15:54.340413+0000","last_became_active":"2024-06-03T02:14:09.753995+0000","last_became_peered":"2024-06-03T02:14:09.753995+0000","last_unstale":"2024-06-03T02:15:54.340413+0000","last_undegraded":"2024-06-03T02:15:54.340413+0000","last_fullsized":"2024-06-03T02:15:54.340413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:19:51.123774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340453+0000","last_change":"2024-06-03T02:14:23.265003+0000","last_active":"2024-06-03T02:15:54.340453+0000","last_peered":"2024-06-03T02:15:54.340453+0000","last_clean":"2024-06-03T02:15:54.340453+0000","last_became_active":"2024-06-03T02:14:23.264529+0000","last_became_peered":"2024-06-03T02:14:23.264529+0000","last_unstale":"2024-06-03T02:15:54.340453+0000","last_undegraded":"2024-06-03T02:15:54.340453+0000","last_fullsized":"2024-06-03T02:15:54.340453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:50:19.838274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270555+0000","last_change":"2024-06-03T02:14:05.236901+0000","last_active":"2024-06-03T02:15:54.270555+0000","last_peered":"2024-06-03T02:15:54.270555+0000","last_clean":"2024-06-03T02:15:54.270555+0000","last_became_active":"2024-06-03T02:14:05.236460+0000","last_became_peered":"2024-06-03T02:14:05.236460+0000","last_unstale":"2024-06-03T02:15:54.270555+0000","last_undegraded":"2024-06-03T02:15:54.270555+0000","last_fullsized":"2024-06-03T02:15:54.270555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:59.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272038+0000","last_change":"2024-06-03T02:14:09.743135+0000","last_active":"2024-06-03T02:15:54.272038+0000","last_peered":"2024-06-03T02:15:54.272038+0000","last_clean":"2024-06-03T02:15:54.272038+0000","last_became_active":"2024-06-03T02:14:09.742506+0000","last_became_peered":"2024-06-03T02:14:09.742506+0000","last_unstale":"2024-06-03T02:15:54.272038+0000","last_undegraded":"2024-06-03T02:15:54.272038+0000","last_fullsized":"2024-06-03T02:15:54.272038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:51.413537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270340+0000","last_change":"2024-06-03T02:14:23.255512+0000","last_active":"2024-06-03T02:15:54.270340+0000","last_peered":"2024-06-03T02:15:54.270340+0000","last_clean":"2024-06-03T02:15:54.270340+0000","last_became_active":"2024-06-03T02:14:23.255103+0000","last_became_peered":"2024-06-03T02:14:23.255103+0000","last_unstale":"2024-06-03T02:15:54.270340+0000","last_undegraded":"2024-06-03T02:15:54.270340+0000","last_fullsized":"2024-06-03T02:15:54.270340+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:12:05.295848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"99'1","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266549+0000","last_change":"2024-06-03T02:14:05.248728+0000","last_active":"2024-06-03T02:15:54.266549+0000","last_peered":"2024-06-03T02:15:54.266549+0000","last_clean":"2024-06-03T02:15:54.266549+0000","last_became_active":"2024-06-03T02:14:05.248144+0000","last_became_peered":"2024-06-03T02:14:05.248144+0000","last_unstale":"2024-06-03T02:15:54.266549+0000","last_undegraded":"2024-06-03T02:15:54.266549+0000","last_fullsized":"2024-06-03T02:15:54.266549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:53:27.814443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263703+0000","last_change":"2024-06-03T02:14:09.752029+0000","last_active":"2024-06-03T02:15:54.263703+0000","last_peered":"2024-06-03T02:15:54.263703+0000","last_clean":"2024-06-03T02:15:54.263703+0000","last_became_active":"2024-06-03T02:14:09.751638+0000","last_became_peered":"2024-06-03T02:14:09.751638+0000","last_unstale":"2024-06-03T02:15:54.263703+0000","last_undegraded":"2024-06-03T02:15:54.263703+0000","last_fullsized":"2024-06-03T02:15:54.263703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:51.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.346477+0000","last_change":"2024-06-03T02:14:23.250406+0000","last_active":"2024-06-03T02:15:54.346477+0000","last_peered":"2024-06-03T02:15:54.346477+0000","last_clean":"2024-06-03T02:15:54.346477+0000","last_became_active":"2024-06-03T02:14:23.249946+0000","last_became_peered":"2024-06-03T02:14:23.249946+0000","last_unstale":"2024-06-03T02:15:54.346477+0000","last_undegraded":"2024-06-03T02:15:54.346477+0000","last_fullsized":"2024-06-03T02:15:54.346477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:04:47.714532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"99'6","reported_seq":84,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:06.390989+0000","last_change":"2024-06-03T02:14:05.241666+0000","last_active":"2024-06-03T02:16:06.390989+0000","last_peered":"2024-06-03T02:16:06.390989+0000","last_clean":"2024-06-03T02:16:06.390989+0000","last_became_active":"2024-06-03T02:14:05.241299+0000","last_became_peered":"2024-06-03T02:14:05.241299+0000","last_unstale":"2024-06-03T02:16:06.390989+0000","last_undegraded":"2024-06-03T02:16:06.390989+0000","last_fullsized":"2024-06-03T02:16:06.390989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:21:16.792022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263546+0000","last_change":"2024-06-03T02:14:09.768602+0000","last_active":"2024-06-03T02:15:54.263546+0000","last_peered":"2024-06-03T02:15:54.263546+0000","last_clean":"2024-06-03T02:15:54.263546+0000","last_became_active":"2024-06-03T02:14:09.768195+0000","last_became_peered":"2024-06-03T02:14:09.768195+0000","last_unstale":"2024-06-03T02:15:54.263546+0000","last_undegraded":"2024-06-03T02:15:54.263546+0000","last_fullsized":"2024-06-03T02:15:54.263546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:45:42.361966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.346593+0000","last_change":"2024-06-03T02:14:23.252472+0000","last_active":"2024-06-03T02:15:54.346593+0000","last_peered":"2024-06-03T02:15:54.346593+0000","last_clean":"2024-06-03T02:15:54.346593+0000","last_became_active":"2024-06-03T02:14:23.252018+0000","last_became_peered":"2024-06-03T02:14:23.252018+0000","last_unstale":"2024-06-03T02:15:54.346593+0000","last_undegraded":"2024-06-03T02:15:54.346593+0000","last_fullsized":"2024-06-03T02:15:54.346593+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:28.468657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340581+0000","last_change":"2024-06-03T02:14:09.744443+0000","last_active":"2024-06-03T02:15:54.340581+0000","last_peered":"2024-06-03T02:15:54.340581+0000","last_clean":"2024-06-03T02:15:54.340581+0000","last_became_active":"2024-06-03T02:14:09.744100+0000","last_became_peered":"2024-06-03T02:14:09.744100+0000","last_unstale":"2024-06-03T02:15:54.340581+0000","last_undegraded":"2024-06-03T02:15:54.340581+0000","last_fullsized":"2024-06-03T02:15:54.340581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:44.566745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":86,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:10.390920+0000","last_change":"2024-06-03T02:14:05.250734+0000","last_active":"2024-06-03T02:16:10.390920+0000","last_peered":"2024-06-03T02:16:10.390920+0000","last_clean":"2024-06-03T02:16:10.390920+0000","last_became_active":"2024-06-03T02:14:05.250300+0000","last_became_peered":"2024-06-03T02:14:05.250300+0000","last_unstale":"2024-06-03T02:16:10.390920+0000","last_undegraded":"2024-06-03T02:16:10.390920+0000","last_fullsized":"2024-06-03T02:16:10.390920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:52.322895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":36,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341425+0000","last_change":"2024-06-03T02:14:53.471397+0000","last_active":"2024-06-03T02:15:54.341425+0000","last_peered":"2024-06-03T02:15:54.341425+0000","last_clean":"2024-06-03T02:15:54.341425+0000","last_became_active":"2024-06-03T02:14:53.470868+0000","last_became_peered":"2024-06-03T02:14:53.470868+0000","last_unstale":"2024-06-03T02:15:54.341425+0000","last_undegraded":"2024-06-03T02:15:54.341425+0000","last_fullsized":"2024-06-03T02:15:54.341425+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:58.726070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268231+0000","last_change":"2024-06-03T02:14:09.765193+0000","last_active":"2024-06-03T02:15:54.268231+0000","last_peered":"2024-06-03T02:15:54.268231+0000","last_clean":"2024-06-03T02:15:54.268231+0000","last_became_active":"2024-06-03T02:14:09.764834+0000","last_became_peered":"2024-06-03T02:14:09.764834+0000","last_unstale":"2024-06-03T02:15:54.268231+0000","last_undegraded":"2024-06-03T02:15:54.268231+0000","last_fullsized":"2024-06-03T02:15:54.268231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:20:37.464286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"99'6","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340500+0000","last_change":"2024-06-03T02:14:05.240118+0000","last_active":"2024-06-03T02:15:54.340500+0000","last_peered":"2024-06-03T02:15:54.340500+0000","last_clean":"2024-06-03T02:15:54.340500+0000","last_became_active":"2024-06-03T02:14:05.239771+0000","last_became_peered":"2024-06-03T02:14:05.239771+0000","last_unstale":"2024-06-03T02:15:54.340500+0000","last_undegraded":"2024-06-03T02:15:54.340500+0000","last_fullsized":"2024-06-03T02:15:54.340500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:08.256222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":370,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340223+0000","last_change":"2024-06-03T02:12:31.842613+0000","last_active":"2024-06-03T02:15:54.340223+0000","last_peered":"2024-06-03T02:15:54.340223+0000","last_clean":"2024-06-03T02:15:54.340223+0000","last_became_active":"2024-06-03T02:12:31.842051+0000","last_became_peered":"2024-06-03T02:12:31.842051+0000","last_unstale":"2024-06-03T02:15:54.340223+0000","last_undegraded":"2024-06-03T02:15:54.340223+0000","last_fullsized":"2024-06-03T02:15:54.340223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_clean_scrub_stamp":"2024-06-03T02:08:38.749702+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:40:29.748789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340282+0000","last_change":"2024-06-03T02:14:23.255849+0000","last_active":"2024-06-03T02:15:54.340282+0000","last_peered":"2024-06-03T02:15:54.340282+0000","last_clean":"2024-06-03T02:15:54.340282+0000","last_became_active":"2024-06-03T02:14:23.255380+0000","last_became_peered":"2024-06-03T02:14:23.255380+0000","last_unstale":"2024-06-03T02:15:54.340282+0000","last_undegraded":"2024-06-03T02:15:54.340282+0000","last_fullsized":"2024-06-03T02:15:54.340282+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:41:16.051326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341820+0000","last_change":"2024-06-03T02:14:53.270423+0000","last_active":"2024-06-03T02:15:54.341820+0000","last_peered":"2024-06-03T02:15:54.341820+0000","last_clean":"2024-06-03T02:15:54.341820+0000","last_became_active":"2024-06-03T02:14:53.269260+0000","last_became_peered":"2024-06-03T02:14:53.269260+0000","last_unstale":"2024-06-03T02:15:54.341820+0000","last_undegraded":"2024-06-03T02:15:54.341820+0000","last_fullsized":"2024-06-03T02:15:54.341820+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:57.392012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"99'4","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341175+0000","last_change":"2024-06-03T02:14:05.235982+0000","last_active":"2024-06-03T02:15:54.341175+0000","last_peered":"2024-06-03T02:15:54.341175+0000","last_clean":"2024-06-03T02:15:54.341175+0000","last_became_active":"2024-06-03T02:14:05.235638+0000","last_became_peered":"2024-06-03T02:14:05.235638+0000","last_unstale":"2024-06-03T02:15:54.341175+0000","last_undegraded":"2024-06-03T02:15:54.341175+0000","last_fullsized":"2024-06-03T02:15:54.341175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:42:07.544005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271797+0000","last_change":"2024-06-03T02:14:23.257790+0000","last_active":"2024-06-03T02:15:54.271797+0000","last_peered":"2024-06-03T02:15:54.271797+0000","last_clean":"2024-06-03T02:15:54.271797+0000","last_became_active":"2024-06-03T02:14:23.257297+0000","last_became_peered":"2024-06-03T02:14:23.257297+0000","last_unstale":"2024-06-03T02:15:54.271797+0000","last_undegraded":"2024-06-03T02:15:54.271797+0000","last_fullsized":"2024-06-03T02:15:54.271797+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:13.715455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"99'4","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272719+0000","last_change":"2024-06-03T02:14:53.268495+0000","last_active":"2024-06-03T02:15:54.272719+0000","last_peered":"2024-06-03T02:15:54.272719+0000","last_clean":"2024-06-03T02:15:54.272719+0000","last_became_active":"2024-06-03T02:14:53.268147+0000","last_became_peered":"2024-06-03T02:14:53.268147+0000","last_unstale":"2024-06-03T02:15:54.272719+0000","last_undegraded":"2024-06-03T02:15:54.272719+0000","last_fullsized":"2024-06-03T02:15:54.272719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:23.076319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345591+0000","last_change":"2024-06-03T02:14:09.758244+0000","last_active":"2024-06-03T02:15:54.345591+0000","last_peered":"2024-06-03T02:15:54.345591+0000","last_clean":"2024-06-03T02:15:54.345591+0000","last_became_active":"2024-06-03T02:14:09.757812+0000","last_became_peered":"2024-06-03T02:14:09.757812+0000","last_unstale":"2024-06-03T02:15:54.345591+0000","last_undegraded":"2024-06-03T02:15:54.345591+0000","last_fullsized":"2024-06-03T02:15:54.345591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:08.243703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266108+0000","last_change":"2024-06-03T02:14:23.259980+0000","last_active":"2024-06-03T02:15:54.266108+0000","last_peered":"2024-06-03T02:15:54.266108+0000","last_clean":"2024-06-03T02:15:54.266108+0000","last_became_active":"2024-06-03T02:14:23.258308+0000","last_became_peered":"2024-06-03T02:14:23.258308+0000","last_unstale":"2024-06-03T02:15:54.266108+0000","last_undegraded":"2024-06-03T02:15:54.266108+0000","last_fullsized":"2024-06-03T02:15:54.266108+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:28.833994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'16","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:01.785970+0000","last_change":"2024-06-03T02:14:05.244855+0000","last_active":"2024-06-03T02:16:01.785970+0000","last_peered":"2024-06-03T02:16:01.785970+0000","last_clean":"2024-06-03T02:16:01.785970+0000","last_became_active":"2024-06-03T02:14:05.244335+0000","last_became_peered":"2024-06-03T02:14:05.244335+0000","last_unstale":"2024-06-03T02:16:01.785970+0000","last_undegraded":"2024-06-03T02:16:01.785970+0000","last_fullsized":"2024-06-03T02:16:01.785970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:53:19.065411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268184+0000","last_change":"2024-06-03T02:14:09.750761+0000","last_active":"2024-06-03T02:15:54.268184+0000","last_peered":"2024-06-03T02:15:54.268184+0000","last_clean":"2024-06-03T02:15:54.268184+0000","last_became_active":"2024-06-03T02:14:09.750336+0000","last_became_peered":"2024-06-03T02:14:09.750336+0000","last_unstale":"2024-06-03T02:15:54.268184+0000","last_undegraded":"2024-06-03T02:15:54.268184+0000","last_fullsized":"2024-06-03T02:15:54.268184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:59:46.651149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270963+0000","last_change":"2024-06-03T02:14:23.243287+0000","last_active":"2024-06-03T02:15:54.270963+0000","last_peered":"2024-06-03T02:15:54.270963+0000","last_clean":"2024-06-03T02:15:54.270963+0000","last_became_active":"2024-06-03T02:14:23.242828+0000","last_became_peered":"2024-06-03T02:14:23.242828+0000","last_unstale":"2024-06-03T02:15:54.270963+0000","last_undegraded":"2024-06-03T02:15:54.270963+0000","last_fullsized":"2024-06-03T02:15:54.270963+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:12.312438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"99'7","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341870+0000","last_change":"2024-06-03T02:14:05.596288+0000","last_active":"2024-06-03T02:15:54.341870+0000","last_peered":"2024-06-03T02:15:54.341870+0000","last_clean":"2024-06-03T02:15:54.341870+0000","last_became_active":"2024-06-03T02:14:05.595868+0000","last_became_peered":"2024-06-03T02:14:05.595868+0000","last_unstale":"2024-06-03T02:15:54.341870+0000","last_undegraded":"2024-06-03T02:15:54.341870+0000","last_fullsized":"2024-06-03T02:15:54.341870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:50.291541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342398+0000","last_change":"2024-06-03T02:14:09.765317+0000","last_active":"2024-06-03T02:15:54.342398+0000","last_peered":"2024-06-03T02:15:54.342398+0000","last_clean":"2024-06-03T02:15:54.342398+0000","last_became_active":"2024-06-03T02:14:09.764983+0000","last_became_peered":"2024-06-03T02:14:09.764983+0000","last_unstale":"2024-06-03T02:15:54.342398+0000","last_undegraded":"2024-06-03T02:15:54.342398+0000","last_fullsized":"2024-06-03T02:15:54.342398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:20.075276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341589+0000","last_change":"2024-06-03T02:14:23.265352+0000","last_active":"2024-06-03T02:15:54.341589+0000","last_peered":"2024-06-03T02:15:54.341589+0000","last_clean":"2024-06-03T02:15:54.341589+0000","last_became_active":"2024-06-03T02:14:23.264932+0000","last_became_peered":"2024-06-03T02:14:23.264932+0000","last_unstale":"2024-06-03T02:15:54.341589+0000","last_undegraded":"2024-06-03T02:15:54.341589+0000","last_fullsized":"2024-06-03T02:15:54.341589+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:43.803170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"99'2","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339930+0000","last_change":"2024-06-03T02:14:05.248563+0000","last_active":"2024-06-03T02:15:54.339930+0000","last_peered":"2024-06-03T02:15:54.339930+0000","last_clean":"2024-06-03T02:15:54.339930+0000","last_became_active":"2024-06-03T02:14:05.248306+0000","last_became_peered":"2024-06-03T02:14:05.248306+0000","last_unstale":"2024-06-03T02:15:54.339930+0000","last_undegraded":"2024-06-03T02:15:54.339930+0000","last_fullsized":"2024-06-03T02:15:54.339930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:23:58.130157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.269836+0000","last_change":"2024-06-03T02:14:09.745948+0000","last_active":"2024-06-03T02:15:54.269836+0000","last_peered":"2024-06-03T02:15:54.269836+0000","last_clean":"2024-06-03T02:15:54.269836+0000","last_became_active":"2024-06-03T02:14:09.745626+0000","last_became_peered":"2024-06-03T02:14:09.745626+0000","last_unstale":"2024-06-03T02:15:54.269836+0000","last_undegraded":"2024-06-03T02:15:54.269836+0000","last_fullsized":"2024-06-03T02:15:54.269836+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:34.742129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342659+0000","last_change":"2024-06-03T02:15:54.341824+0000","last_active":"2024-06-03T02:15:54.342659+0000","last_peered":"2024-06-03T02:15:54.342659+0000","last_clean":"2024-06-03T02:15:54.342659+0000","last_became_active":"2024-06-03T02:15:54.341267+0000","last_became_peered":"2024-06-03T02:15:54.341267+0000","last_unstale":"2024-06-03T02:15:54.342659+0000","last_undegraded":"2024-06-03T02:15:54.342659+0000","last_fullsized":"2024-06-03T02:15:54.342659+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:47.714056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"99'7","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267630+0000","last_change":"2024-06-03T02:14:05.244806+0000","last_active":"2024-06-03T02:15:54.267630+0000","last_peered":"2024-06-03T02:15:54.267630+0000","last_clean":"2024-06-03T02:15:54.267630+0000","last_became_active":"2024-06-03T02:14:05.242719+0000","last_became_peered":"2024-06-03T02:14:05.242719+0000","last_unstale":"2024-06-03T02:15:54.267630+0000","last_undegraded":"2024-06-03T02:15:54.267630+0000","last_fullsized":"2024-06-03T02:15:54.267630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:31.314266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271781+0000","last_change":"2024-06-03T02:14:09.755102+0000","last_active":"2024-06-03T02:15:54.271781+0000","last_peered":"2024-06-03T02:15:54.271781+0000","last_clean":"2024-06-03T02:15:54.271781+0000","last_became_active":"2024-06-03T02:14:09.754748+0000","last_became_peered":"2024-06-03T02:14:09.754748+0000","last_unstale":"2024-06-03T02:15:54.271781+0000","last_undegraded":"2024-06-03T02:15:54.271781+0000","last_fullsized":"2024-06-03T02:15:54.271781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:59.920710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"101'1","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341202+0000","last_change":"2024-06-03T02:14:23.271974+0000","last_active":"2024-06-03T02:15:54.341202+0000","last_peered":"2024-06-03T02:15:54.341202+0000","last_clean":"2024-06-03T02:15:54.341202+0000","last_became_active":"2024-06-03T02:14:23.271470+0000","last_became_peered":"2024-06-03T02:14:23.271470+0000","last_unstale":"2024-06-03T02:15:54.341202+0000","last_undegraded":"2024-06-03T02:15:54.341202+0000","last_fullsized":"2024-06-03T02:15:54.341202+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:03.159351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"99'3","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271407+0000","last_change":"2024-06-03T02:14:05.242261+0000","last_active":"2024-06-03T02:15:54.271407+0000","last_peered":"2024-06-03T02:15:54.271407+0000","last_clean":"2024-06-03T02:15:54.271407+0000","last_became_active":"2024-06-03T02:14:05.241854+0000","last_became_peered":"2024-06-03T02:14:05.241854+0000","last_unstale":"2024-06-03T02:15:54.271407+0000","last_undegraded":"2024-06-03T02:15:54.271407+0000","last_fullsized":"2024-06-03T02:15:54.271407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:21.427703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268011+0000","last_change":"2024-06-03T02:14:09.763591+0000","last_active":"2024-06-03T02:15:54.268011+0000","last_peered":"2024-06-03T02:15:54.268011+0000","last_clean":"2024-06-03T02:15:54.268011+0000","last_became_active":"2024-06-03T02:14:09.763240+0000","last_became_peered":"2024-06-03T02:14:09.763240+0000","last_unstale":"2024-06-03T02:15:54.268011+0000","last_undegraded":"2024-06-03T02:15:54.268011+0000","last_fullsized":"2024-06-03T02:15:54.268011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:40.146898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341547+0000","last_change":"2024-06-03T02:14:23.265011+0000","last_active":"2024-06-03T02:15:54.341547+0000","last_peered":"2024-06-03T02:15:54.341547+0000","last_clean":"2024-06-03T02:15:54.341547+0000","last_became_active":"2024-06-03T02:14:23.263767+0000","last_became_peered":"2024-06-03T02:14:23.263767+0000","last_unstale":"2024-06-03T02:15:54.341547+0000","last_undegraded":"2024-06-03T02:15:54.341547+0000","last_fullsized":"2024-06-03T02:15:54.341547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:34:04.274545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"99'4","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270449+0000","last_change":"2024-06-03T02:14:53.474879+0000","last_active":"2024-06-03T02:15:54.270449+0000","last_peered":"2024-06-03T02:15:54.270449+0000","last_clean":"2024-06-03T02:15:54.270449+0000","last_became_active":"2024-06-03T02:14:53.472209+0000","last_became_peered":"2024-06-03T02:14:53.472209+0000","last_unstale":"2024-06-03T02:15:54.270449+0000","last_undegraded":"2024-06-03T02:15:54.270449+0000","last_fullsized":"2024-06-03T02:15:54.270449+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:36:57.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340200+0000","last_change":"2024-06-03T02:14:09.747756+0000","last_active":"2024-06-03T02:15:54.340200+0000","last_peered":"2024-06-03T02:15:54.340200+0000","last_clean":"2024-06-03T02:15:54.340200+0000","last_became_active":"2024-06-03T02:14:09.747453+0000","last_became_peered":"2024-06-03T02:14:09.747453+0000","last_unstale":"2024-06-03T02:15:54.340200+0000","last_undegraded":"2024-06-03T02:15:54.340200+0000","last_fullsized":"2024-06-03T02:15:54.340200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:19:02.341367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339663+0000","last_change":"2024-06-03T02:14:23.248594+0000","last_active":"2024-06-03T02:15:54.339663+0000","last_peered":"2024-06-03T02:15:54.339663+0000","last_clean":"2024-06-03T02:15:54.339663+0000","last_became_active":"2024-06-03T02:14:23.248080+0000","last_became_peered":"2024-06-03T02:14:23.248080+0000","last_unstale":"2024-06-03T02:15:54.339663+0000","last_undegraded":"2024-06-03T02:15:54.339663+0000","last_fullsized":"2024-06-03T02:15:54.339663+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:24:42.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"99'5","reported_seq":82,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:05.369383+0000","last_change":"2024-06-03T02:14:05.596274+0000","last_active":"2024-06-03T02:16:05.369383+0000","last_peered":"2024-06-03T02:16:05.369383+0000","last_clean":"2024-06-03T02:16:05.369383+0000","last_became_active":"2024-06-03T02:14:05.595678+0000","last_became_peered":"2024-06-03T02:14:05.595678+0000","last_unstale":"2024-06-03T02:16:05.369383+0000","last_undegraded":"2024-06-03T02:16:05.369383+0000","last_fullsized":"2024-06-03T02:16:05.369383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:13:25.436688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341638+0000","last_change":"2024-06-03T02:14:09.744934+0000","last_active":"2024-06-03T02:15:54.341638+0000","last_peered":"2024-06-03T02:15:54.341638+0000","last_clean":"2024-06-03T02:15:54.341638+0000","last_became_active":"2024-06-03T02:14:09.744621+0000","last_became_peered":"2024-06-03T02:14:09.744621+0000","last_unstale":"2024-06-03T02:15:54.341638+0000","last_undegraded":"2024-06-03T02:15:54.341638+0000","last_fullsized":"2024-06-03T02:15:54.341638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:01:51.970160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341780+0000","last_change":"2024-06-03T02:14:23.273123+0000","last_active":"2024-06-03T02:15:54.341780+0000","last_peered":"2024-06-03T02:15:54.341780+0000","last_clean":"2024-06-03T02:15:54.341780+0000","last_became_active":"2024-06-03T02:14:23.272153+0000","last_became_peered":"2024-06-03T02:14:23.272153+0000","last_unstale":"2024-06-03T02:15:54.341780+0000","last_undegraded":"2024-06-03T02:15:54.341780+0000","last_fullsized":"2024-06-03T02:15:54.341780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:01:43.374413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"99'3","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340330+0000","last_change":"2024-06-03T02:14:05.245094+0000","last_active":"2024-06-03T02:15:54.340330+0000","last_peered":"2024-06-03T02:15:54.340330+0000","last_clean":"2024-06-03T02:15:54.340330+0000","last_became_active":"2024-06-03T02:14:05.244678+0000","last_became_peered":"2024-06-03T02:14:05.244678+0000","last_unstale":"2024-06-03T02:15:54.340330+0000","last_undegraded":"2024-06-03T02:15:54.340330+0000","last_fullsized":"2024-06-03T02:15:54.340330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:13:26.546135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340574+0000","last_change":"2024-06-03T02:14:09.745919+0000","last_active":"2024-06-03T02:15:54.340574+0000","last_peered":"2024-06-03T02:15:54.340574+0000","last_clean":"2024-06-03T02:15:54.340574+0000","last_became_active":"2024-06-03T02:14:09.745576+0000","last_became_peered":"2024-06-03T02:14:09.745576+0000","last_unstale":"2024-06-03T02:15:54.340574+0000","last_undegraded":"2024-06-03T02:15:54.340574+0000","last_fullsized":"2024-06-03T02:15:54.340574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:29:31.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339508+0000","last_change":"2024-06-03T02:14:23.244174+0000","last_active":"2024-06-03T02:15:54.339508+0000","last_peered":"2024-06-03T02:15:54.339508+0000","last_clean":"2024-06-03T02:15:54.339508+0000","last_became_active":"2024-06-03T02:14:23.243595+0000","last_became_peered":"2024-06-03T02:14:23.243595+0000","last_unstale":"2024-06-03T02:15:54.339508+0000","last_undegraded":"2024-06-03T02:15:54.339508+0000","last_fullsized":"2024-06-03T02:15:54.339508+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:12:36.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"99'5","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345635+0000","last_change":"2024-06-03T02:14:05.247669+0000","last_active":"2024-06-03T02:15:54.345635+0000","last_peered":"2024-06-03T02:15:54.345635+0000","last_clean":"2024-06-03T02:15:54.345635+0000","last_became_active":"2024-06-03T02:14:05.246685+0000","last_became_peered":"2024-06-03T02:14:05.246685+0000","last_unstale":"2024-06-03T02:15:54.345635+0000","last_undegraded":"2024-06-03T02:15:54.345635+0000","last_fullsized":"2024-06-03T02:15:54.345635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:39.454422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340640+0000","last_change":"2024-06-03T02:14:09.749242+0000","last_active":"2024-06-03T02:15:54.340640+0000","last_peered":"2024-06-03T02:15:54.340640+0000","last_clean":"2024-06-03T02:15:54.340640+0000","last_became_active":"2024-06-03T02:14:09.748370+0000","last_became_peered":"2024-06-03T02:14:09.748370+0000","last_unstale":"2024-06-03T02:15:54.340640+0000","last_undegraded":"2024-06-03T02:15:54.340640+0000","last_fullsized":"2024-06-03T02:15:54.340640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:26.189960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271931+0000","last_change":"2024-06-03T02:14:09.745966+0000","last_active":"2024-06-03T02:15:54.271931+0000","last_peered":"2024-06-03T02:15:54.271931+0000","last_clean":"2024-06-03T02:15:54.271931+0000","last_became_active":"2024-06-03T02:14:09.745590+0000","last_became_peered":"2024-06-03T02:14:09.745590+0000","last_unstale":"2024-06-03T02:15:54.271931+0000","last_undegraded":"2024-06-03T02:15:54.271931+0000","last_fullsized":"2024-06-03T02:15:54.271931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:46.744629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271380+0000","last_change":"2024-06-03T02:14:09.744964+0000","last_active":"2024-06-03T02:15:54.271380+0000","last_peered":"2024-06-03T02:15:54.271380+0000","last_clean":"2024-06-03T02:15:54.271380+0000","last_became_active":"2024-06-03T02:14:09.743877+0000","last_became_peered":"2024-06-03T02:14:09.743877+0000","last_unstale":"2024-06-03T02:15:54.271380+0000","last_undegraded":"2024-06-03T02:15:54.271380+0000","last_fullsized":"2024-06-03T02:15:54.271380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:05:48.020670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271297+0000","last_change":"2024-06-03T02:14:09.761736+0000","last_active":"2024-06-03T02:15:54.271297+0000","last_peered":"2024-06-03T02:15:54.271297+0000","last_clean":"2024-06-03T02:15:54.271297+0000","last_became_active":"2024-06-03T02:14:09.761477+0000","last_became_peered":"2024-06-03T02:14:09.761477+0000","last_unstale":"2024-06-03T02:15:54.271297+0000","last_undegraded":"2024-06-03T02:15:54.271297+0000","last_fullsized":"2024-06-03T02:15:54.271297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:00:04.631330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.343120+0000","last_change":"2024-06-03T02:14:53.266594+0000","last_active":"2024-06-03T02:15:54.343120+0000","last_peered":"2024-06-03T02:15:54.343120+0000","last_clean":"2024-06-03T02:15:54.343120+0000","last_became_active":"2024-06-03T02:14:53.266104+0000","last_became_peered":"2024-06-03T02:14:53.266104+0000","last_unstale":"2024-06-03T02:15:54.343120+0000","last_undegraded":"2024-06-03T02:15:54.343120+0000","last_fullsized":"2024-06-03T02:15:54.343120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:12.018082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345423+0000","last_change":"2024-06-03T02:14:09.772332+0000","last_active":"2024-06-03T02:15:54.345423+0000","last_peered":"2024-06-03T02:15:54.345423+0000","last_clean":"2024-06-03T02:15:54.345423+0000","last_became_active":"2024-06-03T02:14:09.771888+0000","last_became_peered":"2024-06-03T02:14:09.771888+0000","last_unstale":"2024-06-03T02:15:54.345423+0000","last_undegraded":"2024-06-03T02:15:54.345423+0000","last_fullsized":"2024-06-03T02:15:54.345423+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:58:49.414986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.273006+0000","last_change":"2024-06-03T02:14:53.270828+0000","last_active":"2024-06-03T02:15:54.273006+0000","last_peered":"2024-06-03T02:15:54.273006+0000","last_clean":"2024-06-03T02:15:54.273006+0000","last_became_active":"2024-06-03T02:14:53.270365+0000","last_became_peered":"2024-06-03T02:14:53.270365+0000","last_unstale":"2024-06-03T02:15:54.273006+0000","last_undegraded":"2024-06-03T02:15:54.273006+0000","last_fullsized":"2024-06-03T02:15:54.273006+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.051756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272851+0000","last_change":"2024-06-03T02:14:09.747709+0000","last_active":"2024-06-03T02:15:54.272851+0000","last_peered":"2024-06-03T02:15:54.272851+0000","last_clean":"2024-06-03T02:15:54.272851+0000","last_became_active":"2024-06-03T02:14:09.747359+0000","last_became_peered":"2024-06-03T02:14:09.747359+0000","last_unstale":"2024-06-03T02:15:54.272851+0000","last_undegraded":"2024-06-03T02:15:54.272851+0000","last_fullsized":"2024-06-03T02:15:54.272851+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:37:24.873442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266772+0000","last_change":"2024-06-03T02:14:53.473136+0000","last_active":"2024-06-03T02:15:54.266772+0000","last_peered":"2024-06-03T02:15:54.266772+0000","last_clean":"2024-06-03T02:15:54.266772+0000","last_became_active":"2024-06-03T02:14:53.471214+0000","last_became_peered":"2024-06-03T02:14:53.471214+0000","last_unstale":"2024-06-03T02:15:54.266772+0000","last_undegraded":"2024-06-03T02:15:54.266772+0000","last_fullsized":"2024-06-03T02:15:54.266772+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:25.822042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340080+0000","last_change":"2024-06-03T02:14:09.746463+0000","last_active":"2024-06-03T02:15:54.340080+0000","last_peered":"2024-06-03T02:15:54.340080+0000","last_clean":"2024-06-03T02:15:54.340080+0000","last_became_active":"2024-06-03T02:14:09.746115+0000","last_became_peered":"2024-06-03T02:14:09.746115+0000","last_unstale":"2024-06-03T02:15:54.340080+0000","last_undegraded":"2024-06-03T02:15:54.340080+0000","last_fullsized":"2024-06-03T02:15:54.340080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:51:23.050739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267946+0000","last_change":"2024-06-03T02:14:09.755467+0000","last_active":"2024-06-03T02:15:54.267946+0000","last_peered":"2024-06-03T02:15:54.267946+0000","last_clean":"2024-06-03T02:15:54.267946+0000","last_became_active":"2024-06-03T02:14:09.755103+0000","last_became_peered":"2024-06-03T02:14:09.755103+0000","last_unstale":"2024-06-03T02:15:54.267946+0000","last_undegraded":"2024-06-03T02:15:54.267946+0000","last_fullsized":"2024-06-03T02:15:54.267946+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:22.834519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340654+0000","last_change":"2024-06-03T02:14:09.758201+0000","last_active":"2024-06-03T02:15:54.340654+0000","last_peered":"2024-06-03T02:15:54.340654+0000","last_clean":"2024-06-03T02:15:54.340654+0000","last_became_active":"2024-06-03T02:14:09.757819+0000","last_became_peered":"2024-06-03T02:14:09.757819+0000","last_unstale":"2024-06-03T02:15:54.340654+0000","last_undegraded":"2024-06-03T02:15:54.340654+0000","last_fullsized":"2024-06-03T02:15:54.340654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:44.802694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342731+0000","last_change":"2024-06-03T02:14:09.768132+0000","last_active":"2024-06-03T02:15:54.342731+0000","last_peered":"2024-06-03T02:15:54.342731+0000","last_clean":"2024-06-03T02:15:54.342731+0000","last_became_active":"2024-06-03T02:14:09.767642+0000","last_became_peered":"2024-06-03T02:14:09.767642+0000","last_unstale":"2024-06-03T02:15:54.342731+0000","last_undegraded":"2024-06-03T02:15:54.342731+0000","last_fullsized":"2024-06-03T02:15:54.342731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:32:30.455418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272556+0000","last_change":"2024-06-03T02:14:09.745603+0000","last_active":"2024-06-03T02:15:54.272556+0000","last_peered":"2024-06-03T02:15:54.272556+0000","last_clean":"2024-06-03T02:15:54.272556+0000","last_became_active":"2024-06-03T02:14:09.745245+0000","last_became_peered":"2024-06-03T02:14:09.745245+0000","last_unstale":"2024-06-03T02:15:54.272556+0000","last_undegraded":"2024-06-03T02:15:54.272556+0000","last_fullsized":"2024-06-03T02:15:54.272556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:09:14.507101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69806,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":378,"num_read_kb":266,"num_write":251,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":209418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318314,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1179648,"data_stored":578717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37160,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707992,"statfs":{"total":95995035648,"available":95956983808,"internally_reserved":0,"allocated":1867776,"data_stored":1231539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":577953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005114,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":577637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266751,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28764,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716388,"statfs":{"total":95995035648,"available":95965581312,"internally_reserved":0,"allocated":1785856,"data_stored":1175607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430278,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1167360,"data_stored":573847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28232,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716920,"statfs":{"total":95995035648,"available":95966126080,"internally_reserved":0,"allocated":1245184,"data_stored":637203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1191936,"data_stored":574913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1249280,"data_stored":637199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888144,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":577921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691916,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37096,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708056,"statfs":{"total":95995035648,"available":95957049344,"internally_reserved":0,"allocated":1794048,"data_stored":1170409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T02:16:14.960 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-03T02:16:14.960 INFO:tasks.cephadm:Setting up client nodes... 2024-06-03T02:16:14.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-03T02:16:15.447 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:16.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:16 smithi079 ceph-mon[31819]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-06-03T02:16:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:16 smithi135 ceph-mon[36653]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-06-03T02:16:16.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:16 smithi123 ceph-mon[36640]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-06-03T02:16:17.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.759+0000 7fc851800700 1 -- 172.21.15.79:0/712153823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84c109b60 msgr2=0x7fc84c109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:17.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.759+0000 7fc851800700 1 --2- 172.21.15.79:0/712153823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84c109b60 0x7fc84c109fc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8340099c0 tx=0x7fc834009cd0 comp rx=0 tx=0).stop 2024-06-03T02:16:17.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.759+0000 7fc851800700 1 -- 172.21.15.79:0/712153823 shutdown_connections 2024-06-03T02:16:17.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.759+0000 7fc851800700 1 --2- 172.21.15.79:0/712153823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc84c10a500 0x7fc84c110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:17.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.759+0000 7fc851800700 1 --2- 172.21.15.79:0/712153823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc84c108960 0x7fc84c108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:17.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.759+0000 7fc851800700 1 --2- 172.21.15.79:0/712153823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84c109b60 0x7fc84c109fc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:17.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.759+0000 7fc851800700 1 -- 172.21.15.79:0/712153823 >> 172.21.15.79:0/712153823 conn(0x7fc84c077930 msgr2=0x7fc84c079d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:17.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.759+0000 7fc851800700 1 -- 172.21.15.79:0/712153823 shutdown_connections 2024-06-03T02:16:17.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.759+0000 7fc851800700 1 -- 172.21.15.79:0/712153823 wait complete. 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.761+0000 7fc851800700 1 Processor -- start 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.761+0000 7fc851800700 1 -- start start 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.761+0000 7fc851800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc84c108960 0x7fc84c07fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.761+0000 7fc851800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84c109b60 0x7fc84c080090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.761+0000 7fc851800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc84c10a500 0x7fc84c1aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.761+0000 7fc851800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c114520 con 0x7fc84c10a500 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.761+0000 7fc851800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c1143a0 con 0x7fc84c109b60 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.761+0000 7fc851800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c1146a0 con 0x7fc84c108960 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc84c108960 0x7fc84c07fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84c109b60 0x7fc84c080090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc84c108960 0x7fc84c07fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42412/0 (socket says 172.21.15.79:42412) 2024-06-03T02:16:17.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84be00700 1 -- 172.21.15.79:0/696399834 learned_addr learned my addr 172.21.15.79:0/696399834 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:17.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc850e00700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc84c10a500 0x7fc84c1aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:17.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84b400700 1 -- 172.21.15.79:0/696399834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc84c108960 msgr2=0x7fc84c07fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:17.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84b400700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc84c108960 0x7fc84c07fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84b400700 1 -- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc84c10a500 msgr2=0x7fc84c1aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84b400700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc84c10a500 0x7fc84c1aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84b400700 1 -- 172.21.15.79:0/696399834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c0832b0 con 0x7fc84c109b60 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84be00700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc84c108960 0x7fc84c07fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.762+0000 7fc84b400700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84c109b60 0x7fc84c080090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8340098e0 tx=0x7fc83400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.763+0000 7fc848c00700 1 -- 172.21.15.79:0/696399834 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834009e60 con 0x7fc84c109b60 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.763+0000 7fc848c00700 1 -- 172.21.15.79:0/696399834 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83400f460 con 0x7fc84c109b60 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.763+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84c1af1d0 con 0x7fc84c109b60 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.763+0000 7fc848c00700 1 -- 172.21.15.79:0/696399834 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834017670 con 0x7fc84c109b60 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.763+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84c1af760 con 0x7fc84c109b60 2024-06-03T02:16:17.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.764+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84c063af0 con 0x7fc84c109b60 2024-06-03T02:16:17.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.767+0000 7fc848c00700 1 -- 172.21.15.79:0/696399834 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc83400f9c0 con 0x7fc84c109b60 2024-06-03T02:16:17.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.768+0000 7fc848c00700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc8300a2de0 0x7fc8300a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:17.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.769+0000 7fc848c00700 1 -- 172.21.15.79:0/696399834 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc8340d0300 con 0x7fc84c109b60 2024-06-03T02:16:17.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.769+0000 7fc84be00700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc8300a2de0 0x7fc8300a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:17.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.770+0000 7fc84be00700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc8300a2de0 0x7fc8300a52a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0105c0 tx=0x7fc83c00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:17.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:17.773+0000 7fc848c00700 1 -- 172.21.15.79:0/696399834 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc834013070 con 0x7fc84c109b60 2024-06-03T02:16:18.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.085+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc84c02dca0 con 0x7fc84c109b60 2024-06-03T02:16:18.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.094+0000 7fc848c00700 1 -- 172.21.15.79:0/696399834 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc83409cd80 con 0x7fc84c109b60 2024-06-03T02:16:18.094 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2024-06-03T02:16:18.094 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQByJ11mcsxIBRAAwEnjNzX5PLI7SnuyzS5brQ== 2024-06-03T02:16:18.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.097+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc8300a2de0 msgr2=0x7fc8300a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:18.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.097+0000 7fc851800700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc8300a2de0 0x7fc8300a52a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0105c0 tx=0x7fc83c00b4a0 comp rx=0 tx=0).stop 2024-06-03T02:16:18.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.097+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84c109b60 msgr2=0x7fc84c080090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:18.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.097+0000 7fc851800700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84c109b60 0x7fc84c080090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8340098e0 tx=0x7fc83400bf60 comp rx=0 tx=0).stop 2024-06-03T02:16:18.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.098+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 shutdown_connections 2024-06-03T02:16:18.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.098+0000 7fc851800700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc8300a2de0 0x7fc8300a52a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:18.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.098+0000 7fc851800700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc84c10a500 0x7fc84c1aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:18.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.098+0000 7fc851800700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc84c108960 0x7fc84c07fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:18.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.098+0000 7fc851800700 1 --2- 172.21.15.79:0/696399834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84c109b60 0x7fc84c080090 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:18.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.098+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 >> 172.21.15.79:0/696399834 conn(0x7fc84c077930 msgr2=0x7fc84c10ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:18.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.098+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 shutdown_connections 2024-06-03T02:16:18.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:18.098+0000 7fc851800700 1 -- 172.21.15.79:0/696399834 wait complete. 2024-06-03T02:16:18.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:18 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/696399834' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T02:16:18.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:18 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T02:16:18.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:18 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T02:16:18.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:18 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/696399834' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T02:16:18.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:18 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T02:16:18.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:18 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T02:16:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:18 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/696399834' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T02:16:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:18 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T02:16:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:18 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T02:16:19.158 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:16:19.158 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-03T02:16:19.158 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-03T02:16:19.275 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-03T02:16:19.275 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-03T02:16:19.275 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-03T02:16:19.390 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:19 smithi079 ceph-mon[31819]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-06-03T02:16:19.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:19 smithi123 ceph-mon[36640]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-06-03T02:16:19.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:19 smithi135 ceph-mon[36653]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-06-03T02:16:19.766 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:20.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:20 smithi079 ceph-mon[31819]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-03T02:16:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:20 smithi135 ceph-mon[36653]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-03T02:16:20.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:20 smithi123 ceph-mon[36640]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-03T02:16:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.101+0000 7f8e79e00700 1 -- 172.21.15.79:0/1414480391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e74108650 msgr2=0x7f8e74108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.101+0000 7f8e79e00700 1 --2- 172.21.15.79:0/1414480391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e74108650 0x7f8e74108a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400b0a0 tx=0x7f8e6400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.101+0000 7f8e79e00700 1 -- 172.21.15.79:0/1414480391 shutdown_connections 2024-06-03T02:16:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.101+0000 7f8e79e00700 1 --2- 172.21.15.79:0/1414480391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e74102e80 0x7f8e7410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.101+0000 7f8e79e00700 1 --2- 172.21.15.79:0/1414480391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e74108650 0x7f8e74108a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.101+0000 7f8e79e00700 1 --2- 172.21.15.79:0/1414480391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e741024e0 0x7f8e74102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.101+0000 7f8e79e00700 1 -- 172.21.15.79:0/1414480391 >> 172.21.15.79:0/1414480391 conn(0x7f8e740795f0 msgr2=0x7f8e74079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.101+0000 7f8e79e00700 1 -- 172.21.15.79:0/1414480391 shutdown_connections 2024-06-03T02:16:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.101+0000 7f8e79e00700 1 -- 172.21.15.79:0/1414480391 wait complete. 2024-06-03T02:16:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.103+0000 7f8e79e00700 1 Processor -- start 2024-06-03T02:16:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.103+0000 7f8e79e00700 1 -- start start 2024-06-03T02:16:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.103+0000 7f8e79e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e741024e0 0x7f8e7419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.103+0000 7f8e79e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e74102e80 0x7f8e7419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.103+0000 7f8e79e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e74108650 0x7f8e7419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.103+0000 7f8e79e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741127f0 con 0x7f8e74108650 2024-06-03T02:16:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.103+0000 7f8e79e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74112670 con 0x7f8e74102e80 2024-06-03T02:16:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.103+0000 7f8e79e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74112970 con 0x7f8e741024e0 2024-06-03T02:16:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e72a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e741024e0 0x7f8e7419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e72a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e741024e0 0x7f8e7419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42436/0 (socket says 172.21.15.79:42436) 2024-06-03T02:16:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e6a000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e74102e80 0x7f8e7419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:22.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e72a00700 1 -- 172.21.15.79:0/2150999235 learned_addr learned my addr 172.21.15.79:0/2150999235 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:22.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e73400700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e74108650 0x7f8e7419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:22.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e72a00700 1 -- 172.21.15.79:0/2150999235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e74102e80 msgr2=0x7f8e7419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:22.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e72a00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e74102e80 0x7f8e7419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e72a00700 1 -- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e74108650 msgr2=0x7f8e7419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:22.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e72a00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e74108650 0x7f8e7419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e72a00700 1 -- 172.21.15.79:0/2150999235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7419fe60 con 0x7f8e741024e0 2024-06-03T02:16:22.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e73400700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e74108650 0x7f8e7419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:16:22.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e6a000700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e74102e80 0x7f8e7419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:22.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.104+0000 7f8e72a00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e741024e0 0x7f8e7419ae50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400b070 tx=0x7f8e64004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:22.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.105+0000 7f8e78a00700 1 -- 172.21.15.79:0/2150999235 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64021070 con 0x7f8e741024e0 2024-06-03T02:16:22.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.105+0000 7f8e78a00700 1 -- 172.21.15.79:0/2150999235 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e64003c00 con 0x7f8e741024e0 2024-06-03T02:16:22.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.105+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e741a00f0 con 0x7f8e741024e0 2024-06-03T02:16:22.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.105+0000 7f8e78a00700 1 -- 172.21.15.79:0/2150999235 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e640048a0 con 0x7f8e741024e0 2024-06-03T02:16:22.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.105+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e741a79d0 con 0x7f8e741024e0 2024-06-03T02:16:22.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.106+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e74069180 con 0x7f8e741024e0 2024-06-03T02:16:22.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.108+0000 7f8e78a00700 1 -- 172.21.15.79:0/2150999235 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8e64003db0 con 0x7f8e741024e0 2024-06-03T02:16:22.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.109+0000 7f8e78a00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8e580a2ed0 0x7f8e580a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:22.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.110+0000 7f8e78a00700 1 -- 172.21.15.79:0/2150999235 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8e6400f020 con 0x7f8e741024e0 2024-06-03T02:16:22.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.110+0000 7f8e6a000700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8e580a2ed0 0x7f8e580a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:22.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.110+0000 7f8e6a000700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8e580a2ed0 0x7f8e580a5390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e74079f30 tx=0x7f8e5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:22.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.112+0000 7f8e78a00700 1 -- 172.21.15.79:0/2150999235 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e6402a020 con 0x7f8e741024e0 2024-06-03T02:16:22.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:22 smithi079 ceph-mon[31819]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T02:16:22.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.405+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8e741a7ba0 con 0x7f8e741024e0 2024-06-03T02:16:22.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.411+0000 7f8e78a00700 1 -- 172.21.15.79:0/2150999235 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263190 (secure 0 0 0) 0x7f8e6409d8b0 con 0x7f8e741024e0 2024-06-03T02:16:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:16:22.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8e580a2ed0 msgr2=0x7f8e580a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8e580a2ed0 0x7f8e580a5390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e74079f30 tx=0x7f8e5c00b410 comp rx=0 tx=0).stop 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e741024e0 msgr2=0x7f8e7419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e741024e0 0x7f8e7419ae50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400b070 tx=0x7f8e64004cf0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 shutdown_connections 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8e580a2ed0 0x7f8e580a5390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e74108650 0x7f8e7419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e741024e0 0x7f8e7419ae50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 --2- 172.21.15.79:0/2150999235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e74102e80 0x7f8e7419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.417+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 >> 172.21.15.79:0/2150999235 conn(0x7f8e740795f0 msgr2=0x7f8e740748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.418+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 shutdown_connections 2024-06-03T02:16:22.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:22.418+0000 7f8e79e00700 1 -- 172.21.15.79:0/2150999235 wait complete. 2024-06-03T02:16:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:22 smithi135 ceph-mon[36653]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T02:16:22.601 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:22 smithi123 ceph-mon[36640]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T02:16:23.318 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2278164},{"type":"v1","addr":"172.21.15.79:6801","nonce":2278164}]},"active_addr":"172.21.15.79:6801/2278164","active_change":"2024-06-03T02:05:52.009056+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24115,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.79:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3816023784}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3483233144}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3617441405}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":21664158}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3057011856}]}]} 2024-06-03T02:16:23.323 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-03T02:16:23.324 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-03T02:16:23.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:16:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:23 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2150999235' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T02:16:23.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:23 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2150999235' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T02:16:23.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:23 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2150999235' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T02:16:23.813 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:24 smithi079 ceph-mon[31819]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T02:16:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:24 smithi135 ceph-mon[36653]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T02:16:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:24 smithi123 ceph-mon[36640]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T02:16:26.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.025+0000 7f1708e00700 1 -- 172.21.15.79:0/4067233185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1704102740 msgr2=0x7f1704102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.025+0000 7f1708e00700 1 --2- 172.21.15.79:0/4067233185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1704102740 0x7f1704102b20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f16f400b0a0 tx=0x7f16f400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.025+0000 7f1708e00700 1 -- 172.21.15.79:0/4067233185 shutdown_connections 2024-06-03T02:16:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.025+0000 7f1708e00700 1 --2- 172.21.15.79:0/4067233185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704107840 0x7f170410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.025+0000 7f1708e00700 1 --2- 172.21.15.79:0/4067233185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1704102740 0x7f1704102b20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.025+0000 7f1708e00700 1 --2- 172.21.15.79:0/4067233185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17041030f0 0x7f1704107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.025+0000 7f1708e00700 1 -- 172.21.15.79:0/4067233185 >> 172.21.15.79:0/4067233185 conn(0x7f17040795f0 msgr2=0x7f1704079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.025+0000 7f1708e00700 1 -- 172.21.15.79:0/4067233185 shutdown_connections 2024-06-03T02:16:26.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.025+0000 7f1708e00700 1 -- 172.21.15.79:0/4067233185 wait complete. 2024-06-03T02:16:26.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.026+0000 7f1708e00700 1 Processor -- start 2024-06-03T02:16:26.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.026+0000 7f1708e00700 1 -- start start 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1708e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704102740 0x7f1704074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1708e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17041030f0 0x7f1704074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1708e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1704107840 0x7f1704075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1708e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1704112780 con 0x7f1704107840 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1708e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1704112600 con 0x7f1704102740 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1708e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1704112900 con 0x7f17041030f0 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1702800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1704107840 0x7f1704075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1701e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704102740 0x7f1704074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f16f9400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17041030f0 0x7f1704074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1701e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704102740 0x7f1704074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48000/0 (socket says 172.21.15.79:48000) 2024-06-03T02:16:26.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.027+0000 7f1701e00700 1 -- 172.21.15.79:0/224564587 learned_addr learned my addr 172.21.15.79:0/224564587 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.028+0000 7f1702800700 1 -- 172.21.15.79:0/224564587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17041030f0 msgr2=0x7f1704074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.028+0000 7f1702800700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17041030f0 0x7f1704074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.028+0000 7f1702800700 1 -- 172.21.15.79:0/224564587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704102740 msgr2=0x7f1704074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:26.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.028+0000 7f1702800700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704102740 0x7f1704074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.028+0000 7f1702800700 1 -- 172.21.15.79:0/224564587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17041a7100 con 0x7f1704107840 2024-06-03T02:16:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.028+0000 7f16f9400700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17041030f0 0x7f1704074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.028+0000 7f1701e00700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704102740 0x7f1704074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.028+0000 7f1702800700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1704107840 0x7f1704075210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e970 tx=0x7f16fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.029+0000 7f16fb400700 1 -- 172.21.15.79:0/224564587 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc004220 con 0x7f1704107840 2024-06-03T02:16:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.029+0000 7f16fb400700 1 -- 172.21.15.79:0/224564587 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16fc004d10 con 0x7f1704107840 2024-06-03T02:16:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.029+0000 7f16fb400700 1 -- 172.21.15.79:0/224564587 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc010700 con 0x7f1704107840 2024-06-03T02:16:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.029+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17041a73f0 con 0x7f1704107840 2024-06-03T02:16:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.029+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041a7930 con 0x7f1704107840 2024-06-03T02:16:26.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.030+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170410aa80 con 0x7f1704107840 2024-06-03T02:16:26.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.032+0000 7f16fb400700 1 -- 172.21.15.79:0/224564587 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f16fc004750 con 0x7f1704107840 2024-06-03T02:16:26.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.033+0000 7f16fb400700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e80a2cb0 0x7f16e80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:26.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.033+0000 7f16fb400700 1 -- 172.21.15.79:0/224564587 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f16fc0cf2b0 con 0x7f1704107840 2024-06-03T02:16:26.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.034+0000 7f1701e00700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e80a2cb0 0x7f16e80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:26.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.034+0000 7f1701e00700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e80a2cb0 0x7f16e80a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009e60 tx=0x7f16f400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:26.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.038+0000 7f16fb400700 1 -- 172.21.15.79:0/224564587 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f16fc09bca0 con 0x7f1704107840 2024-06-03T02:16:26.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.261+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f170410a460 con 0x7f1704107840 2024-06-03T02:16:26.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.265+0000 7f16fb400700 1 -- 172.21.15.79:0/224564587 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+24805 (secure 0 0 0) 0x7f16fc09b5a0 con 0x7f1704107840 2024-06-03T02:16:26.266 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:16:26.267 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":101,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:15:54.257434+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:16:26.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e80a2cb0 msgr2=0x7f16e80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:26.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e80a2cb0 0x7f16e80a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009e60 tx=0x7f16f400b700 comp rx=0 tx=0).stop 2024-06-03T02:16:26.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1704107840 msgr2=0x7f1704075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:26.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1704107840 0x7f1704075210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e970 tx=0x7f16fc00ec80 comp rx=0 tx=0).stop 2024-06-03T02:16:26.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 shutdown_connections 2024-06-03T02:16:26.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16e80a2cb0 0x7f16e80a5170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1704107840 0x7f1704075210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704102740 0x7f1704074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 --2- 172.21.15.79:0/224564587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17041030f0 0x7f1704074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:26.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 >> 172.21.15.79:0/224564587 conn(0x7f17040795f0 msgr2=0x7f1704108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:26.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.270+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 shutdown_connections 2024-06-03T02:16:26.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:26.271+0000 7f1708e00700 1 -- 172.21.15.79:0/224564587 wait complete. 2024-06-03T02:16:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:26 smithi079 ceph-mon[31819]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T02:16:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:26 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/224564587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:16:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:26 smithi135 ceph-mon[36653]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T02:16:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:26 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/224564587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:16:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:26 smithi123 ceph-mon[36640]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T02:16:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:26 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/224564587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:16:27.240 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-03T02:16:27.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:16:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:28 smithi079 ceph-mon[31819]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:28 smithi135 ceph-mon[36653]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:28 smithi123 ceph-mon[36640]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:30.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.012+0000 7f7736c00700 1 -- 172.21.15.79:0/1418741616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 msgr2=0x7f7730074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:30.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.012+0000 7f7736c00700 1 --2- 172.21.15.79:0/1418741616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 0x7f7730074aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7720009990 tx=0x7f7720009ca0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.012+0000 7f7736c00700 1 -- 172.21.15.79:0/1418741616 shutdown_connections 2024-06-03T02:16:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.012+0000 7f7736c00700 1 --2- 172.21.15.79:0/1418741616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7730113660 0x7f7730115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.012+0000 7f7736c00700 1 --2- 172.21.15.79:0/1418741616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 0x7f7730074aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.012+0000 7f7736c00700 1 --2- 172.21.15.79:0/1418741616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7730074fe0 0x7f7730113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.012+0000 7f7736c00700 1 -- 172.21.15.79:0/1418741616 >> 172.21.15.79:0/1418741616 conn(0x7f77300795f0 msgr2=0x7f7730079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.013+0000 7f7736c00700 1 -- 172.21.15.79:0/1418741616 shutdown_connections 2024-06-03T02:16:30.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.013+0000 7f7736c00700 1 -- 172.21.15.79:0/1418741616 wait complete. 2024-06-03T02:16:30.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.014+0000 7f7736c00700 1 Processor -- start 2024-06-03T02:16:30.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.014+0000 7f7736c00700 1 -- start start 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.014+0000 7f7736c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 0x7f773019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.014+0000 7f7736c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7730074fe0 0x7f773019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.014+0000 7f7736c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7730113660 0x7f77301a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.014+0000 7f7736c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7730118e00 con 0x7f77300746c0 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.014+0000 7f7736c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7730118c80 con 0x7f7730113660 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.014+0000 7f7736c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7730118f80 con 0x7f7730074fe0 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f772fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 0x7f773019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f7727400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7730074fe0 0x7f773019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f772fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 0x7f773019f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57370/0 (socket says 172.21.15.79:57370) 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f772fe00700 1 -- 172.21.15.79:0/1149123386 learned_addr learned my addr 172.21.15.79:0/1149123386 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f7727400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7730074fe0 0x7f773019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40956/0 (socket says 172.21.15.79:40956) 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f7734c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7730113660 0x7f77301a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:30.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f7734c00700 1 -- 172.21.15.79:0/1149123386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7730074fe0 msgr2=0x7f773019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:30.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f7734c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7730074fe0 0x7f773019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f7734c00700 1 -- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 msgr2=0x7f773019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:30.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f7734c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 0x7f773019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f7734c00700 1 -- 172.21.15.79:0/1149123386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77301a4280 con 0x7f7730113660 2024-06-03T02:16:30.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f772fe00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 0x7f773019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:30.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.015+0000 7f7727400700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7730074fe0 0x7f773019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:16:30.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.016+0000 7f7734c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7730113660 0x7f77301a3b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f772800ef90 tx=0x7f772800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:30.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.016+0000 7f772d600700 1 -- 172.21.15.79:0/1149123386 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77280092e0 con 0x7f7730113660 2024-06-03T02:16:30.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.016+0000 7f772d600700 1 -- 172.21.15.79:0/1149123386 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7728004bb0 con 0x7f7730113660 2024-06-03T02:16:30.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.016+0000 7f772d600700 1 -- 172.21.15.79:0/1149123386 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728004280 con 0x7f7730113660 2024-06-03T02:16:30.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.016+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77301a4510 con 0x7f7730113660 2024-06-03T02:16:30.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.016+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77301abdf0 con 0x7f7730113660 2024-06-03T02:16:30.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.017+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7730069180 con 0x7f7730113660 2024-06-03T02:16:30.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.019+0000 7f772d600700 1 -- 172.21.15.79:0/1149123386 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f772801b030 con 0x7f7730113660 2024-06-03T02:16:30.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.021+0000 7f772d600700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f77140a2d30 0x7f77140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:30.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.021+0000 7f772d600700 1 -- 172.21.15.79:0/1149123386 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f77280cf190 con 0x7f7730113660 2024-06-03T02:16:30.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.021+0000 7f772fe00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f77140a2d30 0x7f77140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:30.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.022+0000 7f772fe00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f77140a2d30 0x7f77140a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7720006010 tx=0x7f772000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:30.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.025+0000 7f772d600700 1 -- 172.21.15.79:0/1149123386 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7728022020 con 0x7f7730113660 2024-06-03T02:16:30.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.254+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77301abfc0 con 0x7f7730113660 2024-06-03T02:16:30.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.257+0000 7f772d600700 1 -- 172.21.15.79:0/1149123386 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+24805 (secure 0 0 0) 0x7f7728010040 con 0x7f7730113660 2024-06-03T02:16:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:16:30.258 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":101,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:15:54.257434+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:16:30.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f77140a2d30 msgr2=0x7f77140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:30.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f77140a2d30 0x7f77140a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7720006010 tx=0x7f772000b260 comp rx=0 tx=0).stop 2024-06-03T02:16:30.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7730113660 msgr2=0x7f77301a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:30.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7730113660 0x7f77301a3b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f772800ef90 tx=0x7f772800c3a0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 shutdown_connections 2024-06-03T02:16:30.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7730113660 0x7f77301a3b40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f77140a2d30 0x7f77140a51f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77300746c0 0x7f773019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 --2- 172.21.15.79:0/1149123386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7730074fe0 0x7f773019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:30.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 >> 172.21.15.79:0/1149123386 conn(0x7f77300795f0 msgr2=0x7f7730112430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:30.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 shutdown_connections 2024-06-03T02:16:30.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:30.260+0000 7f7736c00700 1 -- 172.21.15.79:0/1149123386 wait complete. 2024-06-03T02:16:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:30 smithi079 ceph-mon[31819]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:16:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:30 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1149123386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:16:30.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:30 smithi135 ceph-mon[36653]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:16:30.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:30 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1149123386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:16:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:30 smithi123 ceph-mon[36640]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:16:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:30 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1149123386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:16:31.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-03T02:16:31.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-03T02:16:31.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-03T02:16:31.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-03T02:16:31.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-03T02:16:31.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-03T02:16:31.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-03T02:16:31.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-03T02:16:31.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-03T02:16:31.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-03T02:16:31.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-03T02:16:31.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-03T02:16:32.252 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:32.333 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:32.341 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:32.398 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:32.410 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:32.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:32 smithi123 ceph-mon[36640]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:32.528 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:32.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:32 smithi079 ceph-mon[31819]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:32 smithi135 ceph-mon[36653]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:33.764 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:33.778 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:33.778 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:33.778 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:33.781 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:33.786 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:16:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:34 smithi079 ceph-mon[31819]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:34 smithi135 ceph-mon[36653]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:34 smithi123 ceph-mon[36640]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:36 smithi079 ceph-mon[31819]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:16:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:36 smithi135 ceph-mon[36653]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:16:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:36 smithi123 ceph-mon[36640]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:16:38.539 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:38 smithi079 ceph-mon[31819]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:38 smithi135 ceph-mon[36653]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:38.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:38 smithi123 ceph-mon[36640]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:40.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.197+0000 7f966be00700 1 -- 172.21.15.79:0/1842520108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f966c10a650 msgr2=0x7f966c10aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:40.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.197+0000 7f966be00700 1 --2- 172.21.15.79:0/1842520108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f966c10a650 0x7f966c10aa30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f965c009990 tx=0x7f965c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.198+0000 7f966be00700 1 -- 172.21.15.79:0/1842520108 shutdown_connections 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.198+0000 7f966be00700 1 --2- 172.21.15.79:0/1842520108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f966c104f80 0x7f966c111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.198+0000 7f966be00700 1 --2- 172.21.15.79:0/1842520108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f966c10a650 0x7f966c10aa30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.198+0000 7f966be00700 1 --2- 172.21.15.79:0/1842520108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f966c1045c0 0x7f966c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.198+0000 7f966be00700 1 -- 172.21.15.79:0/1842520108 >> 172.21.15.79:0/1842520108 conn(0x7f966c1000b0 msgr2=0x7f966c1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.198+0000 7f966be00700 1 -- 172.21.15.79:0/1842520108 shutdown_connections 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.198+0000 7f966be00700 1 -- 172.21.15.79:0/1842520108 wait complete. 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.199+0000 7f966be00700 1 Processor -- start 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.199+0000 7f966be00700 1 -- start start 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.199+0000 7f966be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f966c1045c0 0x7f966c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.199+0000 7f966be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f966c104f80 0x7f966c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:40.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.199+0000 7f966be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f966c10a650 0x7f966c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.199+0000 7f966be00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f966c1148f0 con 0x7f966c10a650 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.199+0000 7f966be00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f966c114770 con 0x7f966c104f80 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.199+0000 7f966be00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f966c114a70 con 0x7f966c1045c0 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f966c10a650 0x7f966c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966a000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f966c104f80 0x7f966c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f966c1045c0 0x7f966c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966a000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f966c104f80 0x7f966c1a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:55320/0 (socket says 172.21.15.79:55320) 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966a000700 1 -- 172.21.15.79:0/692639682 learned_addr learned my addr 172.21.15.79:0/692639682 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966aa00700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f966c1045c0 0x7f966c1a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37692/0 (socket says 172.21.15.79:37692) 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966b400700 1 -- 172.21.15.79:0/692639682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f966c1045c0 msgr2=0x7f966c1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966b400700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f966c1045c0 0x7f966c1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966b400700 1 -- 172.21.15.79:0/692639682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f966c104f80 msgr2=0x7f966c1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966b400700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f966c104f80 0x7f966c1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:40.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966b400700 1 -- 172.21.15.79:0/692639682 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c19b210 con 0x7f966c10a650 2024-06-03T02:16:40.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.200+0000 7f966a000700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f966c104f80 0x7f966c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:40.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.201+0000 7f966b400700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f966c10a650 0x7f966c19aab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f966400efc0 tx=0x7f966400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:40.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.201+0000 7f9663400700 1 -- 172.21.15.79:0/692639682 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96640092e0 con 0x7f966c10a650 2024-06-03T02:16:40.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.201+0000 7f9663400700 1 -- 172.21.15.79:0/692639682 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9664004bb0 con 0x7f966c10a650 2024-06-03T02:16:40.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.201+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966c19b4a0 con 0x7f966c10a650 2024-06-03T02:16:40.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.201+0000 7f9663400700 1 -- 172.21.15.79:0/692639682 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664004280 con 0x7f966c10a650 2024-06-03T02:16:40.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.201+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966c1a7770 con 0x7f966c10a650 2024-06-03T02:16:40.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.202+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f966c062ba0 con 0x7f966c10a650 2024-06-03T02:16:40.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.206+0000 7f9663400700 1 -- 172.21.15.79:0/692639682 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f966401b030 con 0x7f966c10a650 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.207+0000 7f9663400700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96500a2cb0 0x7f96500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.207+0000 7f9663400700 1 -- 172.21.15.79:0/692639682 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f96640cfa40 con 0x7f966c10a650 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.208+0000 7f966aa00700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96500a2cb0 0x7f96500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.208+0000 7f9663400700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] conn(0x7f96500adf90 0x7f96500b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.208+0000 7f9663400700 1 -- 172.21.15.79:0/692639682 --> [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f966401fe50 con 0x7f96500adf90 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.208+0000 7f9663400700 1 -- 172.21.15.79:0/692639682 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f96640cfe90 con 0x7f966c10a650 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.208+0000 7f966a000700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] conn(0x7f96500adf90 0x7f96500b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.209+0000 7f966aa00700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96500a2cb0 0x7f96500a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b2e0 tx=0x7f965c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.209+0000 7f966a000700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] conn(0x7f96500adf90 0x7f96500b0430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.210+0000 7f9663400700 1 -- 172.21.15.79:0/692639682 <== osd.0 v2:172.21.15.79:6802/1006585875 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f966401fe50 con 0x7f96500adf90 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.228+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 --> [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f966c063020 con 0x7f96500adf90 2024-06-03T02:16:40.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.229+0000 7f9663400700 1 -- 172.21.15.79:0/692639682 <== osd.0 v2:172.21.15.79:6802/1006585875 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f966c063020 con 0x7f96500adf90 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.229+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] conn(0x7f96500adf90 msgr2=0x7f96500b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.229+0000 7f966be00700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] conn(0x7f96500adf90 0x7f96500b0430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.230+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96500a2cb0 msgr2=0x7f96500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.230+0000 7f966be00700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96500a2cb0 0x7f96500a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b2e0 tx=0x7f965c005960 comp rx=0 tx=0).stop 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.230+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f966c10a650 msgr2=0x7f966c19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.230+0000 7f966be00700 1 --2- 172.21.15.79:0/692639682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f966c10a650 0x7f966c19aab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f966400efc0 tx=0x7f966400c3a0 comp rx=0 tx=0).stop 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.230+0000 7f966b400700 1 -- 172.21.15.79:0/692639682 reap_dead start 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.230+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 shutdown_connections 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.230+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 >> 172.21.15.79:0/692639682 conn(0x7f966c1000b0 msgr2=0x7f966c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.230+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 shutdown_connections 2024-06-03T02:16:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:39.230+0000 7f966be00700 1 -- 172.21.15.79:0/692639682 wait complete. 2024-06-03T02:16:40.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:40 smithi079 ceph-mon[31819]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:16:40.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:40 smithi135 ceph-mon[36653]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:16:40.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:40 smithi123 ceph-mon[36640]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:16:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:42 smithi079 ceph-mon[31819]: pgmap v416: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:42 smithi135 ceph-mon[36653]: pgmap v416: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:42 smithi123 ceph-mon[36640]: pgmap v416: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:44.490 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:44 smithi079 ceph-mon[31819]: pgmap v417: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:44 smithi135 ceph-mon[36653]: pgmap v417: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:44 smithi123 ceph-mon[36640]: pgmap v417: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:45.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.388+0000 7f0d07a00700 1 -- 172.21.15.79:0/3541866002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00105060 msgr2=0x7f0d00111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:45.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.388+0000 7f0d07a00700 1 --2- 172.21.15.79:0/3541866002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00105060 0x7f0d00111570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b0a0 tx=0x7f0cf800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:16:45.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.388+0000 7f0d07a00700 1 -- 172.21.15.79:0/3541866002 shutdown_connections 2024-06-03T02:16:45.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.388+0000 7f0d07a00700 1 --2- 172.21.15.79:0/3541866002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00105060 0x7f0d00111570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:45.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.388+0000 7f0d07a00700 1 --2- 172.21.15.79:0/3541866002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0010a6a0 0x7f0d0010aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:45.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.388+0000 7f0d07a00700 1 --2- 172.21.15.79:0/3541866002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d001046a0 0x7f0d00104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:45.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.388+0000 7f0d07a00700 1 -- 172.21.15.79:0/3541866002 >> 172.21.15.79:0/3541866002 conn(0x7f0d00100190 msgr2=0x7f0d001025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:45.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.388+0000 7f0d07a00700 1 -- 172.21.15.79:0/3541866002 shutdown_connections 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.388+0000 7f0d07a00700 1 -- 172.21.15.79:0/3541866002 wait complete. 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.389+0000 7f0d07a00700 1 Processor -- start 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.389+0000 7f0d07a00700 1 -- start start 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.389+0000 7f0d07a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d001046a0 0x7f0d0019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.389+0000 7f0d07a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00105060 0x7f0d0019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.389+0000 7f0d07a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0010a6a0 0x7f0d0019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.389+0000 7f0d07a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00114a30 con 0x7f0d0010a6a0 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.389+0000 7f0d07a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001148b0 con 0x7f0d001046a0 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.389+0000 7f0d07a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00114bb0 con 0x7f0d00105060 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0cffe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00105060 0x7f0d0019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0d05a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0010a6a0 0x7f0d0019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:45.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0cffe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00105060 0x7f0d0019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:50736/0 (socket says 172.21.15.79:50736) 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0cffe00700 1 -- 172.21.15.79:0/2794274929 learned_addr learned my addr 172.21.15.79:0/2794274929 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0d05000700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d001046a0 0x7f0d0019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0d05a00700 1 -- 172.21.15.79:0/2794274929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00105060 msgr2=0x7f0d0019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0d05a00700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00105060 0x7f0d0019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0d05a00700 1 -- 172.21.15.79:0/2794274929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d001046a0 msgr2=0x7f0d0019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0d05a00700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d001046a0 0x7f0d0019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0d05a00700 1 -- 172.21.15.79:0/2794274929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0019fe10 con 0x7f0d0010a6a0 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.390+0000 7f0d05a00700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0010a6a0 0x7f0d0019f6d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b070 tx=0x7f0cf8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.391+0000 7f0cfd600700 1 -- 172.21.15.79:0/2794274929 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8021070 con 0x7f0d0010a6a0 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.391+0000 7f0cfd600700 1 -- 172.21.15.79:0/2794274929 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8007b80 con 0x7f0d0010a6a0 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.391+0000 7f0cfd600700 1 -- 172.21.15.79:0/2794274929 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8005040 con 0x7f0d0010a6a0 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.391+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d001a00a0 con 0x7f0d0010a6a0 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.391+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d001a7980 con 0x7f0d0010a6a0 2024-06-03T02:16:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.392+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d001010d0 con 0x7f0d0010a6a0 2024-06-03T02:16:45.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.394+0000 7f0cfd600700 1 -- 172.21.15.79:0/2794274929 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0cf8024070 con 0x7f0d0010a6a0 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.396+0000 7f0cfd600700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ce80a2cb0 0x7f0ce80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.396+0000 7f0cfd600700 1 -- 172.21.15.79:0/2794274929 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0cf80d03f0 con 0x7f0d0010a6a0 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.396+0000 7f0d05000700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ce80a2cb0 0x7f0ce80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.396+0000 7f0cfd600700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] conn(0x7f0ce80adf90 0x7f0ce80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.396+0000 7f0cfd600700 1 -- 172.21.15.79:0/2794274929 --> [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ce80b0b00 con 0x7f0ce80adf90 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.396+0000 7f0cfd600700 1 -- 172.21.15.79:0/2794274929 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f0cf80d07a0 con 0x7f0d0010a6a0 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.397+0000 7f0cffe00700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] conn(0x7f0ce80adf90 0x7f0ce80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.397+0000 7f0d05000700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ce80a2cb0 0x7f0ce80a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009be0 tx=0x7f0cf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.397+0000 7f0cffe00700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] conn(0x7f0ce80adf90 0x7f0ce80b0430 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:45.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.398+0000 7f0cfd600700 1 -- 172.21.15.79:0/2794274929 <== osd.9 v2:172.21.15.79:6826/1463813812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ce80b0b00 con 0x7f0ce80adf90 2024-06-03T02:16:45.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.416+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 --> [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d00105de0 con 0x7f0ce80adf90 2024-06-03T02:16:45.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.417+0000 7f0cfd600700 1 -- 172.21.15.79:0/2794274929 <== osd.9 v2:172.21.15.79:6826/1463813812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0d00105de0 con 0x7f0ce80adf90 2024-06-03T02:16:45.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.417+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] conn(0x7f0ce80adf90 msgr2=0x7f0ce80b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:45.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.417+0000 7f0d07a00700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] conn(0x7f0ce80adf90 0x7f0ce80b0430 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:45.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.418+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ce80a2cb0 msgr2=0x7f0ce80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:45.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.418+0000 7f0d07a00700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ce80a2cb0 0x7f0ce80a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009be0 tx=0x7f0cf4009260 comp rx=0 tx=0).stop 2024-06-03T02:16:45.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.418+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0010a6a0 msgr2=0x7f0d0019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:45.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.418+0000 7f0d07a00700 1 --2- 172.21.15.79:0/2794274929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d0010a6a0 0x7f0d0019f6d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b070 tx=0x7f0cf8004c80 comp rx=0 tx=0).stop 2024-06-03T02:16:45.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.418+0000 7f0d05a00700 1 -- 172.21.15.79:0/2794274929 reap_dead start 2024-06-03T02:16:45.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.418+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 shutdown_connections 2024-06-03T02:16:45.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.418+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 >> 172.21.15.79:0/2794274929 conn(0x7f0d00100190 msgr2=0x7f0d00100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:45.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.418+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 shutdown_connections 2024-06-03T02:16:45.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:45.418+0000 7f0d07a00700 1 -- 172.21.15.79:0/2794274929 wait complete. 2024-06-03T02:16:46.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:46 smithi079 ceph-mon[31819]: pgmap v418: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-03T02:16:46.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:46 smithi135 ceph-mon[36653]: pgmap v418: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-03T02:16:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:46 smithi123 ceph-mon[36640]: pgmap v418: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-03T02:16:47.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.254+0000 7f8d23c00700 1 -- 172.21.15.79:0/3451113593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d1c105070 msgr2=0x7f8d1c111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:47.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.254+0000 7f8d23c00700 1 --2- 172.21.15.79:0/3451113593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d1c105070 0x7f8d1c111580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400b0a0 tx=0x7f8d1400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:16:47.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.254+0000 7f8d23c00700 1 -- 172.21.15.79:0/3451113593 shutdown_connections 2024-06-03T02:16:47.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.254+0000 7f8d23c00700 1 --2- 172.21.15.79:0/3451113593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d1c105070 0x7f8d1c111580 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:47.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.254+0000 7f8d23c00700 1 --2- 172.21.15.79:0/3451113593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d1c10a6b0 0x7f8d1c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:47.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.254+0000 7f8d23c00700 1 --2- 172.21.15.79:0/3451113593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c1046b0 0x7f8d1c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:47.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.254+0000 7f8d23c00700 1 -- 172.21.15.79:0/3451113593 >> 172.21.15.79:0/3451113593 conn(0x7f8d1c1001a0 msgr2=0x7f8d1c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:47.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.254+0000 7f8d23c00700 1 -- 172.21.15.79:0/3451113593 shutdown_connections 2024-06-03T02:16:47.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.254+0000 7f8d23c00700 1 -- 172.21.15.79:0/3451113593 wait complete. 2024-06-03T02:16:47.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.255+0000 7f8d23c00700 1 Processor -- start 2024-06-03T02:16:47.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.255+0000 7f8d23c00700 1 -- start start 2024-06-03T02:16:47.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d23c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d1c1046b0 0x7f8d1c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:47.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d23c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d1c105070 0x7f8d1c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:47.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d23c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c10a6b0 0x7f8d1c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:47.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d23c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c114a50 con 0x7f8d1c105070 2024-06-03T02:16:47.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d23c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1148d0 con 0x7f8d1c1046b0 2024-06-03T02:16:47.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d23c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c114bd0 con 0x7f8d1c10a6b0 2024-06-03T02:16:47.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d1be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d1c105070 0x7f8d1c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:47.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d1be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d1c105070 0x7f8d1c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53588/0 (socket says 172.21.15.79:53588) 2024-06-03T02:16:47.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d21200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d1c1046b0 0x7f8d1c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:47.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d1be00700 1 -- 172.21.15.79:0/2076548773 learned_addr learned my addr 172.21.15.79:0/2076548773 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:47.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.256+0000 7f8d21c00700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c10a6b0 0x7f8d1c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:47.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.257+0000 7f8d21200700 1 -- 172.21.15.79:0/2076548773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c10a6b0 msgr2=0x7f8d1c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:47.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.257+0000 7f8d21200700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c10a6b0 0x7f8d1c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:47.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.257+0000 7f8d21200700 1 -- 172.21.15.79:0/2076548773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d1c105070 msgr2=0x7f8d1c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:47.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.257+0000 7f8d21200700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d1c105070 0x7f8d1c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:47.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.257+0000 7f8d21200700 1 -- 172.21.15.79:0/2076548773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c19fe10 con 0x7f8d1c1046b0 2024-06-03T02:16:47.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.257+0000 7f8d1be00700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d1c105070 0x7f8d1c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:47.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.257+0000 7f8d21200700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d1c1046b0 0x7f8d1c19ae00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000e9f0 tx=0x7f8d1000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:47.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.257+0000 7f8d19600700 1 -- 172.21.15.79:0/2076548773 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000c9c0 con 0x7f8d1c1046b0 2024-06-03T02:16:47.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.258+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c1a0100 con 0x7f8d1c1046b0 2024-06-03T02:16:47.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.258+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c1a7980 con 0x7f8d1c1046b0 2024-06-03T02:16:47.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.258+0000 7f8d19600700 1 -- 172.21.15.79:0/2076548773 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1000cb60 con 0x7f8d1c1046b0 2024-06-03T02:16:47.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.258+0000 7f8d19600700 1 -- 172.21.15.79:0/2076548773 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10018680 con 0x7f8d1c1046b0 2024-06-03T02:16:47.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.259+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8cfc001030 con 0x7f8d1c1046b0 2024-06-03T02:16:47.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.261+0000 7f8d19600700 1 -- 172.21.15.79:0/2076548773 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8d1000cd10 con 0x7f8d1c1046b0 2024-06-03T02:16:47.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.263+0000 7f8d19600700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8d040a2d30 0x7f8d040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:47.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.263+0000 7f8d19600700 1 -- 172.21.15.79:0/2076548773 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8d100cf4f0 con 0x7f8d1c1046b0 2024-06-03T02:16:47.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.263+0000 7f8d1be00700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8d040a2d30 0x7f8d040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:47.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.263+0000 7f8d19600700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] conn(0x7f8d040ae010 0x7f8d040b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:47.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.263+0000 7f8d19600700 1 -- 172.21.15.79:0/2076548773 --> [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d040b0b80 con 0x7f8d040ae010 2024-06-03T02:16:47.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.264+0000 7f8d19600700 1 -- 172.21.15.79:0/2076548773 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f8d100d6050 con 0x7f8d1c1046b0 2024-06-03T02:16:47.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.264+0000 7f8d21c00700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] conn(0x7f8d040ae010 0x7f8d040b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:47.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.264+0000 7f8d1be00700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8d040a2d30 0x7f8d040a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c109b90 tx=0x7f8d08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:47.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.264+0000 7f8d21c00700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] conn(0x7f8d040ae010 0x7f8d040b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:47.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.265+0000 7f8d19600700 1 -- 172.21.15.79:0/2076548773 <== osd.7 v2:172.21.15.123:6816/283771362 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d040b0b80 con 0x7f8d040ae010 2024-06-03T02:16:47.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.291+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 --> [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8cfc002e60 con 0x7f8d040ae010 2024-06-03T02:16:47.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.292+0000 7f8d19600700 1 -- 172.21.15.79:0/2076548773 <== osd.7 v2:172.21.15.123:6816/283771362 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8cfc002e60 con 0x7f8d040ae010 2024-06-03T02:16:47.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.292+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 >> [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] conn(0x7f8d040ae010 msgr2=0x7f8d040b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:47.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.292+0000 7f8d23c00700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] conn(0x7f8d040ae010 0x7f8d040b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:47.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.292+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8d040a2d30 msgr2=0x7f8d040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:47.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.292+0000 7f8d23c00700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8d040a2d30 0x7f8d040a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c109b90 tx=0x7f8d08009260 comp rx=0 tx=0).stop 2024-06-03T02:16:47.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.292+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d1c1046b0 msgr2=0x7f8d1c19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:47.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.292+0000 7f8d23c00700 1 --2- 172.21.15.79:0/2076548773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d1c1046b0 0x7f8d1c19ae00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000e9f0 tx=0x7f8d1000ed00 comp rx=0 tx=0).stop 2024-06-03T02:16:47.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.292+0000 7f8d21c00700 1 -- 172.21.15.79:0/2076548773 reap_dead start 2024-06-03T02:16:47.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.293+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 shutdown_connections 2024-06-03T02:16:47.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.293+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 >> 172.21.15.79:0/2076548773 conn(0x7f8d1c1001a0 msgr2=0x7f8d1c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:47.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.293+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 shutdown_connections 2024-06-03T02:16:47.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:47.293+0000 7f8d23c00700 1 -- 172.21.15.79:0/2076548773 wait complete. 2024-06-03T02:16:48.555 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:48 smithi079 ceph-mon[31819]: pgmap v419: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:16:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:48 smithi135 ceph-mon[36653]: pgmap v419: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:16:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:48 smithi123 ceph-mon[36640]: pgmap v419: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:16:49.051 INFO:teuthology.orchestra.run.smithi079.stdout:296352743483 2024-06-03T02:16:49.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.004+0000 7fe03da00700 1 -- 172.21.15.79:0/2474827089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe038102740 msgr2=0x7fe038102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.004+0000 7fe03da00700 1 --2- 172.21.15.79:0/2474827089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe038102740 0x7fe038102b20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe02800b0d0 tx=0x7fe02800b3e0 comp rx=0 tx=0).stop 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.004+0000 7fe03da00700 1 -- 172.21.15.79:0/2474827089 shutdown_connections 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.004+0000 7fe03da00700 1 --2- 172.21.15.79:0/2474827089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe038107840 0x7fe03810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.004+0000 7fe03da00700 1 --2- 172.21.15.79:0/2474827089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe038102740 0x7fe038102b20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.004+0000 7fe03da00700 1 --2- 172.21.15.79:0/2474827089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0381030f0 0x7fe038107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.004+0000 7fe03da00700 1 -- 172.21.15.79:0/2474827089 >> 172.21.15.79:0/2474827089 conn(0x7fe0380795f0 msgr2=0x7fe038079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.004+0000 7fe03da00700 1 -- 172.21.15.79:0/2474827089 shutdown_connections 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.005+0000 7fe03da00700 1 -- 172.21.15.79:0/2474827089 wait complete. 2024-06-03T02:16:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.006+0000 7fe03da00700 1 Processor -- start 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.006+0000 7fe03da00700 1 -- start start 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.006+0000 7fe03da00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe038102740 0x7fe038074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.006+0000 7fe03da00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0381030f0 0x7fe038074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.006+0000 7fe03da00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe038107840 0x7fe038075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.006+0000 7fe03da00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe038112760 con 0x7fe0381030f0 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.006+0000 7fe03da00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0381125e0 con 0x7fe038102740 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.006+0000 7fe03da00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0381128e0 con 0x7fe038107840 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.006+0000 7fe02e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0381030f0 0x7fe038074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe02e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0381030f0 0x7fe038074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53602/0 (socket says 172.21.15.79:53602) 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe02e000700 1 -- 172.21.15.79:0/2968609814 learned_addr learned my addr 172.21.15.79:0/2968609814 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe037400700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe038107840 0x7fe038075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe036a00700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe038102740 0x7fe038074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe02e000700 1 -- 172.21.15.79:0/2968609814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe038107840 msgr2=0x7fe038075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:49.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe02e000700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe038107840 0x7fe038075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:49.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe02e000700 1 -- 172.21.15.79:0/2968609814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe038102740 msgr2=0x7fe038074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:49.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe02e000700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe038102740 0x7fe038074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:49.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe02e000700 1 -- 172.21.15.79:0/2968609814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0381a70e0 con 0x7fe0381030f0 2024-06-03T02:16:49.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe037400700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe038107840 0x7fe038075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:49.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe036a00700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe038102740 0x7fe038074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:49.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.007+0000 7fe02e000700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0381030f0 0x7fe038074cd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe02000e9a0 tx=0x7fe02000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:49.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.008+0000 7fe02fe00700 1 -- 172.21.15.79:0/2968609814 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02000c9c0 con 0x7fe0381030f0 2024-06-03T02:16:49.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.008+0000 7fe02fe00700 1 -- 172.21.15.79:0/2968609814 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02000cb60 con 0x7fe0381030f0 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.008+0000 7fe02fe00700 1 -- 172.21.15.79:0/2968609814 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020018660 con 0x7fe0381030f0 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.008+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0381a73d0 con 0x7fe0381030f0 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.008+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0381a7960 con 0x7fe0381030f0 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.009+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe038062c20 con 0x7fe0381030f0 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.011+0000 7fe02fe00700 1 -- 172.21.15.79:0/2968609814 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe02000cd10 con 0x7fe0381030f0 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.012+0000 7fe02fe00700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe01c0a2d30 0x7fe01c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.012+0000 7fe02fe00700 1 -- 172.21.15.79:0/2968609814 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe0200cfbc0 con 0x7fe0381030f0 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.013+0000 7fe036a00700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe01c0a2d30 0x7fe01c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.013+0000 7fe02fe00700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] conn(0x7fe01c0ae010 0x7fe01c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.013+0000 7fe02fe00700 1 -- 172.21.15.79:0/2968609814 --> [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe01c0b0b80 con 0x7fe01c0ae010 2024-06-03T02:16:49.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.013+0000 7fe02fe00700 1 -- 172.21.15.79:0/2968609814 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fe0200cff70 con 0x7fe0381030f0 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.013+0000 7fe037400700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] conn(0x7fe01c0ae010 0x7fe01c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.013+0000 7fe036a00700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe01c0a2d30 0x7fe01c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe028009fd0 tx=0x7fe02800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.014+0000 7fe037400700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] conn(0x7fe01c0ae010 0x7fe01c0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.015+0000 7fe02fe00700 1 -- 172.21.15.79:0/2968609814 <== osd.3 v2:172.21.15.79:6810/1694165026 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe01c0b0b80 con 0x7fe01c0ae010 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.032+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 --> [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe038063020 con 0x7fe01c0ae010 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.033+0000 7fe02fe00700 1 -- 172.21.15.79:0/2968609814 <== osd.3 v2:172.21.15.79:6810/1694165026 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe038063020 con 0x7fe01c0ae010 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.033+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] conn(0x7fe01c0ae010 msgr2=0x7fe01c0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.033+0000 7fe03da00700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] conn(0x7fe01c0ae010 0x7fe01c0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.034+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe01c0a2d30 msgr2=0x7fe01c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.034+0000 7fe03da00700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe01c0a2d30 0x7fe01c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe028009fd0 tx=0x7fe02800b040 comp rx=0 tx=0).stop 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.034+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0381030f0 msgr2=0x7fe038074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.034+0000 7fe03da00700 1 --2- 172.21.15.79:0/2968609814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0381030f0 0x7fe038074cd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe02000e9a0 tx=0x7fe02000ecb0 comp rx=0 tx=0).stop 2024-06-03T02:16:49.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.034+0000 7fe037400700 1 -- 172.21.15.79:0/2968609814 reap_dead start 2024-06-03T02:16:49.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.034+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 shutdown_connections 2024-06-03T02:16:49.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.034+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 >> 172.21.15.79:0/2968609814 conn(0x7fe0380795f0 msgr2=0x7fe038108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:49.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.034+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 shutdown_connections 2024-06-03T02:16:49.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:48.034+0000 7fe03da00700 1 -- 172.21.15.79:0/2968609814 wait complete. 2024-06-03T02:16:50.228 INFO:teuthology.orchestra.run.smithi079.stdout:34359738478 2024-06-03T02:16:50.229 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-03T02:16:50.578 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:50 smithi079 ceph-mon[31819]: pgmap v420: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:50.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:50 smithi135 ceph-mon[36653]: pgmap v420: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:50.647 INFO:teuthology.orchestra.run.smithi079.stdout:227633266759 2024-06-03T02:16:50.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-03T02:16:50.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:50 smithi123 ceph-mon[36640]: pgmap v420: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:51.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.127+0000 7f0b31200700 1 -- 172.21.15.79:0/81101341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2c102ea0 msgr2=0x7f0b2c10f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.127+0000 7f0b31200700 1 --2- 172.21.15.79:0/81101341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2c102ea0 0x7f0b2c10f380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400b0a0 tx=0x7f0b2400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.127+0000 7f0b31200700 1 -- 172.21.15.79:0/81101341 shutdown_connections 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.127+0000 7f0b31200700 1 --2- 172.21.15.79:0/81101341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2c102ea0 0x7f0b2c10f380 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.127+0000 7f0b31200700 1 --2- 172.21.15.79:0/81101341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b2c1085f0 0x7f0b2c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.127+0000 7f0b31200700 1 --2- 172.21.15.79:0/81101341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b2c1024e0 0x7f0b2c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.127+0000 7f0b31200700 1 -- 172.21.15.79:0/81101341 >> 172.21.15.79:0/81101341 conn(0x7f0b2c079640 msgr2=0x7f0b2c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.127+0000 7f0b31200700 1 -- 172.21.15.79:0/81101341 shutdown_connections 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.128+0000 7f0b31200700 1 -- 172.21.15.79:0/81101341 wait complete. 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.129+0000 7f0b31200700 1 Processor -- start 2024-06-03T02:16:51.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.129+0000 7f0b31200700 1 -- start start 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b31200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2c1024e0 0x7f0b2c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b31200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b2c102ea0 0x7f0b2c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b31200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b2c1085f0 0x7f0b2c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b31200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c112790 con 0x7f0b2c1024e0 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b31200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c112610 con 0x7f0b2c1085f0 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b31200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c112910 con 0x7f0b2c102ea0 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b29e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2c1024e0 0x7f0b2c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b29e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2c1024e0 0x7f0b2c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53618/0 (socket says 172.21.15.79:53618) 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b29e00700 1 -- 172.21.15.79:0/731649915 learned_addr learned my addr 172.21.15.79:0/731649915 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b29400700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b2c102ea0 0x7f0b2c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.130+0000 7f0b2a800700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b2c1085f0 0x7f0b2c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b2a800700 1 -- 172.21.15.79:0/731649915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b2c102ea0 msgr2=0x7f0b2c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b2a800700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b2c102ea0 0x7f0b2c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b2a800700 1 -- 172.21.15.79:0/731649915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2c1024e0 msgr2=0x7f0b2c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b2a800700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2c1024e0 0x7f0b2c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b2a800700 1 -- 172.21.15.79:0/731649915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c1a7100 con 0x7f0b2c1085f0 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b2a800700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b2c1085f0 0x7f0b2c075210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400b0a0 tx=0x7f0b24004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b22a00700 1 -- 172.21.15.79:0/731649915 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24021070 con 0x7f0b2c1085f0 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b22a00700 1 -- 172.21.15.79:0/731649915 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b24007b80 con 0x7f0b2c1085f0 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c1a7390 con 0x7f0b2c1085f0 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.131+0000 7f0b22a00700 1 -- 172.21.15.79:0/731649915 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24005040 con 0x7f0b2c1085f0 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.132+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c1a7890 con 0x7f0b2c1085f0 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.132+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b2c062ba0 con 0x7f0b2c1085f0 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.135+0000 7f0b22a00700 1 -- 172.21.15.79:0/731649915 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0b24024070 con 0x7f0b2c1085f0 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.137+0000 7f0b22a00700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0b100a2d30 0x7f0b100a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.137+0000 7f0b22a00700 1 -- 172.21.15.79:0/731649915 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0b2400f020 con 0x7f0b2c1085f0 2024-06-03T02:16:51.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.137+0000 7f0b29e00700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0b100a2d30 0x7f0b100a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.138+0000 7f0b22a00700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] conn(0x7f0b100ae010 0x7f0b100b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.138+0000 7f0b22a00700 1 -- 172.21.15.79:0/731649915 --> [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b100b0b80 con 0x7f0b100ae010 2024-06-03T02:16:51.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.138+0000 7f0b22a00700 1 -- 172.21.15.79:0/731649915 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f0b240d0e70 con 0x7f0b2c1085f0 2024-06-03T02:16:51.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.138+0000 7f0b29e00700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0b100a2d30 0x7f0b100a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009680 tx=0x7f0b1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:51.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.138+0000 7f0b29400700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] conn(0x7f0b100ae010 0x7f0b100b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.138+0000 7f0b29400700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] conn(0x7f0b100ae010 0x7f0b100b04b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.139+0000 7f0b22a00700 1 -- 172.21.15.79:0/731649915 <== osd.4 v2:172.21.15.123:6808/819552045 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b100b0b80 con 0x7f0b100ae010 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.157+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 --> [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b2c062fd0 con 0x7f0b100ae010 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.158+0000 7f0b22a00700 1 -- 172.21.15.79:0/731649915 <== osd.4 v2:172.21.15.123:6808/819552045 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b2c062fd0 con 0x7f0b100ae010 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 >> [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] conn(0x7f0b100ae010 msgr2=0x7f0b100b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] conn(0x7f0b100ae010 0x7f0b100b04b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0b100a2d30 msgr2=0x7f0b100a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0b100a2d30 0x7f0b100a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009680 tx=0x7f0b1c009260 comp rx=0 tx=0).stop 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b2c1085f0 msgr2=0x7f0b2c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 --2- 172.21.15.79:0/731649915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b2c1085f0 0x7f0b2c075210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400b0a0 tx=0x7f0b24004d00 comp rx=0 tx=0).stop 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b2a800700 1 -- 172.21.15.79:0/731649915 reap_dead start 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 shutdown_connections 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 >> 172.21.15.79:0/731649915 conn(0x7f0b2c079640 msgr2=0x7f0b2c100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 shutdown_connections 2024-06-03T02:16:51.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:50.159+0000 7f0b31200700 1 -- 172.21.15.79:0/731649915 wait complete. 2024-06-03T02:16:51.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.656+0000 7f7efb200700 1 -- 172.21.15.79:0/686987823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7efc1089d0 msgr2=0x7f7efc108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.656+0000 7f7efb200700 1 --2- 172.21.15.79:0/686987823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7efc1089d0 0x7f7efc108e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009990 tx=0x7f7ee8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.656+0000 7f7efb200700 1 -- 172.21.15.79:0/686987823 shutdown_connections 2024-06-03T02:16:51.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.656+0000 7f7efb200700 1 --2- 172.21.15.79:0/686987823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc109390 0x7f7efc115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.656+0000 7f7efb200700 1 --2- 172.21.15.79:0/686987823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc10d8a0 0x7f7efc10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.656+0000 7f7efb200700 1 --2- 172.21.15.79:0/686987823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7efc1089d0 0x7f7efc108e50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.656+0000 7f7efb200700 1 -- 172.21.15.79:0/686987823 >> 172.21.15.79:0/686987823 conn(0x7f7efc078220 msgr2=0x7f7efc1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:51.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.656+0000 7f7efb200700 1 -- 172.21.15.79:0/686987823 shutdown_connections 2024-06-03T02:16:51.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.656+0000 7f7efb200700 1 -- 172.21.15.79:0/686987823 wait complete. 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.657+0000 7f7efb200700 1 Processor -- start 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7efb200700 1 -- start start 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7efb200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc1089d0 0x7f7efc19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7efb200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7efc109390 0x7f7efc19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7efb200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc10d8a0 0x7f7efc1a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7efb200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc118cd0 con 0x7f7efc1089d0 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7efb200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc118b50 con 0x7f7efc109390 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7efb200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc118e50 con 0x7f7efc10d8a0 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7ef9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc1089d0 0x7f7efc19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7ef9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc1089d0 0x7f7efc19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53644/0 (socket says 172.21.15.79:53644) 2024-06-03T02:16:51.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.658+0000 7f7ef9e00700 1 -- 172.21.15.79:0/1162930172 learned_addr learned my addr 172.21.15.79:0/1162930172 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:51.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.659+0000 7f7efa800700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc10d8a0 0x7f7efc1a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.659+0000 7f7ef9400700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7efc109390 0x7f7efc19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.659+0000 7f7ef9e00700 1 -- 172.21.15.79:0/1162930172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc10d8a0 msgr2=0x7f7efc1a39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.659+0000 7f7ef9e00700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc10d8a0 0x7f7efc1a39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.659+0000 7f7ef9e00700 1 -- 172.21.15.79:0/1162930172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7efc109390 msgr2=0x7f7efc19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.659+0000 7f7ef9e00700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7efc109390 0x7f7efc19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.659+0000 7f7ef9e00700 1 -- 172.21.15.79:0/1162930172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc1a40e0 con 0x7f7efc1089d0 2024-06-03T02:16:51.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.659+0000 7f7ef9e00700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc1089d0 0x7f7efc19f120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00e9a0 tx=0x7f7eec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:51.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.660+0000 7f7ef2a00700 1 -- 172.21.15.79:0/1162930172 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00ca10 con 0x7f7efc1089d0 2024-06-03T02:16:51.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.660+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc1a43d0 con 0x7f7efc1089d0 2024-06-03T02:16:51.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.660+0000 7f7ef2a00700 1 -- 172.21.15.79:0/1162930172 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eec00cbb0 con 0x7f7efc1089d0 2024-06-03T02:16:51.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.660+0000 7f7ef2a00700 1 -- 172.21.15.79:0/1162930172 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec018680 con 0x7f7efc1089d0 2024-06-03T02:16:51.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.660+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc1abc50 con 0x7f7efc1089d0 2024-06-03T02:16:51.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.661+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7efc062c20 con 0x7f7efc1089d0 2024-06-03T02:16:51.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.663+0000 7f7ef2a00700 1 -- 172.21.15.79:0/1162930172 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7eec010430 con 0x7f7efc1089d0 2024-06-03T02:16:51.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.665+0000 7f7ef2a00700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7ee00a2cb0 0x7f7ee00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.665+0000 7f7ef2a00700 1 -- 172.21.15.79:0/1162930172 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7eec0cff90 con 0x7f7efc1089d0 2024-06-03T02:16:51.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.665+0000 7f7ef9400700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7ee00a2cb0 0x7f7ee00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.665+0000 7f7ef2a00700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] conn(0x7f7ee00adf90 0x7f7ee00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:51.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.665+0000 7f7ef2a00700 1 -- 172.21.15.79:0/1162930172 --> [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7eec01bec0 con 0x7f7ee00adf90 2024-06-03T02:16:51.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.665+0000 7f7ef2a00700 1 -- 172.21.15.79:0/1162930172 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f7eec0d0340 con 0x7f7efc1089d0 2024-06-03T02:16:51.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.666+0000 7f7ef9400700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7ee00a2cb0 0x7f7ee00a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098e0 tx=0x7f7ee8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:51.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.666+0000 7f7efa800700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] conn(0x7f7ee00adf90 0x7f7ee00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:51.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.666+0000 7f7efa800700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] conn(0x7f7ee00adf90 0x7f7ee00b0430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:51.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.668+0000 7f7ef2a00700 1 -- 172.21.15.79:0/1162930172 <== osd.2 v2:172.21.15.135:6800/3876209625 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7eec01bec0 con 0x7f7ee00adf90 2024-06-03T02:16:51.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.685+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 --> [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7efc0630a0 con 0x7f7ee00adf90 2024-06-03T02:16:51.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.686+0000 7f7ef2a00700 1 -- 172.21.15.79:0/1162930172 <== osd.2 v2:172.21.15.135:6800/3876209625 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7efc0630a0 con 0x7f7ee00adf90 2024-06-03T02:16:51.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.686+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 >> [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] conn(0x7f7ee00adf90 msgr2=0x7f7ee00b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.686+0000 7f7efb200700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] conn(0x7f7ee00adf90 0x7f7ee00b0430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.686+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7ee00a2cb0 msgr2=0x7f7ee00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.686+0000 7f7efb200700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7ee00a2cb0 0x7f7ee00a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098e0 tx=0x7f7ee8005960 comp rx=0 tx=0).stop 2024-06-03T02:16:51.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.686+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc1089d0 msgr2=0x7f7efc19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:51.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.686+0000 7f7efb200700 1 --2- 172.21.15.79:0/1162930172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7efc1089d0 0x7f7efc19f120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00e9a0 tx=0x7f7eec00ecb0 comp rx=0 tx=0).stop 2024-06-03T02:16:51.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.686+0000 7f7efa800700 1 -- 172.21.15.79:0/1162930172 reap_dead start 2024-06-03T02:16:51.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.687+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 shutdown_connections 2024-06-03T02:16:51.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.687+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 >> 172.21.15.79:0/1162930172 conn(0x7f7efc078220 msgr2=0x7f7efc0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:51.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.687+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 shutdown_connections 2024-06-03T02:16:51.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:51.687+0000 7f7efb200700 1 -- 172.21.15.79:0/1162930172 wait complete. 2024-06-03T02:16:52.555 INFO:teuthology.orchestra.run.smithi079.stdout:111669149789 2024-06-03T02:16:52.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-03T02:16:52.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:52 smithi079 ceph-mon[31819]: pgmap v421: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:52 smithi135 ceph-mon[36653]: pgmap v421: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:52 smithi123 ceph-mon[36640]: pgmap v421: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:16:53.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.488+0000 7f2d82800700 1 -- 172.21.15.79:0/1358569127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d7c103580 msgr2=0x7f2d7c103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:53.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.488+0000 7f2d82800700 1 --2- 172.21.15.79:0/1358569127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d7c103580 0x7f2d7c103960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009990 tx=0x7f2d6c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:16:53.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.488+0000 7f2d82800700 1 -- 172.21.15.79:0/1358569127 shutdown_connections 2024-06-03T02:16:53.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.488+0000 7f2d82800700 1 --2- 172.21.15.79:0/1358569127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d7c10c9b0 0x7f2d7c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:53.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.488+0000 7f2d82800700 1 --2- 172.21.15.79:0/1358569127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d7c103580 0x7f2d7c103960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:53.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.488+0000 7f2d82800700 1 --2- 172.21.15.79:0/1358569127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7c103ea0 0x7f2d7c10c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:53.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.488+0000 7f2d82800700 1 -- 172.21.15.79:0/1358569127 >> 172.21.15.79:0/1358569127 conn(0x7f2d7c0fcdb0 msgr2=0x7f2d7c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:53.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.488+0000 7f2d82800700 1 -- 172.21.15.79:0/1358569127 shutdown_connections 2024-06-03T02:16:53.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.488+0000 7f2d82800700 1 -- 172.21.15.79:0/1358569127 wait complete. 2024-06-03T02:16:53.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.489+0000 7f2d82800700 1 Processor -- start 2024-06-03T02:16:53.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.489+0000 7f2d82800700 1 -- start start 2024-06-03T02:16:53.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.490+0000 7f2d82800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d7c103580 0x7f2d7c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:53.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.490+0000 7f2d82800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d7c103ea0 0x7f2d7c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:53.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.490+0000 7f2d82800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7c10c9b0 0x7f2d7c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:53.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.490+0000 7f2d82800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c112760 con 0x7f2d7c103ea0 2024-06-03T02:16:53.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.490+0000 7f2d82800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1125e0 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.490+0000 7f2d82800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1128e0 con 0x7f2d7c103580 2024-06-03T02:16:53.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.490+0000 7f2d80a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d7c103ea0 0x7f2d7c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:53.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d80a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d7c103ea0 0x7f2d7c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53664/0 (socket says 172.21.15.79:53664) 2024-06-03T02:16:53.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d81e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7c10c9b0 0x7f2d7c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:53.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d80a00700 1 -- 172.21.15.79:0/3571423545 learned_addr learned my addr 172.21.15.79:0/3571423545 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:53.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d81400700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d7c103580 0x7f2d7c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:53.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d81e00700 1 -- 172.21.15.79:0/3571423545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d7c103580 msgr2=0x7f2d7c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:53.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d81e00700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d7c103580 0x7f2d7c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:53.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d81e00700 1 -- 172.21.15.79:0/3571423545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d7c103ea0 msgr2=0x7f2d7c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:53.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d81e00700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d7c103ea0 0x7f2d7c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:53.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d81e00700 1 -- 172.21.15.79:0/3571423545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c19fdd0 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.491+0000 7f2d81e00700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7c10c9b0 0x7f2d7c19f690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400efc0 tx=0x7f2d7400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:53.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.492+0000 7f2d7a000700 1 -- 172.21.15.79:0/3571423545 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d740092e0 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.492+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c1a0060 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.492+0000 7f2d7a000700 1 -- 172.21.15.79:0/3571423545 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d74004bb0 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.492+0000 7f2d7a000700 1 -- 172.21.15.79:0/3571423545 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74004280 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.492+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c1a7940 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.493+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d7c102ce0 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.495+0000 7f2d7a000700 1 -- 172.21.15.79:0/3571423545 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2d7401b030 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.496+0000 7f2d7a000700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2d600a2cb0 0x7f2d600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:53.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.497+0000 7f2d7a000700 1 -- 172.21.15.79:0/3571423545 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2d740cfa10 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.497+0000 7f2d81400700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2d600a2cb0 0x7f2d600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:53.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.497+0000 7f2d7a000700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] conn(0x7f2d600adf90 0x7f2d600b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:53.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.497+0000 7f2d7a000700 1 -- 172.21.15.79:0/3571423545 --> [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d7401fe50 con 0x7f2d600adf90 2024-06-03T02:16:53.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.497+0000 7f2d7a000700 1 -- 172.21.15.79:0/3571423545 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f2d740cfdc0 con 0x7f2d7c10c9b0 2024-06-03T02:16:53.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.498+0000 7f2d80a00700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] conn(0x7f2d600adf90 0x7f2d600b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:53.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.498+0000 7f2d81400700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2d600a2cb0 0x7f2d600a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00b2e0 tx=0x7f2d6c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:53.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.498+0000 7f2d80a00700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] conn(0x7f2d600adf90 0x7f2d600b0430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:53.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.499+0000 7f2d7a000700 1 -- 172.21.15.79:0/3571423545 <== osd.11 v2:172.21.15.135:6824/2083189453 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d7401fe50 con 0x7f2d600adf90 2024-06-03T02:16:53.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.517+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 --> [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d7c06aa50 con 0x7f2d600adf90 2024-06-03T02:16:53.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.518+0000 7f2d7a000700 1 -- 172.21.15.79:0/3571423545 <== osd.11 v2:172.21.15.135:6824/2083189453 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2d7c06aa50 con 0x7f2d600adf90 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.518+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 >> [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] conn(0x7f2d600adf90 msgr2=0x7f2d600b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.518+0000 7f2d82800700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] conn(0x7f2d600adf90 0x7f2d600b0430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.518+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2d600a2cb0 msgr2=0x7f2d600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.518+0000 7f2d82800700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2d600a2cb0 0x7f2d600a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00b2e0 tx=0x7f2d6c005fd0 comp rx=0 tx=0).stop 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.518+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7c10c9b0 msgr2=0x7f2d7c19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.518+0000 7f2d82800700 1 --2- 172.21.15.79:0/3571423545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7c10c9b0 0x7f2d7c19f690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400efc0 tx=0x7f2d7400c3a0 comp rx=0 tx=0).stop 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.518+0000 7f2d81e00700 1 -- 172.21.15.79:0/3571423545 reap_dead start 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.519+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 shutdown_connections 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.519+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 >> 172.21.15.79:0/3571423545 conn(0x7f2d7c0fcdb0 msgr2=0x7f2d7c10ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.519+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 shutdown_connections 2024-06-03T02:16:53.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:53.519+0000 7f2d82800700 1 -- 172.21.15.79:0/3571423545 wait complete. 2024-06-03T02:16:54.529 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:54 smithi079 ceph-mon[31819]: pgmap v422: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:54 smithi135 ceph-mon[36653]: pgmap v422: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.615+0000 7fb2ebe00700 1 -- 172.21.15.79:0/1448796596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2ec107700 msgr2=0x7fb2ec10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.615+0000 7fb2ebe00700 1 --2- 172.21.15.79:0/1448796596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2ec107700 0x7fb2ec10f2d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400b0d0 tx=0x7fb2e400b3e0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.615+0000 7fb2ebe00700 1 -- 172.21.15.79:0/1448796596 shutdown_connections 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.615+0000 7fb2ebe00700 1 --2- 172.21.15.79:0/1448796596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2ec107700 0x7fb2ec10f2d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.615+0000 7fb2ebe00700 1 --2- 172.21.15.79:0/1448796596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2ec102620 0x7fb2ec102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.615+0000 7fb2ebe00700 1 --2- 172.21.15.79:0/1448796596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2ec102fd0 0x7fb2ec107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.615+0000 7fb2ebe00700 1 -- 172.21.15.79:0/1448796596 >> 172.21.15.79:0/1448796596 conn(0x7fb2ec0fdf10 msgr2=0x7fb2ec100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.615+0000 7fb2ebe00700 1 -- 172.21.15.79:0/1448796596 shutdown_connections 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.615+0000 7fb2ebe00700 1 -- 172.21.15.79:0/1448796596 wait complete. 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.617+0000 7fb2ebe00700 1 Processor -- start 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.617+0000 7fb2ebe00700 1 -- start start 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.617+0000 7fb2ebe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2ec102620 0x7fb2ec19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.617+0000 7fb2ebe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2ec102fd0 0x7fb2ec19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.617+0000 7fb2ebe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2ec107700 0x7fb2ec19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.617+0000 7fb2ebe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec112760 con 0x7fb2ec102fd0 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.617+0000 7fb2ebe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1125e0 con 0x7fb2ec102620 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.617+0000 7fb2ebe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1128e0 con 0x7fb2ec107700 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2ea000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2ec102fd0 0x7fb2ec19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2ea000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2ec102fd0 0x7fb2ec19b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53672/0 (socket says 172.21.15.79:53672) 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2ea000700 1 -- 172.21.15.79:0/2490209519 learned_addr learned my addr 172.21.15.79:0/2490209519 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2eaa00700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2ec102620 0x7fb2ec19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2eb400700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2ec107700 0x7fb2ec19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2ea000700 1 -- 172.21.15.79:0/2490209519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2ec107700 msgr2=0x7fb2ec19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2ea000700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2ec107700 0x7fb2ec19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2ea000700 1 -- 172.21.15.79:0/2490209519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2ec102620 msgr2=0x7fb2ec19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2ea000700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2ec102620 0x7fb2ec19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2ea000700 1 -- 172.21.15.79:0/2490209519 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec19fd40 con 0x7fb2ec102fd0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.618+0000 7fb2eb400700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2ec107700 0x7fb2ec19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.619+0000 7fb2ea000700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2ec102fd0 0x7fb2ec19b270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e9f0 tx=0x7fb2d800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.619+0000 7fb2e3400700 1 -- 172.21.15.79:0/2490209519 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8004a30 con 0x7fb2ec102fd0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.619+0000 7fb2e3400700 1 -- 172.21.15.79:0/2490209519 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d800cca0 con 0x7fb2ec102fd0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.619+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ec1a0030 con 0x7fb2ec102fd0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.619+0000 7fb2e3400700 1 -- 172.21.15.79:0/2490209519 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80056c0 con 0x7fb2ec102fd0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.619+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ec1a78b0 con 0x7fb2ec102fd0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.620+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2ec0ffb20 con 0x7fb2ec102fd0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.623+0000 7fb2e3400700 1 -- 172.21.15.79:0/2490209519 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb2d800c7c0 con 0x7fb2ec102fd0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.624+0000 7fb2e3400700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d00a2d30 0x7fb2d00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.624+0000 7fb2e3400700 1 -- 172.21.15.79:0/2490209519 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb2d80cf6b0 con 0x7fb2ec102fd0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.624+0000 7fb2eaa00700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d00a2d30 0x7fb2d00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.625+0000 7fb2e3400700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] conn(0x7fb2d00ae010 0x7fb2d00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.625+0000 7fb2e3400700 1 -- 172.21.15.79:0/2490209519 --> [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2d00b0b80 con 0x7fb2d00ae010 2024-06-03T02:16:54.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.625+0000 7fb2e3400700 1 -- 172.21.15.79:0/2490209519 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fb2d80d6050 con 0x7fb2ec102fd0 2024-06-03T02:16:54.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.625+0000 7fb2eb400700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] conn(0x7fb2d00ae010 0x7fb2d00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.625+0000 7fb2eaa00700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d00a2d30 0x7fb2d00a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009680 tx=0x7fb2dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:54.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.626+0000 7fb2eb400700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] conn(0x7fb2d00ae010 0x7fb2d00b04b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:54.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.626+0000 7fb2e3400700 1 -- 172.21.15.79:0/2490209519 <== osd.10 v2:172.21.15.123:6824/1137082874 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2d00b0b80 con 0x7fb2d00ae010 2024-06-03T02:16:54.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.662+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 --> [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb2ec103b00 con 0x7fb2d00ae010 2024-06-03T02:16:54.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.663+0000 7fb2e3400700 1 -- 172.21.15.79:0/2490209519 <== osd.10 v2:172.21.15.123:6824/1137082874 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb2ec103b00 con 0x7fb2d00ae010 2024-06-03T02:16:54.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.663+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 >> [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] conn(0x7fb2d00ae010 msgr2=0x7fb2d00b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.664+0000 7fb2ebe00700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] conn(0x7fb2d00ae010 0x7fb2d00b04b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.664+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d00a2d30 msgr2=0x7fb2d00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.664+0000 7fb2ebe00700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d00a2d30 0x7fb2d00a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009680 tx=0x7fb2dc009260 comp rx=0 tx=0).stop 2024-06-03T02:16:54.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.664+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2ec102fd0 msgr2=0x7fb2ec19b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.664+0000 7fb2ebe00700 1 --2- 172.21.15.79:0/2490209519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2ec102fd0 0x7fb2ec19b270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e9f0 tx=0x7fb2d800ed00 comp rx=0 tx=0).stop 2024-06-03T02:16:54.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.664+0000 7fb2eb400700 1 -- 172.21.15.79:0/2490209519 reap_dead start 2024-06-03T02:16:54.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.664+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 shutdown_connections 2024-06-03T02:16:54.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.664+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 >> 172.21.15.79:0/2490209519 conn(0x7fb2ec0fdf10 msgr2=0x7fb2ec10db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:54.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.665+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 shutdown_connections 2024-06-03T02:16:54.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.665+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2490209519 wait complete. 2024-06-03T02:16:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:54 smithi123 ceph-mon[36640]: pgmap v422: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:16:54.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.709+0000 7fbadb000700 1 -- 172.21.15.79:0/1879165523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbad4108540 msgr2=0x7fbad4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.709+0000 7fbadb000700 1 --2- 172.21.15.79:0/1879165523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbad4108540 0x7fbad4108920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbac40099c0 tx=0x7fbac4009cd0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.709+0000 7fbadb000700 1 -- 172.21.15.79:0/1879165523 shutdown_connections 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.709+0000 7fbadb000700 1 --2- 172.21.15.79:0/1879165523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4102e50 0x7fbad410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.709+0000 7fbadb000700 1 --2- 172.21.15.79:0/1879165523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbad4108540 0x7fbad4108920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.709+0000 7fbadb000700 1 --2- 172.21.15.79:0/1879165523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbad4102490 0x7fbad4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.709+0000 7fbadb000700 1 -- 172.21.15.79:0/1879165523 >> 172.21.15.79:0/1879165523 conn(0x7fbad40fdf40 msgr2=0x7fbad4100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.709+0000 7fbadb000700 1 -- 172.21.15.79:0/1879165523 shutdown_connections 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.709+0000 7fbadb000700 1 -- 172.21.15.79:0/1879165523 wait complete. 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.710+0000 7fbadb000700 1 Processor -- start 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.711+0000 7fbadb000700 1 -- start start 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.711+0000 7fbadb000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbad4102490 0x7fbad419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.711+0000 7fbadb000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbad4102e50 0x7fbad419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.711+0000 7fbadb000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4108540 0x7fbad4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.711+0000 7fbadb000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4112770 con 0x7fbad4108540 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.711+0000 7fbadb000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41125f0 con 0x7fbad4102e50 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.711+0000 7fbadb000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41128f0 con 0x7fbad4102490 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbad4102490 0x7fbad419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad9000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4108540 0x7fbad4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbad4102e50 0x7fbad419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbad4102490 0x7fbad419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:50864/0 (socket says 172.21.15.79:50864) 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbad4102e50 0x7fbad419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36052/0 (socket says 172.21.15.79:36052) 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3e00700 1 -- 172.21.15.79:0/2420986870 learned_addr learned my addr 172.21.15.79:0/2420986870 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3e00700 1 -- 172.21.15.79:0/2420986870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbad4102e50 msgr2=0x7fbad419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3e00700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbad4102e50 0x7fbad419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3e00700 1 -- 172.21.15.79:0/2420986870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4108540 msgr2=0x7fbad4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3e00700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbad4108540 0x7fbad4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3e00700 1 -- 172.21.15.79:0/2420986870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad41990b0 con 0x7fbad4102490 2024-06-03T02:16:54.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.712+0000 7fbad3400700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbad4102e50 0x7fbad419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.713+0000 7fbad3e00700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbad4102490 0x7fbad419e790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbac40059d0 tx=0x7fbac400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.713+0000 7fbad0c00700 1 -- 172.21.15.79:0/2420986870 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4027070 con 0x7fbad4102490 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.713+0000 7fbad0c00700 1 -- 172.21.15.79:0/2420986870 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4003680 con 0x7fbad4102490 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.713+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad4199340 con 0x7fbad4102490 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.713+0000 7fbad0c00700 1 -- 172.21.15.79:0/2420986870 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40173f0 con 0x7fbad4102490 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.713+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41a5660 con 0x7fbad4102490 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.714+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbad4062c20 con 0x7fbad4102490 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.717+0000 7fbad0c00700 1 -- 172.21.15.79:0/2420986870 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbac402a070 con 0x7fbad4102490 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.719+0000 7fbad0c00700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbab80a2d30 0x7fbab80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.719+0000 7fbad0c00700 1 -- 172.21.15.79:0/2420986870 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fbac40d0350 con 0x7fbad4102490 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.720+0000 7fbad3400700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbab80a2d30 0x7fbab80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.720+0000 7fbad0c00700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] conn(0x7fbab80ae010 0x7fbab80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.720+0000 7fbad0c00700 1 -- 172.21.15.79:0/2420986870 --> [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbac402ee50 con 0x7fbab80ae010 2024-06-03T02:16:54.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.720+0000 7fbad0c00700 1 -- 172.21.15.79:0/2420986870 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fbac40d0700 con 0x7fbad4102490 2024-06-03T02:16:54.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.720+0000 7fbad9000700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] conn(0x7fbab80ae010 0x7fbab80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:54.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.721+0000 7fbad3400700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbab80a2d30 0x7fbab80a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbad4107ab0 tx=0x7fbac0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:54.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.721+0000 7fbad9000700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] conn(0x7fbab80ae010 0x7fbab80b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:54.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.722+0000 7fbad0c00700 1 -- 172.21.15.79:0/2420986870 <== osd.1 v2:172.21.15.123:6800/1025664412 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbac402ee50 con 0x7fbab80ae010 2024-06-03T02:16:54.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.740+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 --> [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbad4063020 con 0x7fbab80ae010 2024-06-03T02:16:54.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.740+0000 7fbad0c00700 1 -- 172.21.15.79:0/2420986870 <== osd.1 v2:172.21.15.123:6800/1025664412 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbad4063020 con 0x7fbab80ae010 2024-06-03T02:16:54.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 >> [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] conn(0x7fbab80ae010 msgr2=0x7fbab80b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] conn(0x7fbab80ae010 0x7fbab80b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:54.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbab80a2d30 msgr2=0x7fbab80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbab80a2d30 0x7fbab80a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbad4107ab0 tx=0x7fbac0009260 comp rx=0 tx=0).stop 2024-06-03T02:16:54.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbad4102490 msgr2=0x7fbad419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:54.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 --2- 172.21.15.79:0/2420986870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbad4102490 0x7fbad419e790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbac40059d0 tx=0x7fbac400fa00 comp rx=0 tx=0).stop 2024-06-03T02:16:54.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbad9000700 1 -- 172.21.15.79:0/2420986870 reap_dead start 2024-06-03T02:16:54.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 shutdown_connections 2024-06-03T02:16:54.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 >> 172.21.15.79:0/2420986870 conn(0x7fbad40fdf40 msgr2=0x7fbad40ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:54.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 shutdown_connections 2024-06-03T02:16:54.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:54.741+0000 7fbadb000700 1 -- 172.21.15.79:0/2420986870 wait complete. 2024-06-03T02:16:56.491 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:56 smithi079 ceph-mon[31819]: pgmap v423: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-03T02:16:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:56 smithi135 ceph-mon[36653]: pgmap v423: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-03T02:16:56.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:56 smithi123 ceph-mon[36640]: pgmap v423: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-03T02:16:56.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.663+0000 7f0c2a800700 1 -- 172.21.15.79:0/2222671241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c24108ac0 msgr2=0x7f0c24108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.663+0000 7f0c2a800700 1 --2- 172.21.15.79:0/2222671241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c24108ac0 0x7f0c24108f40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009990 tx=0x7f0c10009ca0 comp rx=0 tx=0).stop 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.663+0000 7f0c2a800700 1 -- 172.21.15.79:0/2222671241 shutdown_connections 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.663+0000 7f0c2a800700 1 --2- 172.21.15.79:0/2222671241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c24109480 0x7f0c24115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.663+0000 7f0c2a800700 1 --2- 172.21.15.79:0/2222671241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c2410eac0 0x7f0c2410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.663+0000 7f0c2a800700 1 --2- 172.21.15.79:0/2222671241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c24108ac0 0x7f0c24108f40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.663+0000 7f0c2a800700 1 -- 172.21.15.79:0/2222671241 >> 172.21.15.79:0/2222671241 conn(0x7f0c24075c60 msgr2=0x7f0c241072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.664+0000 7f0c2a800700 1 -- 172.21.15.79:0/2222671241 shutdown_connections 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.664+0000 7f0c2a800700 1 -- 172.21.15.79:0/2222671241 wait complete. 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.665+0000 7f0c2a800700 1 Processor -- start 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.665+0000 7f0c2a800700 1 -- start start 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.665+0000 7f0c2a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c24108ac0 0x7f0c2419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.665+0000 7f0c2a800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c24109480 0x7f0c2419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:56.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.665+0000 7f0c2a800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c2410eac0 0x7f0c241a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.665+0000 7f0c2a800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24118e60 con 0x7f0c24109480 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.665+0000 7f0c2a800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24118ce0 con 0x7f0c2410eac0 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.665+0000 7f0c2a800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24118fe0 con 0x7f0c24108ac0 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c22a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c24109480 0x7f0c2419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c23400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c24108ac0 0x7f0c2419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c22a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c24109480 0x7f0c2419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:45060/0 (socket says 172.21.15.79:45060) 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c22a00700 1 -- 172.21.15.79:0/3603968707 learned_addr learned my addr 172.21.15.79:0/3603968707 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c23e00700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c2410eac0 0x7f0c241a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c23400700 1 -- 172.21.15.79:0/3603968707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c2410eac0 msgr2=0x7f0c241a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c23400700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c2410eac0 0x7f0c241a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c23400700 1 -- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c24109480 msgr2=0x7f0c2419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c23400700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c24109480 0x7f0c2419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:56.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c23400700 1 -- 172.21.15.79:0/3603968707 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c241a4260 con 0x7f0c24108ac0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.666+0000 7f0c23400700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c24108ac0 0x7f0c2419f250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400e9f0 tx=0x7f0c1400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.667+0000 7f0c1be00700 1 -- 172.21.15.79:0/3603968707 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400c9c0 con 0x7f0c24108ac0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.667+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c241a4550 con 0x7f0c24108ac0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.667+0000 7f0c1be00700 1 -- 172.21.15.79:0/3603968707 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1400cb60 con 0x7f0c24108ac0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.667+0000 7f0c1be00700 1 -- 172.21.15.79:0/3603968707 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14018680 con 0x7f0c24108ac0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.667+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c241abdd0 con 0x7f0c24108ac0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.668+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c24062ba0 con 0x7f0c24108ac0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.670+0000 7f0c1be00700 1 -- 172.21.15.79:0/3603968707 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0c14010430 con 0x7f0c24108ac0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.671+0000 7f0c1be00700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c080a2d30 0x7f0c080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.671+0000 7f0c1be00700 1 -- 172.21.15.79:0/3603968707 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0c140cfa60 con 0x7f0c24108ac0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.671+0000 7f0c22a00700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c080a2d30 0x7f0c080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.672+0000 7f0c1be00700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] conn(0x7f0c080ae010 0x7f0c080b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:56.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.672+0000 7f0c1be00700 1 -- 172.21.15.79:0/3603968707 --> [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c1401bec0 con 0x7f0c080ae010 2024-06-03T02:16:56.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.672+0000 7f0c1be00700 1 -- 172.21.15.79:0/3603968707 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f0c140cfe10 con 0x7f0c24108ac0 2024-06-03T02:16:56.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.672+0000 7f0c23e00700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] conn(0x7f0c080ae010 0x7f0c080b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:56.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.672+0000 7f0c22a00700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c080a2d30 0x7f0c080a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009750 tx=0x7f0c1000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:56.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.673+0000 7f0c23e00700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] conn(0x7f0c080ae010 0x7f0c080b04b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:56.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.674+0000 7f0c1be00700 1 -- 172.21.15.79:0/3603968707 <== osd.6 v2:172.21.15.79:6818/2064944325 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c1401bec0 con 0x7f0c080ae010 2024-06-03T02:16:56.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 --> [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c24062fa0 con 0x7f0c080ae010 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c1be00700 1 -- 172.21.15.79:0/3603968707 <== osd.6 v2:172.21.15.79:6818/2064944325 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0c24062fa0 con 0x7f0c080ae010 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] conn(0x7f0c080ae010 msgr2=0x7f0c080b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] conn(0x7f0c080ae010 0x7f0c080b04b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c080a2d30 msgr2=0x7f0c080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c080a2d30 0x7f0c080a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009750 tx=0x7f0c1000b280 comp rx=0 tx=0).stop 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c24108ac0 msgr2=0x7f0c2419f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 --2- 172.21.15.79:0/3603968707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c24108ac0 0x7f0c2419f250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400e9f0 tx=0x7f0c1400ed00 comp rx=0 tx=0).stop 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c23e00700 1 -- 172.21.15.79:0/3603968707 reap_dead start 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 shutdown_connections 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 >> 172.21.15.79:0/3603968707 conn(0x7f0c24075c60 msgr2=0x7f0c24076180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:56.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.693+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 shutdown_connections 2024-06-03T02:16:56.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:56.694+0000 7f0c2a800700 1 -- 172.21.15.79:0/3603968707 wait complete. 2024-06-03T02:16:57.316 INFO:teuthology.orchestra.run.smithi079.stdout:317827579960 2024-06-03T02:16:57.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.555+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1516921367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cec10ec00 msgr2=0x7f5cec111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.555+0000 7f5cf1e00700 1 --2- 172.21.15.79:0/1516921367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cec10ec00 0x7f5cec111070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400b0a0 tx=0x7f5ce400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.555+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1516921367 shutdown_connections 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.555+0000 7f5cf1e00700 1 --2- 172.21.15.79:0/1516921367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cec10ec00 0x7f5cec111070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.555+0000 7f5cf1e00700 1 --2- 172.21.15.79:0/1516921367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cec0748c0 0x7f5cec074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.555+0000 7f5cf1e00700 1 --2- 172.21.15.79:0/1516921367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cec0751e0 0x7f5cec10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.555+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1516921367 >> 172.21.15.79:0/1516921367 conn(0x7f5cec0fdf10 msgr2=0x7f5cec100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.555+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1516921367 shutdown_connections 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.555+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1516921367 wait complete. 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.556+0000 7f5cf1e00700 1 Processor -- start 2024-06-03T02:16:57.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.556+0000 7f5cf1e00700 1 -- start start 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cf1e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cec0748c0 0x7f5cec10e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cf1e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cec0751e0 0x7f5cec10b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cf1e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cec10ec00 0x7f5cec10bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cf1e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec114940 con 0x7f5cec10ec00 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cf1e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1147c0 con 0x7f5cec0748c0 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cf1e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec114ac0 con 0x7f5cec0751e0 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cf0a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cec0748c0 0x7f5cec10e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cf0a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cec0748c0 0x7f5cec10e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49272/0 (socket says 172.21.15.79:49272) 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cf0a00700 1 -- 172.21.15.79:0/1504338125 learned_addr learned my addr 172.21.15.79:0/1504338125 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.557+0000 7f5cebe00700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cec0751e0 0x7f5cec10b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.558+0000 7f5cf0a00700 1 -- 172.21.15.79:0/1504338125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cec0751e0 msgr2=0x7f5cec10b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.558+0000 7f5cf0a00700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cec0751e0 0x7f5cec10b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.558+0000 7f5cf1400700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cec10ec00 0x7f5cec10bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:57.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.558+0000 7f5cf0a00700 1 -- 172.21.15.79:0/1504338125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cec10ec00 msgr2=0x7f5cec10bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:57.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.558+0000 7f5cf0a00700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cec10ec00 0x7f5cec10bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:57.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.558+0000 7f5cf0a00700 1 -- 172.21.15.79:0/1504338125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec1ab680 con 0x7f5cec0748c0 2024-06-03T02:16:57.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.558+0000 7f5cf1400700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cec10ec00 0x7f5cec10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:16:57.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.558+0000 7f5cf0a00700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cec0748c0 0x7f5cec10e530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000e9c0 tx=0x7f5ce000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:57.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.559+0000 7f5ce9600700 1 -- 172.21.15.79:0/1504338125 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce000cbd0 con 0x7f5cec0748c0 2024-06-03T02:16:57.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.559+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cec1ab850 con 0x7f5cec0748c0 2024-06-03T02:16:57.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.559+0000 7f5ce9600700 1 -- 172.21.15.79:0/1504338125 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce000cd70 con 0x7f5cec0748c0 2024-06-03T02:16:57.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.559+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cec1abce0 con 0x7f5cec0748c0 2024-06-03T02:16:57.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.559+0000 7f5ce9600700 1 -- 172.21.15.79:0/1504338125 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0010430 con 0x7f5cec0748c0 2024-06-03T02:16:57.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.560+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ccc001030 con 0x7f5cec0748c0 2024-06-03T02:16:57.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.562+0000 7f5ce9600700 1 -- 172.21.15.79:0/1504338125 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5ce0004750 con 0x7f5cec0748c0 2024-06-03T02:16:57.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.564+0000 7f5ce9600700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5cd40a2cb0 0x7f5cd40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:57.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.564+0000 7f5ce9600700 1 -- 172.21.15.79:0/1504338125 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5ce00cf770 con 0x7f5cec0748c0 2024-06-03T02:16:57.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.564+0000 7f5cebe00700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5cd40a2cb0 0x7f5cd40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:57.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.564+0000 7f5ce9600700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] conn(0x7f5cd40adf90 0x7f5cd40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:57.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.564+0000 7f5ce9600700 1 -- 172.21.15.79:0/1504338125 --> [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5cd40b0b00 con 0x7f5cd40adf90 2024-06-03T02:16:57.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.564+0000 7f5ce9600700 1 -- 172.21.15.79:0/1504338125 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f5ce00d6050 con 0x7f5cec0748c0 2024-06-03T02:16:57.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.565+0000 7f5cf1400700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] conn(0x7f5cd40adf90 0x7f5cd40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:57.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.565+0000 7f5cebe00700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5cd40a2cb0 0x7f5cd40a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009680 tx=0x7f5cd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:57.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.566+0000 7f5cf1400700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] conn(0x7f5cd40adf90 0x7f5cd40b0430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:57.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.566+0000 7f5ce9600700 1 -- 172.21.15.79:0/1504338125 <== osd.5 v2:172.21.15.135:6808/3110965280 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cd40b0b00 con 0x7f5cd40adf90 2024-06-03T02:16:57.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.604+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 --> [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ccc002e60 con 0x7f5cd40adf90 2024-06-03T02:16:57.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.605+0000 7f5ce9600700 1 -- 172.21.15.79:0/1504338125 <== osd.5 v2:172.21.15.135:6808/3110965280 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ccc002e60 con 0x7f5cd40adf90 2024-06-03T02:16:57.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.606+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 >> [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] conn(0x7f5cd40adf90 msgr2=0x7f5cd40b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:57.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.606+0000 7f5cf1e00700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] conn(0x7f5cd40adf90 0x7f5cd40b0430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:57.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.606+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5cd40a2cb0 msgr2=0x7f5cd40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:57.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.606+0000 7f5cf1e00700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5cd40a2cb0 0x7f5cd40a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009680 tx=0x7f5cd8009260 comp rx=0 tx=0).stop 2024-06-03T02:16:57.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.606+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cec0748c0 msgr2=0x7f5cec10e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:57.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.606+0000 7f5cf1e00700 1 --2- 172.21.15.79:0/1504338125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cec0748c0 0x7f5cec10e530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000e9c0 tx=0x7f5ce000ecd0 comp rx=0 tx=0).stop 2024-06-03T02:16:57.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.606+0000 7f5cf1400700 1 -- 172.21.15.79:0/1504338125 reap_dead start 2024-06-03T02:16:57.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.606+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 shutdown_connections 2024-06-03T02:16:57.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.606+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 >> 172.21.15.79:0/1504338125 conn(0x7f5cec0fdf10 msgr2=0x7f5cec0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:57.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.607+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 shutdown_connections 2024-06-03T02:16:57.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.607+0000 7f5cf1e00700 1 -- 172.21.15.79:0/1504338125 wait complete. 2024-06-03T02:16:58.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.628+0000 7f0675000700 1 -- 172.21.15.79:0/2096036676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670105060 msgr2=0x7f0670111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:58.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.628+0000 7f0675000700 1 --2- 172.21.15.79:0/2096036676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670105060 0x7f0670111570 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0668009f90 tx=0x7f066800b330 comp rx=0 tx=0).stop 2024-06-03T02:16:58.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.629+0000 7f0675000700 1 -- 172.21.15.79:0/2096036676 shutdown_connections 2024-06-03T02:16:58.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.629+0000 7f0675000700 1 --2- 172.21.15.79:0/2096036676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670105060 0x7f0670111570 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:58.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.629+0000 7f0675000700 1 --2- 172.21.15.79:0/2096036676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067010a6a0 0x7f067010aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:58.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.629+0000 7f0675000700 1 --2- 172.21.15.79:0/2096036676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06701046a0 0x7f0670104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.629+0000 7f0675000700 1 -- 172.21.15.79:0/2096036676 >> 172.21.15.79:0/2096036676 conn(0x7f0670100190 msgr2=0x7f06701025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.629+0000 7f0675000700 1 -- 172.21.15.79:0/2096036676 shutdown_connections 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.629+0000 7f0675000700 1 -- 172.21.15.79:0/2096036676 wait complete. 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.630+0000 7f0675000700 1 Processor -- start 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.630+0000 7f0675000700 1 -- start start 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.630+0000 7f0675000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06701046a0 0x7f067019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.630+0000 7f0675000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670105060 0x7f067019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.630+0000 7f0675000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067010a6a0 0x7f067019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.630+0000 7f0675000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0670114a40 con 0x7f067010a6a0 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.630+0000 7f0675000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06701148c0 con 0x7f0670105060 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.630+0000 7f0675000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0670114bc0 con 0x7f06701046a0 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.631+0000 7f066de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06701046a0 0x7f067019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:58.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.631+0000 7f066de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06701046a0 0x7f067019ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:48492/0 (socket says 172.21.15.79:48492) 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.631+0000 7f066de00700 1 -- 172.21.15.79:0/790176359 learned_addr learned my addr 172.21.15.79:0/790176359 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.631+0000 7f066e800700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067010a6a0 0x7f067019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.631+0000 7f066de00700 1 -- 172.21.15.79:0/790176359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670105060 msgr2=0x7f067019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.631+0000 7f066de00700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670105060 0x7f067019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.631+0000 7f066de00700 1 -- 172.21.15.79:0/790176359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067010a6a0 msgr2=0x7f067019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.631+0000 7f066de00700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067010a6a0 0x7f067019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.631+0000 7f066de00700 1 -- 172.21.15.79:0/790176359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067019fe50 con 0x7f06701046a0 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.632+0000 7f066de00700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06701046a0 0x7f067019ae90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f066000ec10 tx=0x7f066000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.632+0000 7f0666a00700 1 -- 172.21.15.79:0/790176359 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066000cd40 con 0x7f06701046a0 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.632+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06701a0140 con 0x7f06701046a0 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.632+0000 7f0666a00700 1 -- 172.21.15.79:0/790176359 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0660004500 con 0x7f06701046a0 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.632+0000 7f0666a00700 1 -- 172.21.15.79:0/790176359 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660010690 con 0x7f06701046a0 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.632+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06701a79c0 con 0x7f06701046a0 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.633+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0670062c20 con 0x7f06701046a0 2024-06-03T02:16:58.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.636+0000 7f0666a00700 1 -- 172.21.15.79:0/790176359 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0660003f10 con 0x7f06701046a0 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.639+0000 7f0666a00700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06540a2cb0 0x7f06540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.639+0000 7f0666a00700 1 -- 172.21.15.79:0/790176359 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f06600cfbb0 con 0x7f06701046a0 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.639+0000 7f066d400700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06540a2cb0 0x7f06540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.639+0000 7f0666a00700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] conn(0x7f06540adf90 0x7f06540b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.639+0000 7f0666a00700 1 -- 172.21.15.79:0/790176359 --> [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0660023e50 con 0x7f06540adf90 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.639+0000 7f0666a00700 1 -- 172.21.15.79:0/790176359 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f06600cff60 con 0x7f06701046a0 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.640+0000 7f066d400700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06540a2cb0 0x7f06540a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f065c005950 tx=0x7f065c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.640+0000 7f066e800700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] conn(0x7f06540adf90 0x7f06540b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.641+0000 7f066e800700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] conn(0x7f06540adf90 0x7f06540b0430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:16:58.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.641+0000 7f0666a00700 1 -- 172.21.15.79:0/790176359 <== osd.8 v2:172.21.15.135:6816/2566672579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0660023e50 con 0x7f06540adf90 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.660+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 --> [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0670063020 con 0x7f06540adf90 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.660+0000 7f0666a00700 1 -- 172.21.15.79:0/790176359 <== osd.8 v2:172.21.15.135:6816/2566672579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0670063020 con 0x7f06540adf90 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.661+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 >> [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] conn(0x7f06540adf90 msgr2=0x7f06540b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.661+0000 7f0675000700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] conn(0x7f06540adf90 0x7f06540b0430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.661+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06540a2cb0 msgr2=0x7f06540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.661+0000 7f0675000700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06540a2cb0 0x7f06540a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f065c005950 tx=0x7f065c0058e0 comp rx=0 tx=0).stop 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.661+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06701046a0 msgr2=0x7f067019ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.661+0000 7f0675000700 1 --2- 172.21.15.79:0/790176359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06701046a0 0x7f067019ae90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f066000ec10 tx=0x7f066000ef20 comp rx=0 tx=0).stop 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.661+0000 7f066e800700 1 -- 172.21.15.79:0/790176359 reap_dead start 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.661+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 shutdown_connections 2024-06-03T02:16:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.661+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 >> 172.21.15.79:0/790176359 conn(0x7f0670100190 msgr2=0x7f06701006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:16:58.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.662+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 shutdown_connections 2024-06-03T02:16:58.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:16:57.662+0000 7f0675000700 1 -- 172.21.15.79:0/790176359 wait complete. 2024-06-03T02:16:58.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:16:58 smithi079 ceph-mon[31819]: pgmap v424: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T02:16:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:16:58 smithi135 ceph-mon[36653]: pgmap v424: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T02:16:58.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:16:58 smithi123 ceph-mon[36640]: pgmap v424: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T02:16:58.936 INFO:teuthology.orchestra.run.smithi079.stdout:77309411429 2024-06-03T02:16:58.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-03T02:17:00.111 INFO:teuthology.orchestra.run.smithi079.stdout:171798691926 2024-06-03T02:17:00.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-03T02:17:00.194 INFO:teuthology.orchestra.run.smithi079.stdout:206158430288 2024-06-03T02:17:00.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-03T02:17:00.315 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:00 smithi079 ceph-mon[31819]: pgmap v425: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-06-03T02:17:00.325 INFO:teuthology.orchestra.run.smithi079.stdout:146028888152 2024-06-03T02:17:00.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-03T02:17:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:00 smithi135 ceph-mon[36653]: pgmap v425: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-06-03T02:17:00.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:00 smithi123 ceph-mon[36640]: pgmap v425: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-06-03T02:17:00.755 INFO:teuthology.orchestra.run.smithi079.stdout:352187318323 2024-06-03T02:17:00.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-03T02:17:01.135 INFO:teuthology.orchestra.run.smithi079.stdout:55834574954 2024-06-03T02:17:01.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-03T02:17:01.673 INFO:teuthology.orchestra.run.smithi079.stdout:261993005124 2024-06-03T02:17:01.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-03T02:17:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:01.720 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:01.777 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:01.906 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:01.906 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:02.148 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:02.150 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:02.154 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:02.155 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:02.167 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:02.167 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:02.543 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:02 smithi079 ceph-mon[31819]: pgmap v426: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:02 smithi135 ceph-mon[36653]: pgmap v426: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:02.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:02 smithi123 ceph-mon[36640]: pgmap v426: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:04.522 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:04 smithi079 ceph-mon[31819]: pgmap v427: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:04.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:04 smithi135 ceph-mon[36653]: pgmap v427: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:04 smithi123 ceph-mon[36640]: pgmap v427: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:06 smithi079 ceph-mon[31819]: pgmap v428: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:06 smithi135 ceph-mon[36653]: pgmap v428: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:06.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:06 smithi123 ceph-mon[36640]: pgmap v428: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:08.515 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:08 smithi079 ceph-mon[31819]: pgmap v429: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-06-03T02:17:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:08 smithi135 ceph-mon[36653]: pgmap v429: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-06-03T02:17:08.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:08 smithi123 ceph-mon[36640]: pgmap v429: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-06-03T02:17:09.439 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:09 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4071202245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-03T02:17:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:09 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4071202245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-03T02:17:09.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:09 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4071202245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-03T02:17:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:10 smithi079 ceph-mon[31819]: pgmap v430: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-06-03T02:17:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:10 smithi135 ceph-mon[36653]: pgmap v430: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-06-03T02:17:10.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:10 smithi123 ceph-mon[36640]: pgmap v430: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-06-03T02:17:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:11 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:17:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:11 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:11 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:11 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:11 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:11 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:17:11.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:11 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:11 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:11 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:11 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:11 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:17:11.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:11 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:11 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:11 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:11.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:11 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:12 smithi079 ceph-mon[31819]: pgmap v431: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-06-03T02:17:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:12 smithi135 ceph-mon[36653]: pgmap v431: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-06-03T02:17:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:12 smithi123 ceph-mon[36640]: pgmap v431: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-06-03T02:17:12.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.455+0000 7f085b200700 1 -- 172.21.15.79:0/3356569091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 msgr2=0x7f0854108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:12.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.455+0000 7f085b200700 1 --2- 172.21.15.79:0/3356569091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 0x7f0854108f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f08400098f0 tx=0x7f0840009c00 comp rx=0 tx=0).stop 2024-06-03T02:17:12.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.455+0000 7f085b200700 1 -- 172.21.15.79:0/3356569091 shutdown_connections 2024-06-03T02:17:12.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.455+0000 7f085b200700 1 --2- 172.21.15.79:0/3356569091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0854109460 0x7f0854115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:12.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.455+0000 7f085b200700 1 --2- 172.21.15.79:0/3356569091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f085410eaa0 0x7f085410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:12.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.455+0000 7f085b200700 1 --2- 172.21.15.79:0/3356569091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 0x7f0854108f20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:12.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.455+0000 7f085b200700 1 -- 172.21.15.79:0/3356569091 >> 172.21.15.79:0/3356569091 conn(0x7f0854075c60 msgr2=0x7f08541072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.455+0000 7f085b200700 1 -- 172.21.15.79:0/3356569091 shutdown_connections 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.455+0000 7f085b200700 1 -- 172.21.15.79:0/3356569091 wait complete. 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.456+0000 7f085b200700 1 Processor -- start 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.456+0000 7f085b200700 1 -- start start 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f085b200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 0x7f085419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f085b200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0854109460 0x7f085419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f085b200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f085410eaa0 0x7f08541a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f085b200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0854118df0 con 0x7f085410eaa0 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f085b200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0854118c70 con 0x7f0854108aa0 2024-06-03T02:17:12.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f085b200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0854118f70 con 0x7f0854109460 2024-06-03T02:17:12.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0853e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 0x7f085419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:12.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0853e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 0x7f085419f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49718/0 (socket says 172.21.15.79:49718) 2024-06-03T02:17:12.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0853e00700 1 -- 172.21.15.79:0/875488154 learned_addr learned my addr 172.21.15.79:0/875488154 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:12.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0859200700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f085410eaa0 0x7f08541a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:12.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0853400700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0854109460 0x7f085419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:12.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0853e00700 1 -- 172.21.15.79:0/875488154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0854109460 msgr2=0x7f085419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0853e00700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0854109460 0x7f085419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0853e00700 1 -- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f085410eaa0 msgr2=0x7f08541a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0853e00700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f085410eaa0 0x7f08541a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.457+0000 7f0853e00700 1 -- 172.21.15.79:0/875488154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08541a4250 con 0x7f0854108aa0 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.458+0000 7f0853e00700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 0x7f085419f240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f084400ec10 tx=0x7f084400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.458+0000 7f0850c00700 1 -- 172.21.15.79:0/875488154 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084400cc90 con 0x7f0854108aa0 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.458+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08541a4540 con 0x7f0854108aa0 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.458+0000 7f0850c00700 1 -- 172.21.15.79:0/875488154 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084400ce30 con 0x7f0854108aa0 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.458+0000 7f0850c00700 1 -- 172.21.15.79:0/875488154 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844010690 con 0x7f0854108aa0 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.458+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08541abdc0 con 0x7f0854108aa0 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.459+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0854069180 con 0x7f0854108aa0 2024-06-03T02:17:12.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.461+0000 7f0850c00700 1 -- 172.21.15.79:0/875488154 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0844003680 con 0x7f0854108aa0 2024-06-03T02:17:12.832 INFO:teuthology.orchestra.run.smithi079.stdout:146028888157 2024-06-03T02:17:12.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.462+0000 7f0850c00700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08380a2cb0 0x7f08380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:12.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.462+0000 7f0850c00700 1 -- 172.21.15.79:0/875488154 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f08440cf8c0 con 0x7f0854108aa0 2024-06-03T02:17:12.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.463+0000 7f0853400700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08380a2cb0 0x7f08380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:12.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.463+0000 7f0853400700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08380a2cb0 0x7f08380a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08400098c0 tx=0x7f0840019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:12.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.467+0000 7f0850c00700 1 -- 172.21.15.79:0/875488154 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f084409c2b0 con 0x7f0854108aa0 2024-06-03T02:17:12.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.698+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f085402dca0 con 0x7f0854108aa0 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.699+0000 7f0850c00700 1 -- 172.21.15.79:0/875488154 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f084409bbb0 con 0x7f0854108aa0 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.702+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08380a2cb0 msgr2=0x7f08380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.702+0000 7f085b200700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08380a2cb0 0x7f08380a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08400098c0 tx=0x7f0840019040 comp rx=0 tx=0).stop 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.702+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 msgr2=0x7f085419f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.702+0000 7f085b200700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 0x7f085419f240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f084400ec10 tx=0x7f084400ef20 comp rx=0 tx=0).stop 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.703+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 shutdown_connections 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.703+0000 7f085b200700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08380a2cb0 0x7f08380a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.703+0000 7f085b200700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f085410eaa0 0x7f08541a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.703+0000 7f085b200700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854108aa0 0x7f085419f240 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.703+0000 7f085b200700 1 --2- 172.21.15.79:0/875488154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0854109460 0x7f085419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.703+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 >> 172.21.15.79:0/875488154 conn(0x7f0854075c60 msgr2=0x7f08540761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.703+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 shutdown_connections 2024-06-03T02:17:12.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:12.703+0000 7f085b200700 1 -- 172.21.15.79:0/875488154 wait complete. 2024-06-03T02:17:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:13 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/875488154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T02:17:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:17:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:17:13.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:13.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:13 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/875488154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T02:17:13.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:17:13.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:17:13.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:13 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/875488154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T02:17:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:17:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:17:13.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:17:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.163+0000 7fda75800700 1 -- 172.21.15.79:0/4100755639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70107840 msgr2=0x7fda7010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.163+0000 7fda75800700 1 --2- 172.21.15.79:0/4100755639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70107840 0x7fda7010f3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda6800b0d0 tx=0x7fda6800b3e0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.164+0000 7fda75800700 1 -- 172.21.15.79:0/4100755639 shutdown_connections 2024-06-03T02:17:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.164+0000 7fda75800700 1 --2- 172.21.15.79:0/4100755639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70107840 0x7fda7010f3c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.164+0000 7fda75800700 1 --2- 172.21.15.79:0/4100755639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda70102740 0x7fda70102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.164+0000 7fda75800700 1 --2- 172.21.15.79:0/4100755639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda701030f0 0x7fda70107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.164+0000 7fda75800700 1 -- 172.21.15.79:0/4100755639 >> 172.21.15.79:0/4100755639 conn(0x7fda700795f0 msgr2=0x7fda70079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.164+0000 7fda75800700 1 -- 172.21.15.79:0/4100755639 shutdown_connections 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.164+0000 7fda75800700 1 -- 172.21.15.79:0/4100755639 wait complete. 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.165+0000 7fda75800700 1 Processor -- start 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.165+0000 7fda75800700 1 -- start start 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.165+0000 7fda75800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70102740 0x7fda7019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.165+0000 7fda75800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda701030f0 0x7fda7019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.165+0000 7fda75800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda70107840 0x7fda7019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.165+0000 7fda75800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fda701127f0 con 0x7fda70102740 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.165+0000 7fda75800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda70112670 con 0x7fda70107840 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.165+0000 7fda75800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda70112970 con 0x7fda701030f0 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.165+0000 7fda6ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70102740 0x7fda7019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:13.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda67e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda701030f0 0x7fda7019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda6ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70102740 0x7fda7019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60864/0 (socket says 172.21.15.79:60864) 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda6ea00700 1 -- 172.21.15.79:0/4071202245 learned_addr learned my addr 172.21.15.79:0/4071202245 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda6f400700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda70107840 0x7fda7019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda6ea00700 1 -- 172.21.15.79:0/4071202245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda701030f0 msgr2=0x7fda7019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda6ea00700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda701030f0 0x7fda7019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda6ea00700 1 -- 172.21.15.79:0/4071202245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda70107840 msgr2=0x7fda7019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda6ea00700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda70107840 0x7fda7019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda6ea00700 1 -- 172.21.15.79:0/4071202245 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7019fdb0 con 0x7fda70102740 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda6ea00700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70102740 0x7fda7019ada0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fda6000e9c0 tx=0x7fda6000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.166+0000 7fda67400700 1 -- 172.21.15.79:0/4071202245 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6000cbd0 con 0x7fda70102740 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.167+0000 7fda67400700 1 -- 172.21.15.79:0/4071202245 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6000ee60 con 0x7fda70102740 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.167+0000 7fda67400700 1 -- 172.21.15.79:0/4071202245 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60005630 con 0x7fda70102740 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.167+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda701a00a0 con 0x7fda70102740 2024-06-03T02:17:13.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.167+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda701a7920 con 0x7fda70102740 2024-06-03T02:17:13.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.168+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda70069180 con 0x7fda70102740 2024-06-03T02:17:13.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.170+0000 7fda67400700 1 -- 172.21.15.79:0/4071202245 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fda60004750 con 0x7fda70102740 2024-06-03T02:17:13.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.172+0000 7fda67400700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fda540a2cb0 0x7fda540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:13.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.172+0000 7fda67400700 1 -- 172.21.15.79:0/4071202245 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fda600cf880 con 0x7fda70102740 2024-06-03T02:17:13.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.172+0000 7fda67e00700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fda540a2cb0 0x7fda540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:13.730 INFO:teuthology.orchestra.run.smithi079.stdout:296352743488 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.173+0000 7fda67e00700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fda540a2cb0 0x7fda540a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda70076720 tx=0x7fda58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.176+0000 7fda67400700 1 -- 172.21.15.79:0/4071202245 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fda6009c270 con 0x7fda70102740 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.412+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fda701a7af0 con 0x7fda70102740 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.412+0000 7fda67400700 1 -- 172.21.15.79:0/4071202245 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fda6009bb70 con 0x7fda70102740 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.415+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fda540a2cb0 msgr2=0x7fda540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.415+0000 7fda75800700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fda540a2cb0 0x7fda540a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda70076720 tx=0x7fda58009260 comp rx=0 tx=0).stop 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.415+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70102740 msgr2=0x7fda7019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.415+0000 7fda75800700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70102740 0x7fda7019ada0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fda6000e9c0 tx=0x7fda6000ecd0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.416+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 shutdown_connections 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.416+0000 7fda75800700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda70107840 0x7fda7019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.416+0000 7fda75800700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fda540a2cb0 0x7fda540a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.416+0000 7fda75800700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda70102740 0x7fda7019ada0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.416+0000 7fda75800700 1 --2- 172.21.15.79:0/4071202245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda701030f0 0x7fda7019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:13.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.416+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 >> 172.21.15.79:0/4071202245 conn(0x7fda700795f0 msgr2=0x7fda70101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:13.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.417+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 shutdown_connections 2024-06-03T02:17:13.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:08.417+0000 7fda75800700 1 -- 172.21.15.79:0/4071202245 wait complete. 2024-06-03T02:17:14.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:14 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3961791091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-03T02:17:14.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:14 smithi079 ceph-mon[31819]: pgmap v432: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:14.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:14 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3961791091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-03T02:17:14.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:14 smithi123 ceph-mon[36640]: pgmap v432: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:14.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:14 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3961791091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-03T02:17:14.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:14 smithi135 ceph-mon[36653]: pgmap v432: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:15.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.804+0000 7f5254c00700 1 -- 172.21.15.79:0/1322275807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5250101590 msgr2=0x7f5250101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.804+0000 7f5254c00700 1 --2- 172.21.15.79:0/1322275807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5250101590 0x7f5250101970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f52400098f0 tx=0x7f5240009c00 comp rx=0 tx=0).stop 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.804+0000 7f5254c00700 1 -- 172.21.15.79:0/1322275807 shutdown_connections 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.804+0000 7f5254c00700 1 --2- 172.21.15.79:0/1322275807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5250106670 0x7f525010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.804+0000 7f5254c00700 1 --2- 172.21.15.79:0/1322275807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5250101590 0x7f5250101970 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.804+0000 7f5254c00700 1 --2- 172.21.15.79:0/1322275807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5250101f40 0x7f5250105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.804+0000 7f5254c00700 1 -- 172.21.15.79:0/1322275807 >> 172.21.15.79:0/1322275807 conn(0x7f52500fce40 msgr2=0x7f52500ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.804+0000 7f5254c00700 1 -- 172.21.15.79:0/1322275807 shutdown_connections 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.804+0000 7f5254c00700 1 -- 172.21.15.79:0/1322275807 wait complete. 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.805+0000 7f5254c00700 1 Processor -- start 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f5254c00700 1 -- start start 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f5254c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5250101590 0x7f525019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f5254c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5250101f40 0x7f525019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:15.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f5254c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5250106670 0x7f525019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f5254c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52501116c0 con 0x7f5250101590 2024-06-03T02:17:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f5254c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5250111540 con 0x7f5250101f40 2024-06-03T02:17:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f5254c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5250111840 con 0x7f5250106670 2024-06-03T02:17:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f524f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5250101590 0x7f525019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f524f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5250101590 0x7f525019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60902/0 (socket says 172.21.15.79:60902) 2024-06-03T02:17:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f524f400700 1 -- 172.21.15.79:0/3961791091 learned_addr learned my addr 172.21.15.79:0/3961791091 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.806+0000 7f524ea00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5250101f40 0x7f525019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f524fe00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5250106670 0x7f525019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:15.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f524f400700 1 -- 172.21.15.79:0/3961791091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5250106670 msgr2=0x7f525019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f524f400700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5250106670 0x7f525019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f524f400700 1 -- 172.21.15.79:0/3961791091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5250101f40 msgr2=0x7f525019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f524f400700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5250101f40 0x7f525019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f524f400700 1 -- 172.21.15.79:0/3961791091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525019fe70 con 0x7f5250101590 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f524fe00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5250106670 0x7f525019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f524f400700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5250101590 0x7f525019ae60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5240009fd0 tx=0x7f52400094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f5247e00700 1 -- 172.21.15.79:0/3961791091 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240025070 con 0x7f5250101590 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f5247e00700 1 -- 172.21.15.79:0/3961791091 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f524000ba40 con 0x7f5250101590 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52501a0100 con 0x7f5250101590 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f5247e00700 1 -- 172.21.15.79:0/3961791091 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240004c00 con 0x7f5250101590 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.807+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52501a79e0 con 0x7f5250101590 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.809+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5250069180 con 0x7f5250101590 2024-06-03T02:17:15.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.810+0000 7f5247e00700 1 -- 172.21.15.79:0/3961791091 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5240028070 con 0x7f5250101590 2024-06-03T02:17:15.194 INFO:teuthology.orchestra.run.smithi079.stdout:352187318327 2024-06-03T02:17:15.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.812+0000 7f5247e00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52340a2d30 0x7f52340a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:15.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.812+0000 7f5247e00700 1 -- 172.21.15.79:0/3961791091 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f52400cfd20 con 0x7f5250101590 2024-06-03T02:17:15.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.812+0000 7f524ea00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52340a2d30 0x7f52340a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:15.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.813+0000 7f524ea00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52340a2d30 0x7f52340a51f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5250107fd0 tx=0x7f523c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:15.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:13.817+0000 7f5247e00700 1 -- 172.21.15.79:0/3961791091 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5240013030 con 0x7f5250101590 2024-06-03T02:17:15.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.047+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f525002dca0 con 0x7f5250101590 2024-06-03T02:17:15.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.047+0000 7f5247e00700 1 -- 172.21.15.79:0/3961791091 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f524009c7a0 con 0x7f5250101590 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.050+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52340a2d30 msgr2=0x7f52340a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.050+0000 7f5254c00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52340a2d30 0x7f52340a51f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5250107fd0 tx=0x7f523c009310 comp rx=0 tx=0).stop 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.050+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5250101590 msgr2=0x7f525019ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.050+0000 7f5254c00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5250101590 0x7f525019ae60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5240009fd0 tx=0x7f52400094b0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.051+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 shutdown_connections 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.051+0000 7f5254c00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5250106670 0x7f525019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.051+0000 7f5254c00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52340a2d30 0x7f52340a51f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.051+0000 7f5254c00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5250101590 0x7f525019ae60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.051+0000 7f5254c00700 1 --2- 172.21.15.79:0/3961791091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5250101f40 0x7f525019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.051+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 >> 172.21.15.79:0/3961791091 conn(0x7f52500fce40 msgr2=0x7f52500fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.051+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 shutdown_connections 2024-06-03T02:17:15.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:14.051+0000 7f5254c00700 1 -- 172.21.15.79:0/3961791091 wait complete. 2024-06-03T02:17:16.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:16 smithi079 ceph-mon[31819]: pgmap v433: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:16 smithi135 ceph-mon[36653]: pgmap v433: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:16.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:16 smithi123 ceph-mon[36640]: pgmap v433: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:16.943 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743483 got 296352743488 for osd.9 2024-06-03T02:17:16.943 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:18.002 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:17 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2158570094' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-03T02:17:18.071 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318323 got 352187318327 for osd.11 2024-06-03T02:17:18.072 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:18.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:17 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2158570094' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-03T02:17:18.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:17 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2158570094' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-03T02:17:18.831 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:18 smithi079 ceph-mon[31819]: pgmap v434: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:19.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:18 smithi135 ceph-mon[36653]: pgmap v434: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:19.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:18 smithi123 ceph-mon[36640]: pgmap v434: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:19.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.293+0000 7feab1200700 1 -- 172.21.15.79:0/2269938860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaac1024e0 msgr2=0x7feaac102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.293+0000 7feab1200700 1 --2- 172.21.15.79:0/2269938860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaac1024e0 0x7feaac102960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fea98009990 tx=0x7fea98009ca0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.293+0000 7feab1200700 1 -- 172.21.15.79:0/2269938860 shutdown_connections 2024-06-03T02:17:19.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.293+0000 7feab1200700 1 --2- 172.21.15.79:0/2269938860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feaac102ea0 0x7feaac10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.293+0000 7feab1200700 1 --2- 172.21.15.79:0/2269938860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaac1085f0 0x7feaac1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.293+0000 7feab1200700 1 --2- 172.21.15.79:0/2269938860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaac1024e0 0x7feaac102960 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.293+0000 7feab1200700 1 -- 172.21.15.79:0/2269938860 >> 172.21.15.79:0/2269938860 conn(0x7feaac079640 msgr2=0x7feaac079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:19.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.293+0000 7feab1200700 1 -- 172.21.15.79:0/2269938860 shutdown_connections 2024-06-03T02:17:19.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.293+0000 7feab1200700 1 -- 172.21.15.79:0/2269938860 wait complete. 2024-06-03T02:17:19.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.294+0000 7feab1200700 1 Processor -- start 2024-06-03T02:17:19.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.294+0000 7feab1200700 1 -- start start 2024-06-03T02:17:19.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.294+0000 7feab1200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feaac1024e0 0x7feaac19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.294+0000 7feab1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaac102ea0 0x7feaac19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.294+0000 7feab1200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaac1085f0 0x7feaac19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.294+0000 7feab1200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feaac112800 con 0x7feaac1024e0 2024-06-03T02:17:19.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.294+0000 7feab1200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feaac112680 con 0x7feaac1085f0 2024-06-03T02:17:19.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.294+0000 7feab1200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feaac112980 con 0x7feaac102ea0 2024-06-03T02:17:19.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaa9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feaac1024e0 0x7feaac19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaa9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feaac1024e0 0x7feaac19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60556/0 (socket says 172.21.15.79:60556) 2024-06-03T02:17:19.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaa9e00700 1 -- 172.21.15.79:0/3400843733 learned_addr learned my addr 172.21.15.79:0/3400843733 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:19.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaa9400700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaac102ea0 0x7feaac19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaaa800700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaac1085f0 0x7feaac19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaa9e00700 1 -- 172.21.15.79:0/3400843733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaac102ea0 msgr2=0x7feaac19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaa9e00700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaac102ea0 0x7feaac19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaa9e00700 1 -- 172.21.15.79:0/3400843733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaac1085f0 msgr2=0x7feaac19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaa9e00700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaac1085f0 0x7feaac19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.295+0000 7feaa9e00700 1 -- 172.21.15.79:0/3400843733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac19fe80 con 0x7feaac1024e0 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.296+0000 7feaa9e00700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feaac1024e0 0x7feaac19ae70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9c0 tx=0x7fea9c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.296+0000 7feaa2a00700 1 -- 172.21.15.79:0/3400843733 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00cbd0 con 0x7feaac1024e0 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.296+0000 7feaa2a00700 1 -- 172.21.15.79:0/3400843733 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c00ee60 con 0x7feaac1024e0 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.296+0000 7feaa2a00700 1 -- 172.21.15.79:0/3400843733 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c005630 con 0x7feaac1024e0 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.296+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaac1a0170 con 0x7feaac1024e0 2024-06-03T02:17:19.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.296+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaac1a79f0 con 0x7feaac1024e0 2024-06-03T02:17:19.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.298+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaac069180 con 0x7feaac1024e0 2024-06-03T02:17:19.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.300+0000 7feaa2a00700 1 -- 172.21.15.79:0/3400843733 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fea9c004750 con 0x7feaac1024e0 2024-06-03T02:17:19.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.301+0000 7feaa2a00700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea900a2cb0 0x7fea900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.301+0000 7feaa2a00700 1 -- 172.21.15.79:0/3400843733 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fea9c0cf880 con 0x7feaac1024e0 2024-06-03T02:17:19.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.301+0000 7feaa9400700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea900a2cb0 0x7fea900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.302+0000 7feaa9400700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea900a2cb0 0x7fea900a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feaac079f80 tx=0x7fea9800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:19.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.306+0000 7feaa2a00700 1 -- 172.21.15.79:0/3400843733 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea9c09c270 con 0x7feaac1024e0 2024-06-03T02:17:19.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.181+0000 7f1781800700 1 -- 172.21.15.79:0/3379929246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f177c102740 msgr2=0x7f177c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.181+0000 7f1781800700 1 --2- 172.21.15.79:0/3379929246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f177c102740 0x7f177c102b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f176c009920 tx=0x7f176c009c30 comp rx=0 tx=0).stop 2024-06-03T02:17:19.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.181+0000 7f1781800700 1 -- 172.21.15.79:0/3379929246 shutdown_connections 2024-06-03T02:17:19.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.181+0000 7f1781800700 1 --2- 172.21.15.79:0/3379929246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177c107840 0x7f177c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.181+0000 7f1781800700 1 --2- 172.21.15.79:0/3379929246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f177c102740 0x7f177c102b20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.181+0000 7f1781800700 1 --2- 172.21.15.79:0/3379929246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f177c1030f0 0x7f177c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.181+0000 7f1781800700 1 -- 172.21.15.79:0/3379929246 >> 172.21.15.79:0/3379929246 conn(0x7f177c0795f0 msgr2=0x7f177c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.181+0000 7f1781800700 1 -- 172.21.15.79:0/3379929246 shutdown_connections 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.181+0000 7f1781800700 1 -- 172.21.15.79:0/3379929246 wait complete. 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.182+0000 7f1781800700 1 Processor -- start 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.182+0000 7f1781800700 1 -- start start 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.182+0000 7f1781800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f177c102740 0x7f177c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f1781800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177c1030f0 0x7f177c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f1781800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f177c107840 0x7f177c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f1781800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1127d0 con 0x7f177c102740 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f1781800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f177c112650 con 0x7f177c107840 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f1781800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f177c112950 con 0x7f177c1030f0 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f177c102740 0x7f177c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177c1030f0 0x7f177c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f177c102740 0x7f177c19ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60532/0 (socket says 172.21.15.79:60532) 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177aa00700 1 -- 172.21.15.79:0/3454414660 learned_addr learned my addr 172.21.15.79:0/3454414660 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177b400700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f177c107840 0x7f177c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177a000700 1 -- 172.21.15.79:0/3454414660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f177c107840 msgr2=0x7f177c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177a000700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f177c107840 0x7f177c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177a000700 1 -- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f177c102740 msgr2=0x7f177c19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177a000700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f177c102740 0x7f177c19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177a000700 1 -- 172.21.15.79:0/3454414660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c1993f0 con 0x7f177c1030f0 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177aa00700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f177c102740 0x7f177c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.183+0000 7f177b400700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f177c107840 0x7f177c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.184+0000 7f177a000700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177c1030f0 0x7f177c19ef80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f176800eb90 tx=0x7f176800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.184+0000 7f1773400700 1 -- 172.21.15.79:0/3454414660 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176800cc90 con 0x7f177c1030f0 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.184+0000 7f1773400700 1 -- 172.21.15.79:0/3454414660 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f176800ce30 con 0x7f177c1030f0 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.184+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c1996e0 con 0x7f177c1030f0 2024-06-03T02:17:19.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.184+0000 7f1773400700 1 -- 172.21.15.79:0/3454414660 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768010430 con 0x7f177c1030f0 2024-06-03T02:17:19.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.184+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c1a59c0 con 0x7f177c1030f0 2024-06-03T02:17:19.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.185+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c10aa80 con 0x7f177c1030f0 2024-06-03T02:17:19.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.188+0000 7f1773400700 1 -- 172.21.15.79:0/3454414660 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1768003680 con 0x7f177c1030f0 2024-06-03T02:17:19.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.189+0000 7f1773400700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f17600a2cb0 0x7f17600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.190+0000 7f1773400700 1 -- 172.21.15.79:0/3454414660 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f17680cf970 con 0x7f177c1030f0 2024-06-03T02:17:19.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.190+0000 7f177aa00700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f17600a2cb0 0x7f17600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.190+0000 7f177aa00700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f17600a2cb0 0x7f17600a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f176c009680 tx=0x7f176c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:19.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.191+0000 7f1773400700 1 -- 172.21.15.79:0/3454414660 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f176801f340 con 0x7f177c1030f0 2024-06-03T02:17:19.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.432+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f177c199fd0 con 0x7f177c1030f0 2024-06-03T02:17:19.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.433+0000 7f1773400700 1 -- 172.21.15.79:0/3454414660 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f176809c3f0 con 0x7f177c1030f0 2024-06-03T02:17:19.433 INFO:teuthology.orchestra.run.smithi079.stdout:171798691930 2024-06-03T02:17:19.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.435+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f17600a2cb0 msgr2=0x7f17600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.435+0000 7f1781800700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f17600a2cb0 0x7f17600a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f176c009680 tx=0x7f176c019040 comp rx=0 tx=0).stop 2024-06-03T02:17:19.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.435+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177c1030f0 msgr2=0x7f177c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.435+0000 7f1781800700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177c1030f0 0x7f177c19ef80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f176800eb90 tx=0x7f176800eea0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.436+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 shutdown_connections 2024-06-03T02:17:19.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.436+0000 7f1781800700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f177c107840 0x7f177c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.436+0000 7f1781800700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f17600a2cb0 0x7f17600a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.436+0000 7f1781800700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f177c102740 0x7f177c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.436+0000 7f1781800700 1 --2- 172.21.15.79:0/3454414660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177c1030f0 0x7f177c19ef80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.436+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 >> 172.21.15.79:0/3454414660 conn(0x7f177c0795f0 msgr2=0x7f177c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:19.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.436+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 shutdown_connections 2024-06-03T02:17:19.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.436+0000 7f1781800700 1 -- 172.21.15.79:0/3454414660 wait complete. 2024-06-03T02:17:19.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.544+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7feaac19c1d0 con 0x7feaac1024e0 2024-06-03T02:17:19.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.545+0000 7feaa2a00700 1 -- 172.21.15.79:0/3400843733 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fea9c09bb70 con 0x7feaac1024e0 2024-06-03T02:17:19.546 INFO:teuthology.orchestra.run.smithi079.stdout:77309411435 2024-06-03T02:17:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea900a2cb0 msgr2=0x7fea900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea900a2cb0 0x7fea900a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feaac079f80 tx=0x7fea9800b260 comp rx=0 tx=0).stop 2024-06-03T02:17:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feaac1024e0 msgr2=0x7feaac19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feaac1024e0 0x7feaac19ae70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9c0 tx=0x7fea9c00ecd0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 shutdown_connections 2024-06-03T02:17:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaac1085f0 0x7feaac19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea900a2cb0 0x7fea900a5170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feaac1024e0 0x7feaac19ae70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 --2- 172.21.15.79:0/3400843733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaac102ea0 0x7feaac19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.549+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 >> 172.21.15.79:0/3400843733 conn(0x7feaac079640 msgr2=0x7feaac10d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:19.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.550+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 shutdown_connections 2024-06-03T02:17:19.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.550+0000 7feab1200700 1 -- 172.21.15.79:0/3400843733 wait complete. 2024-06-03T02:17:19.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.831+0000 7fbd9ec00700 1 -- 172.21.15.79:0/2896719734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 msgr2=0x7fbd98102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.831+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/2896719734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 0x7fbd98102b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009990 tx=0x7fbd88009ca0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.831+0000 7fbd9ec00700 1 -- 172.21.15.79:0/2896719734 shutdown_connections 2024-06-03T02:17:19.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.831+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/2896719734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd98107840 0x7fbd9810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.831+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/2896719734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 0x7fbd98102b20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.831+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/2896719734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd981030f0 0x7fbd98107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.831+0000 7fbd9ec00700 1 -- 172.21.15.79:0/2896719734 >> 172.21.15.79:0/2896719734 conn(0x7fbd980795f0 msgr2=0x7fbd98079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:19.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.831+0000 7fbd9ec00700 1 -- 172.21.15.79:0/2896719734 shutdown_connections 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.831+0000 7fbd9ec00700 1 -- 172.21.15.79:0/2896719734 wait complete. 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.832+0000 7fbd9ec00700 1 Processor -- start 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.832+0000 7fbd9ec00700 1 -- start start 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd9ec00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 0x7fbd9819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd9ec00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd981030f0 0x7fbd9819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd9ec00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd98107840 0x7fbd9819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd9ec00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981127b0 con 0x7fbd981030f0 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd9ec00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98112630 con 0x7fbd98107840 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd9ec00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98112930 con 0x7fbd98102740 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd97e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 0x7fbd9819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd97e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 0x7fbd9819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54438/0 (socket says 172.21.15.79:54438) 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd97e00700 1 -- 172.21.15.79:0/4022104524 learned_addr learned my addr 172.21.15.79:0/4022104524 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd8fe00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd981030f0 0x7fbd9819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd9cc00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd98107840 0x7fbd9819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd97e00700 1 -- 172.21.15.79:0/4022104524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd98107840 msgr2=0x7fbd9819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd97e00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd98107840 0x7fbd9819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd97e00700 1 -- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd981030f0 msgr2=0x7fbd9819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd97e00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd981030f0 0x7fbd9819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.833+0000 7fbd97e00700 1 -- 172.21.15.79:0/4022104524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9819fe20 con 0x7fbd98102740 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.834+0000 7fbd97e00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 0x7fbd9819ae10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbd88000c00 tx=0x7fbd88005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.834+0000 7fbd95600700 1 -- 172.21.15.79:0/4022104524 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88027070 con 0x7fbd98102740 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.834+0000 7fbd95600700 1 -- 172.21.15.79:0/4022104524 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd880056b0 con 0x7fbd98102740 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.834+0000 7fbd95600700 1 -- 172.21.15.79:0/4022104524 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd880173f0 con 0x7fbd98102740 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.834+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd981a00b0 con 0x7fbd98102740 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.834+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981a7990 con 0x7fbd98102740 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.835+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd98103bd0 con 0x7fbd98102740 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.837+0000 7fbd95600700 1 -- 172.21.15.79:0/4022104524 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbd8802a070 con 0x7fbd98102740 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.838+0000 7fbd95600700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbd7c0a2cb0 0x7fbd7c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.839+0000 7fbd95600700 1 -- 172.21.15.79:0/4022104524 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fbd880d0220 con 0x7fbd98102740 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.839+0000 7fbd8fe00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbd7c0a2cb0 0x7fbd7c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.839+0000 7fbd8fe00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbd7c0a2cb0 0x7fbd7c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbd981021e0 tx=0x7fbd80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:18.843+0000 7fbd95600700 1 -- 172.21.15.79:0/4022104524 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd88013070 con 0x7fbd98102740 2024-06-03T02:17:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.075+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fbd9819c170 con 0x7fbd98102740 2024-06-03T02:17:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.076+0000 7fbd95600700 1 -- 172.21.15.79:0/4022104524 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbd8809cca0 con 0x7fbd98102740 2024-06-03T02:17:19.706 INFO:teuthology.orchestra.run.smithi079.stdout:317827579964 2024-06-03T02:17:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbd7c0a2cb0 msgr2=0x7fbd7c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbd7c0a2cb0 0x7fbd7c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbd981021e0 tx=0x7fbd80009260 comp rx=0 tx=0).stop 2024-06-03T02:17:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 msgr2=0x7fbd9819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 0x7fbd9819ae10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbd88000c00 tx=0x7fbd88005590 comp rx=0 tx=0).stop 2024-06-03T02:17:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 shutdown_connections 2024-06-03T02:17:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd98107840 0x7fbd9819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd98102740 0x7fbd9819ae10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fbd7c0a2cb0 0x7fbd7c0a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 --2- 172.21.15.79:0/4022104524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd981030f0 0x7fbd9819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 >> 172.21.15.79:0/4022104524 conn(0x7fbd980795f0 msgr2=0x7fbd98100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 shutdown_connections 2024-06-03T02:17:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:19.079+0000 7fbd9ec00700 1 -- 172.21.15.79:0/4022104524 wait complete. 2024-06-03T02:17:20.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:19 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4022104524' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-03T02:17:20.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:19 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3454414660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T02:17:20.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:19 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3400843733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T02:17:20.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:19 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4022104524' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-03T02:17:20.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:19 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3454414660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T02:17:20.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:19 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3400843733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T02:17:20.124 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:19 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4022104524' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-03T02:17:20.124 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:19 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3454414660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T02:17:20.124 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:19 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3400843733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T02:17:21.052 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:20 smithi079 ceph-mon[31819]: pgmap v435: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:17:21.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:20 smithi135 ceph-mon[36653]: pgmap v435: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:17:21.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:20 smithi123 ceph-mon[36640]: pgmap v435: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:17:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.486+0000 7f05e7e00700 1 -- 172.21.15.79:0/2867216241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05e8074830 msgr2=0x7f05e8074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.486+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2867216241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05e8074830 0x7f05e8074c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009990 tx=0x7f05d8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.486+0000 7f05e7e00700 1 -- 172.21.15.79:0/2867216241 shutdown_connections 2024-06-03T02:17:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.486+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2867216241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e81133d0 0x7f05e81157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.486+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2867216241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05e8074830 0x7f05e8074c10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.486+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2867216241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05e8075150 0x7f05e8112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.486+0000 7f05e7e00700 1 -- 172.21.15.79:0/2867216241 >> 172.21.15.79:0/2867216241 conn(0x7f05e80fdf10 msgr2=0x7f05e8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.486+0000 7f05e7e00700 1 -- 172.21.15.79:0/2867216241 shutdown_connections 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.486+0000 7f05e7e00700 1 -- 172.21.15.79:0/2867216241 wait complete. 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.487+0000 7f05e7e00700 1 Processor -- start 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.488+0000 7f05e7e00700 1 -- start start 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.488+0000 7f05e7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e8074830 0x7f05e819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.488+0000 7f05e7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05e8075150 0x7f05e819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.488+0000 7f05e7e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05e81133d0 0x7f05e81a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.488+0000 7f05e7e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8118d40 con 0x7f05e81133d0 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.488+0000 7f05e7e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8118bc0 con 0x7f05e8074830 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.488+0000 7f05e7e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8118ec0 con 0x7f05e8075150 2024-06-03T02:17:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e8074830 0x7f05e819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e8074830 0x7f05e819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49326/0 (socket says 172.21.15.79:49326) 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6a00700 1 -- 172.21.15.79:0/2158570094 learned_addr learned my addr 172.21.15.79:0/2158570094 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6000700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05e8075150 0x7f05e819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e7400700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05e81133d0 0x7f05e81a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6a00700 1 -- 172.21.15.79:0/2158570094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05e8075150 msgr2=0x7f05e819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6a00700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05e8075150 0x7f05e819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6a00700 1 -- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05e81133d0 msgr2=0x7f05e81a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6a00700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05e81133d0 0x7f05e81a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6a00700 1 -- 172.21.15.79:0/2158570094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e81a41c0 con 0x7f05e8074830 2024-06-03T02:17:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.489+0000 7f05e6a00700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e8074830 0x7f05e819f1b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05d8005910 tx=0x7f05d800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:21.470 INFO:teuthology.orchestra.run.smithi079.stdout:206158430292 2024-06-03T02:17:21.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.490+0000 7f05df400700 1 -- 172.21.15.79:0/2158570094 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8026070 con 0x7f05e8074830 2024-06-03T02:17:21.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.490+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e81a4450 con 0x7f05e8074830 2024-06-03T02:17:21.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.490+0000 7f05df400700 1 -- 172.21.15.79:0/2158570094 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d8003d70 con 0x7f05e8074830 2024-06-03T02:17:21.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.490+0000 7f05df400700 1 -- 172.21.15.79:0/2158570094 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8004a90 con 0x7f05e8074830 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.490+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e81abd30 con 0x7f05e8074830 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.491+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e8069180 con 0x7f05e8074830 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.493+0000 7f05df400700 1 -- 172.21.15.79:0/2158570094 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f05d8029070 con 0x7f05e8074830 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.494+0000 7f05df400700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f05cc0a2d30 0x7f05cc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.495+0000 7f05df400700 1 -- 172.21.15.79:0/2158570094 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f05d80cff20 con 0x7f05e8074830 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.495+0000 7f05e6000700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f05cc0a2d30 0x7f05cc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.495+0000 7f05e6000700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f05cc0a2d30 0x7f05cc0a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f05e81061c0 tx=0x7f05d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.499+0000 7f05df400700 1 -- 172.21.15.79:0/2158570094 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f05d805d850 con 0x7f05e8074830 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.732+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f05e81a0510 con 0x7f05e8074830 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.733+0000 7f05df400700 1 -- 172.21.15.79:0/2158570094 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f05d8013070 con 0x7f05e8074830 2024-06-03T02:17:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f05cc0a2d30 msgr2=0x7f05cc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f05cc0a2d30 0x7f05cc0a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f05e81061c0 tx=0x7f05d4009260 comp rx=0 tx=0).stop 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e8074830 msgr2=0x7f05e819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e8074830 0x7f05e819f1b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05d8005910 tx=0x7f05d800bab0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 shutdown_connections 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f05cc0a2d30 0x7f05cc0a51f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05e81133d0 0x7f05e81a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e8074830 0x7f05e819f1b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 --2- 172.21.15.79:0/2158570094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05e8075150 0x7f05e819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 >> 172.21.15.79:0/2158570094 conn(0x7f05e80fdf10 msgr2=0x7f05e80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 shutdown_connections 2024-06-03T02:17:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:17.736+0000 7f05e7e00700 1 -- 172.21.15.79:0/2158570094 wait complete. 2024-06-03T02:17:22.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:21 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2779814913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T02:17:22.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:21 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2779814913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T02:17:22.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:21 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2779814913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T02:17:23.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:22 smithi079 ceph-mon[31819]: pgmap v436: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:23.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:22 smithi135 ceph-mon[36653]: pgmap v436: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:23.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:22 smithi123 ceph-mon[36640]: pgmap v436: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:23.684 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411429 got 77309411435 for osd.2 2024-06-03T02:17:23.684 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:23.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.667+0000 7f05bee00700 1 -- 172.21.15.79:0/3333330659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05b8108500 msgr2=0x7f05b81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:23.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.667+0000 7f05bee00700 1 --2- 172.21.15.79:0/3333330659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05b8108500 0x7f05b81088e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f05a80098f0 tx=0x7f05a8009c00 comp rx=0 tx=0).stop 2024-06-03T02:17:23.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.667+0000 7f05bee00700 1 -- 172.21.15.79:0/3333330659 shutdown_connections 2024-06-03T02:17:23.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.667+0000 7f05bee00700 1 --2- 172.21.15.79:0/3333330659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05b806ab30 0x7f05b810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:23.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.667+0000 7f05bee00700 1 --2- 172.21.15.79:0/3333330659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05b8108500 0x7f05b81088e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.667+0000 7f05bee00700 1 --2- 172.21.15.79:0/3333330659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b806a170 0x7f05b806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.667+0000 7f05bee00700 1 -- 172.21.15.79:0/3333330659 >> 172.21.15.79:0/3333330659 conn(0x7f05b8078340 msgr2=0x7f05b8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.667+0000 7f05bee00700 1 -- 172.21.15.79:0/3333330659 shutdown_connections 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.667+0000 7f05bee00700 1 -- 172.21.15.79:0/3333330659 wait complete. 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.668+0000 7f05bee00700 1 Processor -- start 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.668+0000 7f05bee00700 1 -- start start 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05bee00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b806a170 0x7f05b819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05bee00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05b806ab30 0x7f05b819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05bee00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05b8108500 0x7f05b819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05bee00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81127d0 con 0x7f05b806ab30 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05bee00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8112650 con 0x7f05b806a170 2024-06-03T02:17:23.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05bee00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8112950 con 0x7f05b8108500 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05afe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05b806ab30 0x7f05b819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05afe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05b806ab30 0x7f05b819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60600/0 (socket says 172.21.15.79:60600) 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05afe00700 1 -- 172.21.15.79:0/2715748400 learned_addr learned my addr 172.21.15.79:0/2715748400 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05b7e00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b806a170 0x7f05b819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.669+0000 7f05bce00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05b8108500 0x7f05b819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.670+0000 7f05b7e00700 1 -- 172.21.15.79:0/2715748400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05b8108500 msgr2=0x7f05b819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.670+0000 7f05b7e00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05b8108500 0x7f05b819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.670+0000 7f05b7e00700 1 -- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05b806ab30 msgr2=0x7f05b819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.670+0000 7f05b7e00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05b806ab30 0x7f05b819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.670+0000 7f05b7e00700 1 -- 172.21.15.79:0/2715748400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b819ff10 con 0x7f05b806a170 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.670+0000 7f05afe00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05b806ab30 0x7f05b819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.670+0000 7f05b7e00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b806a170 0x7f05b819af00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000c00 tx=0x7f05a800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.671+0000 7f05b5600700 1 -- 172.21.15.79:0/2715748400 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8005040 con 0x7f05b806a170 2024-06-03T02:17:23.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.671+0000 7f05b5600700 1 -- 172.21.15.79:0/2715748400 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a80056c0 con 0x7f05b806a170 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.671+0000 7f05b5600700 1 -- 172.21.15.79:0/2715748400 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8003a00 con 0x7f05b806a170 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.671+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b81a01a0 con 0x7f05b806a170 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.671+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b81a7a80 con 0x7f05b806a170 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.672+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b8103bc0 con 0x7f05b806a170 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.675+0000 7f05b5600700 1 -- 172.21.15.79:0/2715748400 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f05a800b560 con 0x7f05b806a170 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.676+0000 7f05b5600700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f059c0a2ca0 0x7f059c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.677+0000 7f05b5600700 1 -- 172.21.15.79:0/2715748400 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f05a80d0510 con 0x7f05b806a170 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.677+0000 7f05afe00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f059c0a2ca0 0x7f059c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.678+0000 7f05b5600700 1 -- 172.21.15.79:0/2715748400 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f05a8013030 con 0x7f05b806a170 2024-06-03T02:17:23.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.678+0000 7f05afe00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f059c0a2ca0 0x7f059c0a5160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05b8103450 tx=0x7f05a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:23.876 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691926 got 171798691930 for osd.5 2024-06-03T02:17:23.877 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:23.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.917+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f05b81a7c50 con 0x7f05b806a170 2024-06-03T02:17:23.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.918+0000 7f05b5600700 1 -- 172.21.15.79:0/2715748400 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f05a809d130 con 0x7f05b806a170 2024-06-03T02:17:23.918 INFO:teuthology.orchestra.run.smithi079.stdout:34359738486 2024-06-03T02:17:23.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f059c0a2ca0 msgr2=0x7f059c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:23.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f059c0a2ca0 0x7f059c0a5160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05b8103450 tx=0x7f05a0009260 comp rx=0 tx=0).stop 2024-06-03T02:17:23.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b806a170 msgr2=0x7f05b819af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:23.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b806a170 0x7f05b819af00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000c00 tx=0x7f05a800fa00 comp rx=0 tx=0).stop 2024-06-03T02:17:23.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 shutdown_connections 2024-06-03T02:17:23.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05b8108500 0x7f05b819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:23.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05b806a170 0x7f05b819af00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:23.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f059c0a2ca0 0x7f059c0a5160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:23.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 --2- 172.21.15.79:0/2715748400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05b806ab30 0x7f05b819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:23.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 >> 172.21.15.79:0/2715748400 conn(0x7f05b8078340 msgr2=0x7f05b810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:23.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 shutdown_connections 2024-06-03T02:17:23.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:23.921+0000 7f05bee00700 1 -- 172.21.15.79:0/2715748400 wait complete. 2024-06-03T02:17:24.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:23 smithi079 ceph-mon[31819]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-03T02:17:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:23 smithi135 ceph-mon[36653]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-03T02:17:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:23 smithi123 ceph-mon[36640]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-03T02:17:24.852 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579960 got 317827579964 for osd.10 2024-06-03T02:17:24.852 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:24.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.521+0000 7efc50a00700 1 -- 172.21.15.79:0/2183249361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c104f80 msgr2=0x7efc4c111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.521+0000 7efc50a00700 1 --2- 172.21.15.79:0/2183249361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c104f80 0x7efc4c111440 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b0d0 tx=0x7efc4400b3e0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.521+0000 7efc50a00700 1 -- 172.21.15.79:0/2183249361 shutdown_connections 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.521+0000 7efc50a00700 1 --2- 172.21.15.79:0/2183249361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c104f80 0x7efc4c111440 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.521+0000 7efc50a00700 1 --2- 172.21.15.79:0/2183249361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c10a670 0x7efc4c10aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.521+0000 7efc50a00700 1 --2- 172.21.15.79:0/2183249361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc4c1045c0 0x7efc4c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.521+0000 7efc50a00700 1 -- 172.21.15.79:0/2183249361 >> 172.21.15.79:0/2183249361 conn(0x7efc4c100070 msgr2=0x7efc4c102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.521+0000 7efc50a00700 1 -- 172.21.15.79:0/2183249361 shutdown_connections 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.522+0000 7efc50a00700 1 -- 172.21.15.79:0/2183249361 wait complete. 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.523+0000 7efc50a00700 1 Processor -- start 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.523+0000 7efc50a00700 1 -- start start 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.523+0000 7efc50a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c1045c0 0x7efc4c1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.523+0000 7efc50a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc4c104f80 0x7efc4c1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.523+0000 7efc50a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c10a670 0x7efc4c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.523+0000 7efc50a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c114970 con 0x7efc4c1045c0 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.523+0000 7efc50a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1147f0 con 0x7efc4c10a670 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.523+0000 7efc50a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c114af0 con 0x7efc4c104f80 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.523+0000 7efc4b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c1045c0 0x7efc4c1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c1045c0 0x7efc4c1a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60580/0 (socket says 172.21.15.79:60580) 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4b400700 1 -- 172.21.15.79:0/2779814913 learned_addr learned my addr 172.21.15.79:0/2779814913 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4aa00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc4c104f80 0x7efc4c1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4be00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c10a670 0x7efc4c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4b400700 1 -- 172.21.15.79:0/2779814913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc4c104f80 msgr2=0x7efc4c1a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4b400700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc4c104f80 0x7efc4c1a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4b400700 1 -- 172.21.15.79:0/2779814913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c10a670 msgr2=0x7efc4c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4b400700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c10a670 0x7efc4c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4b400700 1 -- 172.21.15.79:0/2779814913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c19b250 con 0x7efc4c1045c0 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4aa00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc4c104f80 0x7efc4c1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4be00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c10a670 0x7efc4c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.524+0000 7efc4b400700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c1045c0 0x7efc4c1a08e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00eb30 tx=0x7efc3c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.525+0000 7efc43e00700 1 -- 172.21.15.79:0/2779814913 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c00cbd0 con 0x7efc4c1045c0 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.525+0000 7efc43e00700 1 -- 172.21.15.79:0/2779814913 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c004500 con 0x7efc4c1045c0 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.525+0000 7efc43e00700 1 -- 172.21.15.79:0/2779814913 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c010430 con 0x7efc4c1045c0 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.525+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c19b540 con 0x7efc4c1045c0 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.525+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1a77b0 con 0x7efc4c1045c0 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.526+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c069180 con 0x7efc4c1045c0 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.528+0000 7efc43e00700 1 -- 172.21.15.79:0/2779814913 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efc3c003f00 con 0x7efc4c1045c0 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.529+0000 7efc43e00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efc300a2d30 0x7efc300a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.530+0000 7efc43e00700 1 -- 172.21.15.79:0/2779814913 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7efc3c0cf8c0 con 0x7efc4c1045c0 2024-06-03T02:17:24.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.530+0000 7efc4aa00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efc300a2d30 0x7efc300a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:24.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.530+0000 7efc4aa00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efc300a2d30 0x7efc300a51f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc4c109bf0 tx=0x7efc38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:24.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.534+0000 7efc43e00700 1 -- 172.21.15.79:0/2779814913 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efc3c09c2b0 con 0x7efc4c1045c0 2024-06-03T02:17:24.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.761+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7efc4c02dca0 con 0x7efc4c1045c0 2024-06-03T02:17:24.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.762+0000 7efc43e00700 1 -- 172.21.15.79:0/2779814913 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc3c09bbb0 con 0x7efc4c1045c0 2024-06-03T02:17:24.976 INFO:teuthology.orchestra.run.smithi079.stdout:55834574959 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.765+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efc300a2d30 msgr2=0x7efc300a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.765+0000 7efc50a00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efc300a2d30 0x7efc300a51f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc4c109bf0 tx=0x7efc38009260 comp rx=0 tx=0).stop 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.765+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c1045c0 msgr2=0x7efc4c1a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.765+0000 7efc50a00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c1045c0 0x7efc4c1a08e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00eb30 tx=0x7efc3c00ee40 comp rx=0 tx=0).stop 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.766+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 shutdown_connections 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.766+0000 7efc50a00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc4c10a670 0x7efc4c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.766+0000 7efc50a00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efc300a2d30 0x7efc300a51f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.766+0000 7efc50a00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc4c1045c0 0x7efc4c1a08e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.766+0000 7efc50a00700 1 --2- 172.21.15.79:0/2779814913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc4c104f80 0x7efc4c1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.766+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 >> 172.21.15.79:0/2779814913 conn(0x7efc4c100070 msgr2=0x7efc4c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:24.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.766+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 shutdown_connections 2024-06-03T02:17:24.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:21.766+0000 7efc50a00700 1 -- 172.21.15.79:0/2779814913 wait complete. 2024-06-03T02:17:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:24 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2715748400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T02:17:25.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:24 smithi079 ceph-mon[31819]: pgmap v437: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T02:17:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:24 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2715748400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T02:17:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:24 smithi135 ceph-mon[36653]: pgmap v437: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T02:17:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:24 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2715748400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T02:17:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:24 smithi123 ceph-mon[36640]: pgmap v437: 129 pgs: 129 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T02:17:25.506 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430288 got 206158430292 for osd.6 2024-06-03T02:17:25.506 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:25.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.515+0000 7f8877800700 1 -- 172.21.15.79:0/2253633128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887006abc0 msgr2=0x7f887006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:25.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.515+0000 7f8877800700 1 --2- 172.21.15.79:0/2253633128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887006abc0 0x7f887006b040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f885c009990 tx=0x7f885c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.515+0000 7f8877800700 1 -- 172.21.15.79:0/2253633128 shutdown_connections 2024-06-03T02:17:25.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.515+0000 7f8877800700 1 --2- 172.21.15.79:0/2253633128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f887006b580 0x7f887010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.515+0000 7f8877800700 1 --2- 172.21.15.79:0/2253633128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8870108500 0x7f88701088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.515+0000 7f8877800700 1 --2- 172.21.15.79:0/2253633128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f887006abc0 0x7f887006b040 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.515+0000 7f8877800700 1 -- 172.21.15.79:0/2253633128 >> 172.21.15.79:0/2253633128 conn(0x7f8870078340 msgr2=0x7f8870078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:25.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.515+0000 7f8877800700 1 -- 172.21.15.79:0/2253633128 shutdown_connections 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.515+0000 7f8877800700 1 -- 172.21.15.79:0/2253633128 wait complete. 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8877800700 1 Processor -- start 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8877800700 1 -- start start 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8877800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f887006abc0 0x7f8870101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8877800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f887006b580 0x7f88701019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8877800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8870108500 0x7f8870101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8877800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8870112780 con 0x7f8870108500 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8877800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8870112600 con 0x7f887006abc0 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8877800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8870112900 con 0x7f887006b580 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8875800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8870108500 0x7f8870101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f886fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f887006b580 0x7f88701019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8875800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8870108500 0x7f8870101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37120/0 (socket says 172.21.15.79:37120) 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.517+0000 7f8875800700 1 -- 172.21.15.79:0/1127481734 learned_addr learned my addr 172.21.15.79:0/1127481734 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:25.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.518+0000 7f8874e00700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f887006abc0 0x7f8870101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.518+0000 7f8875800700 1 -- 172.21.15.79:0/1127481734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f887006b580 msgr2=0x7f88701019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.518+0000 7f8875800700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f887006b580 0x7f88701019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.518+0000 7f8875800700 1 -- 172.21.15.79:0/1127481734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f887006abc0 msgr2=0x7f8870101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.518+0000 7f8875800700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f887006abc0 0x7f8870101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.518+0000 7f8875800700 1 -- 172.21.15.79:0/1127481734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88701a7220 con 0x7f8870108500 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.518+0000 7f8874e00700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f887006abc0 0x7f8870101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.518+0000 7f8875800700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8870108500 0x7f8870101f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f886800ef30 tx=0x7f886800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.519+0000 7f886d600700 1 -- 172.21.15.79:0/1127481734 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88680092e0 con 0x7f8870108500 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.519+0000 7f886d600700 1 -- 172.21.15.79:0/1127481734 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8868004bb0 con 0x7f8870108500 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.519+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88701a74b0 con 0x7f8870108500 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.519+0000 7f886d600700 1 -- 172.21.15.79:0/1127481734 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868004280 con 0x7f8870108500 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.519+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88701a79c0 con 0x7f8870108500 2024-06-03T02:17:25.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.520+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f887010a9e0 con 0x7f8870108500 2024-06-03T02:17:25.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.522+0000 7f886d600700 1 -- 172.21.15.79:0/1127481734 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f886801b030 con 0x7f8870108500 2024-06-03T02:17:25.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.523+0000 7f886d600700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f88580a2c20 0x7f88580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:25.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.524+0000 7f886d600700 1 -- 172.21.15.79:0/1127481734 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f88680ceed0 con 0x7f8870108500 2024-06-03T02:17:25.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.524+0000 7f8874e00700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f88580a2c20 0x7f88580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:25.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.524+0000 7f8874e00700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f88580a2c20 0x7f88580a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8864009fd0 tx=0x7f8864009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:25.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.529+0000 7f886d600700 1 -- 172.21.15.79:0/1127481734 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8868022020 con 0x7f8870108500 2024-06-03T02:17:25.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.759+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f887010a3c0 con 0x7f8870108500 2024-06-03T02:17:25.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.760+0000 7f886d600700 1 -- 172.21.15.79:0/1127481734 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8868010040 con 0x7f8870108500 2024-06-03T02:17:25.760 INFO:teuthology.orchestra.run.smithi079.stdout:111669149796 2024-06-03T02:17:25.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f88580a2c20 msgr2=0x7f88580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:25.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f88580a2c20 0x7f88580a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8864009fd0 tx=0x7f8864009260 comp rx=0 tx=0).stop 2024-06-03T02:17:25.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8870108500 msgr2=0x7f8870101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:25.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8870108500 0x7f8870101f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f886800ef30 tx=0x7f886800c3a0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 shutdown_connections 2024-06-03T02:17:25.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f88580a2c20 0x7f88580a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8870108500 0x7f8870101f10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f887006abc0 0x7f8870101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 --2- 172.21.15.79:0/1127481734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f887006b580 0x7f88701019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:25.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 >> 172.21.15.79:0/1127481734 conn(0x7f8870078340 msgr2=0x7f88701000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:25.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 shutdown_connections 2024-06-03T02:17:25.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:25.763+0000 7f8877800700 1 -- 172.21.15.79:0/1127481734 wait complete. 2024-06-03T02:17:25.952 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:25.962 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:25 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1127481734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T02:17:25.988 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888152 got 146028888157 for osd.4 2024-06-03T02:17:25.988 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:26.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:25 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1127481734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T02:17:26.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:25 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1127481734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T02:17:26.785 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149789 got 111669149796 for osd.3 2024-06-03T02:17:26.786 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:27.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:26 smithi079 ceph-mon[31819]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-03T02:17:27.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574954 got 55834574959 for osd.1 2024-06-03T02:17:27.172 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:27.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:26 smithi123 ceph-mon[36640]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-03T02:17:27.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:26 smithi135 ceph-mon[36653]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-03T02:17:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.467+0000 7fd77be00700 1 -- 172.21.15.79:0/256978991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd77c107700 msgr2=0x7fd77c10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.467+0000 7fd77be00700 1 --2- 172.21.15.79:0/256978991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd77c107700 0x7fd77c10f2d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd7740099c0 tx=0x7fd774009cd0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.467+0000 7fd77be00700 1 -- 172.21.15.79:0/256978991 shutdown_connections 2024-06-03T02:17:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.467+0000 7fd77be00700 1 --2- 172.21.15.79:0/256978991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd77c107700 0x7fd77c10f2d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.467+0000 7fd77be00700 1 --2- 172.21.15.79:0/256978991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd77c102620 0x7fd77c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.467+0000 7fd77be00700 1 --2- 172.21.15.79:0/256978991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd77c102fd0 0x7fd77c107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.467+0000 7fd77be00700 1 -- 172.21.15.79:0/256978991 >> 172.21.15.79:0/256978991 conn(0x7fd77c0fdf10 msgr2=0x7fd77c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.467+0000 7fd77be00700 1 -- 172.21.15.79:0/256978991 shutdown_connections 2024-06-03T02:17:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.468+0000 7fd77be00700 1 -- 172.21.15.79:0/256978991 wait complete. 2024-06-03T02:17:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.469+0000 7fd77be00700 1 Processor -- start 2024-06-03T02:17:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.469+0000 7fd77be00700 1 -- start start 2024-06-03T02:17:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.469+0000 7fd77be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd77c102620 0x7fd77c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.469+0000 7fd77be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd77c102fd0 0x7fd77c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.469+0000 7fd77be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd77c107700 0x7fd77c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.469+0000 7fd77be00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c112740 con 0x7fd77c102620 2024-06-03T02:17:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.469+0000 7fd77be00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1125c0 con 0x7fd77c102fd0 2024-06-03T02:17:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.469+0000 7fd77be00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1128c0 con 0x7fd77c107700 2024-06-03T02:17:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd77c102620 0x7fd77c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd77c102620 0x7fd77c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37138/0 (socket says 172.21.15.79:37138) 2024-06-03T02:17:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77aa00700 1 -- 172.21.15.79:0/1532090081 learned_addr learned my addr 172.21.15.79:0/1532090081 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd772000700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd77c102fd0 0x7fd77c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77b400700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd77c107700 0x7fd77c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77aa00700 1 -- 172.21.15.79:0/1532090081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd77c107700 msgr2=0x7fd77c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77aa00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd77c107700 0x7fd77c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77aa00700 1 -- 172.21.15.79:0/1532090081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd77c102fd0 msgr2=0x7fd77c19b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77aa00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd77c102fd0 0x7fd77c19b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77aa00700 1 -- 172.21.15.79:0/1532090081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c19fd90 con 0x7fd77c102620 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd772000700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd77c102fd0 0x7fd77c19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.470+0000 7fd77aa00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd77c102620 0x7fd77c19ad80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00c370 tx=0x7fd76c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.471+0000 7fd773e00700 1 -- 172.21.15.79:0/1532090081 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0092e0 con 0x7fd77c102620 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.471+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c1a0020 con 0x7fd77c102620 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.471+0000 7fd773e00700 1 -- 172.21.15.79:0/1532090081 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76c00cbe0 con 0x7fd77c102620 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.471+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c1a7900 con 0x7fd77c102620 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.471+0000 7fd773e00700 1 -- 172.21.15.79:0/1532090081 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c004780 con 0x7fd77c102620 2024-06-03T02:17:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.472+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd77c069180 con 0x7fd77c102620 2024-06-03T02:17:27.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.474+0000 7fd773e00700 1 -- 172.21.15.79:0/1532090081 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd76c01a020 con 0x7fd77c102620 2024-06-03T02:17:27.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.475+0000 7fd773e00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7600a2c20 0x7fd7600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:27.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.475+0000 7fd773e00700 1 -- 172.21.15.79:0/1532090081 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd76c0cedb0 con 0x7fd77c102620 2024-06-03T02:17:27.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.476+0000 7fd772000700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7600a2c20 0x7fd7600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:27.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.476+0000 7fd772000700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7600a2c20 0x7fd7600a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd77c076720 tx=0x7fd764009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:27.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.480+0000 7fd773e00700 1 -- 172.21.15.79:0/1532090081 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd76c0d3070 con 0x7fd77c102620 2024-06-03T02:17:27.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.712+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd77c1a7b20 con 0x7fd77c102620 2024-06-03T02:17:27.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.713+0000 7fd773e00700 1 -- 172.21.15.79:0/1532090081 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd76c010040 con 0x7fd77c102620 2024-06-03T02:17:27.713 INFO:teuthology.orchestra.run.smithi079.stdout:227633266767 2024-06-03T02:17:27.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7600a2c20 msgr2=0x7fd7600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:27.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7600a2c20 0x7fd7600a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd77c076720 tx=0x7fd764009260 comp rx=0 tx=0).stop 2024-06-03T02:17:27.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd77c102620 msgr2=0x7fd77c19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:27.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd77c102620 0x7fd77c19ad80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00c370 tx=0x7fd76c00c680 comp rx=0 tx=0).stop 2024-06-03T02:17:27.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 shutdown_connections 2024-06-03T02:17:27.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd77c107700 0x7fd77c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd7600a2c20 0x7fd7600a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd77c102620 0x7fd77c19ad80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 --2- 172.21.15.79:0/1532090081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd77c102fd0 0x7fd77c19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:27.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 >> 172.21.15.79:0/1532090081 conn(0x7fd77c0fdf10 msgr2=0x7fd77c10d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:27.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 shutdown_connections 2024-06-03T02:17:27.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:27.716+0000 7fd77be00700 1 -- 172.21.15.79:0/1532090081 wait complete. 2024-06-03T02:17:27.949 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:27 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1532090081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-03T02:17:28.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:27 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1532090081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-03T02:17:28.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:27 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1532090081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-03T02:17:28.718 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738478 got 34359738486 for osd.0 2024-06-03T02:17:28.718 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:28.962 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:28 smithi079 ceph-mon[31819]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:29.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:28 smithi123 ceph-mon[36640]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:29.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:28 smithi135 ceph-mon[36653]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:29.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266759 got 227633266767 for osd.7 2024-06-03T02:17:29.362 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:30.584 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:30 smithi079 ceph-mon[31819]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:30.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:30 smithi135 ceph-mon[36653]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.639+0000 7f78a4000700 1 -- 172.21.15.79:0/550922570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c0746c0 msgr2=0x7f789c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.639+0000 7f78a4000700 1 --2- 172.21.15.79:0/550922570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c0746c0 0x7f789c074aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7890009990 tx=0x7f7890009ca0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.639+0000 7f78a4000700 1 -- 172.21.15.79:0/550922570 shutdown_connections 2024-06-03T02:17:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.639+0000 7f78a4000700 1 --2- 172.21.15.79:0/550922570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f789c113660 0x7f789c115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.639+0000 7f78a4000700 1 --2- 172.21.15.79:0/550922570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c0746c0 0x7f789c074aa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.639+0000 7f78a4000700 1 --2- 172.21.15.79:0/550922570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f789c074fe0 0x7f789c113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.639+0000 7f78a4000700 1 -- 172.21.15.79:0/550922570 >> 172.21.15.79:0/550922570 conn(0x7f789c0795f0 msgr2=0x7f789c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:30.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.639+0000 7f78a4000700 1 -- 172.21.15.79:0/550922570 shutdown_connections 2024-06-03T02:17:30.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.639+0000 7f78a4000700 1 -- 172.21.15.79:0/550922570 wait complete. 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.640+0000 7f78a4000700 1 Processor -- start 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a4000700 1 -- start start 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a4000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f789c0746c0 0x7f789c19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a4000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f789c074fe0 0x7f789c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a4000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c113660 0x7f789c1a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a4000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f789c118e10 con 0x7f789c074fe0 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a4000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f789c118c90 con 0x7f789c113660 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a4000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f789c118f90 con 0x7f789c0746c0 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a0c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f789c074fe0 0x7f789c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:30.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a0c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f789c074fe0 0x7f789c19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37170/0 (socket says 172.21.15.79:37170) 2024-06-03T02:17:30.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a0c00700 1 -- 172.21.15.79:0/2692430023 learned_addr learned my addr 172.21.15.79:0/2692430023 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:30.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a2000700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c113660 0x7f789c1a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:30.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.641+0000 7f78a1600700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f789c0746c0 0x7f789c19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:30.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.642+0000 7f78a1600700 1 -- 172.21.15.79:0/2692430023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c113660 msgr2=0x7f789c1a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:30.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.642+0000 7f78a1600700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c113660 0x7f789c1a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.642+0000 7f78a1600700 1 -- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f789c074fe0 msgr2=0x7f789c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:30.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.642+0000 7f78a1600700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f789c074fe0 0x7f789c19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.642+0000 7f78a1600700 1 -- 172.21.15.79:0/2692430023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c1a4310 con 0x7f789c0746c0 2024-06-03T02:17:30.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.642+0000 7f78a2000700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c113660 0x7f789c1a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:30.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.642+0000 7f78a0c00700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f789c074fe0 0x7f789c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:30.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.642+0000 7f78a1600700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f789c0746c0 0x7f789c19f300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7890000c00 tx=0x7f789000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:30.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.643+0000 7f789a000700 1 -- 172.21.15.79:0/2692430023 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890026070 con 0x7f789c0746c0 2024-06-03T02:17:30.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.643+0000 7f789a000700 1 -- 172.21.15.79:0/2692430023 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7890003d70 con 0x7f789c0746c0 2024-06-03T02:17:30.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.643+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789c1a45a0 con 0x7f789c0746c0 2024-06-03T02:17:30.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.643+0000 7f789a000700 1 -- 172.21.15.79:0/2692430023 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789000fdc0 con 0x7f789c0746c0 2024-06-03T02:17:30.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.643+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789c1abe80 con 0x7f789c0746c0 2024-06-03T02:17:30.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.644+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f789c069180 con 0x7f789c0746c0 2024-06-03T02:17:30.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.646+0000 7f789a000700 1 -- 172.21.15.79:0/2692430023 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7890029070 con 0x7f789c0746c0 2024-06-03T02:17:30.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.647+0000 7f789a000700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f78840a2ca0 0x7f78840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:30.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.647+0000 7f789a000700 1 -- 172.21.15.79:0/2692430023 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f78900d0420 con 0x7f789c0746c0 2024-06-03T02:17:30.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.648+0000 7f78a0c00700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f78840a2ca0 0x7f78840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:30.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.648+0000 7f78a0c00700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f78840a2ca0 0x7f78840a5160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f789c112b70 tx=0x7f7888009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:30.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.650+0000 7f789a000700 1 -- 172.21.15.79:0/2692430023 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7890013070 con 0x7f789c0746c0 2024-06-03T02:17:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:30 smithi123 ceph-mon[36640]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:30.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.874+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f789c1ac050 con 0x7f789c0746c0 2024-06-03T02:17:30.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.875+0000 7f789a000700 1 -- 172.21.15.79:0/2692430023 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f789009d000 con 0x7f789c0746c0 2024-06-03T02:17:30.875 INFO:teuthology.orchestra.run.smithi079.stdout:261993005130 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f78840a2ca0 msgr2=0x7f78840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f78840a2ca0 0x7f78840a5160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f789c112b70 tx=0x7f7888009260 comp rx=0 tx=0).stop 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f789c0746c0 msgr2=0x7f789c19f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f789c0746c0 0x7f789c19f300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7890000c00 tx=0x7f789000bab0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 shutdown_connections 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c113660 0x7f789c1a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f789c0746c0 0x7f789c19f300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f78840a2ca0 0x7f78840a5160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 --2- 172.21.15.79:0/2692430023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f789c074fe0 0x7f789c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.877+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 >> 172.21.15.79:0/2692430023 conn(0x7f789c0795f0 msgr2=0x7f789c10f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.878+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 shutdown_connections 2024-06-03T02:17:30.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:30.878+0000 7f78a4000700 1 -- 172.21.15.79:0/2692430023 wait complete. 2024-06-03T02:17:31.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:31 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2692430023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-03T02:17:31.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:31 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2692430023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-03T02:17:31.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:31 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2692430023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-03T02:17:31.765 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005124 got 261993005130 for osd.8 2024-06-03T02:17:31.765 DEBUG:teuthology.parallel:result is None 2024-06-03T02:17:31.765 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-03T02:17:31.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T02:17:32.214 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:32.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:32 smithi079 ceph-mon[31819]: pgmap v441: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:32 smithi135 ceph-mon[36653]: pgmap v441: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:32 smithi123 ceph-mon[36640]: pgmap v441: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:34.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.433+0000 7f9c8c800700 1 -- 172.21.15.79:0/2042880467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8406aa30 msgr2=0x7f9c8406ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:34.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.433+0000 7f9c8c800700 1 --2- 172.21.15.79:0/2042880467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8406aa30 0x7f9c8406ae10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9c780098f0 tx=0x7f9c78009c00 comp rx=0 tx=0).stop 2024-06-03T02:17:34.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.433+0000 7f9c8c800700 1 -- 172.21.15.79:0/2042880467 shutdown_connections 2024-06-03T02:17:34.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.433+0000 7f9c8c800700 1 --2- 172.21.15.79:0/2042880467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c8410ef30 0x7f9c84111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:34.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.433+0000 7f9c8c800700 1 --2- 172.21.15.79:0/2042880467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8406aa30 0x7f9c8406ae10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:34.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.433+0000 7f9c8c800700 1 --2- 172.21.15.79:0/2042880467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c8406b350 0x7f9c8410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:34.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.433+0000 7f9c8c800700 1 -- 172.21.15.79:0/2042880467 >> 172.21.15.79:0/2042880467 conn(0x7f9c84078300 msgr2=0x7f9c84078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:34.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.434+0000 7f9c8c800700 1 -- 172.21.15.79:0/2042880467 shutdown_connections 2024-06-03T02:17:34.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.434+0000 7f9c8c800700 1 -- 172.21.15.79:0/2042880467 wait complete. 2024-06-03T02:17:34.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.435+0000 7f9c8c800700 1 Processor -- start 2024-06-03T02:17:34.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.435+0000 7f9c8c800700 1 -- start start 2024-06-03T02:17:34.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.435+0000 7f9c8c800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8406aa30 0x7f9c8419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:34.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.435+0000 7f9c8c800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c8406b350 0x7f9c8419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.435+0000 7f9c8c800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c8410ef30 0x7f9c8419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.435+0000 7f9c8c800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841149c0 con 0x7f9c8406b350 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.435+0000 7f9c8c800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84114840 con 0x7f9c8406aa30 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.435+0000 7f9c8c800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84114b40 con 0x7f9c8410ef30 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.435+0000 7f9c89400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c8406b350 0x7f9c8419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c8a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c8410ef30 0x7f9c8419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c89400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c8406b350 0x7f9c8419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37190/0 (socket says 172.21.15.79:37190) 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c89400700 1 -- 172.21.15.79:0/1970148152 learned_addr learned my addr 172.21.15.79:0/1970148152 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c89e00700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8406aa30 0x7f9c8419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c89e00700 1 -- 172.21.15.79:0/1970148152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c8410ef30 msgr2=0x7f9c8419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c89e00700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c8410ef30 0x7f9c8419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c89e00700 1 -- 172.21.15.79:0/1970148152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c8406b350 msgr2=0x7f9c8419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c89e00700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c8406b350 0x7f9c8419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c89e00700 1 -- 172.21.15.79:0/1970148152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8419ff10 con 0x7f9c8406aa30 2024-06-03T02:17:34.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.436+0000 7f9c89e00700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8406aa30 0x7f9c8419af50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800c010 tx=0x7f9c7800fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:34.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.437+0000 7f9c7ea00700 1 -- 172.21.15.79:0/1970148152 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78005180 con 0x7f9c8406aa30 2024-06-03T02:17:34.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.437+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c841a01a0 con 0x7f9c8406aa30 2024-06-03T02:17:34.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.437+0000 7f9c7ea00700 1 -- 172.21.15.79:0/1970148152 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7800b560 con 0x7f9c8406aa30 2024-06-03T02:17:34.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.437+0000 7f9c7ea00700 1 -- 172.21.15.79:0/1970148152 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78003a80 con 0x7f9c8406aa30 2024-06-03T02:17:34.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.437+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841a7a80 con 0x7f9c8406aa30 2024-06-03T02:17:34.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.438+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c84069180 con 0x7f9c8406aa30 2024-06-03T02:17:34.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.440+0000 7f9c7ea00700 1 -- 172.21.15.79:0/1970148152 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9c7800b710 con 0x7f9c8406aa30 2024-06-03T02:17:34.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.441+0000 7f9c7ea00700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9c6c0a2c20 0x7f9c6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:34.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.441+0000 7f9c7ea00700 1 -- 172.21.15.79:0/1970148152 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f9c780d0320 con 0x7f9c8406aa30 2024-06-03T02:17:34.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.441+0000 7f9c89400700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9c6c0a2c20 0x7f9c6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:34.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.442+0000 7f9c89400700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9c6c0a2c20 0x7f9c6c0a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c84102940 tx=0x7f9c74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:34.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.446+0000 7f9c7ea00700 1 -- 172.21.15.79:0/1970148152 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c78013030 con 0x7f9c8406aa30 2024-06-03T02:17:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:34 smithi079 ceph-mon[31819]: pgmap v442: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:34 smithi135 ceph-mon[36653]: pgmap v442: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:34.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.665+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c8419c260 con 0x7f9c6c0a2c20 2024-06-03T02:17:34.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.683+0000 7f9c7ea00700 1 -- 172.21.15.79:0/1970148152 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340741 (secure 0 0 0) 0x7f9c8419c260 con 0x7f9c6c0a2c20 2024-06-03T02:17:34.691 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.691+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9c6c0a2c20 msgr2=0x7f9c6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.691+0000 7f9c8c800700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9c6c0a2c20 0x7f9c6c0a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c84102940 tx=0x7f9c74009260 comp rx=0 tx=0).stop 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.691+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8406aa30 msgr2=0x7f9c8419af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.691+0000 7f9c8c800700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8406aa30 0x7f9c8419af50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800c010 tx=0x7f9c7800fa70 comp rx=0 tx=0).stop 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.691+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 shutdown_connections 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.691+0000 7f9c8c800700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c8410ef30 0x7f9c8419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.691+0000 7f9c8c800700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8406aa30 0x7f9c8419af50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.691+0000 7f9c8c800700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9c6c0a2c20 0x7f9c6c0a50e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.691+0000 7f9c8c800700 1 --2- 172.21.15.79:0/1970148152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c8406b350 0x7f9c8419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.692+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 >> 172.21.15.79:0/1970148152 conn(0x7f9c84078300 msgr2=0x7f9c84110420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.692+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 shutdown_connections 2024-06-03T02:17:34.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:34.692+0000 7f9c8c800700 1 -- 172.21.15.79:0/1970148152 wait complete. 2024-06-03T02:17:34.695 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-06-03T02:17:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:34 smithi123 ceph-mon[36640]: pgmap v442: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T02:17:35.452 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:35 smithi079 ceph-mon[31819]: from='client.24929 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:17:35.524 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-06-03T02:17:34.246517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665345,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1338,"num_read_kb":799,"num_write":446,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360828,"kb_used_data":15836,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580996,"statfs":{"total":1151940427776,"available":1151570939904,"internally_reserved":0,"allocated":16216064,"data_stored":8900921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":69,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005098"},"pg_stats":[{"pgid":"3.3f","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186075+0000","last_change":"2024-06-03T02:14:09.768431+0000","last_active":"2024-06-03T02:17:22.186075+0000","last_peered":"2024-06-03T02:17:22.186075+0000","last_clean":"2024-06-03T02:17:22.186075+0000","last_became_active":"2024-06-03T02:14:09.768059+0000","last_became_peered":"2024-06-03T02:14:09.768059+0000","last_unstale":"2024-06-03T02:17:22.186075+0000","last_undegraded":"2024-06-03T02:17:22.186075+0000","last_fullsized":"2024-06-03T02:17:22.186075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:12.265896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"101'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187232+0000","last_change":"2024-06-03T02:14:09.771842+0000","last_active":"2024-06-03T02:17:22.187232+0000","last_peered":"2024-06-03T02:17:22.187232+0000","last_clean":"2024-06-03T02:17:22.187232+0000","last_became_active":"2024-06-03T02:14:09.770961+0000","last_became_peered":"2024-06-03T02:14:09.770961+0000","last_unstale":"2024-06-03T02:17:22.187232+0000","last_undegraded":"2024-06-03T02:17:22.187232+0000","last_fullsized":"2024-06-03T02:17:22.187232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:49:02.297128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.182425+0000","last_change":"2024-06-03T02:14:09.758407+0000","last_active":"2024-06-03T02:17:22.182425+0000","last_peered":"2024-06-03T02:17:22.182425+0000","last_clean":"2024-06-03T02:17:22.182425+0000","last_became_active":"2024-06-03T02:14:09.758045+0000","last_became_peered":"2024-06-03T02:14:09.758045+0000","last_unstale":"2024-06-03T02:17:22.182425+0000","last_undegraded":"2024-06-03T02:17:22.182425+0000","last_fullsized":"2024-06-03T02:17:22.182425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:01:08.166971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.151507+0000","last_change":"2024-06-03T02:14:09.747658+0000","last_active":"2024-06-03T02:17:22.151507+0000","last_peered":"2024-06-03T02:17:22.151507+0000","last_clean":"2024-06-03T02:17:22.151507+0000","last_became_active":"2024-06-03T02:14:09.747321+0000","last_became_peered":"2024-06-03T02:14:09.747321+0000","last_unstale":"2024-06-03T02:17:22.151507+0000","last_undegraded":"2024-06-03T02:17:22.151507+0000","last_fullsized":"2024-06-03T02:17:22.151507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:08.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"101'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184802+0000","last_change":"2024-06-03T02:14:09.758161+0000","last_active":"2024-06-03T02:17:22.184802+0000","last_peered":"2024-06-03T02:17:22.184802+0000","last_clean":"2024-06-03T02:17:22.184802+0000","last_became_active":"2024-06-03T02:14:09.757800+0000","last_became_peered":"2024-06-03T02:14:09.757800+0000","last_unstale":"2024-06-03T02:17:22.184802+0000","last_undegraded":"2024-06-03T02:17:22.184802+0000","last_fullsized":"2024-06-03T02:17:22.184802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:47.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.160331+0000","last_change":"2024-06-03T02:14:09.741818+0000","last_active":"2024-06-03T02:17:22.160331+0000","last_peered":"2024-06-03T02:17:22.160331+0000","last_clean":"2024-06-03T02:17:22.160331+0000","last_became_active":"2024-06-03T02:14:09.741219+0000","last_became_peered":"2024-06-03T02:14:09.741219+0000","last_unstale":"2024-06-03T02:17:22.160331+0000","last_undegraded":"2024-06-03T02:17:22.160331+0000","last_fullsized":"2024-06-03T02:17:22.160331+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:16:30.585245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.159623+0000","last_change":"2024-06-03T02:14:09.744863+0000","last_active":"2024-06-03T02:17:22.159623+0000","last_peered":"2024-06-03T02:17:22.159623+0000","last_clean":"2024-06-03T02:17:22.159623+0000","last_became_active":"2024-06-03T02:14:09.744514+0000","last_became_peered":"2024-06-03T02:14:09.744514+0000","last_unstale":"2024-06-03T02:17:22.159623+0000","last_undegraded":"2024-06-03T02:17:22.159623+0000","last_fullsized":"2024-06-03T02:17:22.159623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:51:42.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187514+0000","last_change":"2024-06-03T02:14:09.766897+0000","last_active":"2024-06-03T02:17:22.187514+0000","last_peered":"2024-06-03T02:17:22.187514+0000","last_clean":"2024-06-03T02:17:22.187514+0000","last_became_active":"2024-06-03T02:14:09.765461+0000","last_became_peered":"2024-06-03T02:14:09.765461+0000","last_unstale":"2024-06-03T02:17:22.187514+0000","last_undegraded":"2024-06-03T02:17:22.187514+0000","last_fullsized":"2024-06-03T02:17:22.187514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:39:28.641517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"101'11","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171704+0000","last_change":"2024-06-03T02:14:09.746451+0000","last_active":"2024-06-03T02:17:22.171704+0000","last_peered":"2024-06-03T02:17:22.171704+0000","last_clean":"2024-06-03T02:17:22.171704+0000","last_became_active":"2024-06-03T02:14:09.746117+0000","last_became_peered":"2024-06-03T02:14:09.746117+0000","last_unstale":"2024-06-03T02:17:22.171704+0000","last_undegraded":"2024-06-03T02:17:22.171704+0000","last_fullsized":"2024-06-03T02:17:22.171704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:42.599993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"101'16","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.162197+0000","last_change":"2024-06-03T02:14:09.750680+0000","last_active":"2024-06-03T02:17:22.162197+0000","last_peered":"2024-06-03T02:17:22.162197+0000","last_clean":"2024-06-03T02:17:22.162197+0000","last_became_active":"2024-06-03T02:14:09.746138+0000","last_became_peered":"2024-06-03T02:14:09.746138+0000","last_unstale":"2024-06-03T02:17:22.162197+0000","last_undegraded":"2024-06-03T02:17:22.162197+0000","last_fullsized":"2024-06-03T02:17:22.162197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:04:28.188323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172319+0000","last_change":"2024-06-03T02:14:09.743467+0000","last_active":"2024-06-03T02:17:22.172319+0000","last_peered":"2024-06-03T02:17:22.172319+0000","last_clean":"2024-06-03T02:17:22.172319+0000","last_became_active":"2024-06-03T02:14:09.743152+0000","last_became_peered":"2024-06-03T02:14:09.743152+0000","last_unstale":"2024-06-03T02:17:22.172319+0000","last_undegraded":"2024-06-03T02:17:22.172319+0000","last_fullsized":"2024-06-03T02:17:22.172319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:27.336630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.148154+0000","last_change":"2024-06-03T02:14:09.747142+0000","last_active":"2024-06-03T02:17:22.148154+0000","last_peered":"2024-06-03T02:17:22.148154+0000","last_clean":"2024-06-03T02:17:22.148154+0000","last_became_active":"2024-06-03T02:14:09.746694+0000","last_became_peered":"2024-06-03T02:14:09.746694+0000","last_unstale":"2024-06-03T02:17:22.148154+0000","last_undegraded":"2024-06-03T02:17:22.148154+0000","last_fullsized":"2024-06-03T02:17:22.148154+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:04.507667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"101'10","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179040+0000","last_change":"2024-06-03T02:14:09.759906+0000","last_active":"2024-06-03T02:17:22.179040+0000","last_peered":"2024-06-03T02:17:22.179040+0000","last_clean":"2024-06-03T02:17:22.179040+0000","last_became_active":"2024-06-03T02:14:09.755450+0000","last_became_peered":"2024-06-03T02:14:09.755450+0000","last_unstale":"2024-06-03T02:17:22.179040+0000","last_undegraded":"2024-06-03T02:17:22.179040+0000","last_fullsized":"2024-06-03T02:17:22.179040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:23.057956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187103+0000","last_change":"2024-06-03T02:14:09.769388+0000","last_active":"2024-06-03T02:17:22.187103+0000","last_peered":"2024-06-03T02:17:22.187103+0000","last_clean":"2024-06-03T02:17:22.187103+0000","last_became_active":"2024-06-03T02:14:09.769029+0000","last_became_peered":"2024-06-03T02:14:09.769029+0000","last_unstale":"2024-06-03T02:17:22.187103+0000","last_undegraded":"2024-06-03T02:17:22.187103+0000","last_fullsized":"2024-06-03T02:17:22.187103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:24.751039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181114+0000","last_change":"2024-06-03T02:14:09.768584+0000","last_active":"2024-06-03T02:17:22.181114+0000","last_peered":"2024-06-03T02:17:22.181114+0000","last_clean":"2024-06-03T02:17:22.181114+0000","last_became_active":"2024-06-03T02:14:09.765737+0000","last_became_peered":"2024-06-03T02:14:09.765737+0000","last_unstale":"2024-06-03T02:17:22.181114+0000","last_undegraded":"2024-06-03T02:17:22.181114+0000","last_fullsized":"2024-06-03T02:17:22.181114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:36:18.807762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181296+0000","last_change":"2024-06-03T02:14:09.757828+0000","last_active":"2024-06-03T02:17:22.181296+0000","last_peered":"2024-06-03T02:17:22.181296+0000","last_clean":"2024-06-03T02:17:22.181296+0000","last_became_active":"2024-06-03T02:14:09.757040+0000","last_became_peered":"2024-06-03T02:14:09.757040+0000","last_unstale":"2024-06-03T02:17:22.181296+0000","last_undegraded":"2024-06-03T02:17:22.181296+0000","last_fullsized":"2024-06-03T02:17:22.181296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:26:28.694022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188466+0000","last_change":"2024-06-03T02:14:09.768263+0000","last_active":"2024-06-03T02:17:22.188466+0000","last_peered":"2024-06-03T02:17:22.188466+0000","last_clean":"2024-06-03T02:17:22.188466+0000","last_became_active":"2024-06-03T02:14:09.767988+0000","last_became_peered":"2024-06-03T02:14:09.767988+0000","last_unstale":"2024-06-03T02:17:22.188466+0000","last_undegraded":"2024-06-03T02:17:22.188466+0000","last_fullsized":"2024-06-03T02:17:22.188466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:28:37.432659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"101'10","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171695+0000","last_change":"2024-06-03T02:14:09.747118+0000","last_active":"2024-06-03T02:17:22.171695+0000","last_peered":"2024-06-03T02:17:22.171695+0000","last_clean":"2024-06-03T02:17:22.171695+0000","last_became_active":"2024-06-03T02:14:09.746580+0000","last_became_peered":"2024-06-03T02:14:09.746580+0000","last_unstale":"2024-06-03T02:17:22.171695+0000","last_undegraded":"2024-06-03T02:17:22.171695+0000","last_fullsized":"2024-06-03T02:17:22.171695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:33:04.607285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185237+0000","last_change":"2024-06-03T02:14:09.755755+0000","last_active":"2024-06-03T02:17:22.185237+0000","last_peered":"2024-06-03T02:17:22.185237+0000","last_clean":"2024-06-03T02:17:22.185237+0000","last_became_active":"2024-06-03T02:14:09.754503+0000","last_became_peered":"2024-06-03T02:14:09.754503+0000","last_unstale":"2024-06-03T02:17:22.185237+0000","last_undegraded":"2024-06-03T02:17:22.185237+0000","last_fullsized":"2024-06-03T02:17:22.185237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:30:58.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271241+0000","last_change":"2024-06-03T02:14:23.260021+0000","last_active":"2024-06-03T02:15:54.271241+0000","last_peered":"2024-06-03T02:15:54.271241+0000","last_clean":"2024-06-03T02:15:54.271241+0000","last_became_active":"2024-06-03T02:14:23.259587+0000","last_became_peered":"2024-06-03T02:14:23.259587+0000","last_unstale":"2024-06-03T02:15:54.271241+0000","last_undegraded":"2024-06-03T02:15:54.271241+0000","last_fullsized":"2024-06-03T02:15:54.271241+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:09:02.403936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"101'14","reported_seq":175,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:29.577336+0000","last_change":"2024-06-03T02:14:05.239623+0000","last_active":"2024-06-03T02:17:29.577336+0000","last_peered":"2024-06-03T02:17:29.577336+0000","last_clean":"2024-06-03T02:17:29.577336+0000","last_became_active":"2024-06-03T02:14:05.239112+0000","last_became_peered":"2024-06-03T02:14:05.239112+0000","last_unstale":"2024-06-03T02:17:29.577336+0000","last_undegraded":"2024-06-03T02:17:29.577336+0000","last_fullsized":"2024-06-03T02:17:29.577336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:12:14.020068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184909+0000","last_change":"2024-06-03T02:14:09.764972+0000","last_active":"2024-06-03T02:17:22.184909+0000","last_peered":"2024-06-03T02:17:22.184909+0000","last_clean":"2024-06-03T02:17:22.184909+0000","last_became_active":"2024-06-03T02:14:09.764435+0000","last_became_peered":"2024-06-03T02:14:09.764435+0000","last_unstale":"2024-06-03T02:17:22.184909+0000","last_undegraded":"2024-06-03T02:17:22.184909+0000","last_fullsized":"2024-06-03T02:17:22.184909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:46:58.941554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339964+0000","last_change":"2024-06-03T02:14:23.272076+0000","last_active":"2024-06-03T02:15:54.339964+0000","last_peered":"2024-06-03T02:15:54.339964+0000","last_clean":"2024-06-03T02:15:54.339964+0000","last_became_active":"2024-06-03T02:14:23.271578+0000","last_became_peered":"2024-06-03T02:14:23.271578+0000","last_unstale":"2024-06-03T02:15:54.339964+0000","last_undegraded":"2024-06-03T02:15:54.339964+0000","last_fullsized":"2024-06-03T02:15:54.339964+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:17:59.000291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"99'2","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268287+0000","last_change":"2024-06-03T02:14:05.251952+0000","last_active":"2024-06-03T02:15:54.268287+0000","last_peered":"2024-06-03T02:15:54.268287+0000","last_clean":"2024-06-03T02:15:54.268287+0000","last_became_active":"2024-06-03T02:14:05.251152+0000","last_became_peered":"2024-06-03T02:14:05.251152+0000","last_unstale":"2024-06-03T02:15:54.268287+0000","last_undegraded":"2024-06-03T02:15:54.268287+0000","last_fullsized":"2024-06-03T02:15:54.268287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:24:32.802236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.190220+0000","last_change":"2024-06-03T02:14:09.764230+0000","last_active":"2024-06-03T02:17:22.190220+0000","last_peered":"2024-06-03T02:17:22.190220+0000","last_clean":"2024-06-03T02:17:22.190220+0000","last_became_active":"2024-06-03T02:14:09.761868+0000","last_became_peered":"2024-06-03T02:14:09.761868+0000","last_unstale":"2024-06-03T02:17:22.190220+0000","last_undegraded":"2024-06-03T02:17:22.190220+0000","last_fullsized":"2024-06-03T02:17:22.190220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:39:23.828546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340393+0000","last_change":"2024-06-03T02:14:23.272169+0000","last_active":"2024-06-03T02:15:54.340393+0000","last_peered":"2024-06-03T02:15:54.340393+0000","last_clean":"2024-06-03T02:15:54.340393+0000","last_became_active":"2024-06-03T02:14:23.271636+0000","last_became_peered":"2024-06-03T02:14:23.271636+0000","last_unstale":"2024-06-03T02:15:54.340393+0000","last_undegraded":"2024-06-03T02:15:54.340393+0000","last_fullsized":"2024-06-03T02:15:54.340393+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:51:25.478028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"99'12","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270076+0000","last_change":"2024-06-03T02:14:05.238590+0000","last_active":"2024-06-03T02:15:54.270076+0000","last_peered":"2024-06-03T02:15:54.270076+0000","last_clean":"2024-06-03T02:15:54.270076+0000","last_became_active":"2024-06-03T02:14:05.238128+0000","last_became_peered":"2024-06-03T02:14:05.238128+0000","last_unstale":"2024-06-03T02:15:54.270076+0000","last_undegraded":"2024-06-03T02:15:54.270076+0000","last_fullsized":"2024-06-03T02:15:54.270076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:21.298255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174637+0000","last_change":"2024-06-03T02:14:09.754489+0000","last_active":"2024-06-03T02:17:22.174637+0000","last_peered":"2024-06-03T02:17:22.174637+0000","last_clean":"2024-06-03T02:17:22.174637+0000","last_became_active":"2024-06-03T02:14:09.754155+0000","last_became_peered":"2024-06-03T02:14:09.754155+0000","last_unstale":"2024-06-03T02:17:22.174637+0000","last_undegraded":"2024-06-03T02:17:22.174637+0000","last_fullsized":"2024-06-03T02:17:22.174637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:20:43.841271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340612+0000","last_change":"2024-06-03T02:14:23.261232+0000","last_active":"2024-06-03T02:15:54.340612+0000","last_peered":"2024-06-03T02:15:54.340612+0000","last_clean":"2024-06-03T02:15:54.340612+0000","last_became_active":"2024-06-03T02:14:23.260772+0000","last_became_peered":"2024-06-03T02:14:23.260772+0000","last_unstale":"2024-06-03T02:15:54.340612+0000","last_undegraded":"2024-06-03T02:15:54.340612+0000","last_fullsized":"2024-06-03T02:15:54.340612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:52.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"99'5","reported_seq":24,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272244+0000","last_change":"2024-06-03T02:14:53.274084+0000","last_active":"2024-06-03T02:15:54.272244+0000","last_peered":"2024-06-03T02:15:54.272244+0000","last_clean":"2024-06-03T02:15:54.272244+0000","last_became_active":"2024-06-03T02:14:53.265253+0000","last_became_peered":"2024-06-03T02:14:53.265253+0000","last_unstale":"2024-06-03T02:15:54.272244+0000","last_undegraded":"2024-06-03T02:15:54.272244+0000","last_fullsized":"2024-06-03T02:15:54.272244+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:25:56.189308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"101'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.155955+0000","last_change":"2024-06-03T02:14:09.743450+0000","last_active":"2024-06-03T02:17:22.155955+0000","last_peered":"2024-06-03T02:17:22.155955+0000","last_clean":"2024-06-03T02:17:22.155955+0000","last_became_active":"2024-06-03T02:14:09.743071+0000","last_became_peered":"2024-06-03T02:14:09.743071+0000","last_unstale":"2024-06-03T02:17:22.155955+0000","last_undegraded":"2024-06-03T02:17:22.155955+0000","last_fullsized":"2024-06-03T02:17:22.155955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:43:44.278633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":36,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.343527+0000","last_change":"2024-06-03T02:15:54.342978+0000","last_active":"2024-06-03T02:15:54.343527+0000","last_peered":"2024-06-03T02:15:54.343527+0000","last_clean":"2024-06-03T02:15:54.343527+0000","last_became_active":"2024-06-03T02:15:54.342477+0000","last_became_peered":"2024-06-03T02:15:54.342477+0000","last_unstale":"2024-06-03T02:15:54.343527+0000","last_undegraded":"2024-06-03T02:15:54.343527+0000","last_fullsized":"2024-06-03T02:15:54.343527+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:47.980513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"101'7","reported_seq":172,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:30.551899+0000","last_change":"2024-06-03T02:14:05.237464+0000","last_active":"2024-06-03T02:17:30.551899+0000","last_peered":"2024-06-03T02:17:30.551899+0000","last_clean":"2024-06-03T02:17:30.551899+0000","last_became_active":"2024-06-03T02:14:05.237067+0000","last_became_peered":"2024-06-03T02:14:05.237067+0000","last_unstale":"2024-06-03T02:17:30.551899+0000","last_undegraded":"2024-06-03T02:17:30.551899+0000","last_fullsized":"2024-06-03T02:17:30.551899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:06:38.850484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.149882+0000","last_change":"2024-06-03T02:14:09.748741+0000","last_active":"2024-06-03T02:17:22.149882+0000","last_peered":"2024-06-03T02:17:22.149882+0000","last_clean":"2024-06-03T02:17:22.149882+0000","last_became_active":"2024-06-03T02:14:09.748402+0000","last_became_peered":"2024-06-03T02:14:09.748402+0000","last_unstale":"2024-06-03T02:17:22.149882+0000","last_undegraded":"2024-06-03T02:17:22.149882+0000","last_fullsized":"2024-06-03T02:17:22.149882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:03:25.664415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271843+0000","last_change":"2024-06-03T02:14:23.260397+0000","last_active":"2024-06-03T02:15:54.271843+0000","last_peered":"2024-06-03T02:15:54.271843+0000","last_clean":"2024-06-03T02:15:54.271843+0000","last_became_active":"2024-06-03T02:14:23.259886+0000","last_became_peered":"2024-06-03T02:14:23.259886+0000","last_unstale":"2024-06-03T02:15:54.271843+0000","last_undegraded":"2024-06-03T02:15:54.271843+0000","last_fullsized":"2024-06-03T02:15:54.271843+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:46:18.562706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"99'7","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263532+0000","last_change":"2024-06-03T02:14:05.241536+0000","last_active":"2024-06-03T02:15:54.263532+0000","last_peered":"2024-06-03T02:15:54.263532+0000","last_clean":"2024-06-03T02:15:54.263532+0000","last_became_active":"2024-06-03T02:14:05.241203+0000","last_became_peered":"2024-06-03T02:14:05.241203+0000","last_unstale":"2024-06-03T02:15:54.263532+0000","last_undegraded":"2024-06-03T02:15:54.263532+0000","last_fullsized":"2024-06-03T02:15:54.263532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:47.052563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"101'8","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185090+0000","last_change":"2024-06-03T02:14:53.475893+0000","last_active":"2024-06-03T02:17:22.185090+0000","last_peered":"2024-06-03T02:17:22.185090+0000","last_clean":"2024-06-03T02:17:22.185090+0000","last_became_active":"2024-06-03T02:14:53.475494+0000","last_became_peered":"2024-06-03T02:14:53.475494+0000","last_unstale":"2024-06-03T02:17:22.185090+0000","last_undegraded":"2024-06-03T02:17:22.185090+0000","last_fullsized":"2024-06-03T02:17:22.185090+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:27.113842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345061+0000","last_change":"2024-06-03T02:15:54.344417+0000","last_active":"2024-06-03T02:15:54.345061+0000","last_peered":"2024-06-03T02:15:54.345061+0000","last_clean":"2024-06-03T02:15:54.345061+0000","last_became_active":"2024-06-03T02:15:54.343904+0000","last_became_peered":"2024-06-03T02:15:54.343904+0000","last_unstale":"2024-06-03T02:15:54.345061+0000","last_undegraded":"2024-06-03T02:15:54.345061+0000","last_fullsized":"2024-06-03T02:15:54.345061+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:42:18.121696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"99'3","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.343922+0000","last_change":"2024-06-03T02:14:05.237743+0000","last_active":"2024-06-03T02:15:54.343922+0000","last_peered":"2024-06-03T02:15:54.343922+0000","last_clean":"2024-06-03T02:15:54.343922+0000","last_became_active":"2024-06-03T02:14:05.236944+0000","last_became_peered":"2024-06-03T02:14:05.236944+0000","last_unstale":"2024-06-03T02:15:54.343922+0000","last_undegraded":"2024-06-03T02:15:54.343922+0000","last_fullsized":"2024-06-03T02:15:54.343922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:14:09.850862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185374+0000","last_change":"2024-06-03T02:14:09.761694+0000","last_active":"2024-06-03T02:17:22.185374+0000","last_peered":"2024-06-03T02:17:22.185374+0000","last_clean":"2024-06-03T02:17:22.185374+0000","last_became_active":"2024-06-03T02:14:09.761361+0000","last_became_peered":"2024-06-03T02:14:09.761361+0000","last_unstale":"2024-06-03T02:17:22.185374+0000","last_undegraded":"2024-06-03T02:17:22.185374+0000","last_fullsized":"2024-06-03T02:17:22.185374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:19:17.803088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342311+0000","last_change":"2024-06-03T02:14:23.258634+0000","last_active":"2024-06-03T02:15:54.342311+0000","last_peered":"2024-06-03T02:15:54.342311+0000","last_clean":"2024-06-03T02:15:54.342311+0000","last_became_active":"2024-06-03T02:14:23.257799+0000","last_became_peered":"2024-06-03T02:14:23.257799+0000","last_unstale":"2024-06-03T02:15:54.342311+0000","last_undegraded":"2024-06-03T02:15:54.342311+0000","last_fullsized":"2024-06-03T02:15:54.342311+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:38:33.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"99'3","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339907+0000","last_change":"2024-06-03T02:14:05.236790+0000","last_active":"2024-06-03T02:15:54.339907+0000","last_peered":"2024-06-03T02:15:54.339907+0000","last_clean":"2024-06-03T02:15:54.339907+0000","last_became_active":"2024-06-03T02:14:05.236419+0000","last_became_peered":"2024-06-03T02:14:05.236419+0000","last_unstale":"2024-06-03T02:15:54.339907+0000","last_undegraded":"2024-06-03T02:15:54.339907+0000","last_fullsized":"2024-06-03T02:15:54.339907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:44:00.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"101'11","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183235+0000","last_change":"2024-06-03T02:14:09.744680+0000","last_active":"2024-06-03T02:17:22.183235+0000","last_peered":"2024-06-03T02:17:22.183235+0000","last_clean":"2024-06-03T02:17:22.183235+0000","last_became_active":"2024-06-03T02:14:09.744250+0000","last_became_peered":"2024-06-03T02:14:09.744250+0000","last_unstale":"2024-06-03T02:17:22.183235+0000","last_undegraded":"2024-06-03T02:17:22.183235+0000","last_fullsized":"2024-06-03T02:17:22.183235+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:58.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267519+0000","last_change":"2024-06-03T02:14:23.259325+0000","last_active":"2024-06-03T02:15:54.267519+0000","last_peered":"2024-06-03T02:15:54.267519+0000","last_clean":"2024-06-03T02:15:54.267519+0000","last_became_active":"2024-06-03T02:14:23.258925+0000","last_became_peered":"2024-06-03T02:14:23.258925+0000","last_unstale":"2024-06-03T02:15:54.267519+0000","last_undegraded":"2024-06-03T02:15:54.267519+0000","last_fullsized":"2024-06-03T02:15:54.267519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:32.038564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"101'5","reported_seq":286,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:33.493338+0000","last_change":"2024-06-03T02:14:05.242690+0000","last_active":"2024-06-03T02:17:33.493338+0000","last_peered":"2024-06-03T02:17:33.493338+0000","last_clean":"2024-06-03T02:17:33.493338+0000","last_became_active":"2024-06-03T02:14:05.242320+0000","last_became_peered":"2024-06-03T02:14:05.242320+0000","last_unstale":"2024-06-03T02:17:33.493338+0000","last_undegraded":"2024-06-03T02:17:33.493338+0000","last_fullsized":"2024-06-03T02:17:33.493338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:02.793186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"101'10","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.170581+0000","last_change":"2024-06-03T02:14:09.747146+0000","last_active":"2024-06-03T02:17:22.170581+0000","last_peered":"2024-06-03T02:17:22.170581+0000","last_clean":"2024-06-03T02:17:22.170581+0000","last_became_active":"2024-06-03T02:14:09.746803+0000","last_became_peered":"2024-06-03T02:14:09.746803+0000","last_unstale":"2024-06-03T02:17:22.170581+0000","last_undegraded":"2024-06-03T02:17:22.170581+0000","last_fullsized":"2024-06-03T02:17:22.170581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:04:36.262223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.265005+0000","last_change":"2024-06-03T02:14:23.270328+0000","last_active":"2024-06-03T02:15:54.265005+0000","last_peered":"2024-06-03T02:15:54.265005+0000","last_clean":"2024-06-03T02:15:54.265005+0000","last_became_active":"2024-06-03T02:14:23.264189+0000","last_became_peered":"2024-06-03T02:14:23.264189+0000","last_unstale":"2024-06-03T02:15:54.265005+0000","last_undegraded":"2024-06-03T02:15:54.265005+0000","last_fullsized":"2024-06-03T02:15:54.265005+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:36.248884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"99'4","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340154+0000","last_change":"2024-06-03T02:14:05.238898+0000","last_active":"2024-06-03T02:15:54.340154+0000","last_peered":"2024-06-03T02:15:54.340154+0000","last_clean":"2024-06-03T02:15:54.340154+0000","last_became_active":"2024-06-03T02:14:05.238552+0000","last_became_peered":"2024-06-03T02:14:05.238552+0000","last_unstale":"2024-06-03T02:15:54.340154+0000","last_undegraded":"2024-06-03T02:15:54.340154+0000","last_fullsized":"2024-06-03T02:15:54.340154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:29:33.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.189485+0000","last_change":"2024-06-03T02:14:09.769973+0000","last_active":"2024-06-03T02:17:22.189485+0000","last_peered":"2024-06-03T02:17:22.189485+0000","last_clean":"2024-06-03T02:17:22.189485+0000","last_became_active":"2024-06-03T02:14:09.768984+0000","last_became_peered":"2024-06-03T02:14:09.768984+0000","last_unstale":"2024-06-03T02:17:22.189485+0000","last_undegraded":"2024-06-03T02:17:22.189485+0000","last_fullsized":"2024-06-03T02:17:22.189485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:48:33.720441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340613+0000","last_change":"2024-06-03T02:14:23.265947+0000","last_active":"2024-06-03T02:15:54.340613+0000","last_peered":"2024-06-03T02:15:54.340613+0000","last_clean":"2024-06-03T02:15:54.340613+0000","last_became_active":"2024-06-03T02:14:23.265412+0000","last_became_peered":"2024-06-03T02:14:23.265412+0000","last_unstale":"2024-06-03T02:15:54.340613+0000","last_undegraded":"2024-06-03T02:15:54.340613+0000","last_fullsized":"2024-06-03T02:15:54.340613+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:35:08.664884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"99'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339912+0000","last_change":"2024-06-03T02:14:05.252751+0000","last_active":"2024-06-03T02:15:54.339912+0000","last_peered":"2024-06-03T02:15:54.339912+0000","last_clean":"2024-06-03T02:15:54.339912+0000","last_became_active":"2024-06-03T02:14:05.252267+0000","last_became_peered":"2024-06-03T02:14:05.252267+0000","last_unstale":"2024-06-03T02:15:54.339912+0000","last_undegraded":"2024-06-03T02:15:54.339912+0000","last_fullsized":"2024-06-03T02:15:54.339912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:58.091711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"101'11","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171318+0000","last_change":"2024-06-03T02:14:09.750757+0000","last_active":"2024-06-03T02:17:22.171318+0000","last_peered":"2024-06-03T02:17:22.171318+0000","last_clean":"2024-06-03T02:17:22.171318+0000","last_became_active":"2024-06-03T02:14:09.750414+0000","last_became_peered":"2024-06-03T02:14:09.750414+0000","last_unstale":"2024-06-03T02:17:22.171318+0000","last_undegraded":"2024-06-03T02:17:22.171318+0000","last_fullsized":"2024-06-03T02:17:22.171318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:29:55.435408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.265175+0000","last_change":"2024-06-03T02:14:23.263440+0000","last_active":"2024-06-03T02:15:54.265175+0000","last_peered":"2024-06-03T02:15:54.265175+0000","last_clean":"2024-06-03T02:15:54.265175+0000","last_became_active":"2024-06-03T02:14:23.254019+0000","last_became_peered":"2024-06-03T02:14:23.254019+0000","last_unstale":"2024-06-03T02:15:54.265175+0000","last_undegraded":"2024-06-03T02:15:54.265175+0000","last_fullsized":"2024-06-03T02:15:54.265175+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:48:58.745990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'39","reported_seq":331,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:33.493334+0000","last_change":"2024-06-03T02:14:05.239353+0000","last_active":"2024-06-03T02:17:33.493334+0000","last_peered":"2024-06-03T02:17:33.493334+0000","last_clean":"2024-06-03T02:17:33.493334+0000","last_became_active":"2024-06-03T02:14:05.238976+0000","last_became_peered":"2024-06-03T02:14:05.238976+0000","last_unstale":"2024-06-03T02:17:33.493334+0000","last_undegraded":"2024-06-03T02:17:33.493334+0000","last_fullsized":"2024-06-03T02:17:33.493334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:51.598533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66728,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":188,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.138882+0000","last_change":"2024-06-03T02:14:09.746546+0000","last_active":"2024-06-03T02:17:22.138882+0000","last_peered":"2024-06-03T02:17:22.138882+0000","last_clean":"2024-06-03T02:17:22.138882+0000","last_became_active":"2024-06-03T02:14:09.746049+0000","last_became_peered":"2024-06-03T02:14:09.746049+0000","last_unstale":"2024-06-03T02:17:22.138882+0000","last_undegraded":"2024-06-03T02:17:22.138882+0000","last_fullsized":"2024-06-03T02:17:22.138882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:34.667133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340598+0000","last_change":"2024-06-03T02:14:23.257296+0000","last_active":"2024-06-03T02:15:54.340598+0000","last_peered":"2024-06-03T02:15:54.340598+0000","last_clean":"2024-06-03T02:15:54.340598+0000","last_became_active":"2024-06-03T02:14:23.256858+0000","last_became_peered":"2024-06-03T02:14:23.256858+0000","last_unstale":"2024-06-03T02:15:54.340598+0000","last_undegraded":"2024-06-03T02:15:54.340598+0000","last_fullsized":"2024-06-03T02:15:54.340598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:00.464902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"99'9","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271662+0000","last_change":"2024-06-03T02:14:05.245163+0000","last_active":"2024-06-03T02:15:54.271662+0000","last_peered":"2024-06-03T02:15:54.271662+0000","last_clean":"2024-06-03T02:15:54.271662+0000","last_became_active":"2024-06-03T02:14:05.244813+0000","last_became_peered":"2024-06-03T02:14:05.244813+0000","last_unstale":"2024-06-03T02:15:54.271662+0000","last_undegraded":"2024-06-03T02:15:54.271662+0000","last_fullsized":"2024-06-03T02:15:54.271662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:46:47.017859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181591+0000","last_change":"2024-06-03T02:14:09.757200+0000","last_active":"2024-06-03T02:17:22.181591+0000","last_peered":"2024-06-03T02:17:22.181591+0000","last_clean":"2024-06-03T02:17:22.181591+0000","last_became_active":"2024-06-03T02:14:09.756380+0000","last_became_peered":"2024-06-03T02:14:09.756380+0000","last_unstale":"2024-06-03T02:17:22.181591+0000","last_undegraded":"2024-06-03T02:17:22.181591+0000","last_fullsized":"2024-06-03T02:17:22.181591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:56.045338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272398+0000","last_change":"2024-06-03T02:14:23.258441+0000","last_active":"2024-06-03T02:15:54.272398+0000","last_peered":"2024-06-03T02:15:54.272398+0000","last_clean":"2024-06-03T02:15:54.272398+0000","last_became_active":"2024-06-03T02:14:23.258015+0000","last_became_peered":"2024-06-03T02:14:23.258015+0000","last_unstale":"2024-06-03T02:15:54.272398+0000","last_undegraded":"2024-06-03T02:15:54.272398+0000","last_fullsized":"2024-06-03T02:15:54.272398+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:03.123511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"99'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340165+0000","last_change":"2024-06-03T02:14:05.240456+0000","last_active":"2024-06-03T02:15:54.340165+0000","last_peered":"2024-06-03T02:15:54.340165+0000","last_clean":"2024-06-03T02:15:54.340165+0000","last_became_active":"2024-06-03T02:14:05.240078+0000","last_became_peered":"2024-06-03T02:14:05.240078+0000","last_unstale":"2024-06-03T02:15:54.340165+0000","last_undegraded":"2024-06-03T02:15:54.340165+0000","last_fullsized":"2024-06-03T02:15:54.340165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:00:26.491773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"101'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179593+0000","last_change":"2024-06-03T02:14:09.750876+0000","last_active":"2024-06-03T02:17:22.179593+0000","last_peered":"2024-06-03T02:17:22.179593+0000","last_clean":"2024-06-03T02:17:22.179593+0000","last_became_active":"2024-06-03T02:14:09.750547+0000","last_became_peered":"2024-06-03T02:14:09.750547+0000","last_unstale":"2024-06-03T02:17:22.179593+0000","last_undegraded":"2024-06-03T02:17:22.179593+0000","last_fullsized":"2024-06-03T02:17:22.179593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:08:41.136225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268761+0000","last_change":"2024-06-03T02:14:23.276172+0000","last_active":"2024-06-03T02:15:54.268761+0000","last_peered":"2024-06-03T02:15:54.268761+0000","last_clean":"2024-06-03T02:15:54.268761+0000","last_became_active":"2024-06-03T02:14:23.275410+0000","last_became_peered":"2024-06-03T02:14:23.275410+0000","last_unstale":"2024-06-03T02:15:54.268761+0000","last_undegraded":"2024-06-03T02:15:54.268761+0000","last_fullsized":"2024-06-03T02:15:54.268761+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:16:30.859453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"101'7","reported_seq":171,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:29.550293+0000","last_change":"2024-06-03T02:14:05.237263+0000","last_active":"2024-06-03T02:17:29.550293+0000","last_peered":"2024-06-03T02:17:29.550293+0000","last_clean":"2024-06-03T02:17:29.550293+0000","last_became_active":"2024-06-03T02:14:05.236890+0000","last_became_peered":"2024-06-03T02:14:05.236890+0000","last_unstale":"2024-06-03T02:17:29.550293+0000","last_undegraded":"2024-06-03T02:17:29.550293+0000","last_fullsized":"2024-06-03T02:17:29.550293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:42.292545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171473+0000","last_change":"2024-06-03T02:14:09.746260+0000","last_active":"2024-06-03T02:17:22.171473+0000","last_peered":"2024-06-03T02:17:22.171473+0000","last_clean":"2024-06-03T02:17:22.171473+0000","last_became_active":"2024-06-03T02:14:09.745940+0000","last_became_peered":"2024-06-03T02:14:09.745940+0000","last_unstale":"2024-06-03T02:17:22.171473+0000","last_undegraded":"2024-06-03T02:17:22.171473+0000","last_fullsized":"2024-06-03T02:17:22.171473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:08:55.851464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345467+0000","last_change":"2024-06-03T02:14:23.265440+0000","last_active":"2024-06-03T02:15:54.345467+0000","last_peered":"2024-06-03T02:15:54.345467+0000","last_clean":"2024-06-03T02:15:54.345467+0000","last_became_active":"2024-06-03T02:14:23.265028+0000","last_became_peered":"2024-06-03T02:14:23.265028+0000","last_unstale":"2024-06-03T02:15:54.345467+0000","last_undegraded":"2024-06-03T02:15:54.345467+0000","last_fullsized":"2024-06-03T02:15:54.345467+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:56.237500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"99'17","reported_seq":79,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270007+0000","last_change":"2024-06-03T02:14:53.271821+0000","last_active":"2024-06-03T02:15:54.270007+0000","last_peered":"2024-06-03T02:15:54.270007+0000","last_clean":"2024-06-03T02:15:54.270007+0000","last_became_active":"2024-06-03T02:14:53.263047+0000","last_became_peered":"2024-06-03T02:14:53.263047+0000","last_unstale":"2024-06-03T02:15:54.270007+0000","last_undegraded":"2024-06-03T02:15:54.270007+0000","last_fullsized":"2024-06-03T02:15:54.270007+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:02:25.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.148053+0000","last_change":"2024-06-03T02:14:09.754819+0000","last_active":"2024-06-03T02:17:22.148053+0000","last_peered":"2024-06-03T02:17:22.148053+0000","last_clean":"2024-06-03T02:17:22.148053+0000","last_became_active":"2024-06-03T02:14:09.753995+0000","last_became_peered":"2024-06-03T02:14:09.753995+0000","last_unstale":"2024-06-03T02:17:22.148053+0000","last_undegraded":"2024-06-03T02:17:22.148053+0000","last_fullsized":"2024-06-03T02:17:22.148053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:19:51.123774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340529+0000","last_change":"2024-06-03T02:14:23.265003+0000","last_active":"2024-06-03T02:15:54.340529+0000","last_peered":"2024-06-03T02:15:54.340529+0000","last_clean":"2024-06-03T02:15:54.340529+0000","last_became_active":"2024-06-03T02:14:23.264529+0000","last_became_peered":"2024-06-03T02:14:23.264529+0000","last_unstale":"2024-06-03T02:15:54.340529+0000","last_undegraded":"2024-06-03T02:15:54.340529+0000","last_fullsized":"2024-06-03T02:15:54.340529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:50:19.838274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270619+0000","last_change":"2024-06-03T02:14:05.236901+0000","last_active":"2024-06-03T02:15:54.270619+0000","last_peered":"2024-06-03T02:15:54.270619+0000","last_clean":"2024-06-03T02:15:54.270619+0000","last_became_active":"2024-06-03T02:14:05.236460+0000","last_became_peered":"2024-06-03T02:14:05.236460+0000","last_unstale":"2024-06-03T02:15:54.270619+0000","last_undegraded":"2024-06-03T02:15:54.270619+0000","last_fullsized":"2024-06-03T02:15:54.270619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:59.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.155988+0000","last_change":"2024-06-03T02:14:09.743135+0000","last_active":"2024-06-03T02:17:22.155988+0000","last_peered":"2024-06-03T02:17:22.155988+0000","last_clean":"2024-06-03T02:17:22.155988+0000","last_became_active":"2024-06-03T02:14:09.742506+0000","last_became_peered":"2024-06-03T02:14:09.742506+0000","last_unstale":"2024-06-03T02:17:22.155988+0000","last_undegraded":"2024-06-03T02:17:22.155988+0000","last_fullsized":"2024-06-03T02:17:22.155988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:51.413537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270403+0000","last_change":"2024-06-03T02:14:23.255512+0000","last_active":"2024-06-03T02:15:54.270403+0000","last_peered":"2024-06-03T02:15:54.270403+0000","last_clean":"2024-06-03T02:15:54.270403+0000","last_became_active":"2024-06-03T02:14:23.255103+0000","last_became_peered":"2024-06-03T02:14:23.255103+0000","last_unstale":"2024-06-03T02:15:54.270403+0000","last_undegraded":"2024-06-03T02:15:54.270403+0000","last_fullsized":"2024-06-03T02:15:54.270403+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:12:05.295848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"99'1","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266549+0000","last_change":"2024-06-03T02:14:05.248728+0000","last_active":"2024-06-03T02:15:54.266549+0000","last_peered":"2024-06-03T02:15:54.266549+0000","last_clean":"2024-06-03T02:15:54.266549+0000","last_became_active":"2024-06-03T02:14:05.248144+0000","last_became_peered":"2024-06-03T02:14:05.248144+0000","last_unstale":"2024-06-03T02:15:54.266549+0000","last_undegraded":"2024-06-03T02:15:54.266549+0000","last_fullsized":"2024-06-03T02:15:54.266549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:53:27.814443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"101'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184758+0000","last_change":"2024-06-03T02:14:09.752029+0000","last_active":"2024-06-03T02:17:22.184758+0000","last_peered":"2024-06-03T02:17:22.184758+0000","last_clean":"2024-06-03T02:17:22.184758+0000","last_became_active":"2024-06-03T02:14:09.751638+0000","last_became_peered":"2024-06-03T02:14:09.751638+0000","last_unstale":"2024-06-03T02:17:22.184758+0000","last_undegraded":"2024-06-03T02:17:22.184758+0000","last_fullsized":"2024-06-03T02:17:22.184758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:51.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.346566+0000","last_change":"2024-06-03T02:14:23.250406+0000","last_active":"2024-06-03T02:15:54.346566+0000","last_peered":"2024-06-03T02:15:54.346566+0000","last_clean":"2024-06-03T02:15:54.346566+0000","last_became_active":"2024-06-03T02:14:23.249946+0000","last_became_peered":"2024-06-03T02:14:23.249946+0000","last_unstale":"2024-06-03T02:15:54.346566+0000","last_undegraded":"2024-06-03T02:15:54.346566+0000","last_fullsized":"2024-06-03T02:15:54.346566+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:04:47.714532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"101'7","reported_seq":171,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:30.580645+0000","last_change":"2024-06-03T02:14:05.241666+0000","last_active":"2024-06-03T02:17:30.580645+0000","last_peered":"2024-06-03T02:17:30.580645+0000","last_clean":"2024-06-03T02:17:30.580645+0000","last_became_active":"2024-06-03T02:14:05.241299+0000","last_became_peered":"2024-06-03T02:14:05.241299+0000","last_unstale":"2024-06-03T02:17:30.580645+0000","last_undegraded":"2024-06-03T02:17:30.580645+0000","last_fullsized":"2024-06-03T02:17:30.580645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:21:16.792022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.176162+0000","last_change":"2024-06-03T02:14:09.768602+0000","last_active":"2024-06-03T02:17:22.176162+0000","last_peered":"2024-06-03T02:17:22.176162+0000","last_clean":"2024-06-03T02:17:22.176162+0000","last_became_active":"2024-06-03T02:14:09.768195+0000","last_became_peered":"2024-06-03T02:14:09.768195+0000","last_unstale":"2024-06-03T02:17:22.176162+0000","last_undegraded":"2024-06-03T02:17:22.176162+0000","last_fullsized":"2024-06-03T02:17:22.176162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:45:42.361966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.346730+0000","last_change":"2024-06-03T02:14:23.252472+0000","last_active":"2024-06-03T02:15:54.346730+0000","last_peered":"2024-06-03T02:15:54.346730+0000","last_clean":"2024-06-03T02:15:54.346730+0000","last_became_active":"2024-06-03T02:14:23.252018+0000","last_became_peered":"2024-06-03T02:14:23.252018+0000","last_unstale":"2024-06-03T02:15:54.346730+0000","last_undegraded":"2024-06-03T02:15:54.346730+0000","last_fullsized":"2024-06-03T02:15:54.346730+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:28.468657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163069+0000","last_change":"2024-06-03T02:14:09.744443+0000","last_active":"2024-06-03T02:17:22.163069+0000","last_peered":"2024-06-03T02:17:22.163069+0000","last_clean":"2024-06-03T02:17:22.163069+0000","last_became_active":"2024-06-03T02:14:09.744100+0000","last_became_peered":"2024-06-03T02:14:09.744100+0000","last_unstale":"2024-06-03T02:17:22.163069+0000","last_undegraded":"2024-06-03T02:17:22.163069+0000","last_fullsized":"2024-06-03T02:17:22.163069+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:44.566745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"101'6","reported_seq":167,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:29.549695+0000","last_change":"2024-06-03T02:14:05.250734+0000","last_active":"2024-06-03T02:17:29.549695+0000","last_peered":"2024-06-03T02:17:29.549695+0000","last_clean":"2024-06-03T02:17:29.549695+0000","last_became_active":"2024-06-03T02:14:05.250300+0000","last_became_peered":"2024-06-03T02:14:05.250300+0000","last_unstale":"2024-06-03T02:17:29.549695+0000","last_undegraded":"2024-06-03T02:17:29.549695+0000","last_fullsized":"2024-06-03T02:17:29.549695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:52.322895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341486+0000","last_change":"2024-06-03T02:14:53.471397+0000","last_active":"2024-06-03T02:15:54.341486+0000","last_peered":"2024-06-03T02:15:54.341486+0000","last_clean":"2024-06-03T02:15:54.341486+0000","last_became_active":"2024-06-03T02:14:53.470868+0000","last_became_peered":"2024-06-03T02:14:53.470868+0000","last_unstale":"2024-06-03T02:15:54.341486+0000","last_undegraded":"2024-06-03T02:15:54.341486+0000","last_fullsized":"2024-06-03T02:15:54.341486+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:58.726070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188094+0000","last_change":"2024-06-03T02:14:09.765193+0000","last_active":"2024-06-03T02:17:22.188094+0000","last_peered":"2024-06-03T02:17:22.188094+0000","last_clean":"2024-06-03T02:17:22.188094+0000","last_became_active":"2024-06-03T02:14:09.764834+0000","last_became_peered":"2024-06-03T02:14:09.764834+0000","last_unstale":"2024-06-03T02:17:22.188094+0000","last_undegraded":"2024-06-03T02:17:22.188094+0000","last_fullsized":"2024-06-03T02:17:22.188094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:20:37.464286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"99'6","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340500+0000","last_change":"2024-06-03T02:14:05.240118+0000","last_active":"2024-06-03T02:15:54.340500+0000","last_peered":"2024-06-03T02:15:54.340500+0000","last_clean":"2024-06-03T02:15:54.340500+0000","last_became_active":"2024-06-03T02:14:05.239771+0000","last_became_peered":"2024-06-03T02:14:05.239771+0000","last_unstale":"2024-06-03T02:15:54.340500+0000","last_undegraded":"2024-06-03T02:15:54.340500+0000","last_fullsized":"2024-06-03T02:15:54.340500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:08.256222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":371,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340279+0000","last_change":"2024-06-03T02:12:31.842613+0000","last_active":"2024-06-03T02:15:54.340279+0000","last_peered":"2024-06-03T02:15:54.340279+0000","last_clean":"2024-06-03T02:15:54.340279+0000","last_became_active":"2024-06-03T02:12:31.842051+0000","last_became_peered":"2024-06-03T02:12:31.842051+0000","last_unstale":"2024-06-03T02:15:54.340279+0000","last_undegraded":"2024-06-03T02:15:54.340279+0000","last_fullsized":"2024-06-03T02:15:54.340279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_clean_scrub_stamp":"2024-06-03T02:08:38.749702+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:40:29.748789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340391+0000","last_change":"2024-06-03T02:14:23.255849+0000","last_active":"2024-06-03T02:15:54.340391+0000","last_peered":"2024-06-03T02:15:54.340391+0000","last_clean":"2024-06-03T02:15:54.340391+0000","last_became_active":"2024-06-03T02:14:23.255380+0000","last_became_peered":"2024-06-03T02:14:23.255380+0000","last_unstale":"2024-06-03T02:15:54.340391+0000","last_undegraded":"2024-06-03T02:15:54.340391+0000","last_fullsized":"2024-06-03T02:15:54.340391+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:41:16.051326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"101'5","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.169445+0000","last_change":"2024-06-03T02:14:53.270423+0000","last_active":"2024-06-03T02:17:22.169445+0000","last_peered":"2024-06-03T02:17:22.169445+0000","last_clean":"2024-06-03T02:17:22.169445+0000","last_became_active":"2024-06-03T02:14:53.269260+0000","last_became_peered":"2024-06-03T02:14:53.269260+0000","last_unstale":"2024-06-03T02:17:22.169445+0000","last_undegraded":"2024-06-03T02:17:22.169445+0000","last_fullsized":"2024-06-03T02:17:22.169445+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:57.392012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"99'4","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341175+0000","last_change":"2024-06-03T02:14:05.235982+0000","last_active":"2024-06-03T02:15:54.341175+0000","last_peered":"2024-06-03T02:15:54.341175+0000","last_clean":"2024-06-03T02:15:54.341175+0000","last_became_active":"2024-06-03T02:14:05.235638+0000","last_became_peered":"2024-06-03T02:14:05.235638+0000","last_unstale":"2024-06-03T02:15:54.341175+0000","last_undegraded":"2024-06-03T02:15:54.341175+0000","last_fullsized":"2024-06-03T02:15:54.341175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:42:07.544005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271863+0000","last_change":"2024-06-03T02:14:23.257790+0000","last_active":"2024-06-03T02:15:54.271863+0000","last_peered":"2024-06-03T02:15:54.271863+0000","last_clean":"2024-06-03T02:15:54.271863+0000","last_became_active":"2024-06-03T02:14:23.257297+0000","last_became_peered":"2024-06-03T02:14:23.257297+0000","last_unstale":"2024-06-03T02:15:54.271863+0000","last_undegraded":"2024-06-03T02:15:54.271863+0000","last_fullsized":"2024-06-03T02:15:54.271863+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:13.715455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"99'4","reported_seq":21,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272792+0000","last_change":"2024-06-03T02:14:53.268495+0000","last_active":"2024-06-03T02:15:54.272792+0000","last_peered":"2024-06-03T02:15:54.272792+0000","last_clean":"2024-06-03T02:15:54.272792+0000","last_became_active":"2024-06-03T02:14:53.268147+0000","last_became_peered":"2024-06-03T02:14:53.268147+0000","last_unstale":"2024-06-03T02:15:54.272792+0000","last_undegraded":"2024-06-03T02:15:54.272792+0000","last_fullsized":"2024-06-03T02:15:54.272792+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:23.076319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.169542+0000","last_change":"2024-06-03T02:14:09.758244+0000","last_active":"2024-06-03T02:17:22.169542+0000","last_peered":"2024-06-03T02:17:22.169542+0000","last_clean":"2024-06-03T02:17:22.169542+0000","last_became_active":"2024-06-03T02:14:09.757812+0000","last_became_peered":"2024-06-03T02:14:09.757812+0000","last_unstale":"2024-06-03T02:17:22.169542+0000","last_undegraded":"2024-06-03T02:17:22.169542+0000","last_fullsized":"2024-06-03T02:17:22.169542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:08.243703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266267+0000","last_change":"2024-06-03T02:14:23.259980+0000","last_active":"2024-06-03T02:15:54.266267+0000","last_peered":"2024-06-03T02:15:54.266267+0000","last_clean":"2024-06-03T02:15:54.266267+0000","last_became_active":"2024-06-03T02:14:23.258308+0000","last_became_peered":"2024-06-03T02:14:23.258308+0000","last_unstale":"2024-06-03T02:15:54.266267+0000","last_undegraded":"2024-06-03T02:15:54.266267+0000","last_fullsized":"2024-06-03T02:15:54.266267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:28.833994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'17","reported_seq":62,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:16.235959+0000","last_change":"2024-06-03T02:14:05.244855+0000","last_active":"2024-06-03T02:16:16.235959+0000","last_peered":"2024-06-03T02:16:16.235959+0000","last_clean":"2024-06-03T02:16:16.235959+0000","last_became_active":"2024-06-03T02:14:05.244335+0000","last_became_peered":"2024-06-03T02:14:05.244335+0000","last_unstale":"2024-06-03T02:16:16.235959+0000","last_undegraded":"2024-06-03T02:16:16.235959+0000","last_fullsized":"2024-06-03T02:16:16.235959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:53:19.065411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185762+0000","last_change":"2024-06-03T02:14:09.750761+0000","last_active":"2024-06-03T02:17:22.185762+0000","last_peered":"2024-06-03T02:17:22.185762+0000","last_clean":"2024-06-03T02:17:22.185762+0000","last_became_active":"2024-06-03T02:14:09.750336+0000","last_became_peered":"2024-06-03T02:14:09.750336+0000","last_unstale":"2024-06-03T02:17:22.185762+0000","last_undegraded":"2024-06-03T02:17:22.185762+0000","last_fullsized":"2024-06-03T02:17:22.185762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:59:46.651149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271019+0000","last_change":"2024-06-03T02:14:23.243287+0000","last_active":"2024-06-03T02:15:54.271019+0000","last_peered":"2024-06-03T02:15:54.271019+0000","last_clean":"2024-06-03T02:15:54.271019+0000","last_became_active":"2024-06-03T02:14:23.242828+0000","last_became_peered":"2024-06-03T02:14:23.242828+0000","last_unstale":"2024-06-03T02:15:54.271019+0000","last_undegraded":"2024-06-03T02:15:54.271019+0000","last_fullsized":"2024-06-03T02:15:54.271019+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:12.312438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"99'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341908+0000","last_change":"2024-06-03T02:14:05.596288+0000","last_active":"2024-06-03T02:15:54.341908+0000","last_peered":"2024-06-03T02:15:54.341908+0000","last_clean":"2024-06-03T02:15:54.341908+0000","last_became_active":"2024-06-03T02:14:05.595868+0000","last_became_peered":"2024-06-03T02:14:05.595868+0000","last_unstale":"2024-06-03T02:15:54.341908+0000","last_undegraded":"2024-06-03T02:15:54.341908+0000","last_fullsized":"2024-06-03T02:15:54.341908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:50.291541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.189215+0000","last_change":"2024-06-03T02:14:09.765317+0000","last_active":"2024-06-03T02:17:22.189215+0000","last_peered":"2024-06-03T02:17:22.189215+0000","last_clean":"2024-06-03T02:17:22.189215+0000","last_became_active":"2024-06-03T02:14:09.764983+0000","last_became_peered":"2024-06-03T02:14:09.764983+0000","last_unstale":"2024-06-03T02:17:22.189215+0000","last_undegraded":"2024-06-03T02:17:22.189215+0000","last_fullsized":"2024-06-03T02:17:22.189215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:20.075276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341648+0000","last_change":"2024-06-03T02:14:23.265352+0000","last_active":"2024-06-03T02:15:54.341648+0000","last_peered":"2024-06-03T02:15:54.341648+0000","last_clean":"2024-06-03T02:15:54.341648+0000","last_became_active":"2024-06-03T02:14:23.264932+0000","last_became_peered":"2024-06-03T02:14:23.264932+0000","last_unstale":"2024-06-03T02:15:54.341648+0000","last_undegraded":"2024-06-03T02:15:54.341648+0000","last_fullsized":"2024-06-03T02:15:54.341648+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:43.803170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"99'2","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339982+0000","last_change":"2024-06-03T02:14:05.248563+0000","last_active":"2024-06-03T02:15:54.339982+0000","last_peered":"2024-06-03T02:15:54.339982+0000","last_clean":"2024-06-03T02:15:54.339982+0000","last_became_active":"2024-06-03T02:14:05.248306+0000","last_became_peered":"2024-06-03T02:14:05.248306+0000","last_unstale":"2024-06-03T02:15:54.339982+0000","last_undegraded":"2024-06-03T02:15:54.339982+0000","last_fullsized":"2024-06-03T02:15:54.339982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:23:58.130157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"101'2","reported_seq":41,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.123863+0000","last_change":"2024-06-03T02:14:09.745948+0000","last_active":"2024-06-03T02:17:22.123863+0000","last_peered":"2024-06-03T02:17:22.123863+0000","last_clean":"2024-06-03T02:17:22.123863+0000","last_became_active":"2024-06-03T02:14:09.745626+0000","last_became_peered":"2024-06-03T02:14:09.745626+0000","last_unstale":"2024-06-03T02:17:22.123863+0000","last_undegraded":"2024-06-03T02:17:22.123863+0000","last_fullsized":"2024-06-03T02:17:22.123863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:34.742129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342659+0000","last_change":"2024-06-03T02:15:54.341824+0000","last_active":"2024-06-03T02:15:54.342659+0000","last_peered":"2024-06-03T02:15:54.342659+0000","last_clean":"2024-06-03T02:15:54.342659+0000","last_became_active":"2024-06-03T02:15:54.341267+0000","last_became_peered":"2024-06-03T02:15:54.341267+0000","last_unstale":"2024-06-03T02:15:54.342659+0000","last_undegraded":"2024-06-03T02:15:54.342659+0000","last_fullsized":"2024-06-03T02:15:54.342659+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:47.714056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"99'7","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267775+0000","last_change":"2024-06-03T02:14:05.244806+0000","last_active":"2024-06-03T02:15:54.267775+0000","last_peered":"2024-06-03T02:15:54.267775+0000","last_clean":"2024-06-03T02:15:54.267775+0000","last_became_active":"2024-06-03T02:14:05.242719+0000","last_became_peered":"2024-06-03T02:14:05.242719+0000","last_unstale":"2024-06-03T02:15:54.267775+0000","last_undegraded":"2024-06-03T02:15:54.267775+0000","last_fullsized":"2024-06-03T02:15:54.267775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:31.314266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.176658+0000","last_change":"2024-06-03T02:14:09.755102+0000","last_active":"2024-06-03T02:17:22.176658+0000","last_peered":"2024-06-03T02:17:22.176658+0000","last_clean":"2024-06-03T02:17:22.176658+0000","last_became_active":"2024-06-03T02:14:09.754748+0000","last_became_peered":"2024-06-03T02:14:09.754748+0000","last_unstale":"2024-06-03T02:17:22.176658+0000","last_undegraded":"2024-06-03T02:17:22.176658+0000","last_fullsized":"2024-06-03T02:17:22.176658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:59.920710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"101'1","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:55.297902+0000","last_change":"2024-06-03T02:14:23.271974+0000","last_active":"2024-06-03T02:15:55.297902+0000","last_peered":"2024-06-03T02:15:55.297902+0000","last_clean":"2024-06-03T02:15:55.297902+0000","last_became_active":"2024-06-03T02:14:23.271470+0000","last_became_peered":"2024-06-03T02:14:23.271470+0000","last_unstale":"2024-06-03T02:15:55.297902+0000","last_undegraded":"2024-06-03T02:15:55.297902+0000","last_fullsized":"2024-06-03T02:15:55.297902+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:03.159351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"99'3","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271461+0000","last_change":"2024-06-03T02:14:05.242261+0000","last_active":"2024-06-03T02:15:54.271461+0000","last_peered":"2024-06-03T02:15:54.271461+0000","last_clean":"2024-06-03T02:15:54.271461+0000","last_became_active":"2024-06-03T02:14:05.241854+0000","last_became_peered":"2024-06-03T02:14:05.241854+0000","last_unstale":"2024-06-03T02:15:54.271461+0000","last_undegraded":"2024-06-03T02:15:54.271461+0000","last_fullsized":"2024-06-03T02:15:54.271461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:21.427703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185157+0000","last_change":"2024-06-03T02:14:09.763591+0000","last_active":"2024-06-03T02:17:22.185157+0000","last_peered":"2024-06-03T02:17:22.185157+0000","last_clean":"2024-06-03T02:17:22.185157+0000","last_became_active":"2024-06-03T02:14:09.763240+0000","last_became_peered":"2024-06-03T02:14:09.763240+0000","last_unstale":"2024-06-03T02:17:22.185157+0000","last_undegraded":"2024-06-03T02:17:22.185157+0000","last_fullsized":"2024-06-03T02:17:22.185157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:40.146898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341606+0000","last_change":"2024-06-03T02:14:23.265011+0000","last_active":"2024-06-03T02:15:54.341606+0000","last_peered":"2024-06-03T02:15:54.341606+0000","last_clean":"2024-06-03T02:15:54.341606+0000","last_became_active":"2024-06-03T02:14:23.263767+0000","last_became_peered":"2024-06-03T02:14:23.263767+0000","last_unstale":"2024-06-03T02:15:54.341606+0000","last_undegraded":"2024-06-03T02:15:54.341606+0000","last_fullsized":"2024-06-03T02:15:54.341606+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:34:04.274545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"99'4","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270508+0000","last_change":"2024-06-03T02:14:53.474879+0000","last_active":"2024-06-03T02:15:54.270508+0000","last_peered":"2024-06-03T02:15:54.270508+0000","last_clean":"2024-06-03T02:15:54.270508+0000","last_became_active":"2024-06-03T02:14:53.472209+0000","last_became_peered":"2024-06-03T02:14:53.472209+0000","last_unstale":"2024-06-03T02:15:54.270508+0000","last_undegraded":"2024-06-03T02:15:54.270508+0000","last_fullsized":"2024-06-03T02:15:54.270508+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:36:57.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172586+0000","last_change":"2024-06-03T02:14:09.747756+0000","last_active":"2024-06-03T02:17:22.172586+0000","last_peered":"2024-06-03T02:17:22.172586+0000","last_clean":"2024-06-03T02:17:22.172586+0000","last_became_active":"2024-06-03T02:14:09.747453+0000","last_became_peered":"2024-06-03T02:14:09.747453+0000","last_unstale":"2024-06-03T02:17:22.172586+0000","last_undegraded":"2024-06-03T02:17:22.172586+0000","last_fullsized":"2024-06-03T02:17:22.172586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:19:02.341367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339730+0000","last_change":"2024-06-03T02:14:23.248594+0000","last_active":"2024-06-03T02:15:54.339730+0000","last_peered":"2024-06-03T02:15:54.339730+0000","last_clean":"2024-06-03T02:15:54.339730+0000","last_became_active":"2024-06-03T02:14:23.248080+0000","last_became_peered":"2024-06-03T02:14:23.248080+0000","last_unstale":"2024-06-03T02:15:54.339730+0000","last_undegraded":"2024-06-03T02:15:54.339730+0000","last_fullsized":"2024-06-03T02:15:54.339730+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:24:42.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"99'5","reported_seq":168,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:30.489719+0000","last_change":"2024-06-03T02:14:05.596274+0000","last_active":"2024-06-03T02:17:30.489719+0000","last_peered":"2024-06-03T02:17:30.489719+0000","last_clean":"2024-06-03T02:17:30.489719+0000","last_became_active":"2024-06-03T02:14:05.595678+0000","last_became_peered":"2024-06-03T02:14:05.595678+0000","last_unstale":"2024-06-03T02:17:30.489719+0000","last_undegraded":"2024-06-03T02:17:30.489719+0000","last_fullsized":"2024-06-03T02:17:30.489719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:13:25.436688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172440+0000","last_change":"2024-06-03T02:14:09.744934+0000","last_active":"2024-06-03T02:17:22.172440+0000","last_peered":"2024-06-03T02:17:22.172440+0000","last_clean":"2024-06-03T02:17:22.172440+0000","last_became_active":"2024-06-03T02:14:09.744621+0000","last_became_peered":"2024-06-03T02:14:09.744621+0000","last_unstale":"2024-06-03T02:17:22.172440+0000","last_undegraded":"2024-06-03T02:17:22.172440+0000","last_fullsized":"2024-06-03T02:17:22.172440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:01:51.970160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341842+0000","last_change":"2024-06-03T02:14:23.273123+0000","last_active":"2024-06-03T02:15:54.341842+0000","last_peered":"2024-06-03T02:15:54.341842+0000","last_clean":"2024-06-03T02:15:54.341842+0000","last_became_active":"2024-06-03T02:14:23.272153+0000","last_became_peered":"2024-06-03T02:14:23.272153+0000","last_unstale":"2024-06-03T02:15:54.341842+0000","last_undegraded":"2024-06-03T02:15:54.341842+0000","last_fullsized":"2024-06-03T02:15:54.341842+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:01:43.374413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"99'3","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340388+0000","last_change":"2024-06-03T02:14:05.245094+0000","last_active":"2024-06-03T02:15:54.340388+0000","last_peered":"2024-06-03T02:15:54.340388+0000","last_clean":"2024-06-03T02:15:54.340388+0000","last_became_active":"2024-06-03T02:14:05.244678+0000","last_became_peered":"2024-06-03T02:14:05.244678+0000","last_unstale":"2024-06-03T02:15:54.340388+0000","last_undegraded":"2024-06-03T02:15:54.340388+0000","last_fullsized":"2024-06-03T02:15:54.340388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:13:26.546135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184748+0000","last_change":"2024-06-03T02:14:09.745919+0000","last_active":"2024-06-03T02:17:22.184748+0000","last_peered":"2024-06-03T02:17:22.184748+0000","last_clean":"2024-06-03T02:17:22.184748+0000","last_became_active":"2024-06-03T02:14:09.745576+0000","last_became_peered":"2024-06-03T02:14:09.745576+0000","last_unstale":"2024-06-03T02:17:22.184748+0000","last_undegraded":"2024-06-03T02:17:22.184748+0000","last_fullsized":"2024-06-03T02:17:22.184748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:29:31.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339570+0000","last_change":"2024-06-03T02:14:23.244174+0000","last_active":"2024-06-03T02:15:54.339570+0000","last_peered":"2024-06-03T02:15:54.339570+0000","last_clean":"2024-06-03T02:15:54.339570+0000","last_became_active":"2024-06-03T02:14:23.243595+0000","last_became_peered":"2024-06-03T02:14:23.243595+0000","last_unstale":"2024-06-03T02:15:54.339570+0000","last_undegraded":"2024-06-03T02:15:54.339570+0000","last_fullsized":"2024-06-03T02:15:54.339570+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:12:36.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"99'5","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345783+0000","last_change":"2024-06-03T02:14:05.247669+0000","last_active":"2024-06-03T02:15:54.345783+0000","last_peered":"2024-06-03T02:15:54.345783+0000","last_clean":"2024-06-03T02:15:54.345783+0000","last_became_active":"2024-06-03T02:14:05.246685+0000","last_became_peered":"2024-06-03T02:14:05.246685+0000","last_unstale":"2024-06-03T02:15:54.345783+0000","last_undegraded":"2024-06-03T02:15:54.345783+0000","last_fullsized":"2024-06-03T02:15:54.345783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:39.454422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171553+0000","last_change":"2024-06-03T02:14:09.749242+0000","last_active":"2024-06-03T02:17:22.171553+0000","last_peered":"2024-06-03T02:17:22.171553+0000","last_clean":"2024-06-03T02:17:22.171553+0000","last_became_active":"2024-06-03T02:14:09.748370+0000","last_became_peered":"2024-06-03T02:14:09.748370+0000","last_unstale":"2024-06-03T02:17:22.171553+0000","last_undegraded":"2024-06-03T02:17:22.171553+0000","last_fullsized":"2024-06-03T02:17:22.171553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:26.189960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.161980+0000","last_change":"2024-06-03T02:14:09.745966+0000","last_active":"2024-06-03T02:17:22.161980+0000","last_peered":"2024-06-03T02:17:22.161980+0000","last_clean":"2024-06-03T02:17:22.161980+0000","last_became_active":"2024-06-03T02:14:09.745590+0000","last_became_peered":"2024-06-03T02:14:09.745590+0000","last_unstale":"2024-06-03T02:17:22.161980+0000","last_undegraded":"2024-06-03T02:17:22.161980+0000","last_fullsized":"2024-06-03T02:17:22.161980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:46.744629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"101'12","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163231+0000","last_change":"2024-06-03T02:14:09.744964+0000","last_active":"2024-06-03T02:17:22.163231+0000","last_peered":"2024-06-03T02:17:22.163231+0000","last_clean":"2024-06-03T02:17:22.163231+0000","last_became_active":"2024-06-03T02:14:09.743877+0000","last_became_peered":"2024-06-03T02:14:09.743877+0000","last_unstale":"2024-06-03T02:17:22.163231+0000","last_undegraded":"2024-06-03T02:17:22.163231+0000","last_fullsized":"2024-06-03T02:17:22.163231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:05:48.020670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186713+0000","last_change":"2024-06-03T02:14:09.761736+0000","last_active":"2024-06-03T02:17:22.186713+0000","last_peered":"2024-06-03T02:17:22.186713+0000","last_clean":"2024-06-03T02:17:22.186713+0000","last_became_active":"2024-06-03T02:14:09.761477+0000","last_became_peered":"2024-06-03T02:14:09.761477+0000","last_unstale":"2024-06-03T02:17:22.186713+0000","last_undegraded":"2024-06-03T02:17:22.186713+0000","last_fullsized":"2024-06-03T02:17:22.186713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:00:04.631330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"101'7","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183182+0000","last_change":"2024-06-03T02:14:53.266594+0000","last_active":"2024-06-03T02:17:22.183182+0000","last_peered":"2024-06-03T02:17:22.183182+0000","last_clean":"2024-06-03T02:17:22.183182+0000","last_became_active":"2024-06-03T02:14:53.266104+0000","last_became_peered":"2024-06-03T02:14:53.266104+0000","last_unstale":"2024-06-03T02:17:22.183182+0000","last_undegraded":"2024-06-03T02:17:22.183182+0000","last_fullsized":"2024-06-03T02:17:22.183182+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:12.018082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"101'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174788+0000","last_change":"2024-06-03T02:14:09.772332+0000","last_active":"2024-06-03T02:17:22.174788+0000","last_peered":"2024-06-03T02:17:22.174788+0000","last_clean":"2024-06-03T02:17:22.174788+0000","last_became_active":"2024-06-03T02:14:09.771888+0000","last_became_peered":"2024-06-03T02:14:09.771888+0000","last_unstale":"2024-06-03T02:17:22.174788+0000","last_undegraded":"2024-06-03T02:17:22.174788+0000","last_fullsized":"2024-06-03T02:17:22.174788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:58:49.414986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"101'6","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.156755+0000","last_change":"2024-06-03T02:14:53.270828+0000","last_active":"2024-06-03T02:17:22.156755+0000","last_peered":"2024-06-03T02:17:22.156755+0000","last_clean":"2024-06-03T02:17:22.156755+0000","last_became_active":"2024-06-03T02:14:53.270365+0000","last_became_peered":"2024-06-03T02:14:53.270365+0000","last_unstale":"2024-06-03T02:17:22.156755+0000","last_undegraded":"2024-06-03T02:17:22.156755+0000","last_fullsized":"2024-06-03T02:17:22.156755+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.051756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"101'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.161480+0000","last_change":"2024-06-03T02:14:09.747709+0000","last_active":"2024-06-03T02:17:22.161480+0000","last_peered":"2024-06-03T02:17:22.161480+0000","last_clean":"2024-06-03T02:17:22.161480+0000","last_became_active":"2024-06-03T02:14:09.747359+0000","last_became_peered":"2024-06-03T02:14:09.747359+0000","last_unstale":"2024-06-03T02:17:22.161480+0000","last_undegraded":"2024-06-03T02:17:22.161480+0000","last_fullsized":"2024-06-03T02:17:22.161480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:37:24.873442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"101'2","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.145262+0000","last_change":"2024-06-03T02:14:53.473136+0000","last_active":"2024-06-03T02:17:22.145262+0000","last_peered":"2024-06-03T02:17:22.145262+0000","last_clean":"2024-06-03T02:17:22.145262+0000","last_became_active":"2024-06-03T02:14:53.471214+0000","last_became_peered":"2024-06-03T02:14:53.471214+0000","last_unstale":"2024-06-03T02:17:22.145262+0000","last_undegraded":"2024-06-03T02:17:22.145262+0000","last_fullsized":"2024-06-03T02:17:22.145262+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:25.822042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184565+0000","last_change":"2024-06-03T02:14:09.746463+0000","last_active":"2024-06-03T02:17:22.184565+0000","last_peered":"2024-06-03T02:17:22.184565+0000","last_clean":"2024-06-03T02:17:22.184565+0000","last_became_active":"2024-06-03T02:14:09.746115+0000","last_became_peered":"2024-06-03T02:14:09.746115+0000","last_unstale":"2024-06-03T02:17:22.184565+0000","last_undegraded":"2024-06-03T02:17:22.184565+0000","last_fullsized":"2024-06-03T02:17:22.184565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:51:23.050739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184552+0000","last_change":"2024-06-03T02:14:09.755467+0000","last_active":"2024-06-03T02:17:22.184552+0000","last_peered":"2024-06-03T02:17:22.184552+0000","last_clean":"2024-06-03T02:17:22.184552+0000","last_became_active":"2024-06-03T02:14:09.755103+0000","last_became_peered":"2024-06-03T02:14:09.755103+0000","last_unstale":"2024-06-03T02:17:22.184552+0000","last_undegraded":"2024-06-03T02:17:22.184552+0000","last_fullsized":"2024-06-03T02:17:22.184552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:22.834519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"101'10","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185923+0000","last_change":"2024-06-03T02:14:09.758201+0000","last_active":"2024-06-03T02:17:22.185923+0000","last_peered":"2024-06-03T02:17:22.185923+0000","last_clean":"2024-06-03T02:17:22.185923+0000","last_became_active":"2024-06-03T02:14:09.757819+0000","last_became_peered":"2024-06-03T02:14:09.757819+0000","last_unstale":"2024-06-03T02:17:22.185923+0000","last_undegraded":"2024-06-03T02:17:22.185923+0000","last_fullsized":"2024-06-03T02:17:22.185923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:44.802694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"101'11","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183299+0000","last_change":"2024-06-03T02:14:09.768132+0000","last_active":"2024-06-03T02:17:22.183299+0000","last_peered":"2024-06-03T02:17:22.183299+0000","last_clean":"2024-06-03T02:17:22.183299+0000","last_became_active":"2024-06-03T02:14:09.767642+0000","last_became_peered":"2024-06-03T02:14:09.767642+0000","last_unstale":"2024-06-03T02:17:22.183299+0000","last_undegraded":"2024-06-03T02:17:22.183299+0000","last_fullsized":"2024-06-03T02:17:22.183299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:32:30.455418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.159027+0000","last_change":"2024-06-03T02:14:09.745603+0000","last_active":"2024-06-03T02:17:22.159027+0000","last_peered":"2024-06-03T02:17:22.159027+0000","last_clean":"2024-06-03T02:17:22.159027+0000","last_became_active":"2024-06-03T02:14:09.745245+0000","last_became_peered":"2024-06-03T02:14:09.745245+0000","last_unstale":"2024-06-03T02:17:22.159027+0000","last_undegraded":"2024-06-03T02:17:22.159027+0000","last_fullsized":"2024-06-03T02:17:22.159027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:09:14.507101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74743,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1212,"num_read_kb":690,"num_write":263,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":224229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318332,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":578717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37164,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707988,"statfs":{"total":95995035648,"available":95956979712,"internally_reserved":0,"allocated":1871872,"data_stored":1236194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":577953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005131,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":577637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266769,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1785856,"data_stored":1176030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430295,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":573988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1249280,"data_stored":641858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":574913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28240,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716912,"statfs":{"total":95995035648,"available":95966117888,"internally_reserved":0,"allocated":1253376,"data_stored":641854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149798,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888161,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":578062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691933,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37088,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708064,"statfs":{"total":95995035648,"available":95957057536,"internally_reserved":0,"allocated":1794048,"data_stored":1170550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T02:17:35.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T02:17:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:35 smithi135 ceph-mon[36653]: from='client.24929 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:17:35.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:35 smithi123 ceph-mon[36640]: from='client.24929 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:17:36.003 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:36.506 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:36 smithi079 ceph-mon[31819]: pgmap v443: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-03T02:17:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:36 smithi135 ceph-mon[36653]: pgmap v443: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-03T02:17:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:36 smithi123 ceph-mon[36640]: pgmap v443: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-03T02:17:38.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.168+0000 7fb3eb200700 1 -- 172.21.15.79:0/1410778021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4105040 msgr2=0x7fb3e4111550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:38.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.168+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1410778021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4105040 0x7fb3e4111550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00b0a0 tx=0x7fb3dc00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.168+0000 7fb3eb200700 1 -- 172.21.15.79:0/1410778021 shutdown_connections 2024-06-03T02:17:38.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.168+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1410778021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4105040 0x7fb3e4111550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.168+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1410778021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e410a680 0x7fb3e410aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.168+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1410778021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e4104680 0x7fb3e4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.168+0000 7fb3eb200700 1 -- 172.21.15.79:0/1410778021 >> 172.21.15.79:0/1410778021 conn(0x7fb3e4100170 msgr2=0x7fb3e4102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:38.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.168+0000 7fb3eb200700 1 -- 172.21.15.79:0/1410778021 shutdown_connections 2024-06-03T02:17:38.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.168+0000 7fb3eb200700 1 -- 172.21.15.79:0/1410778021 wait complete. 2024-06-03T02:17:38.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.169+0000 7fb3eb200700 1 Processor -- start 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3eb200700 1 -- start start 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3eb200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4104680 0x7fb3e419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3eb200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e4105040 0x7fb3e419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3eb200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e410a680 0x7fb3e419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3eb200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4114a10 con 0x7fb3e4104680 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3eb200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4114890 con 0x7fb3e410a680 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3eb200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4114b90 con 0x7fb3e4105040 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3e3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4104680 0x7fb3e419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3e3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4104680 0x7fb3e419ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54392/0 (socket says 172.21.15.79:54392) 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3e3e00700 1 -- 172.21.15.79:0/1652051700 learned_addr learned my addr 172.21.15.79:0/1652051700 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.170+0000 7fb3dbe00700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e4105040 0x7fb3e419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3e9200700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e410a680 0x7fb3e419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3e3e00700 1 -- 172.21.15.79:0/1652051700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e4105040 msgr2=0x7fb3e419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3e3e00700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e4105040 0x7fb3e419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3e3e00700 1 -- 172.21.15.79:0/1652051700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e410a680 msgr2=0x7fb3e419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3e3e00700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e410a680 0x7fb3e419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3e3e00700 1 -- 172.21.15.79:0/1652051700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e419fd70 con 0x7fb3e4104680 2024-06-03T02:17:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3e3e00700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4104680 0x7fb3e419ad60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400e9a0 tx=0x7fb3d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3e1600700 1 -- 172.21.15.79:0/1652051700 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d400ca10 con 0x7fb3e4104680 2024-06-03T02:17:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e41a0060 con 0x7fb3e4104680 2024-06-03T02:17:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.171+0000 7fb3e1600700 1 -- 172.21.15.79:0/1652051700 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d400cbb0 con 0x7fb3e4104680 2024-06-03T02:17:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.172+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e41a78e0 con 0x7fb3e4104680 2024-06-03T02:17:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.172+0000 7fb3e1600700 1 -- 172.21.15.79:0/1652051700 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4010430 con 0x7fb3e4104680 2024-06-03T02:17:38.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.173+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e4069180 con 0x7fb3e4104680 2024-06-03T02:17:38.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.175+0000 7fb3e1600700 1 -- 172.21.15.79:0/1652051700 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb3d40105d0 con 0x7fb3e4104680 2024-06-03T02:17:38.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.176+0000 7fb3e1600700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb3c80a2c20 0x7fb3c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:38.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.176+0000 7fb3e1600700 1 -- 172.21.15.79:0/1652051700 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb3d40cfb90 con 0x7fb3e4104680 2024-06-03T02:17:38.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.176+0000 7fb3dbe00700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb3c80a2c20 0x7fb3c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:38.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.177+0000 7fb3dbe00700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb3c80a2c20 0x7fb3c80a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4109b60 tx=0x7fb3cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:38.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.180+0000 7fb3e1600700 1 -- 172.21.15.79:0/1652051700 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb3d409c740 con 0x7fb3e4104680 2024-06-03T02:17:38.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.401+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3e4100d00 con 0x7fb3c80a2c20 2024-06-03T02:17:38.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.416+0000 7fb3e1600700 1 -- 172.21.15.79:0/1652051700 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340740 (secure 0 0 0) 0x7fb3e4100d00 con 0x7fb3c80a2c20 2024-06-03T02:17:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:17:38.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.424+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb3c80a2c20 msgr2=0x7fb3c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:38.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.424+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb3c80a2c20 0x7fb3c80a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4109b60 tx=0x7fb3cc009260 comp rx=0 tx=0).stop 2024-06-03T02:17:38.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.424+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4104680 msgr2=0x7fb3e419ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.424+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4104680 0x7fb3e419ad60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400e9a0 tx=0x7fb3d400ecb0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.425+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 shutdown_connections 2024-06-03T02:17:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.425+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e410a680 0x7fb3e419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.425+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb3c80a2c20 0x7fb3c80a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.425+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4104680 0x7fb3e419ad60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.425+0000 7fb3eb200700 1 --2- 172.21.15.79:0/1652051700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3e4105040 0x7fb3e419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.425+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 >> 172.21.15.79:0/1652051700 conn(0x7fb3e4100170 msgr2=0x7fb3e41005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.425+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 shutdown_connections 2024-06-03T02:17:38.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:38.425+0000 7fb3eb200700 1 -- 172.21.15.79:0/1652051700 wait complete. 2024-06-03T02:17:38.429 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-06-03T02:17:38.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:38 smithi079 ceph-mon[31819]: pgmap v444: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-03T02:17:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:38 smithi135 ceph-mon[36653]: pgmap v444: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-03T02:17:38.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:38 smithi123 ceph-mon[36640]: pgmap v444: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-03T02:17:39.306 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-06-03T02:17:38.248591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665345,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1368,"num_read_kb":809,"num_write":446,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360828,"kb_used_data":15836,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580996,"statfs":{"total":1151940427776,"available":1151570939904,"internally_reserved":0,"allocated":16216064,"data_stored":8900921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":57,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005455"},"pg_stats":[{"pgid":"3.3f","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186075+0000","last_change":"2024-06-03T02:14:09.768431+0000","last_active":"2024-06-03T02:17:22.186075+0000","last_peered":"2024-06-03T02:17:22.186075+0000","last_clean":"2024-06-03T02:17:22.186075+0000","last_became_active":"2024-06-03T02:14:09.768059+0000","last_became_peered":"2024-06-03T02:14:09.768059+0000","last_unstale":"2024-06-03T02:17:22.186075+0000","last_undegraded":"2024-06-03T02:17:22.186075+0000","last_fullsized":"2024-06-03T02:17:22.186075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:12.265896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"101'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187232+0000","last_change":"2024-06-03T02:14:09.771842+0000","last_active":"2024-06-03T02:17:22.187232+0000","last_peered":"2024-06-03T02:17:22.187232+0000","last_clean":"2024-06-03T02:17:22.187232+0000","last_became_active":"2024-06-03T02:14:09.770961+0000","last_became_peered":"2024-06-03T02:14:09.770961+0000","last_unstale":"2024-06-03T02:17:22.187232+0000","last_undegraded":"2024-06-03T02:17:22.187232+0000","last_fullsized":"2024-06-03T02:17:22.187232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:49:02.297128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.182425+0000","last_change":"2024-06-03T02:14:09.758407+0000","last_active":"2024-06-03T02:17:22.182425+0000","last_peered":"2024-06-03T02:17:22.182425+0000","last_clean":"2024-06-03T02:17:22.182425+0000","last_became_active":"2024-06-03T02:14:09.758045+0000","last_became_peered":"2024-06-03T02:14:09.758045+0000","last_unstale":"2024-06-03T02:17:22.182425+0000","last_undegraded":"2024-06-03T02:17:22.182425+0000","last_fullsized":"2024-06-03T02:17:22.182425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:01:08.166971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.151507+0000","last_change":"2024-06-03T02:14:09.747658+0000","last_active":"2024-06-03T02:17:22.151507+0000","last_peered":"2024-06-03T02:17:22.151507+0000","last_clean":"2024-06-03T02:17:22.151507+0000","last_became_active":"2024-06-03T02:14:09.747321+0000","last_became_peered":"2024-06-03T02:14:09.747321+0000","last_unstale":"2024-06-03T02:17:22.151507+0000","last_undegraded":"2024-06-03T02:17:22.151507+0000","last_fullsized":"2024-06-03T02:17:22.151507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:08.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"101'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184802+0000","last_change":"2024-06-03T02:14:09.758161+0000","last_active":"2024-06-03T02:17:22.184802+0000","last_peered":"2024-06-03T02:17:22.184802+0000","last_clean":"2024-06-03T02:17:22.184802+0000","last_became_active":"2024-06-03T02:14:09.757800+0000","last_became_peered":"2024-06-03T02:14:09.757800+0000","last_unstale":"2024-06-03T02:17:22.184802+0000","last_undegraded":"2024-06-03T02:17:22.184802+0000","last_fullsized":"2024-06-03T02:17:22.184802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:47.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.160331+0000","last_change":"2024-06-03T02:14:09.741818+0000","last_active":"2024-06-03T02:17:22.160331+0000","last_peered":"2024-06-03T02:17:22.160331+0000","last_clean":"2024-06-03T02:17:22.160331+0000","last_became_active":"2024-06-03T02:14:09.741219+0000","last_became_peered":"2024-06-03T02:14:09.741219+0000","last_unstale":"2024-06-03T02:17:22.160331+0000","last_undegraded":"2024-06-03T02:17:22.160331+0000","last_fullsized":"2024-06-03T02:17:22.160331+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:16:30.585245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.159623+0000","last_change":"2024-06-03T02:14:09.744863+0000","last_active":"2024-06-03T02:17:22.159623+0000","last_peered":"2024-06-03T02:17:22.159623+0000","last_clean":"2024-06-03T02:17:22.159623+0000","last_became_active":"2024-06-03T02:14:09.744514+0000","last_became_peered":"2024-06-03T02:14:09.744514+0000","last_unstale":"2024-06-03T02:17:22.159623+0000","last_undegraded":"2024-06-03T02:17:22.159623+0000","last_fullsized":"2024-06-03T02:17:22.159623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:51:42.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187514+0000","last_change":"2024-06-03T02:14:09.766897+0000","last_active":"2024-06-03T02:17:22.187514+0000","last_peered":"2024-06-03T02:17:22.187514+0000","last_clean":"2024-06-03T02:17:22.187514+0000","last_became_active":"2024-06-03T02:14:09.765461+0000","last_became_peered":"2024-06-03T02:14:09.765461+0000","last_unstale":"2024-06-03T02:17:22.187514+0000","last_undegraded":"2024-06-03T02:17:22.187514+0000","last_fullsized":"2024-06-03T02:17:22.187514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:39:28.641517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"101'11","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171704+0000","last_change":"2024-06-03T02:14:09.746451+0000","last_active":"2024-06-03T02:17:22.171704+0000","last_peered":"2024-06-03T02:17:22.171704+0000","last_clean":"2024-06-03T02:17:22.171704+0000","last_became_active":"2024-06-03T02:14:09.746117+0000","last_became_peered":"2024-06-03T02:14:09.746117+0000","last_unstale":"2024-06-03T02:17:22.171704+0000","last_undegraded":"2024-06-03T02:17:22.171704+0000","last_fullsized":"2024-06-03T02:17:22.171704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:42.599993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"101'16","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.162197+0000","last_change":"2024-06-03T02:14:09.750680+0000","last_active":"2024-06-03T02:17:22.162197+0000","last_peered":"2024-06-03T02:17:22.162197+0000","last_clean":"2024-06-03T02:17:22.162197+0000","last_became_active":"2024-06-03T02:14:09.746138+0000","last_became_peered":"2024-06-03T02:14:09.746138+0000","last_unstale":"2024-06-03T02:17:22.162197+0000","last_undegraded":"2024-06-03T02:17:22.162197+0000","last_fullsized":"2024-06-03T02:17:22.162197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:04:28.188323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172319+0000","last_change":"2024-06-03T02:14:09.743467+0000","last_active":"2024-06-03T02:17:22.172319+0000","last_peered":"2024-06-03T02:17:22.172319+0000","last_clean":"2024-06-03T02:17:22.172319+0000","last_became_active":"2024-06-03T02:14:09.743152+0000","last_became_peered":"2024-06-03T02:14:09.743152+0000","last_unstale":"2024-06-03T02:17:22.172319+0000","last_undegraded":"2024-06-03T02:17:22.172319+0000","last_fullsized":"2024-06-03T02:17:22.172319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:27.336630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.148154+0000","last_change":"2024-06-03T02:14:09.747142+0000","last_active":"2024-06-03T02:17:22.148154+0000","last_peered":"2024-06-03T02:17:22.148154+0000","last_clean":"2024-06-03T02:17:22.148154+0000","last_became_active":"2024-06-03T02:14:09.746694+0000","last_became_peered":"2024-06-03T02:14:09.746694+0000","last_unstale":"2024-06-03T02:17:22.148154+0000","last_undegraded":"2024-06-03T02:17:22.148154+0000","last_fullsized":"2024-06-03T02:17:22.148154+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:04.507667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"101'10","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179040+0000","last_change":"2024-06-03T02:14:09.759906+0000","last_active":"2024-06-03T02:17:22.179040+0000","last_peered":"2024-06-03T02:17:22.179040+0000","last_clean":"2024-06-03T02:17:22.179040+0000","last_became_active":"2024-06-03T02:14:09.755450+0000","last_became_peered":"2024-06-03T02:14:09.755450+0000","last_unstale":"2024-06-03T02:17:22.179040+0000","last_undegraded":"2024-06-03T02:17:22.179040+0000","last_fullsized":"2024-06-03T02:17:22.179040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:23.057956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187103+0000","last_change":"2024-06-03T02:14:09.769388+0000","last_active":"2024-06-03T02:17:22.187103+0000","last_peered":"2024-06-03T02:17:22.187103+0000","last_clean":"2024-06-03T02:17:22.187103+0000","last_became_active":"2024-06-03T02:14:09.769029+0000","last_became_peered":"2024-06-03T02:14:09.769029+0000","last_unstale":"2024-06-03T02:17:22.187103+0000","last_undegraded":"2024-06-03T02:17:22.187103+0000","last_fullsized":"2024-06-03T02:17:22.187103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:24.751039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181114+0000","last_change":"2024-06-03T02:14:09.768584+0000","last_active":"2024-06-03T02:17:22.181114+0000","last_peered":"2024-06-03T02:17:22.181114+0000","last_clean":"2024-06-03T02:17:22.181114+0000","last_became_active":"2024-06-03T02:14:09.765737+0000","last_became_peered":"2024-06-03T02:14:09.765737+0000","last_unstale":"2024-06-03T02:17:22.181114+0000","last_undegraded":"2024-06-03T02:17:22.181114+0000","last_fullsized":"2024-06-03T02:17:22.181114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:36:18.807762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181296+0000","last_change":"2024-06-03T02:14:09.757828+0000","last_active":"2024-06-03T02:17:22.181296+0000","last_peered":"2024-06-03T02:17:22.181296+0000","last_clean":"2024-06-03T02:17:22.181296+0000","last_became_active":"2024-06-03T02:14:09.757040+0000","last_became_peered":"2024-06-03T02:14:09.757040+0000","last_unstale":"2024-06-03T02:17:22.181296+0000","last_undegraded":"2024-06-03T02:17:22.181296+0000","last_fullsized":"2024-06-03T02:17:22.181296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:26:28.694022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188466+0000","last_change":"2024-06-03T02:14:09.768263+0000","last_active":"2024-06-03T02:17:22.188466+0000","last_peered":"2024-06-03T02:17:22.188466+0000","last_clean":"2024-06-03T02:17:22.188466+0000","last_became_active":"2024-06-03T02:14:09.767988+0000","last_became_peered":"2024-06-03T02:14:09.767988+0000","last_unstale":"2024-06-03T02:17:22.188466+0000","last_undegraded":"2024-06-03T02:17:22.188466+0000","last_fullsized":"2024-06-03T02:17:22.188466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:28:37.432659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"101'10","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171695+0000","last_change":"2024-06-03T02:14:09.747118+0000","last_active":"2024-06-03T02:17:22.171695+0000","last_peered":"2024-06-03T02:17:22.171695+0000","last_clean":"2024-06-03T02:17:22.171695+0000","last_became_active":"2024-06-03T02:14:09.746580+0000","last_became_peered":"2024-06-03T02:14:09.746580+0000","last_unstale":"2024-06-03T02:17:22.171695+0000","last_undegraded":"2024-06-03T02:17:22.171695+0000","last_fullsized":"2024-06-03T02:17:22.171695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:33:04.607285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185237+0000","last_change":"2024-06-03T02:14:09.755755+0000","last_active":"2024-06-03T02:17:22.185237+0000","last_peered":"2024-06-03T02:17:22.185237+0000","last_clean":"2024-06-03T02:17:22.185237+0000","last_became_active":"2024-06-03T02:14:09.754503+0000","last_became_peered":"2024-06-03T02:14:09.754503+0000","last_unstale":"2024-06-03T02:17:22.185237+0000","last_undegraded":"2024-06-03T02:17:22.185237+0000","last_fullsized":"2024-06-03T02:17:22.185237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:30:58.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271241+0000","last_change":"2024-06-03T02:14:23.260021+0000","last_active":"2024-06-03T02:15:54.271241+0000","last_peered":"2024-06-03T02:15:54.271241+0000","last_clean":"2024-06-03T02:15:54.271241+0000","last_became_active":"2024-06-03T02:14:23.259587+0000","last_became_peered":"2024-06-03T02:14:23.259587+0000","last_unstale":"2024-06-03T02:15:54.271241+0000","last_undegraded":"2024-06-03T02:15:54.271241+0000","last_fullsized":"2024-06-03T02:15:54.271241+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:09:02.403936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"101'14","reported_seq":180,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:34.589558+0000","last_change":"2024-06-03T02:14:05.239623+0000","last_active":"2024-06-03T02:17:34.589558+0000","last_peered":"2024-06-03T02:17:34.589558+0000","last_clean":"2024-06-03T02:17:34.589558+0000","last_became_active":"2024-06-03T02:14:05.239112+0000","last_became_peered":"2024-06-03T02:14:05.239112+0000","last_unstale":"2024-06-03T02:17:34.589558+0000","last_undegraded":"2024-06-03T02:17:34.589558+0000","last_fullsized":"2024-06-03T02:17:34.589558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:12:14.020068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184909+0000","last_change":"2024-06-03T02:14:09.764972+0000","last_active":"2024-06-03T02:17:22.184909+0000","last_peered":"2024-06-03T02:17:22.184909+0000","last_clean":"2024-06-03T02:17:22.184909+0000","last_became_active":"2024-06-03T02:14:09.764435+0000","last_became_peered":"2024-06-03T02:14:09.764435+0000","last_unstale":"2024-06-03T02:17:22.184909+0000","last_undegraded":"2024-06-03T02:17:22.184909+0000","last_fullsized":"2024-06-03T02:17:22.184909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:46:58.941554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339964+0000","last_change":"2024-06-03T02:14:23.272076+0000","last_active":"2024-06-03T02:15:54.339964+0000","last_peered":"2024-06-03T02:15:54.339964+0000","last_clean":"2024-06-03T02:15:54.339964+0000","last_became_active":"2024-06-03T02:14:23.271578+0000","last_became_peered":"2024-06-03T02:14:23.271578+0000","last_unstale":"2024-06-03T02:15:54.339964+0000","last_undegraded":"2024-06-03T02:15:54.339964+0000","last_fullsized":"2024-06-03T02:15:54.339964+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:17:59.000291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"99'2","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268287+0000","last_change":"2024-06-03T02:14:05.251952+0000","last_active":"2024-06-03T02:15:54.268287+0000","last_peered":"2024-06-03T02:15:54.268287+0000","last_clean":"2024-06-03T02:15:54.268287+0000","last_became_active":"2024-06-03T02:14:05.251152+0000","last_became_peered":"2024-06-03T02:14:05.251152+0000","last_unstale":"2024-06-03T02:15:54.268287+0000","last_undegraded":"2024-06-03T02:15:54.268287+0000","last_fullsized":"2024-06-03T02:15:54.268287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:24:32.802236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.190220+0000","last_change":"2024-06-03T02:14:09.764230+0000","last_active":"2024-06-03T02:17:22.190220+0000","last_peered":"2024-06-03T02:17:22.190220+0000","last_clean":"2024-06-03T02:17:22.190220+0000","last_became_active":"2024-06-03T02:14:09.761868+0000","last_became_peered":"2024-06-03T02:14:09.761868+0000","last_unstale":"2024-06-03T02:17:22.190220+0000","last_undegraded":"2024-06-03T02:17:22.190220+0000","last_fullsized":"2024-06-03T02:17:22.190220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:39:23.828546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340393+0000","last_change":"2024-06-03T02:14:23.272169+0000","last_active":"2024-06-03T02:15:54.340393+0000","last_peered":"2024-06-03T02:15:54.340393+0000","last_clean":"2024-06-03T02:15:54.340393+0000","last_became_active":"2024-06-03T02:14:23.271636+0000","last_became_peered":"2024-06-03T02:14:23.271636+0000","last_unstale":"2024-06-03T02:15:54.340393+0000","last_undegraded":"2024-06-03T02:15:54.340393+0000","last_fullsized":"2024-06-03T02:15:54.340393+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:51:25.478028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"99'12","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270076+0000","last_change":"2024-06-03T02:14:05.238590+0000","last_active":"2024-06-03T02:15:54.270076+0000","last_peered":"2024-06-03T02:15:54.270076+0000","last_clean":"2024-06-03T02:15:54.270076+0000","last_became_active":"2024-06-03T02:14:05.238128+0000","last_became_peered":"2024-06-03T02:14:05.238128+0000","last_unstale":"2024-06-03T02:15:54.270076+0000","last_undegraded":"2024-06-03T02:15:54.270076+0000","last_fullsized":"2024-06-03T02:15:54.270076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:21.298255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174637+0000","last_change":"2024-06-03T02:14:09.754489+0000","last_active":"2024-06-03T02:17:22.174637+0000","last_peered":"2024-06-03T02:17:22.174637+0000","last_clean":"2024-06-03T02:17:22.174637+0000","last_became_active":"2024-06-03T02:14:09.754155+0000","last_became_peered":"2024-06-03T02:14:09.754155+0000","last_unstale":"2024-06-03T02:17:22.174637+0000","last_undegraded":"2024-06-03T02:17:22.174637+0000","last_fullsized":"2024-06-03T02:17:22.174637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:20:43.841271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340612+0000","last_change":"2024-06-03T02:14:23.261232+0000","last_active":"2024-06-03T02:15:54.340612+0000","last_peered":"2024-06-03T02:15:54.340612+0000","last_clean":"2024-06-03T02:15:54.340612+0000","last_became_active":"2024-06-03T02:14:23.260772+0000","last_became_peered":"2024-06-03T02:14:23.260772+0000","last_unstale":"2024-06-03T02:15:54.340612+0000","last_undegraded":"2024-06-03T02:15:54.340612+0000","last_fullsized":"2024-06-03T02:15:54.340612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:52.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"99'5","reported_seq":24,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272244+0000","last_change":"2024-06-03T02:14:53.274084+0000","last_active":"2024-06-03T02:15:54.272244+0000","last_peered":"2024-06-03T02:15:54.272244+0000","last_clean":"2024-06-03T02:15:54.272244+0000","last_became_active":"2024-06-03T02:14:53.265253+0000","last_became_peered":"2024-06-03T02:14:53.265253+0000","last_unstale":"2024-06-03T02:15:54.272244+0000","last_undegraded":"2024-06-03T02:15:54.272244+0000","last_fullsized":"2024-06-03T02:15:54.272244+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:25:56.189308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"101'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.155955+0000","last_change":"2024-06-03T02:14:09.743450+0000","last_active":"2024-06-03T02:17:22.155955+0000","last_peered":"2024-06-03T02:17:22.155955+0000","last_clean":"2024-06-03T02:17:22.155955+0000","last_became_active":"2024-06-03T02:14:09.743071+0000","last_became_peered":"2024-06-03T02:14:09.743071+0000","last_unstale":"2024-06-03T02:17:22.155955+0000","last_undegraded":"2024-06-03T02:17:22.155955+0000","last_fullsized":"2024-06-03T02:17:22.155955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:43:44.278633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":36,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.343527+0000","last_change":"2024-06-03T02:15:54.342978+0000","last_active":"2024-06-03T02:15:54.343527+0000","last_peered":"2024-06-03T02:15:54.343527+0000","last_clean":"2024-06-03T02:15:54.343527+0000","last_became_active":"2024-06-03T02:15:54.342477+0000","last_became_peered":"2024-06-03T02:15:54.342477+0000","last_unstale":"2024-06-03T02:15:54.343527+0000","last_undegraded":"2024-06-03T02:15:54.343527+0000","last_fullsized":"2024-06-03T02:15:54.343527+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:47.980513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"101'7","reported_seq":177,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:35.563221+0000","last_change":"2024-06-03T02:14:05.237464+0000","last_active":"2024-06-03T02:17:35.563221+0000","last_peered":"2024-06-03T02:17:35.563221+0000","last_clean":"2024-06-03T02:17:35.563221+0000","last_became_active":"2024-06-03T02:14:05.237067+0000","last_became_peered":"2024-06-03T02:14:05.237067+0000","last_unstale":"2024-06-03T02:17:35.563221+0000","last_undegraded":"2024-06-03T02:17:35.563221+0000","last_fullsized":"2024-06-03T02:17:35.563221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:06:38.850484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.149882+0000","last_change":"2024-06-03T02:14:09.748741+0000","last_active":"2024-06-03T02:17:22.149882+0000","last_peered":"2024-06-03T02:17:22.149882+0000","last_clean":"2024-06-03T02:17:22.149882+0000","last_became_active":"2024-06-03T02:14:09.748402+0000","last_became_peered":"2024-06-03T02:14:09.748402+0000","last_unstale":"2024-06-03T02:17:22.149882+0000","last_undegraded":"2024-06-03T02:17:22.149882+0000","last_fullsized":"2024-06-03T02:17:22.149882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:03:25.664415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271843+0000","last_change":"2024-06-03T02:14:23.260397+0000","last_active":"2024-06-03T02:15:54.271843+0000","last_peered":"2024-06-03T02:15:54.271843+0000","last_clean":"2024-06-03T02:15:54.271843+0000","last_became_active":"2024-06-03T02:14:23.259886+0000","last_became_peered":"2024-06-03T02:14:23.259886+0000","last_unstale":"2024-06-03T02:15:54.271843+0000","last_undegraded":"2024-06-03T02:15:54.271843+0000","last_fullsized":"2024-06-03T02:15:54.271843+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:46:18.562706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"99'7","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.263532+0000","last_change":"2024-06-03T02:14:05.241536+0000","last_active":"2024-06-03T02:15:54.263532+0000","last_peered":"2024-06-03T02:15:54.263532+0000","last_clean":"2024-06-03T02:15:54.263532+0000","last_became_active":"2024-06-03T02:14:05.241203+0000","last_became_peered":"2024-06-03T02:14:05.241203+0000","last_unstale":"2024-06-03T02:15:54.263532+0000","last_undegraded":"2024-06-03T02:15:54.263532+0000","last_fullsized":"2024-06-03T02:15:54.263532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:47.052563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"101'8","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185090+0000","last_change":"2024-06-03T02:14:53.475893+0000","last_active":"2024-06-03T02:17:22.185090+0000","last_peered":"2024-06-03T02:17:22.185090+0000","last_clean":"2024-06-03T02:17:22.185090+0000","last_became_active":"2024-06-03T02:14:53.475494+0000","last_became_peered":"2024-06-03T02:14:53.475494+0000","last_unstale":"2024-06-03T02:17:22.185090+0000","last_undegraded":"2024-06-03T02:17:22.185090+0000","last_fullsized":"2024-06-03T02:17:22.185090+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:27.113842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345061+0000","last_change":"2024-06-03T02:15:54.344417+0000","last_active":"2024-06-03T02:15:54.345061+0000","last_peered":"2024-06-03T02:15:54.345061+0000","last_clean":"2024-06-03T02:15:54.345061+0000","last_became_active":"2024-06-03T02:15:54.343904+0000","last_became_peered":"2024-06-03T02:15:54.343904+0000","last_unstale":"2024-06-03T02:15:54.345061+0000","last_undegraded":"2024-06-03T02:15:54.345061+0000","last_fullsized":"2024-06-03T02:15:54.345061+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:42:18.121696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"99'3","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.343922+0000","last_change":"2024-06-03T02:14:05.237743+0000","last_active":"2024-06-03T02:15:54.343922+0000","last_peered":"2024-06-03T02:15:54.343922+0000","last_clean":"2024-06-03T02:15:54.343922+0000","last_became_active":"2024-06-03T02:14:05.236944+0000","last_became_peered":"2024-06-03T02:14:05.236944+0000","last_unstale":"2024-06-03T02:15:54.343922+0000","last_undegraded":"2024-06-03T02:15:54.343922+0000","last_fullsized":"2024-06-03T02:15:54.343922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:14:09.850862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185374+0000","last_change":"2024-06-03T02:14:09.761694+0000","last_active":"2024-06-03T02:17:22.185374+0000","last_peered":"2024-06-03T02:17:22.185374+0000","last_clean":"2024-06-03T02:17:22.185374+0000","last_became_active":"2024-06-03T02:14:09.761361+0000","last_became_peered":"2024-06-03T02:14:09.761361+0000","last_unstale":"2024-06-03T02:17:22.185374+0000","last_undegraded":"2024-06-03T02:17:22.185374+0000","last_fullsized":"2024-06-03T02:17:22.185374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:19:17.803088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342311+0000","last_change":"2024-06-03T02:14:23.258634+0000","last_active":"2024-06-03T02:15:54.342311+0000","last_peered":"2024-06-03T02:15:54.342311+0000","last_clean":"2024-06-03T02:15:54.342311+0000","last_became_active":"2024-06-03T02:14:23.257799+0000","last_became_peered":"2024-06-03T02:14:23.257799+0000","last_unstale":"2024-06-03T02:15:54.342311+0000","last_undegraded":"2024-06-03T02:15:54.342311+0000","last_fullsized":"2024-06-03T02:15:54.342311+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:38:33.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"99'3","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339907+0000","last_change":"2024-06-03T02:14:05.236790+0000","last_active":"2024-06-03T02:15:54.339907+0000","last_peered":"2024-06-03T02:15:54.339907+0000","last_clean":"2024-06-03T02:15:54.339907+0000","last_became_active":"2024-06-03T02:14:05.236419+0000","last_became_peered":"2024-06-03T02:14:05.236419+0000","last_unstale":"2024-06-03T02:15:54.339907+0000","last_undegraded":"2024-06-03T02:15:54.339907+0000","last_fullsized":"2024-06-03T02:15:54.339907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:44:00.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"101'11","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183235+0000","last_change":"2024-06-03T02:14:09.744680+0000","last_active":"2024-06-03T02:17:22.183235+0000","last_peered":"2024-06-03T02:17:22.183235+0000","last_clean":"2024-06-03T02:17:22.183235+0000","last_became_active":"2024-06-03T02:14:09.744250+0000","last_became_peered":"2024-06-03T02:14:09.744250+0000","last_unstale":"2024-06-03T02:17:22.183235+0000","last_undegraded":"2024-06-03T02:17:22.183235+0000","last_fullsized":"2024-06-03T02:17:22.183235+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:58.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267519+0000","last_change":"2024-06-03T02:14:23.259325+0000","last_active":"2024-06-03T02:15:54.267519+0000","last_peered":"2024-06-03T02:15:54.267519+0000","last_clean":"2024-06-03T02:15:54.267519+0000","last_became_active":"2024-06-03T02:14:23.258925+0000","last_became_peered":"2024-06-03T02:14:23.258925+0000","last_unstale":"2024-06-03T02:15:54.267519+0000","last_undegraded":"2024-06-03T02:15:54.267519+0000","last_fullsized":"2024-06-03T02:15:54.267519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:32.038564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"101'5","reported_seq":286,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:33.493338+0000","last_change":"2024-06-03T02:14:05.242690+0000","last_active":"2024-06-03T02:17:33.493338+0000","last_peered":"2024-06-03T02:17:33.493338+0000","last_clean":"2024-06-03T02:17:33.493338+0000","last_became_active":"2024-06-03T02:14:05.242320+0000","last_became_peered":"2024-06-03T02:14:05.242320+0000","last_unstale":"2024-06-03T02:17:33.493338+0000","last_undegraded":"2024-06-03T02:17:33.493338+0000","last_fullsized":"2024-06-03T02:17:33.493338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:02.793186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"101'10","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.170581+0000","last_change":"2024-06-03T02:14:09.747146+0000","last_active":"2024-06-03T02:17:22.170581+0000","last_peered":"2024-06-03T02:17:22.170581+0000","last_clean":"2024-06-03T02:17:22.170581+0000","last_became_active":"2024-06-03T02:14:09.746803+0000","last_became_peered":"2024-06-03T02:14:09.746803+0000","last_unstale":"2024-06-03T02:17:22.170581+0000","last_undegraded":"2024-06-03T02:17:22.170581+0000","last_fullsized":"2024-06-03T02:17:22.170581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:04:36.262223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.265005+0000","last_change":"2024-06-03T02:14:23.270328+0000","last_active":"2024-06-03T02:15:54.265005+0000","last_peered":"2024-06-03T02:15:54.265005+0000","last_clean":"2024-06-03T02:15:54.265005+0000","last_became_active":"2024-06-03T02:14:23.264189+0000","last_became_peered":"2024-06-03T02:14:23.264189+0000","last_unstale":"2024-06-03T02:15:54.265005+0000","last_undegraded":"2024-06-03T02:15:54.265005+0000","last_fullsized":"2024-06-03T02:15:54.265005+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:36.248884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"99'4","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340154+0000","last_change":"2024-06-03T02:14:05.238898+0000","last_active":"2024-06-03T02:15:54.340154+0000","last_peered":"2024-06-03T02:15:54.340154+0000","last_clean":"2024-06-03T02:15:54.340154+0000","last_became_active":"2024-06-03T02:14:05.238552+0000","last_became_peered":"2024-06-03T02:14:05.238552+0000","last_unstale":"2024-06-03T02:15:54.340154+0000","last_undegraded":"2024-06-03T02:15:54.340154+0000","last_fullsized":"2024-06-03T02:15:54.340154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:29:33.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.189485+0000","last_change":"2024-06-03T02:14:09.769973+0000","last_active":"2024-06-03T02:17:22.189485+0000","last_peered":"2024-06-03T02:17:22.189485+0000","last_clean":"2024-06-03T02:17:22.189485+0000","last_became_active":"2024-06-03T02:14:09.768984+0000","last_became_peered":"2024-06-03T02:14:09.768984+0000","last_unstale":"2024-06-03T02:17:22.189485+0000","last_undegraded":"2024-06-03T02:17:22.189485+0000","last_fullsized":"2024-06-03T02:17:22.189485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:48:33.720441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340613+0000","last_change":"2024-06-03T02:14:23.265947+0000","last_active":"2024-06-03T02:15:54.340613+0000","last_peered":"2024-06-03T02:15:54.340613+0000","last_clean":"2024-06-03T02:15:54.340613+0000","last_became_active":"2024-06-03T02:14:23.265412+0000","last_became_peered":"2024-06-03T02:14:23.265412+0000","last_unstale":"2024-06-03T02:15:54.340613+0000","last_undegraded":"2024-06-03T02:15:54.340613+0000","last_fullsized":"2024-06-03T02:15:54.340613+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:35:08.664884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"99'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339912+0000","last_change":"2024-06-03T02:14:05.252751+0000","last_active":"2024-06-03T02:15:54.339912+0000","last_peered":"2024-06-03T02:15:54.339912+0000","last_clean":"2024-06-03T02:15:54.339912+0000","last_became_active":"2024-06-03T02:14:05.252267+0000","last_became_peered":"2024-06-03T02:14:05.252267+0000","last_unstale":"2024-06-03T02:15:54.339912+0000","last_undegraded":"2024-06-03T02:15:54.339912+0000","last_fullsized":"2024-06-03T02:15:54.339912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:58.091711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"101'11","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171318+0000","last_change":"2024-06-03T02:14:09.750757+0000","last_active":"2024-06-03T02:17:22.171318+0000","last_peered":"2024-06-03T02:17:22.171318+0000","last_clean":"2024-06-03T02:17:22.171318+0000","last_became_active":"2024-06-03T02:14:09.750414+0000","last_became_peered":"2024-06-03T02:14:09.750414+0000","last_unstale":"2024-06-03T02:17:22.171318+0000","last_undegraded":"2024-06-03T02:17:22.171318+0000","last_fullsized":"2024-06-03T02:17:22.171318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:29:55.435408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.265175+0000","last_change":"2024-06-03T02:14:23.263440+0000","last_active":"2024-06-03T02:15:54.265175+0000","last_peered":"2024-06-03T02:15:54.265175+0000","last_clean":"2024-06-03T02:15:54.265175+0000","last_became_active":"2024-06-03T02:14:23.254019+0000","last_became_peered":"2024-06-03T02:14:23.254019+0000","last_unstale":"2024-06-03T02:15:54.265175+0000","last_undegraded":"2024-06-03T02:15:54.265175+0000","last_fullsized":"2024-06-03T02:15:54.265175+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:48:58.745990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'39","reported_seq":331,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:33.493334+0000","last_change":"2024-06-03T02:14:05.239353+0000","last_active":"2024-06-03T02:17:33.493334+0000","last_peered":"2024-06-03T02:17:33.493334+0000","last_clean":"2024-06-03T02:17:33.493334+0000","last_became_active":"2024-06-03T02:14:05.238976+0000","last_became_peered":"2024-06-03T02:14:05.238976+0000","last_unstale":"2024-06-03T02:17:33.493334+0000","last_undegraded":"2024-06-03T02:17:33.493334+0000","last_fullsized":"2024-06-03T02:17:33.493334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:51.598533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66728,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":188,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.138882+0000","last_change":"2024-06-03T02:14:09.746546+0000","last_active":"2024-06-03T02:17:22.138882+0000","last_peered":"2024-06-03T02:17:22.138882+0000","last_clean":"2024-06-03T02:17:22.138882+0000","last_became_active":"2024-06-03T02:14:09.746049+0000","last_became_peered":"2024-06-03T02:14:09.746049+0000","last_unstale":"2024-06-03T02:17:22.138882+0000","last_undegraded":"2024-06-03T02:17:22.138882+0000","last_fullsized":"2024-06-03T02:17:22.138882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:34.667133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340598+0000","last_change":"2024-06-03T02:14:23.257296+0000","last_active":"2024-06-03T02:15:54.340598+0000","last_peered":"2024-06-03T02:15:54.340598+0000","last_clean":"2024-06-03T02:15:54.340598+0000","last_became_active":"2024-06-03T02:14:23.256858+0000","last_became_peered":"2024-06-03T02:14:23.256858+0000","last_unstale":"2024-06-03T02:15:54.340598+0000","last_undegraded":"2024-06-03T02:15:54.340598+0000","last_fullsized":"2024-06-03T02:15:54.340598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:00.464902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"99'9","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271662+0000","last_change":"2024-06-03T02:14:05.245163+0000","last_active":"2024-06-03T02:15:54.271662+0000","last_peered":"2024-06-03T02:15:54.271662+0000","last_clean":"2024-06-03T02:15:54.271662+0000","last_became_active":"2024-06-03T02:14:05.244813+0000","last_became_peered":"2024-06-03T02:14:05.244813+0000","last_unstale":"2024-06-03T02:15:54.271662+0000","last_undegraded":"2024-06-03T02:15:54.271662+0000","last_fullsized":"2024-06-03T02:15:54.271662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:46:47.017859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181591+0000","last_change":"2024-06-03T02:14:09.757200+0000","last_active":"2024-06-03T02:17:22.181591+0000","last_peered":"2024-06-03T02:17:22.181591+0000","last_clean":"2024-06-03T02:17:22.181591+0000","last_became_active":"2024-06-03T02:14:09.756380+0000","last_became_peered":"2024-06-03T02:14:09.756380+0000","last_unstale":"2024-06-03T02:17:22.181591+0000","last_undegraded":"2024-06-03T02:17:22.181591+0000","last_fullsized":"2024-06-03T02:17:22.181591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:56.045338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272398+0000","last_change":"2024-06-03T02:14:23.258441+0000","last_active":"2024-06-03T02:15:54.272398+0000","last_peered":"2024-06-03T02:15:54.272398+0000","last_clean":"2024-06-03T02:15:54.272398+0000","last_became_active":"2024-06-03T02:14:23.258015+0000","last_became_peered":"2024-06-03T02:14:23.258015+0000","last_unstale":"2024-06-03T02:15:54.272398+0000","last_undegraded":"2024-06-03T02:15:54.272398+0000","last_fullsized":"2024-06-03T02:15:54.272398+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:03.123511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"99'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340165+0000","last_change":"2024-06-03T02:14:05.240456+0000","last_active":"2024-06-03T02:15:54.340165+0000","last_peered":"2024-06-03T02:15:54.340165+0000","last_clean":"2024-06-03T02:15:54.340165+0000","last_became_active":"2024-06-03T02:14:05.240078+0000","last_became_peered":"2024-06-03T02:14:05.240078+0000","last_unstale":"2024-06-03T02:15:54.340165+0000","last_undegraded":"2024-06-03T02:15:54.340165+0000","last_fullsized":"2024-06-03T02:15:54.340165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:00:26.491773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"101'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179593+0000","last_change":"2024-06-03T02:14:09.750876+0000","last_active":"2024-06-03T02:17:22.179593+0000","last_peered":"2024-06-03T02:17:22.179593+0000","last_clean":"2024-06-03T02:17:22.179593+0000","last_became_active":"2024-06-03T02:14:09.750547+0000","last_became_peered":"2024-06-03T02:14:09.750547+0000","last_unstale":"2024-06-03T02:17:22.179593+0000","last_undegraded":"2024-06-03T02:17:22.179593+0000","last_fullsized":"2024-06-03T02:17:22.179593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:08:41.136225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.268761+0000","last_change":"2024-06-03T02:14:23.276172+0000","last_active":"2024-06-03T02:15:54.268761+0000","last_peered":"2024-06-03T02:15:54.268761+0000","last_clean":"2024-06-03T02:15:54.268761+0000","last_became_active":"2024-06-03T02:14:23.275410+0000","last_became_peered":"2024-06-03T02:14:23.275410+0000","last_unstale":"2024-06-03T02:15:54.268761+0000","last_undegraded":"2024-06-03T02:15:54.268761+0000","last_fullsized":"2024-06-03T02:15:54.268761+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:16:30.859453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"101'7","reported_seq":176,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:34.561505+0000","last_change":"2024-06-03T02:14:05.237263+0000","last_active":"2024-06-03T02:17:34.561505+0000","last_peered":"2024-06-03T02:17:34.561505+0000","last_clean":"2024-06-03T02:17:34.561505+0000","last_became_active":"2024-06-03T02:14:05.236890+0000","last_became_peered":"2024-06-03T02:14:05.236890+0000","last_unstale":"2024-06-03T02:17:34.561505+0000","last_undegraded":"2024-06-03T02:17:34.561505+0000","last_fullsized":"2024-06-03T02:17:34.561505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:42.292545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171473+0000","last_change":"2024-06-03T02:14:09.746260+0000","last_active":"2024-06-03T02:17:22.171473+0000","last_peered":"2024-06-03T02:17:22.171473+0000","last_clean":"2024-06-03T02:17:22.171473+0000","last_became_active":"2024-06-03T02:14:09.745940+0000","last_became_peered":"2024-06-03T02:14:09.745940+0000","last_unstale":"2024-06-03T02:17:22.171473+0000","last_undegraded":"2024-06-03T02:17:22.171473+0000","last_fullsized":"2024-06-03T02:17:22.171473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:08:55.851464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345467+0000","last_change":"2024-06-03T02:14:23.265440+0000","last_active":"2024-06-03T02:15:54.345467+0000","last_peered":"2024-06-03T02:15:54.345467+0000","last_clean":"2024-06-03T02:15:54.345467+0000","last_became_active":"2024-06-03T02:14:23.265028+0000","last_became_peered":"2024-06-03T02:14:23.265028+0000","last_unstale":"2024-06-03T02:15:54.345467+0000","last_undegraded":"2024-06-03T02:15:54.345467+0000","last_fullsized":"2024-06-03T02:15:54.345467+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:56.237500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"99'17","reported_seq":79,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270007+0000","last_change":"2024-06-03T02:14:53.271821+0000","last_active":"2024-06-03T02:15:54.270007+0000","last_peered":"2024-06-03T02:15:54.270007+0000","last_clean":"2024-06-03T02:15:54.270007+0000","last_became_active":"2024-06-03T02:14:53.263047+0000","last_became_peered":"2024-06-03T02:14:53.263047+0000","last_unstale":"2024-06-03T02:15:54.270007+0000","last_undegraded":"2024-06-03T02:15:54.270007+0000","last_fullsized":"2024-06-03T02:15:54.270007+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:02:25.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.148053+0000","last_change":"2024-06-03T02:14:09.754819+0000","last_active":"2024-06-03T02:17:22.148053+0000","last_peered":"2024-06-03T02:17:22.148053+0000","last_clean":"2024-06-03T02:17:22.148053+0000","last_became_active":"2024-06-03T02:14:09.753995+0000","last_became_peered":"2024-06-03T02:14:09.753995+0000","last_unstale":"2024-06-03T02:17:22.148053+0000","last_undegraded":"2024-06-03T02:17:22.148053+0000","last_fullsized":"2024-06-03T02:17:22.148053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:19:51.123774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340529+0000","last_change":"2024-06-03T02:14:23.265003+0000","last_active":"2024-06-03T02:15:54.340529+0000","last_peered":"2024-06-03T02:15:54.340529+0000","last_clean":"2024-06-03T02:15:54.340529+0000","last_became_active":"2024-06-03T02:14:23.264529+0000","last_became_peered":"2024-06-03T02:14:23.264529+0000","last_unstale":"2024-06-03T02:15:54.340529+0000","last_undegraded":"2024-06-03T02:15:54.340529+0000","last_fullsized":"2024-06-03T02:15:54.340529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:50:19.838274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270619+0000","last_change":"2024-06-03T02:14:05.236901+0000","last_active":"2024-06-03T02:15:54.270619+0000","last_peered":"2024-06-03T02:15:54.270619+0000","last_clean":"2024-06-03T02:15:54.270619+0000","last_became_active":"2024-06-03T02:14:05.236460+0000","last_became_peered":"2024-06-03T02:14:05.236460+0000","last_unstale":"2024-06-03T02:15:54.270619+0000","last_undegraded":"2024-06-03T02:15:54.270619+0000","last_fullsized":"2024-06-03T02:15:54.270619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:59.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.155988+0000","last_change":"2024-06-03T02:14:09.743135+0000","last_active":"2024-06-03T02:17:22.155988+0000","last_peered":"2024-06-03T02:17:22.155988+0000","last_clean":"2024-06-03T02:17:22.155988+0000","last_became_active":"2024-06-03T02:14:09.742506+0000","last_became_peered":"2024-06-03T02:14:09.742506+0000","last_unstale":"2024-06-03T02:17:22.155988+0000","last_undegraded":"2024-06-03T02:17:22.155988+0000","last_fullsized":"2024-06-03T02:17:22.155988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:51.413537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270403+0000","last_change":"2024-06-03T02:14:23.255512+0000","last_active":"2024-06-03T02:15:54.270403+0000","last_peered":"2024-06-03T02:15:54.270403+0000","last_clean":"2024-06-03T02:15:54.270403+0000","last_became_active":"2024-06-03T02:14:23.255103+0000","last_became_peered":"2024-06-03T02:14:23.255103+0000","last_unstale":"2024-06-03T02:15:54.270403+0000","last_undegraded":"2024-06-03T02:15:54.270403+0000","last_fullsized":"2024-06-03T02:15:54.270403+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:12:05.295848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"99'1","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266549+0000","last_change":"2024-06-03T02:14:05.248728+0000","last_active":"2024-06-03T02:15:54.266549+0000","last_peered":"2024-06-03T02:15:54.266549+0000","last_clean":"2024-06-03T02:15:54.266549+0000","last_became_active":"2024-06-03T02:14:05.248144+0000","last_became_peered":"2024-06-03T02:14:05.248144+0000","last_unstale":"2024-06-03T02:15:54.266549+0000","last_undegraded":"2024-06-03T02:15:54.266549+0000","last_fullsized":"2024-06-03T02:15:54.266549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:53:27.814443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"101'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184758+0000","last_change":"2024-06-03T02:14:09.752029+0000","last_active":"2024-06-03T02:17:22.184758+0000","last_peered":"2024-06-03T02:17:22.184758+0000","last_clean":"2024-06-03T02:17:22.184758+0000","last_became_active":"2024-06-03T02:14:09.751638+0000","last_became_peered":"2024-06-03T02:14:09.751638+0000","last_unstale":"2024-06-03T02:17:22.184758+0000","last_undegraded":"2024-06-03T02:17:22.184758+0000","last_fullsized":"2024-06-03T02:17:22.184758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:51.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.346566+0000","last_change":"2024-06-03T02:14:23.250406+0000","last_active":"2024-06-03T02:15:54.346566+0000","last_peered":"2024-06-03T02:15:54.346566+0000","last_clean":"2024-06-03T02:15:54.346566+0000","last_became_active":"2024-06-03T02:14:23.249946+0000","last_became_peered":"2024-06-03T02:14:23.249946+0000","last_unstale":"2024-06-03T02:15:54.346566+0000","last_undegraded":"2024-06-03T02:15:54.346566+0000","last_fullsized":"2024-06-03T02:15:54.346566+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:04:47.714532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"101'7","reported_seq":176,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:35.592573+0000","last_change":"2024-06-03T02:14:05.241666+0000","last_active":"2024-06-03T02:17:35.592573+0000","last_peered":"2024-06-03T02:17:35.592573+0000","last_clean":"2024-06-03T02:17:35.592573+0000","last_became_active":"2024-06-03T02:14:05.241299+0000","last_became_peered":"2024-06-03T02:14:05.241299+0000","last_unstale":"2024-06-03T02:17:35.592573+0000","last_undegraded":"2024-06-03T02:17:35.592573+0000","last_fullsized":"2024-06-03T02:17:35.592573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:21:16.792022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.176162+0000","last_change":"2024-06-03T02:14:09.768602+0000","last_active":"2024-06-03T02:17:22.176162+0000","last_peered":"2024-06-03T02:17:22.176162+0000","last_clean":"2024-06-03T02:17:22.176162+0000","last_became_active":"2024-06-03T02:14:09.768195+0000","last_became_peered":"2024-06-03T02:14:09.768195+0000","last_unstale":"2024-06-03T02:17:22.176162+0000","last_undegraded":"2024-06-03T02:17:22.176162+0000","last_fullsized":"2024-06-03T02:17:22.176162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:45:42.361966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.346730+0000","last_change":"2024-06-03T02:14:23.252472+0000","last_active":"2024-06-03T02:15:54.346730+0000","last_peered":"2024-06-03T02:15:54.346730+0000","last_clean":"2024-06-03T02:15:54.346730+0000","last_became_active":"2024-06-03T02:14:23.252018+0000","last_became_peered":"2024-06-03T02:14:23.252018+0000","last_unstale":"2024-06-03T02:15:54.346730+0000","last_undegraded":"2024-06-03T02:15:54.346730+0000","last_fullsized":"2024-06-03T02:15:54.346730+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:28.468657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163069+0000","last_change":"2024-06-03T02:14:09.744443+0000","last_active":"2024-06-03T02:17:22.163069+0000","last_peered":"2024-06-03T02:17:22.163069+0000","last_clean":"2024-06-03T02:17:22.163069+0000","last_became_active":"2024-06-03T02:14:09.744100+0000","last_became_peered":"2024-06-03T02:14:09.744100+0000","last_unstale":"2024-06-03T02:17:22.163069+0000","last_undegraded":"2024-06-03T02:17:22.163069+0000","last_fullsized":"2024-06-03T02:17:22.163069+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:44.566745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"101'6","reported_seq":172,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:34.560720+0000","last_change":"2024-06-03T02:14:05.250734+0000","last_active":"2024-06-03T02:17:34.560720+0000","last_peered":"2024-06-03T02:17:34.560720+0000","last_clean":"2024-06-03T02:17:34.560720+0000","last_became_active":"2024-06-03T02:14:05.250300+0000","last_became_peered":"2024-06-03T02:14:05.250300+0000","last_unstale":"2024-06-03T02:17:34.560720+0000","last_undegraded":"2024-06-03T02:17:34.560720+0000","last_fullsized":"2024-06-03T02:17:34.560720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:52.322895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341486+0000","last_change":"2024-06-03T02:14:53.471397+0000","last_active":"2024-06-03T02:15:54.341486+0000","last_peered":"2024-06-03T02:15:54.341486+0000","last_clean":"2024-06-03T02:15:54.341486+0000","last_became_active":"2024-06-03T02:14:53.470868+0000","last_became_peered":"2024-06-03T02:14:53.470868+0000","last_unstale":"2024-06-03T02:15:54.341486+0000","last_undegraded":"2024-06-03T02:15:54.341486+0000","last_fullsized":"2024-06-03T02:15:54.341486+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:58.726070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188094+0000","last_change":"2024-06-03T02:14:09.765193+0000","last_active":"2024-06-03T02:17:22.188094+0000","last_peered":"2024-06-03T02:17:22.188094+0000","last_clean":"2024-06-03T02:17:22.188094+0000","last_became_active":"2024-06-03T02:14:09.764834+0000","last_became_peered":"2024-06-03T02:14:09.764834+0000","last_unstale":"2024-06-03T02:17:22.188094+0000","last_undegraded":"2024-06-03T02:17:22.188094+0000","last_fullsized":"2024-06-03T02:17:22.188094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:20:37.464286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"99'6","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340500+0000","last_change":"2024-06-03T02:14:05.240118+0000","last_active":"2024-06-03T02:15:54.340500+0000","last_peered":"2024-06-03T02:15:54.340500+0000","last_clean":"2024-06-03T02:15:54.340500+0000","last_became_active":"2024-06-03T02:14:05.239771+0000","last_became_peered":"2024-06-03T02:14:05.239771+0000","last_unstale":"2024-06-03T02:15:54.340500+0000","last_undegraded":"2024-06-03T02:15:54.340500+0000","last_fullsized":"2024-06-03T02:15:54.340500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:08.256222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":371,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340279+0000","last_change":"2024-06-03T02:12:31.842613+0000","last_active":"2024-06-03T02:15:54.340279+0000","last_peered":"2024-06-03T02:15:54.340279+0000","last_clean":"2024-06-03T02:15:54.340279+0000","last_became_active":"2024-06-03T02:12:31.842051+0000","last_became_peered":"2024-06-03T02:12:31.842051+0000","last_unstale":"2024-06-03T02:15:54.340279+0000","last_undegraded":"2024-06-03T02:15:54.340279+0000","last_fullsized":"2024-06-03T02:15:54.340279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_clean_scrub_stamp":"2024-06-03T02:08:38.749702+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:40:29.748789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340391+0000","last_change":"2024-06-03T02:14:23.255849+0000","last_active":"2024-06-03T02:15:54.340391+0000","last_peered":"2024-06-03T02:15:54.340391+0000","last_clean":"2024-06-03T02:15:54.340391+0000","last_became_active":"2024-06-03T02:14:23.255380+0000","last_became_peered":"2024-06-03T02:14:23.255380+0000","last_unstale":"2024-06-03T02:15:54.340391+0000","last_undegraded":"2024-06-03T02:15:54.340391+0000","last_fullsized":"2024-06-03T02:15:54.340391+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:41:16.051326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"101'5","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.169445+0000","last_change":"2024-06-03T02:14:53.270423+0000","last_active":"2024-06-03T02:17:22.169445+0000","last_peered":"2024-06-03T02:17:22.169445+0000","last_clean":"2024-06-03T02:17:22.169445+0000","last_became_active":"2024-06-03T02:14:53.269260+0000","last_became_peered":"2024-06-03T02:14:53.269260+0000","last_unstale":"2024-06-03T02:17:22.169445+0000","last_undegraded":"2024-06-03T02:17:22.169445+0000","last_fullsized":"2024-06-03T02:17:22.169445+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:57.392012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"99'4","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341175+0000","last_change":"2024-06-03T02:14:05.235982+0000","last_active":"2024-06-03T02:15:54.341175+0000","last_peered":"2024-06-03T02:15:54.341175+0000","last_clean":"2024-06-03T02:15:54.341175+0000","last_became_active":"2024-06-03T02:14:05.235638+0000","last_became_peered":"2024-06-03T02:14:05.235638+0000","last_unstale":"2024-06-03T02:15:54.341175+0000","last_undegraded":"2024-06-03T02:15:54.341175+0000","last_fullsized":"2024-06-03T02:15:54.341175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:42:07.544005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271863+0000","last_change":"2024-06-03T02:14:23.257790+0000","last_active":"2024-06-03T02:15:54.271863+0000","last_peered":"2024-06-03T02:15:54.271863+0000","last_clean":"2024-06-03T02:15:54.271863+0000","last_became_active":"2024-06-03T02:14:23.257297+0000","last_became_peered":"2024-06-03T02:14:23.257297+0000","last_unstale":"2024-06-03T02:15:54.271863+0000","last_undegraded":"2024-06-03T02:15:54.271863+0000","last_fullsized":"2024-06-03T02:15:54.271863+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:13.715455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"99'4","reported_seq":21,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272792+0000","last_change":"2024-06-03T02:14:53.268495+0000","last_active":"2024-06-03T02:15:54.272792+0000","last_peered":"2024-06-03T02:15:54.272792+0000","last_clean":"2024-06-03T02:15:54.272792+0000","last_became_active":"2024-06-03T02:14:53.268147+0000","last_became_peered":"2024-06-03T02:14:53.268147+0000","last_unstale":"2024-06-03T02:15:54.272792+0000","last_undegraded":"2024-06-03T02:15:54.272792+0000","last_fullsized":"2024-06-03T02:15:54.272792+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:23.076319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.169542+0000","last_change":"2024-06-03T02:14:09.758244+0000","last_active":"2024-06-03T02:17:22.169542+0000","last_peered":"2024-06-03T02:17:22.169542+0000","last_clean":"2024-06-03T02:17:22.169542+0000","last_became_active":"2024-06-03T02:14:09.757812+0000","last_became_peered":"2024-06-03T02:14:09.757812+0000","last_unstale":"2024-06-03T02:17:22.169542+0000","last_undegraded":"2024-06-03T02:17:22.169542+0000","last_fullsized":"2024-06-03T02:17:22.169542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:08.243703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.266267+0000","last_change":"2024-06-03T02:14:23.259980+0000","last_active":"2024-06-03T02:15:54.266267+0000","last_peered":"2024-06-03T02:15:54.266267+0000","last_clean":"2024-06-03T02:15:54.266267+0000","last_became_active":"2024-06-03T02:14:23.258308+0000","last_became_peered":"2024-06-03T02:14:23.258308+0000","last_unstale":"2024-06-03T02:15:54.266267+0000","last_undegraded":"2024-06-03T02:15:54.266267+0000","last_fullsized":"2024-06-03T02:15:54.266267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:28.833994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'17","reported_seq":62,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:16.235959+0000","last_change":"2024-06-03T02:14:05.244855+0000","last_active":"2024-06-03T02:16:16.235959+0000","last_peered":"2024-06-03T02:16:16.235959+0000","last_clean":"2024-06-03T02:16:16.235959+0000","last_became_active":"2024-06-03T02:14:05.244335+0000","last_became_peered":"2024-06-03T02:14:05.244335+0000","last_unstale":"2024-06-03T02:16:16.235959+0000","last_undegraded":"2024-06-03T02:16:16.235959+0000","last_fullsized":"2024-06-03T02:16:16.235959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:53:19.065411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185762+0000","last_change":"2024-06-03T02:14:09.750761+0000","last_active":"2024-06-03T02:17:22.185762+0000","last_peered":"2024-06-03T02:17:22.185762+0000","last_clean":"2024-06-03T02:17:22.185762+0000","last_became_active":"2024-06-03T02:14:09.750336+0000","last_became_peered":"2024-06-03T02:14:09.750336+0000","last_unstale":"2024-06-03T02:17:22.185762+0000","last_undegraded":"2024-06-03T02:17:22.185762+0000","last_fullsized":"2024-06-03T02:17:22.185762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:59:46.651149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271019+0000","last_change":"2024-06-03T02:14:23.243287+0000","last_active":"2024-06-03T02:15:54.271019+0000","last_peered":"2024-06-03T02:15:54.271019+0000","last_clean":"2024-06-03T02:15:54.271019+0000","last_became_active":"2024-06-03T02:14:23.242828+0000","last_became_peered":"2024-06-03T02:14:23.242828+0000","last_unstale":"2024-06-03T02:15:54.271019+0000","last_undegraded":"2024-06-03T02:15:54.271019+0000","last_fullsized":"2024-06-03T02:15:54.271019+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:12.312438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"99'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341908+0000","last_change":"2024-06-03T02:14:05.596288+0000","last_active":"2024-06-03T02:15:54.341908+0000","last_peered":"2024-06-03T02:15:54.341908+0000","last_clean":"2024-06-03T02:15:54.341908+0000","last_became_active":"2024-06-03T02:14:05.595868+0000","last_became_peered":"2024-06-03T02:14:05.595868+0000","last_unstale":"2024-06-03T02:15:54.341908+0000","last_undegraded":"2024-06-03T02:15:54.341908+0000","last_fullsized":"2024-06-03T02:15:54.341908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:50.291541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.189215+0000","last_change":"2024-06-03T02:14:09.765317+0000","last_active":"2024-06-03T02:17:22.189215+0000","last_peered":"2024-06-03T02:17:22.189215+0000","last_clean":"2024-06-03T02:17:22.189215+0000","last_became_active":"2024-06-03T02:14:09.764983+0000","last_became_peered":"2024-06-03T02:14:09.764983+0000","last_unstale":"2024-06-03T02:17:22.189215+0000","last_undegraded":"2024-06-03T02:17:22.189215+0000","last_fullsized":"2024-06-03T02:17:22.189215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:20.075276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341648+0000","last_change":"2024-06-03T02:14:23.265352+0000","last_active":"2024-06-03T02:15:54.341648+0000","last_peered":"2024-06-03T02:15:54.341648+0000","last_clean":"2024-06-03T02:15:54.341648+0000","last_became_active":"2024-06-03T02:14:23.264932+0000","last_became_peered":"2024-06-03T02:14:23.264932+0000","last_unstale":"2024-06-03T02:15:54.341648+0000","last_undegraded":"2024-06-03T02:15:54.341648+0000","last_fullsized":"2024-06-03T02:15:54.341648+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:43.803170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"99'2","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339982+0000","last_change":"2024-06-03T02:14:05.248563+0000","last_active":"2024-06-03T02:15:54.339982+0000","last_peered":"2024-06-03T02:15:54.339982+0000","last_clean":"2024-06-03T02:15:54.339982+0000","last_became_active":"2024-06-03T02:14:05.248306+0000","last_became_peered":"2024-06-03T02:14:05.248306+0000","last_unstale":"2024-06-03T02:15:54.339982+0000","last_undegraded":"2024-06-03T02:15:54.339982+0000","last_fullsized":"2024-06-03T02:15:54.339982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:23:58.130157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"101'2","reported_seq":41,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.123863+0000","last_change":"2024-06-03T02:14:09.745948+0000","last_active":"2024-06-03T02:17:22.123863+0000","last_peered":"2024-06-03T02:17:22.123863+0000","last_clean":"2024-06-03T02:17:22.123863+0000","last_became_active":"2024-06-03T02:14:09.745626+0000","last_became_peered":"2024-06-03T02:14:09.745626+0000","last_unstale":"2024-06-03T02:17:22.123863+0000","last_undegraded":"2024-06-03T02:17:22.123863+0000","last_fullsized":"2024-06-03T02:17:22.123863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:34.742129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.342659+0000","last_change":"2024-06-03T02:15:54.341824+0000","last_active":"2024-06-03T02:15:54.342659+0000","last_peered":"2024-06-03T02:15:54.342659+0000","last_clean":"2024-06-03T02:15:54.342659+0000","last_became_active":"2024-06-03T02:15:54.341267+0000","last_became_peered":"2024-06-03T02:15:54.341267+0000","last_unstale":"2024-06-03T02:15:54.342659+0000","last_undegraded":"2024-06-03T02:15:54.342659+0000","last_fullsized":"2024-06-03T02:15:54.342659+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:47.714056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"99'7","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.267775+0000","last_change":"2024-06-03T02:14:05.244806+0000","last_active":"2024-06-03T02:15:54.267775+0000","last_peered":"2024-06-03T02:15:54.267775+0000","last_clean":"2024-06-03T02:15:54.267775+0000","last_became_active":"2024-06-03T02:14:05.242719+0000","last_became_peered":"2024-06-03T02:14:05.242719+0000","last_unstale":"2024-06-03T02:15:54.267775+0000","last_undegraded":"2024-06-03T02:15:54.267775+0000","last_fullsized":"2024-06-03T02:15:54.267775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:31.314266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.176658+0000","last_change":"2024-06-03T02:14:09.755102+0000","last_active":"2024-06-03T02:17:22.176658+0000","last_peered":"2024-06-03T02:17:22.176658+0000","last_clean":"2024-06-03T02:17:22.176658+0000","last_became_active":"2024-06-03T02:14:09.754748+0000","last_became_peered":"2024-06-03T02:14:09.754748+0000","last_unstale":"2024-06-03T02:17:22.176658+0000","last_undegraded":"2024-06-03T02:17:22.176658+0000","last_fullsized":"2024-06-03T02:17:22.176658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:59.920710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"101'1","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:55.297902+0000","last_change":"2024-06-03T02:14:23.271974+0000","last_active":"2024-06-03T02:15:55.297902+0000","last_peered":"2024-06-03T02:15:55.297902+0000","last_clean":"2024-06-03T02:15:55.297902+0000","last_became_active":"2024-06-03T02:14:23.271470+0000","last_became_peered":"2024-06-03T02:14:23.271470+0000","last_unstale":"2024-06-03T02:15:55.297902+0000","last_undegraded":"2024-06-03T02:15:55.297902+0000","last_fullsized":"2024-06-03T02:15:55.297902+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:03.159351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"99'3","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.271461+0000","last_change":"2024-06-03T02:14:05.242261+0000","last_active":"2024-06-03T02:15:54.271461+0000","last_peered":"2024-06-03T02:15:54.271461+0000","last_clean":"2024-06-03T02:15:54.271461+0000","last_became_active":"2024-06-03T02:14:05.241854+0000","last_became_peered":"2024-06-03T02:14:05.241854+0000","last_unstale":"2024-06-03T02:15:54.271461+0000","last_undegraded":"2024-06-03T02:15:54.271461+0000","last_fullsized":"2024-06-03T02:15:54.271461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:21.427703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185157+0000","last_change":"2024-06-03T02:14:09.763591+0000","last_active":"2024-06-03T02:17:22.185157+0000","last_peered":"2024-06-03T02:17:22.185157+0000","last_clean":"2024-06-03T02:17:22.185157+0000","last_became_active":"2024-06-03T02:14:09.763240+0000","last_became_peered":"2024-06-03T02:14:09.763240+0000","last_unstale":"2024-06-03T02:17:22.185157+0000","last_undegraded":"2024-06-03T02:17:22.185157+0000","last_fullsized":"2024-06-03T02:17:22.185157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:40.146898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341606+0000","last_change":"2024-06-03T02:14:23.265011+0000","last_active":"2024-06-03T02:15:54.341606+0000","last_peered":"2024-06-03T02:15:54.341606+0000","last_clean":"2024-06-03T02:15:54.341606+0000","last_became_active":"2024-06-03T02:14:23.263767+0000","last_became_peered":"2024-06-03T02:14:23.263767+0000","last_unstale":"2024-06-03T02:15:54.341606+0000","last_undegraded":"2024-06-03T02:15:54.341606+0000","last_fullsized":"2024-06-03T02:15:54.341606+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:34:04.274545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"99'4","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.270508+0000","last_change":"2024-06-03T02:14:53.474879+0000","last_active":"2024-06-03T02:15:54.270508+0000","last_peered":"2024-06-03T02:15:54.270508+0000","last_clean":"2024-06-03T02:15:54.270508+0000","last_became_active":"2024-06-03T02:14:53.472209+0000","last_became_peered":"2024-06-03T02:14:53.472209+0000","last_unstale":"2024-06-03T02:15:54.270508+0000","last_undegraded":"2024-06-03T02:15:54.270508+0000","last_fullsized":"2024-06-03T02:15:54.270508+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:36:57.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"101'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172586+0000","last_change":"2024-06-03T02:14:09.747756+0000","last_active":"2024-06-03T02:17:22.172586+0000","last_peered":"2024-06-03T02:17:22.172586+0000","last_clean":"2024-06-03T02:17:22.172586+0000","last_became_active":"2024-06-03T02:14:09.747453+0000","last_became_peered":"2024-06-03T02:14:09.747453+0000","last_unstale":"2024-06-03T02:17:22.172586+0000","last_undegraded":"2024-06-03T02:17:22.172586+0000","last_fullsized":"2024-06-03T02:17:22.172586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:19:02.341367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339730+0000","last_change":"2024-06-03T02:14:23.248594+0000","last_active":"2024-06-03T02:15:54.339730+0000","last_peered":"2024-06-03T02:15:54.339730+0000","last_clean":"2024-06-03T02:15:54.339730+0000","last_became_active":"2024-06-03T02:14:23.248080+0000","last_became_peered":"2024-06-03T02:14:23.248080+0000","last_unstale":"2024-06-03T02:15:54.339730+0000","last_undegraded":"2024-06-03T02:15:54.339730+0000","last_fullsized":"2024-06-03T02:15:54.339730+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:24:42.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"99'5","reported_seq":173,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:35.496818+0000","last_change":"2024-06-03T02:14:05.596274+0000","last_active":"2024-06-03T02:17:35.496818+0000","last_peered":"2024-06-03T02:17:35.496818+0000","last_clean":"2024-06-03T02:17:35.496818+0000","last_became_active":"2024-06-03T02:14:05.595678+0000","last_became_peered":"2024-06-03T02:14:05.595678+0000","last_unstale":"2024-06-03T02:17:35.496818+0000","last_undegraded":"2024-06-03T02:17:35.496818+0000","last_fullsized":"2024-06-03T02:17:35.496818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:13:25.436688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"101'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172440+0000","last_change":"2024-06-03T02:14:09.744934+0000","last_active":"2024-06-03T02:17:22.172440+0000","last_peered":"2024-06-03T02:17:22.172440+0000","last_clean":"2024-06-03T02:17:22.172440+0000","last_became_active":"2024-06-03T02:14:09.744621+0000","last_became_peered":"2024-06-03T02:14:09.744621+0000","last_unstale":"2024-06-03T02:17:22.172440+0000","last_undegraded":"2024-06-03T02:17:22.172440+0000","last_fullsized":"2024-06-03T02:17:22.172440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:01:51.970160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.341842+0000","last_change":"2024-06-03T02:14:23.273123+0000","last_active":"2024-06-03T02:15:54.341842+0000","last_peered":"2024-06-03T02:15:54.341842+0000","last_clean":"2024-06-03T02:15:54.341842+0000","last_became_active":"2024-06-03T02:14:23.272153+0000","last_became_peered":"2024-06-03T02:14:23.272153+0000","last_unstale":"2024-06-03T02:15:54.341842+0000","last_undegraded":"2024-06-03T02:15:54.341842+0000","last_fullsized":"2024-06-03T02:15:54.341842+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:01:43.374413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"99'3","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.340388+0000","last_change":"2024-06-03T02:14:05.245094+0000","last_active":"2024-06-03T02:15:54.340388+0000","last_peered":"2024-06-03T02:15:54.340388+0000","last_clean":"2024-06-03T02:15:54.340388+0000","last_became_active":"2024-06-03T02:14:05.244678+0000","last_became_peered":"2024-06-03T02:14:05.244678+0000","last_unstale":"2024-06-03T02:15:54.340388+0000","last_undegraded":"2024-06-03T02:15:54.340388+0000","last_fullsized":"2024-06-03T02:15:54.340388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:13:26.546135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184748+0000","last_change":"2024-06-03T02:14:09.745919+0000","last_active":"2024-06-03T02:17:22.184748+0000","last_peered":"2024-06-03T02:17:22.184748+0000","last_clean":"2024-06-03T02:17:22.184748+0000","last_became_active":"2024-06-03T02:14:09.745576+0000","last_became_peered":"2024-06-03T02:14:09.745576+0000","last_unstale":"2024-06-03T02:17:22.184748+0000","last_undegraded":"2024-06-03T02:17:22.184748+0000","last_fullsized":"2024-06-03T02:17:22.184748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:29:31.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.339570+0000","last_change":"2024-06-03T02:14:23.244174+0000","last_active":"2024-06-03T02:15:54.339570+0000","last_peered":"2024-06-03T02:15:54.339570+0000","last_clean":"2024-06-03T02:15:54.339570+0000","last_became_active":"2024-06-03T02:14:23.243595+0000","last_became_peered":"2024-06-03T02:14:23.243595+0000","last_unstale":"2024-06-03T02:15:54.339570+0000","last_undegraded":"2024-06-03T02:15:54.339570+0000","last_fullsized":"2024-06-03T02:15:54.339570+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:12:36.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"99'5","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.345783+0000","last_change":"2024-06-03T02:14:05.247669+0000","last_active":"2024-06-03T02:15:54.345783+0000","last_peered":"2024-06-03T02:15:54.345783+0000","last_clean":"2024-06-03T02:15:54.345783+0000","last_became_active":"2024-06-03T02:14:05.246685+0000","last_became_peered":"2024-06-03T02:14:05.246685+0000","last_unstale":"2024-06-03T02:15:54.345783+0000","last_undegraded":"2024-06-03T02:15:54.345783+0000","last_fullsized":"2024-06-03T02:15:54.345783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:39.454422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171553+0000","last_change":"2024-06-03T02:14:09.749242+0000","last_active":"2024-06-03T02:17:22.171553+0000","last_peered":"2024-06-03T02:17:22.171553+0000","last_clean":"2024-06-03T02:17:22.171553+0000","last_became_active":"2024-06-03T02:14:09.748370+0000","last_became_peered":"2024-06-03T02:14:09.748370+0000","last_unstale":"2024-06-03T02:17:22.171553+0000","last_undegraded":"2024-06-03T02:17:22.171553+0000","last_fullsized":"2024-06-03T02:17:22.171553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:26.189960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"101'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.161980+0000","last_change":"2024-06-03T02:14:09.745966+0000","last_active":"2024-06-03T02:17:22.161980+0000","last_peered":"2024-06-03T02:17:22.161980+0000","last_clean":"2024-06-03T02:17:22.161980+0000","last_became_active":"2024-06-03T02:14:09.745590+0000","last_became_peered":"2024-06-03T02:14:09.745590+0000","last_unstale":"2024-06-03T02:17:22.161980+0000","last_undegraded":"2024-06-03T02:17:22.161980+0000","last_fullsized":"2024-06-03T02:17:22.161980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:46.744629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"101'12","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163231+0000","last_change":"2024-06-03T02:14:09.744964+0000","last_active":"2024-06-03T02:17:22.163231+0000","last_peered":"2024-06-03T02:17:22.163231+0000","last_clean":"2024-06-03T02:17:22.163231+0000","last_became_active":"2024-06-03T02:14:09.743877+0000","last_became_peered":"2024-06-03T02:14:09.743877+0000","last_unstale":"2024-06-03T02:17:22.163231+0000","last_undegraded":"2024-06-03T02:17:22.163231+0000","last_fullsized":"2024-06-03T02:17:22.163231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:05:48.020670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186713+0000","last_change":"2024-06-03T02:14:09.761736+0000","last_active":"2024-06-03T02:17:22.186713+0000","last_peered":"2024-06-03T02:17:22.186713+0000","last_clean":"2024-06-03T02:17:22.186713+0000","last_became_active":"2024-06-03T02:14:09.761477+0000","last_became_peered":"2024-06-03T02:14:09.761477+0000","last_unstale":"2024-06-03T02:17:22.186713+0000","last_undegraded":"2024-06-03T02:17:22.186713+0000","last_fullsized":"2024-06-03T02:17:22.186713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:00:04.631330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"101'7","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183182+0000","last_change":"2024-06-03T02:14:53.266594+0000","last_active":"2024-06-03T02:17:22.183182+0000","last_peered":"2024-06-03T02:17:22.183182+0000","last_clean":"2024-06-03T02:17:22.183182+0000","last_became_active":"2024-06-03T02:14:53.266104+0000","last_became_peered":"2024-06-03T02:14:53.266104+0000","last_unstale":"2024-06-03T02:17:22.183182+0000","last_undegraded":"2024-06-03T02:17:22.183182+0000","last_fullsized":"2024-06-03T02:17:22.183182+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:12.018082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"101'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174788+0000","last_change":"2024-06-03T02:14:09.772332+0000","last_active":"2024-06-03T02:17:22.174788+0000","last_peered":"2024-06-03T02:17:22.174788+0000","last_clean":"2024-06-03T02:17:22.174788+0000","last_became_active":"2024-06-03T02:14:09.771888+0000","last_became_peered":"2024-06-03T02:14:09.771888+0000","last_unstale":"2024-06-03T02:17:22.174788+0000","last_undegraded":"2024-06-03T02:17:22.174788+0000","last_fullsized":"2024-06-03T02:17:22.174788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:58:49.414986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"101'6","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.156755+0000","last_change":"2024-06-03T02:14:53.270828+0000","last_active":"2024-06-03T02:17:22.156755+0000","last_peered":"2024-06-03T02:17:22.156755+0000","last_clean":"2024-06-03T02:17:22.156755+0000","last_became_active":"2024-06-03T02:14:53.270365+0000","last_became_peered":"2024-06-03T02:14:53.270365+0000","last_unstale":"2024-06-03T02:17:22.156755+0000","last_undegraded":"2024-06-03T02:17:22.156755+0000","last_fullsized":"2024-06-03T02:17:22.156755+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.051756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"101'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.161480+0000","last_change":"2024-06-03T02:14:09.747709+0000","last_active":"2024-06-03T02:17:22.161480+0000","last_peered":"2024-06-03T02:17:22.161480+0000","last_clean":"2024-06-03T02:17:22.161480+0000","last_became_active":"2024-06-03T02:14:09.747359+0000","last_became_peered":"2024-06-03T02:14:09.747359+0000","last_unstale":"2024-06-03T02:17:22.161480+0000","last_undegraded":"2024-06-03T02:17:22.161480+0000","last_fullsized":"2024-06-03T02:17:22.161480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:37:24.873442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"101'2","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.145262+0000","last_change":"2024-06-03T02:14:53.473136+0000","last_active":"2024-06-03T02:17:22.145262+0000","last_peered":"2024-06-03T02:17:22.145262+0000","last_clean":"2024-06-03T02:17:22.145262+0000","last_became_active":"2024-06-03T02:14:53.471214+0000","last_became_peered":"2024-06-03T02:14:53.471214+0000","last_unstale":"2024-06-03T02:17:22.145262+0000","last_undegraded":"2024-06-03T02:17:22.145262+0000","last_fullsized":"2024-06-03T02:17:22.145262+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:25.822042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"101'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184565+0000","last_change":"2024-06-03T02:14:09.746463+0000","last_active":"2024-06-03T02:17:22.184565+0000","last_peered":"2024-06-03T02:17:22.184565+0000","last_clean":"2024-06-03T02:17:22.184565+0000","last_became_active":"2024-06-03T02:14:09.746115+0000","last_became_peered":"2024-06-03T02:14:09.746115+0000","last_unstale":"2024-06-03T02:17:22.184565+0000","last_undegraded":"2024-06-03T02:17:22.184565+0000","last_fullsized":"2024-06-03T02:17:22.184565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:51:23.050739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184552+0000","last_change":"2024-06-03T02:14:09.755467+0000","last_active":"2024-06-03T02:17:22.184552+0000","last_peered":"2024-06-03T02:17:22.184552+0000","last_clean":"2024-06-03T02:17:22.184552+0000","last_became_active":"2024-06-03T02:14:09.755103+0000","last_became_peered":"2024-06-03T02:14:09.755103+0000","last_unstale":"2024-06-03T02:17:22.184552+0000","last_undegraded":"2024-06-03T02:17:22.184552+0000","last_fullsized":"2024-06-03T02:17:22.184552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:22.834519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"101'10","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185923+0000","last_change":"2024-06-03T02:14:09.758201+0000","last_active":"2024-06-03T02:17:22.185923+0000","last_peered":"2024-06-03T02:17:22.185923+0000","last_clean":"2024-06-03T02:17:22.185923+0000","last_became_active":"2024-06-03T02:14:09.757819+0000","last_became_peered":"2024-06-03T02:14:09.757819+0000","last_unstale":"2024-06-03T02:17:22.185923+0000","last_undegraded":"2024-06-03T02:17:22.185923+0000","last_fullsized":"2024-06-03T02:17:22.185923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:44.802694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"101'11","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183299+0000","last_change":"2024-06-03T02:14:09.768132+0000","last_active":"2024-06-03T02:17:22.183299+0000","last_peered":"2024-06-03T02:17:22.183299+0000","last_clean":"2024-06-03T02:17:22.183299+0000","last_became_active":"2024-06-03T02:14:09.767642+0000","last_became_peered":"2024-06-03T02:14:09.767642+0000","last_unstale":"2024-06-03T02:17:22.183299+0000","last_undegraded":"2024-06-03T02:17:22.183299+0000","last_fullsized":"2024-06-03T02:17:22.183299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:32:30.455418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"101'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.159027+0000","last_change":"2024-06-03T02:14:09.745603+0000","last_active":"2024-06-03T02:17:22.159027+0000","last_peered":"2024-06-03T02:17:22.159027+0000","last_clean":"2024-06-03T02:17:22.159027+0000","last_became_active":"2024-06-03T02:14:09.745245+0000","last_became_peered":"2024-06-03T02:14:09.745245+0000","last_unstale":"2024-06-03T02:17:22.159027+0000","last_undegraded":"2024-06-03T02:17:22.159027+0000","last_fullsized":"2024-06-03T02:17:22.159027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:09:14.507101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74743,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1242,"num_read_kb":700,"num_write":263,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":224229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318332,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":578717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37164,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707988,"statfs":{"total":95995035648,"available":95956979712,"internally_reserved":0,"allocated":1871872,"data_stored":1236194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743494,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":577953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005132,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":577637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266770,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1785856,"data_stored":1176030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430296,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":573988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1249280,"data_stored":641858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":574913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28240,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716912,"statfs":{"total":95995035648,"available":95966117888,"internally_reserved":0,"allocated":1253376,"data_stored":641854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149799,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888162,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":578062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691934,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37088,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708064,"statfs":{"total":95995035648,"available":95957057536,"internally_reserved":0,"allocated":1794048,"data_stored":1170550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T02:17:39.311 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-03T02:17:39.311 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-03T02:17:39.311 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-03T02:17:39.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-03T02:17:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:39 smithi079 ceph-mon[31819]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:17:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:39 smithi135 ceph-mon[36653]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:17:39.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:39 smithi123 ceph-mon[36640]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:17:39.793 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:40.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:40 smithi079 ceph-mon[31819]: pgmap v445: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:17:40.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:40 smithi135 ceph-mon[36653]: pgmap v445: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:17:40.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:40 smithi123 ceph-mon[36640]: pgmap v445: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:17:41.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.961+0000 7fc42b200700 1 -- 172.21.15.79:0/2225996154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc424105040 msgr2=0x7fc424111550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:41.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.961+0000 7fc42b200700 1 --2- 172.21.15.79:0/2225996154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc424105040 0x7fc424111550 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00b0a0 tx=0x7fc41c00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:17:41.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.961+0000 7fc42b200700 1 -- 172.21.15.79:0/2225996154 shutdown_connections 2024-06-03T02:17:41.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.961+0000 7fc42b200700 1 --2- 172.21.15.79:0/2225996154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc424105040 0x7fc424111550 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:41.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.961+0000 7fc42b200700 1 --2- 172.21.15.79:0/2225996154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc42410a680 0x7fc42410aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.961+0000 7fc42b200700 1 --2- 172.21.15.79:0/2225996154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc424104680 0x7fc424104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.961+0000 7fc42b200700 1 -- 172.21.15.79:0/2225996154 >> 172.21.15.79:0/2225996154 conn(0x7fc424100170 msgr2=0x7fc424102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.961+0000 7fc42b200700 1 -- 172.21.15.79:0/2225996154 shutdown_connections 2024-06-03T02:17:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.961+0000 7fc42b200700 1 -- 172.21.15.79:0/2225996154 wait complete. 2024-06-03T02:17:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.962+0000 7fc42b200700 1 Processor -- start 2024-06-03T02:17:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.963+0000 7fc42b200700 1 -- start start 2024-06-03T02:17:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.963+0000 7fc42b200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc424104680 0x7fc42419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.963+0000 7fc42b200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc424105040 0x7fc42419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.963+0000 7fc42b200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc42410a680 0x7fc42419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.963+0000 7fc42b200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc424114a60 con 0x7fc424105040 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.963+0000 7fc42b200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241148e0 con 0x7fc42410a680 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.963+0000 7fc42b200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc424114be0 con 0x7fc424104680 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.963+0000 7fc423400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc424105040 0x7fc42419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.963+0000 7fc423e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc424104680 0x7fc42419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.964+0000 7fc423e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc424104680 0x7fc42419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:43164/0 (socket says 172.21.15.79:43164) 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.964+0000 7fc423e00700 1 -- 172.21.15.79:0/3943673890 learned_addr learned my addr 172.21.15.79:0/3943673890 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.964+0000 7fc429200700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc42410a680 0x7fc42419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.964+0000 7fc423e00700 1 -- 172.21.15.79:0/3943673890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc42410a680 msgr2=0x7fc42419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.964+0000 7fc423e00700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc42410a680 0x7fc42419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.964+0000 7fc423e00700 1 -- 172.21.15.79:0/3943673890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc424105040 msgr2=0x7fc42419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.964+0000 7fc423e00700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc424105040 0x7fc42419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:41.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.964+0000 7fc423e00700 1 -- 172.21.15.79:0/3943673890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42419fe00 con 0x7fc424104680 2024-06-03T02:17:41.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.964+0000 7fc423e00700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc424104680 0x7fc42419adf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc41400e9a0 tx=0x7fc41400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:41.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.965+0000 7fc420c00700 1 -- 172.21.15.79:0/3943673890 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41400ca10 con 0x7fc424104680 2024-06-03T02:17:41.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.965+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4241a00f0 con 0x7fc424104680 2024-06-03T02:17:41.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.965+0000 7fc420c00700 1 -- 172.21.15.79:0/3943673890 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41400cbb0 con 0x7fc424104680 2024-06-03T02:17:41.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.965+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4241a7970 con 0x7fc424104680 2024-06-03T02:17:41.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.965+0000 7fc420c00700 1 -- 172.21.15.79:0/3943673890 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414018680 con 0x7fc424104680 2024-06-03T02:17:41.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.966+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc424069180 con 0x7fc424104680 2024-06-03T02:17:41.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.968+0000 7fc420c00700 1 -- 172.21.15.79:0/3943673890 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc414010430 con 0x7fc424104680 2024-06-03T02:17:41.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.969+0000 7fc420c00700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc4080a2ca0 0x7fc4080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:41.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.970+0000 7fc420c00700 1 -- 172.21.15.79:0/3943673890 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc4140cf9e0 con 0x7fc424104680 2024-06-03T02:17:41.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.970+0000 7fc423400700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc4080a2ca0 0x7fc4080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:41.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.971+0000 7fc423400700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc4080a2ca0 0x7fc4080a5160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc424109c20 tx=0x7fc410009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:41.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:41.972+0000 7fc420c00700 1 -- 172.21.15.79:0/3943673890 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc41409c530 con 0x7fc424104680 2024-06-03T02:17:42.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.248+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc42419c150 con 0x7fc424104680 2024-06-03T02:17:42.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.249+0000 7fc420c00700 1 -- 172.21.15.79:0/3943673890 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc41409be30 con 0x7fc424104680 2024-06-03T02:17:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:17:42.249 INFO:teuthology.orchestra.run.smithi079.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-03T02:17:42.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc4080a2ca0 msgr2=0x7fc4080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:42.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc4080a2ca0 0x7fc4080a5160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc424109c20 tx=0x7fc410009260 comp rx=0 tx=0).stop 2024-06-03T02:17:42.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc424104680 msgr2=0x7fc42419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:42.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc424104680 0x7fc42419adf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc41400e9a0 tx=0x7fc41400ecb0 comp rx=0 tx=0).stop 2024-06-03T02:17:42.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 shutdown_connections 2024-06-03T02:17:42.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc42410a680 0x7fc42419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:42.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc424104680 0x7fc42419adf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:42.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc4080a2ca0 0x7fc4080a5160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:42.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 --2- 172.21.15.79:0/3943673890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc424105040 0x7fc42419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:42.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.253+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 >> 172.21.15.79:0/3943673890 conn(0x7fc424100170 msgr2=0x7fc4241006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:42.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.254+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 shutdown_connections 2024-06-03T02:17:42.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:42.254+0000 7fc42b200700 1 -- 172.21.15.79:0/3943673890 wait complete. 2024-06-03T02:17:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:42 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3943673890' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T02:17:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:42 smithi079 ceph-mon[31819]: pgmap v446: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:42 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3943673890' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T02:17:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:42 smithi135 ceph-mon[36653]: pgmap v446: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:42 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3943673890' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T02:17:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:42 smithi123 ceph-mon[36640]: pgmap v446: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:43.138 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-03T02:17:43.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:17:43.586 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:44.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:44 smithi079 ceph-mon[31819]: pgmap v447: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:44 smithi135 ceph-mon[36653]: pgmap v447: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:44 smithi123 ceph-mon[36640]: pgmap v447: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:45.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.646+0000 7f3a2c600700 1 -- 172.21.15.79:0/2377866042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406a170 msgr2=0x7f3a2406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:45.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.646+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2377866042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406a170 0x7f3a2406a5f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009990 tx=0x7f3a10009ca0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.647+0000 7f3a2c600700 1 -- 172.21.15.79:0/2377866042 shutdown_connections 2024-06-03T02:17:45.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.647+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2377866042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a2406ab30 0x7f3a2410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.647+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2377866042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a24108500 0x7f3a241088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.647+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2377866042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406a170 0x7f3a2406a5f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.647+0000 7f3a2c600700 1 -- 172.21.15.79:0/2377866042 >> 172.21.15.79:0/2377866042 conn(0x7f3a24078340 msgr2=0x7f3a24078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:45.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.647+0000 7f3a2c600700 1 -- 172.21.15.79:0/2377866042 shutdown_connections 2024-06-03T02:17:45.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.647+0000 7f3a2c600700 1 -- 172.21.15.79:0/2377866042 wait complete. 2024-06-03T02:17:45.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.648+0000 7f3a2c600700 1 Processor -- start 2024-06-03T02:17:45.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.648+0000 7f3a2c600700 1 -- start start 2024-06-03T02:17:45.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a2c600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a2406a170 0x7f3a2419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:45.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a2c600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406ab30 0x7f3a2419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:45.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a2c600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a24108500 0x7f3a2419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:45.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a2c600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24112780 con 0x7f3a2406ab30 2024-06-03T02:17:45.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a2c600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24112600 con 0x7f3a24108500 2024-06-03T02:17:45.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a2c600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24112900 con 0x7f3a2406a170 2024-06-03T02:17:45.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a29200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406ab30 0x7f3a2419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:45.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a29c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a2406a170 0x7f3a2419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:45.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a29200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406ab30 0x7f3a2419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50230/0 (socket says 172.21.15.79:50230) 2024-06-03T02:17:45.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.649+0000 7f3a29200700 1 -- 172.21.15.79:0/2772031525 learned_addr learned my addr 172.21.15.79:0/2772031525 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:45.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a29200700 1 -- 172.21.15.79:0/2772031525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a2406a170 msgr2=0x7f3a2419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a29200700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a2406a170 0x7f3a2419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a29200700 1 -- 172.21.15.79:0/2772031525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a24108500 msgr2=0x7f3a2419f770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a29200700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a24108500 0x7f3a2419f770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a29200700 1 -- 172.21.15.79:0/2772031525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2419feb0 con 0x7f3a2406ab30 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a29200700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406ab30 0x7f3a2419b3e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000bae0 tx=0x7f3a1000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a22000700 1 -- 172.21.15.79:0/2772031525 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10026070 con 0x7f3a2406ab30 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a22000700 1 -- 172.21.15.79:0/2772031525 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a10003d70 con 0x7f3a2406ab30 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a241a0140 con 0x7f3a2406ab30 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.650+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a241a7a20 con 0x7f3a2406ab30 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.651+0000 7f3a22000700 1 -- 172.21.15.79:0/2772031525 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10021410 con 0x7f3a2406ab30 2024-06-03T02:17:45.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.652+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a24069180 con 0x7f3a2406ab30 2024-06-03T02:17:45.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.655+0000 7f3a22000700 1 -- 172.21.15.79:0/2772031525 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a10029070 con 0x7f3a2406ab30 2024-06-03T02:17:45.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.657+0000 7f3a22000700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3a0c0a2c20 0x7f3a0c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:45.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.657+0000 7f3a22000700 1 -- 172.21.15.79:0/2772031525 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3a100d0a90 con 0x7f3a2406ab30 2024-06-03T02:17:45.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.657+0000 7f3a29c00700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3a0c0a2c20 0x7f3a0c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:45.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.658+0000 7f3a29c00700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3a0c0a2c20 0x7f3a0c0a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009be0 tx=0x7f3a18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:45.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.660+0000 7f3a22000700 1 -- 172.21.15.79:0/2772031525 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3a10013070 con 0x7f3a2406ab30 2024-06-03T02:17:45.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.934+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a241a7c50 con 0x7f3a2406ab30 2024-06-03T02:17:45.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.936+0000 7f3a22000700 1 -- 172.21.15.79:0/2772031525 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7f3a1009d670 con 0x7f3a2406ab30 2024-06-03T02:17:45.937 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:17:45.937 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:15:40.029293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24490,24482,24500,24515]},"id":1}]} 2024-06-03T02:17:45.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.939+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3a0c0a2c20 msgr2=0x7f3a0c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:45.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.939+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3a0c0a2c20 0x7f3a0c0a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009be0 tx=0x7f3a18009260 comp rx=0 tx=0).stop 2024-06-03T02:17:45.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.939+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406ab30 msgr2=0x7f3a2419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:45.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.939+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406ab30 0x7f3a2419b3e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000bae0 tx=0x7f3a1000bb10 comp rx=0 tx=0).stop 2024-06-03T02:17:45.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.940+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 shutdown_connections 2024-06-03T02:17:45.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.940+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a24108500 0x7f3a2419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.940+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a2406a170 0x7f3a2419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.940+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3a0c0a2c20 0x7f3a0c0a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.940+0000 7f3a2c600700 1 --2- 172.21.15.79:0/2772031525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a2406ab30 0x7f3a2419b3e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:45.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.940+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 >> 172.21.15.79:0/2772031525 conn(0x7f3a24078340 msgr2=0x7f3a240ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:45.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.940+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 shutdown_connections 2024-06-03T02:17:45.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:45.940+0000 7f3a2c600700 1 -- 172.21.15.79:0/2772031525 wait complete. 2024-06-03T02:17:45.942 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 29 2024-06-03T02:17:46.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:45 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2772031525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:17:46.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:45 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2772031525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:17:46.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:45 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2772031525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:17:46.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:17:47.243 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:47.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:46 smithi079 ceph-mon[31819]: pgmap v448: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:17:47.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:46 smithi135 ceph-mon[36653]: pgmap v448: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:17:47.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:46 smithi123 ceph-mon[36640]: pgmap v448: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:17:48.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:48 smithi079 ceph-mon[31819]: pgmap v449: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:48 smithi135 ceph-mon[36653]: pgmap v449: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:48 smithi123 ceph-mon[36640]: pgmap v449: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T02:17:49.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.432+0000 7f9631600700 1 -- 172.21.15.79:0/2177966922 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 msgr2=0x7f962c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:49.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.432+0000 7f9631600700 1 --2- 172.21.15.79:0/2177966922 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 0x7f962c102b20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f961c0099c0 tx=0x7f961c009cd0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.433+0000 7f9631600700 1 -- 172.21.15.79:0/2177966922 shutdown_connections 2024-06-03T02:17:49.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.433+0000 7f9631600700 1 --2- 172.21.15.79:0/2177966922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f962c107840 0x7f962c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.433+0000 7f9631600700 1 --2- 172.21.15.79:0/2177966922 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 0x7f962c102b20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.433+0000 7f9631600700 1 --2- 172.21.15.79:0/2177966922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f962c1030f0 0x7f962c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.433+0000 7f9631600700 1 -- 172.21.15.79:0/2177966922 >> 172.21.15.79:0/2177966922 conn(0x7f962c0795f0 msgr2=0x7f962c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:49.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.433+0000 7f9631600700 1 -- 172.21.15.79:0/2177966922 shutdown_connections 2024-06-03T02:17:49.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.433+0000 7f9631600700 1 -- 172.21.15.79:0/2177966922 wait complete. 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.434+0000 7f9631600700 1 Processor -- start 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.434+0000 7f9631600700 1 -- start start 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.434+0000 7f9631600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 0x7f962c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.434+0000 7f9631600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f962c1030f0 0x7f962c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.434+0000 7f9631600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f962c107840 0x7f962c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.434+0000 7f9631600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1127b0 con 0x7f962c102740 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.434+0000 7f9631600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f962c112630 con 0x7f962c1030f0 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.434+0000 7f9631600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f962c112930 con 0x7f962c107840 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f962aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 0x7f962c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:49.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f9622000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f962c1030f0 0x7f962c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f962aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 0x7f962c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50246/0 (socket says 172.21.15.79:50246) 2024-06-03T02:17:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f9622000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f962c1030f0 0x7f962c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:57970/0 (socket says 172.21.15.79:57970) 2024-06-03T02:17:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f962aa00700 1 -- 172.21.15.79:0/2779494255 learned_addr learned my addr 172.21.15.79:0/2779494255 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:49.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f962b400700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f962c107840 0x7f962c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f9622000700 1 -- 172.21.15.79:0/2779494255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f962c107840 msgr2=0x7f962c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f9622000700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f962c107840 0x7f962c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f9622000700 1 -- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 msgr2=0x7f962c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f9622000700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 0x7f962c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f9622000700 1 -- 172.21.15.79:0/2779494255 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c19fdc0 con 0x7f962c1030f0 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.435+0000 7f962aa00700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 0x7f962c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.436+0000 7f9622000700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f962c1030f0 0x7f962c19b2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9f0 tx=0x7f961400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.436+0000 7f9623e00700 1 -- 172.21.15.79:0/2779494255 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400c9c0 con 0x7f962c1030f0 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.436+0000 7f9623e00700 1 -- 172.21.15.79:0/2779494255 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961400cb60 con 0x7f962c1030f0 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.436+0000 7f9623e00700 1 -- 172.21.15.79:0/2779494255 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614018680 con 0x7f962c1030f0 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.436+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c1a00b0 con 0x7f962c1030f0 2024-06-03T02:17:49.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.436+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c1a7930 con 0x7f962c1030f0 2024-06-03T02:17:49.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.437+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962c069180 con 0x7f962c1030f0 2024-06-03T02:17:49.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.439+0000 7f9623e00700 1 -- 172.21.15.79:0/2779494255 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9614010430 con 0x7f962c1030f0 2024-06-03T02:17:49.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.440+0000 7f9623e00700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96100a2c20 0x7f96100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:49.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.441+0000 7f9623e00700 1 -- 172.21.15.79:0/2779494255 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f96140cf560 con 0x7f962c1030f0 2024-06-03T02:17:49.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.441+0000 7f962aa00700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96100a2c20 0x7f96100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:49.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.441+0000 7f962aa00700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96100a2c20 0x7f96100a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f961c000c00 tx=0x7f961c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:49.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.445+0000 7f9623e00700 1 -- 172.21.15.79:0/2779494255 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f961409c0b0 con 0x7f962c1030f0 2024-06-03T02:17:49.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.717+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f962c1a7b30 con 0x7f962c1030f0 2024-06-03T02:17:49.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.719+0000 7f9623e00700 1 -- 172.21.15.79:0/2779494255 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7f961409b9b0 con 0x7f962c1030f0 2024-06-03T02:17:49.719 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:17:49.719 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:15:40.029293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24490,24482,24500,24515]},"id":1}]} 2024-06-03T02:17:49.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96100a2c20 msgr2=0x7f96100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:49.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96100a2c20 0x7f96100a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f961c000c00 tx=0x7f961c005fb0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f962c1030f0 msgr2=0x7f962c19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:49.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f962c1030f0 0x7f962c19b2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9f0 tx=0x7f961400ed00 comp rx=0 tx=0).stop 2024-06-03T02:17:49.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 shutdown_connections 2024-06-03T02:17:49.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f962c107840 0x7f962c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96100a2c20 0x7f96100a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f962c102740 0x7f962c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 --2- 172.21.15.79:0/2779494255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f962c1030f0 0x7f962c19b2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:49.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.722+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 >> 172.21.15.79:0/2779494255 conn(0x7f962c0795f0 msgr2=0x7f962c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:49.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.723+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 shutdown_connections 2024-06-03T02:17:49.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:49.723+0000 7f9631600700 1 -- 172.21.15.79:0/2779494255 wait complete. 2024-06-03T02:17:49.724 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 29 2024-06-03T02:17:50.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:49 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2779494255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:17:50.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:49 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2779494255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:17:50.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:49 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2779494255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:17:50.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:17:50.968 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:51.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:50 smithi079 ceph-mon[31819]: pgmap v450: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:17:51.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:50 smithi135 ceph-mon[36653]: pgmap v450: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:17:51.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:50 smithi123 ceph-mon[36640]: pgmap v450: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:17:52.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:52 smithi079 ceph-mon[31819]: pgmap v451: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:52 smithi135 ceph-mon[36653]: pgmap v451: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:52 smithi123 ceph-mon[36640]: pgmap v451: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:17:53.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.362+0000 7f23e3000700 1 -- 172.21.15.79:0/965803165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc102740 msgr2=0x7f23dc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:53.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.362+0000 7f23e3000700 1 --2- 172.21.15.79:0/965803165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc102740 0x7f23dc102b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009920 tx=0x7f23cc009c30 comp rx=0 tx=0).stop 2024-06-03T02:17:53.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.362+0000 7f23e3000700 1 -- 172.21.15.79:0/965803165 shutdown_connections 2024-06-03T02:17:53.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.362+0000 7f23e3000700 1 --2- 172.21.15.79:0/965803165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23dc107840 0x7f23dc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:53.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.362+0000 7f23e3000700 1 --2- 172.21.15.79:0/965803165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc102740 0x7f23dc102b20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:53.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.362+0000 7f23e3000700 1 --2- 172.21.15.79:0/965803165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23dc1030f0 0x7f23dc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:53.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.362+0000 7f23e3000700 1 -- 172.21.15.79:0/965803165 >> 172.21.15.79:0/965803165 conn(0x7f23dc0795f0 msgr2=0x7f23dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:53.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.362+0000 7f23e3000700 1 -- 172.21.15.79:0/965803165 shutdown_connections 2024-06-03T02:17:53.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.362+0000 7f23e3000700 1 -- 172.21.15.79:0/965803165 wait complete. 2024-06-03T02:17:53.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.363+0000 7f23e3000700 1 Processor -- start 2024-06-03T02:17:53.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.363+0000 7f23e3000700 1 -- start start 2024-06-03T02:17:53.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23e3000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23dc102740 0x7f23dc19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:53.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23e3000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23dc1030f0 0x7f23dc19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:53.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23e3000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc107840 0x7f23dc19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:53.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23e3000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc074fc0 con 0x7f23dc102740 2024-06-03T02:17:53.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23e3000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc074e40 con 0x7f23dc107840 2024-06-03T02:17:53.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23e3000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc075140 con 0x7f23dc1030f0 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23dbe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23dc102740 0x7f23dc19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23dbe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23dc102740 0x7f23dc19b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50278/0 (socket says 172.21.15.79:50278) 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23dbe00700 1 -- 172.21.15.79:0/1079492216 learned_addr learned my addr 172.21.15.79:0/1079492216 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.364+0000 7f23e1000700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc107840 0x7f23dc19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.365+0000 7f23dbe00700 1 -- 172.21.15.79:0/1079492216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23dc1030f0 msgr2=0x7f23dc19b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.365+0000 7f23dbe00700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23dc1030f0 0x7f23dc19b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.365+0000 7f23dbe00700 1 -- 172.21.15.79:0/1079492216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc107840 msgr2=0x7f23dc19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.365+0000 7f23dbe00700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc107840 0x7f23dc19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.365+0000 7f23dbe00700 1 -- 172.21.15.79:0/1079492216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc19fe10 con 0x7f23dc102740 2024-06-03T02:17:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.365+0000 7f23dbe00700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23dc102740 0x7f23dc19b1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23cc000c00 tx=0x7f23cc00fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.365+0000 7f23d8c00700 1 -- 172.21.15.79:0/1079492216 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc027070 con 0x7f23dc102740 2024-06-03T02:17:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.365+0000 7f23d8c00700 1 -- 172.21.15.79:0/1079492216 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23cc00b560 con 0x7f23dc102740 2024-06-03T02:17:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.365+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc1a00a0 con 0x7f23dc102740 2024-06-03T02:17:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.366+0000 7f23d8c00700 1 -- 172.21.15.79:0/1079492216 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc004a50 con 0x7f23dc102740 2024-06-03T02:17:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.366+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc1a0530 con 0x7f23dc102740 2024-06-03T02:17:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.367+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc069180 con 0x7f23dc102740 2024-06-03T02:17:53.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.369+0000 7f23d8c00700 1 -- 172.21.15.79:0/1079492216 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f23cc02a070 con 0x7f23dc102740 2024-06-03T02:17:53.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.370+0000 7f23d8c00700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f23c00a2c20 0x7f23c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:53.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.371+0000 7f23d8c00700 1 -- 172.21.15.79:0/1079492216 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f23cc0d0410 con 0x7f23dc102740 2024-06-03T02:17:53.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.371+0000 7f23db400700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f23c00a2c20 0x7f23c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:53.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.372+0000 7f23db400700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f23c00a2c20 0x7f23c00a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009750 tx=0x7f23c8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:53.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.378+0000 7f23d8c00700 1 -- 172.21.15.79:0/1079492216 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f23cc031080 con 0x7f23dc102740 2024-06-03T02:17:53.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.606+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23dc1a79b0 con 0x7f23dc102740 2024-06-03T02:17:53.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.610+0000 7f23d8c00700 1 -- 172.21.15.79:0/1079492216 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+24805 (secure 0 0 0) 0x7f23dc1a0530 con 0x7f23dc102740 2024-06-03T02:17:53.611 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:17:53.611 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":101,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:15:54.257434+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f23c00a2c20 msgr2=0x7f23c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f23c00a2c20 0x7f23c00a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009750 tx=0x7f23c8009310 comp rx=0 tx=0).stop 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23dc102740 msgr2=0x7f23dc19b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23dc102740 0x7f23dc19b1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23cc000c00 tx=0x7f23cc00fb20 comp rx=0 tx=0).stop 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 shutdown_connections 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23dc107840 0x7f23dc19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f23c00a2c20 0x7f23c00a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23dc102740 0x7f23dc19b1a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 --2- 172.21.15.79:0/1079492216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23dc1030f0 0x7f23dc19b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:53.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.613+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 >> 172.21.15.79:0/1079492216 conn(0x7f23dc0795f0 msgr2=0x7f23dc106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:53.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.614+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 shutdown_connections 2024-06-03T02:17:53.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:53.614+0000 7f23e3000700 1 -- 172.21.15.79:0/1079492216 wait complete. 2024-06-03T02:17:53.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:53 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1079492216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:17:53.996 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:53 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1079492216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:17:54.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:53 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1079492216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:17:54.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:17:54.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:54 smithi079 ceph-mon[31819]: pgmap v452: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:54.963 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:54.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:54 smithi123 ceph-mon[36640]: pgmap v452: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:55.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:54 smithi135 ceph-mon[36653]: pgmap v452: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:17:56.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:56 smithi079 ceph-mon[31819]: pgmap v453: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:17:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:56 smithi135 ceph-mon[36653]: pgmap v453: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:17:56.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:56 smithi123 ceph-mon[36640]: pgmap v453: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:17:57.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.383+0000 7f835ca00700 1 -- 172.21.15.79:0/1832179171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8358074fe0 msgr2=0x7f835810e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:57.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.383+0000 7f835ca00700 1 --2- 172.21.15.79:0/1832179171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8358074fe0 0x7f835810e980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8340009990 tx=0x7f8340009ca0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.384+0000 7f835ca00700 1 -- 172.21.15.79:0/1832179171 shutdown_connections 2024-06-03T02:17:57.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.384+0000 7f835ca00700 1 --2- 172.21.15.79:0/1832179171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f835810eec0 0x7f83581112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.384+0000 7f835ca00700 1 --2- 172.21.15.79:0/1832179171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83580746c0 0x7f8358074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.384+0000 7f835ca00700 1 --2- 172.21.15.79:0/1832179171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8358074fe0 0x7f835810e980 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.384+0000 7f835ca00700 1 -- 172.21.15.79:0/1832179171 >> 172.21.15.79:0/1832179171 conn(0x7f8358079560 msgr2=0x7f8358079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:57.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.384+0000 7f835ca00700 1 -- 172.21.15.79:0/1832179171 shutdown_connections 2024-06-03T02:17:57.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.384+0000 7f835ca00700 1 -- 172.21.15.79:0/1832179171 wait complete. 2024-06-03T02:17:57.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.385+0000 7f835ca00700 1 Processor -- start 2024-06-03T02:17:57.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.385+0000 7f835ca00700 1 -- start start 2024-06-03T02:17:57.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f835ca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83580746c0 0x7f835819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:57.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f835ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8358074fe0 0x7f835819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:57.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f835ca00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f835810eec0 0x7f835819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:57.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f835ca00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83581149d0 con 0x7f835810eec0 2024-06-03T02:17:57.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f835ca00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8358114850 con 0x7f83580746c0 2024-06-03T02:17:57.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f835ca00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8358114b50 con 0x7f8358074fe0 2024-06-03T02:17:57.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f8356800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f835810eec0 0x7f835819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:57.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f8356800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f835810eec0 0x7f835819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36782/0 (socket says 172.21.15.79:36782) 2024-06-03T02:17:57.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f8356800700 1 -- 172.21.15.79:0/2146656202 learned_addr learned my addr 172.21.15.79:0/2146656202 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:17:57.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f8355e00700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83580746c0 0x7f835819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:57.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.386+0000 7f834d400700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8358074fe0 0x7f835819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f8356800700 1 -- 172.21.15.79:0/2146656202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8358074fe0 msgr2=0x7f835819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f8356800700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8358074fe0 0x7f835819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f8356800700 1 -- 172.21.15.79:0/2146656202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83580746c0 msgr2=0x7f835819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f8356800700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83580746c0 0x7f835819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f8356800700 1 -- 172.21.15.79:0/2146656202 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835819fe00 con 0x7f835810eec0 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f8356800700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f835810eec0 0x7f835819f6c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f835000c110 tx=0x7f835000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f834f400700 1 -- 172.21.15.79:0/2146656202 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83500092e0 con 0x7f835810eec0 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f834f400700 1 -- 172.21.15.79:0/2146656202 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8350003a00 con 0x7f835810eec0 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f834f400700 1 -- 172.21.15.79:0/2146656202 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350004680 con 0x7f835810eec0 2024-06-03T02:17:57.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.387+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83581a0090 con 0x7f835810eec0 2024-06-03T02:17:57.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.388+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83581a7970 con 0x7f835810eec0 2024-06-03T02:17:57.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.389+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8358075db0 con 0x7f835810eec0 2024-06-03T02:17:57.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.391+0000 7f834f400700 1 -- 172.21.15.79:0/2146656202 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f835001a020 con 0x7f835810eec0 2024-06-03T02:17:57.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.392+0000 7f834f400700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f833c0a2ca0 0x7f833c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:17:57.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.392+0000 7f834f400700 1 -- 172.21.15.79:0/2146656202 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f83500ced00 con 0x7f835810eec0 2024-06-03T02:17:57.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.393+0000 7f8355e00700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f833c0a2ca0 0x7f833c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:17:57.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.393+0000 7f8355e00700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f833c0a2ca0 0x7f833c0a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8348009c40 tx=0x7f8348009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:17:57.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.397+0000 7f834f400700 1 -- 172.21.15.79:0/2146656202 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8350010040 con 0x7f835810eec0 2024-06-03T02:17:57.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.674+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83581a7b40 con 0x7f835810eec0 2024-06-03T02:17:57.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.676+0000 7f834f400700 1 -- 172.21.15.79:0/2146656202 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7f835009b850 con 0x7f835810eec0 2024-06-03T02:17:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:17:57.676 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:15:40.029293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24490,24482,24500,24515]},"id":1}]} 2024-06-03T02:17:57.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f833c0a2ca0 msgr2=0x7f833c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:57.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f833c0a2ca0 0x7f833c0a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8348009c40 tx=0x7f8348009260 comp rx=0 tx=0).stop 2024-06-03T02:17:57.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f835810eec0 msgr2=0x7f835819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:17:57.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f835810eec0 0x7f835819f6c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f835000c110 tx=0x7f835000c420 comp rx=0 tx=0).stop 2024-06-03T02:17:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 shutdown_connections 2024-06-03T02:17:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f833c0a2ca0 0x7f833c0a5160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f835810eec0 0x7f835819f6c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83580746c0 0x7f835819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 --2- 172.21.15.79:0/2146656202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8358074fe0 0x7f835819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:17:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 >> 172.21.15.79:0/2146656202 conn(0x7f8358079560 msgr2=0x7f83580ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:17:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 shutdown_connections 2024-06-03T02:17:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:17:57.679+0000 7f835ca00700 1 -- 172.21.15.79:0/2146656202 wait complete. 2024-06-03T02:17:57.681 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 29 2024-06-03T02:17:58.065 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:57 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2146656202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:17:58.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:57 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2146656202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:17:58.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:57 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2146656202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:17:58.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-03T02:14:12.550646+0000', 'modified': '2024-06-03T02:15:40.029293+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24515, 'mds_2': 24500, 'mds_3': 24490, 'mds_4': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24448': {'gid': 24448, 'name': 'j', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/859282294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 859282294}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 859282294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/296204924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 296204924}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 296204924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6835/3846200406', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 3846200406}, {'type': 'v1', 'addr': '172.21.15.135:6835', 'nonce': 3846200406}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6837/476881413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 476881413}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 476881413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6837/755294241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6836', 'nonce': 755294241}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 755294241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 34, 'addr': '172.21.15.123:6835/4173566972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 4173566972}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 4173566972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.135:6839/414076797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 414076797}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 414076797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.123:6837/963974438', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 963974438}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 963974438}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.79:6839/2483452059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 2483452059}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 2483452059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.123:6839/1178082108', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 1178082108}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 1178082108}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24539, 24490, 24482, 24500, 24515]} 2024-06-03T02:17:58.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-03T02:17:58.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-06-03T02:17:58.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:17:58 smithi079 ceph-mon[31819]: pgmap v454: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:17:59.049 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:17:59.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:17:58 smithi135 ceph-mon[36653]: pgmap v454: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:17:59.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:17:58 smithi123 ceph-mon[36640]: pgmap v454: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:00 smithi079 ceph-mon[31819]: pgmap v455: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:00 smithi135 ceph-mon[36653]: pgmap v455: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:00.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:00 smithi123 ceph-mon[36640]: pgmap v455: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:01.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.304+0000 7fd0ac000700 1 -- 172.21.15.79:0/2721063984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a4108500 msgr2=0x7fd0a41088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:01.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.304+0000 7fd0ac000700 1 --2- 172.21.15.79:0/2721063984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a4108500 0x7fd0a41088e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b0a0 tx=0x7fd09800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.304+0000 7fd0ac000700 1 -- 172.21.15.79:0/2721063984 shutdown_connections 2024-06-03T02:18:01.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.304+0000 7fd0ac000700 1 --2- 172.21.15.79:0/2721063984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0a406ab30 0x7fd0a410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.304+0000 7fd0ac000700 1 --2- 172.21.15.79:0/2721063984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a4108500 0x7fd0a41088e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.304+0000 7fd0ac000700 1 --2- 172.21.15.79:0/2721063984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0a406a170 0x7fd0a406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.304+0000 7fd0ac000700 1 -- 172.21.15.79:0/2721063984 >> 172.21.15.79:0/2721063984 conn(0x7fd0a4078340 msgr2=0x7fd0a4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:01.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.304+0000 7fd0ac000700 1 -- 172.21.15.79:0/2721063984 shutdown_connections 2024-06-03T02:18:01.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.304+0000 7fd0ac000700 1 -- 172.21.15.79:0/2721063984 wait complete. 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.305+0000 7fd0ac000700 1 Processor -- start 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0ac000700 1 -- start start 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0ac000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0a406a170 0x7fd0a419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0ac000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a406ab30 0x7fd0a419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0ac000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0a4108500 0x7fd0a419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0ac000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41127d0 con 0x7fd0a406ab30 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0ac000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4112650 con 0x7fd0a406a170 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0ac000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4112950 con 0x7fd0a4108500 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0a8c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a406ab30 0x7fd0a419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0a8c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a406ab30 0x7fd0a419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36814/0 (socket says 172.21.15.79:36814) 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0a8c00700 1 -- 172.21.15.79:0/1821195773 learned_addr learned my addr 172.21.15.79:0/1821195773 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0a9600700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0a406a170 0x7fd0a419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.306+0000 7fd0aa000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0a4108500 0x7fd0a419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.307+0000 7fd0a8c00700 1 -- 172.21.15.79:0/1821195773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0a4108500 msgr2=0x7fd0a419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.307+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0a4108500 0x7fd0a419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.307+0000 7fd0a8c00700 1 -- 172.21.15.79:0/1821195773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0a406a170 msgr2=0x7fd0a419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.307+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0a406a170 0x7fd0a419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.307+0000 7fd0a8c00700 1 -- 172.21.15.79:0/1821195773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a419fe70 con 0x7fd0a406ab30 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.307+0000 7fd0a9600700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0a406a170 0x7fd0a419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.307+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a406ab30 0x7fd0a419b3a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e9a0 tx=0x7fd09000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.307+0000 7fd0a2000700 1 -- 172.21.15.79:0/1821195773 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09000ca10 con 0x7fd0a406ab30 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.308+0000 7fd0a2000700 1 -- 172.21.15.79:0/1821195773 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09000cbb0 con 0x7fd0a406ab30 2024-06-03T02:18:01.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.308+0000 7fd0ac000700 1 -- 172.21.15.79:0/1821195773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a41a0160 con 0x7fd0a406ab30 2024-06-03T02:18:01.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.308+0000 7fd0a2000700 1 -- 172.21.15.79:0/1821195773 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090018680 con 0x7fd0a406ab30 2024-06-03T02:18:01.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.308+0000 7fd0ac000700 1 -- 172.21.15.79:0/1821195773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a79e0 con 0x7fd0a406ab30 2024-06-03T02:18:01.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.312+0000 7fd0a2000700 1 -- 172.21.15.79:0/1821195773 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd090010430 con 0x7fd0a406ab30 2024-06-03T02:18:01.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.312+0000 7fd0b696fb80 1 Processor -- start 2024-06-03T02:18:01.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.312+0000 7fd0b696fb80 1 -- start start 2024-06-03T02:18:01.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 0x564efd375cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd376630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564efd2849c0 con 0x564efd3761f0 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564efd2c7ab0 con 0x564efd37c460 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564efd2828a0 con 0x564efd37bb40 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0a2000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd08c0a2c20 0x7fd08c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0a2000700 1 -- 172.21.15.79:0/1821195773 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd0900cf630 con 0x7fd0a406ab30 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0aa000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd08c0a2c20 0x7fd08c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0a9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 0x564efd375cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0a8c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd376630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0a9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 0x564efd375cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38932/0 (socket says 172.21.15.79:38932) 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.313+0000 7fd0a9600700 1 -- 172.21.15.79:0/2335226700 learned_addr learned my addr 172.21.15.79:0/2335226700 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0aa000700 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0a9600700 1 -- 172.21.15.79:0/2335226700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 msgr2=0x564efd37bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0a9600700 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0a9600700 1 -- 172.21.15.79:0/2335226700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 msgr2=0x564efd376630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:01.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0a9600700 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd376630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0a9600700 1 -- 172.21.15.79:0/2335226700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564efd26ff40 con 0x564efd37c460 2024-06-03T02:18:01.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0aa000700 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:18:01.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd376630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:18:01.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0a9600700 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 0x564efd375cb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b0a0 tx=0x7fd098009e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bdda08517362c55 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:01.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.314+0000 7fd0aa000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd08c0a2c20 0x7fd08c0a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009ee0 tx=0x7fd09c009980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:01.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.315+0000 7fd096a00700 1 -- 172.21.15.79:0/2335226700 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098021070 con 0x564efd37c460 2024-06-03T02:18:01.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.315+0000 7fd096a00700 1 -- 172.21.15.79:0/2335226700 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd098009460 con 0x564efd37c460 2024-06-03T02:18:01.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.315+0000 7fd096a00700 1 -- 172.21.15.79:0/2335226700 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980052a0 con 0x564efd37c460 2024-06-03T02:18:01.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 -- 172.21.15.79:0/2335226700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 msgr2=0x564efd375cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:01.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 0x564efd375cb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b0a0 tx=0x7fd098009e10 comp rx=0 tx=0).stop 2024-06-03T02:18:01.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 -- 172.21.15.79:0/2335226700 shutdown_connections 2024-06-03T02:18:01.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd376630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/2335226700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 0x564efd375cb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 -- 172.21.15.79:0/2335226700 >> 172.21.15.79:0/2335226700 conn(0x564efd2a2120 msgr2=0x564efd370fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:01.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 -- 172.21.15.79:0/2335226700 shutdown_connections 2024-06-03T02:18:01.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 -- 172.21.15.79:0/2335226700 wait complete. 2024-06-03T02:18:01.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.316+0000 7fd0b696fb80 1 -- start start 2024-06-03T02:18:01.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.317+0000 7fd0b696fb80 1 Processor -- start 2024-06-03T02:18:01.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.317+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd382140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.317+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.317+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37c460 0x564efd37f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.317+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564efd263f10 con 0x564efd37c460 2024-06-03T02:18:01.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.317+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564efd2c3a70 con 0x564efd3761f0 2024-06-03T02:18:01.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.317+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564efd382de0 con 0x564efd37bb40 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.317+0000 7fd0b696fb80 10 client.0 _ll_register_callbacks cb 0x564efd25d330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a8c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd382140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd382140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38944/0 (socket says 172.21.15.79:38944) 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a8c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56380/0 (socket says 172.21.15.79:56380) 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a9600700 1 -- 172.21.15.79:0/4210712509 learned_addr learned my addr 172.21.15.79:0/4210712509 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0aa000700 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37c460 0x564efd37f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a9600700 1 -- 172.21.15.79:0/4210712509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 msgr2=0x564efd37f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a9600700 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a9600700 1 -- 172.21.15.79:0/4210712509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37c460 msgr2=0x564efd37f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a9600700 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37c460 0x564efd37f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a9600700 1 -- 172.21.15.79:0/4210712509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564efd216880 con 0x564efd3761f0 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:01.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd0a9600700 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd382140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b070 tx=0x7fd09800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04ba9908da6a6e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.318+0000 7fd08b400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.319+0000 7fd08b400700 1 -- 172.21.15.79:0/4210712509 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098021040 con 0x564efd3761f0 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.319+0000 7fd0b696fb80 1 -- 172.21.15.79:0/4210712509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd098009460 con 0x564efd3761f0 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.319+0000 7fd08b400700 1 -- 172.21.15.79:0/4210712509 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd098005250 con 0x564efd3761f0 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.319+0000 7fd08b400700 1 -- 172.21.15.79:0/4210712509 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd098038050 con 0x564efd3761f0 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.319+0000 7fd0b696fb80 10 client.24974 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.319+0000 7fd0b696fb80 1 -- 172.21.15.79:0/4210712509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564efd38d440 con 0x564efd3761f0 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.320+0000 7fd08b400700 1 -- 172.21.15.79:0/4210712509 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd098009460 con 0x564efd3761f0 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.320+0000 7fd08b400700 1 -- 172.21.15.79:0/4210712509 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd098011070 con 0x564efd3761f0 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.320+0000 7fd0b696fb80 10 client.24974 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.320+0000 7fd0b696fb80 10 client.24974 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.320+0000 7fd0b696fb80 20 client.24974 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:01.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.320+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x564efd38f550 0x564efd3909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.320+0000 7fd0b696fb80 4 client.24974 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:18:01.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.320+0000 7fd0b696fb80 1 -- 172.21.15.79:0/4210712509 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564efd2d4fb0 con 0x564efd38f550 2024-06-03T02:18:01.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.320+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x564efd38f550 0x564efd3909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.321+0000 7fd08b400700 10 client.24974 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.321+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x564efd38f550 0x564efd3909c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5aea030c07356952 server_cookie=4d95ec2c25321ee6 in_seq=0 out_seq=0 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd08b400700 1 -- 172.21.15.79:0/4210712509 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564efd2d4fb0 con 0x564efd38f550 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd08b400700 10 client.24974 handle_command_reply: tid=0 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 client.24974 shutdown 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 -- 172.21.15.79:0/4210712509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 msgr2=0x564efd382140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd382140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b070 tx=0x7fd09800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 -- 172.21.15.79:0/4210712509 shutdown_connections 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x564efd38f550 0x564efd3909c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37c460 0x564efd37f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd382140 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd08b400700 0 client.24974 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:18:01.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/4210712509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd37f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 -- 172.21.15.79:0/4210712509 >> 172.21.15.79:0/4210712509 conn(0x564efd2a2120 msgr2=0x564efd36a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:01.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 -- 172.21.15.79:0/4210712509 shutdown_connections 2024-06-03T02:18:01.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 1 -- 172.21.15.79:0/4210712509 wait complete. 2024-06-03T02:18:01.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.322+0000 7fd0b696fb80 20 client.24974 trim_cache size 0 max 16384 2024-06-03T02:18:01.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.345+0000 7fd0b696fb80 1 Processor -- start 2024-06-03T02:18:01.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.345+0000 7fd0b696fb80 1 -- start start 2024-06-03T02:18:01.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd370db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37bb40 0x564efd37d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37c460 0x564efd36b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564efd38ed00 con 0x564efd37bb40 2024-06-03T02:18:01.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564efd38f3d0 con 0x564efd3761f0 2024-06-03T02:18:01.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564efd38faa0 con 0x564efd37c460 2024-06-03T02:18:01.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0a8c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37bb40 0x564efd37d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0a8c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37bb40 0x564efd37d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36846/0 (socket says 172.21.15.79:36846) 2024-06-03T02:18:01.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0a8c00700 1 -- 172.21.15.79:0/1646659931 learned_addr learned my addr 172.21.15.79:0/1646659931 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:01.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0a9600700 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd370db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.346+0000 7fd0aa000700 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37c460 0x564efd36b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd0a8c00700 1 -- 172.21.15.79:0/1646659931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37c460 msgr2=0x564efd36b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:01.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37c460 0x564efd36b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd0a8c00700 1 -- 172.21.15.79:0/1646659931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 msgr2=0x564efd370db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd370db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd0a8c00700 1 -- 172.21.15.79:0/1646659931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564efd37b590 con 0x564efd37bb40 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd0a9600700 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd370db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37bb40 0x564efd37d0d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd0900ccdd0 tx=0x7fd09009b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba85df066c10d4b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd096000700 1 -- 172.21.15.79:0/1646659931 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09009fe90 con 0x564efd37bb40 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd096000700 1 -- 172.21.15.79:0/1646659931 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0900a4750 con 0x564efd37bb40 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.347+0000 7fd096000700 1 -- 172.21.15.79:0/1646659931 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09009bd40 con 0x564efd37bb40 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.348+0000 7fd0b696fb80 1 -- 172.21.15.79:0/1646659931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37bb40 msgr2=0x564efd37d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.348+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37bb40 0x564efd37d0d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd0900ccdd0 tx=0x7fd09009b8d0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.348+0000 7fd0b696fb80 1 -- 172.21.15.79:0/1646659931 shutdown_connections 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.348+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37c460 0x564efd36b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.348+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd3761f0 0x564efd370db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.348+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/1646659931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd37bb40 0x564efd37d0d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.348+0000 7fd0b696fb80 1 -- 172.21.15.79:0/1646659931 >> 172.21.15.79:0/1646659931 conn(0x564efd2a2120 msgr2=0x564efd38b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:01.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.348+0000 7fd0b696fb80 1 -- 172.21.15.79:0/1646659931 shutdown_connections 2024-06-03T02:18:01.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.348+0000 7fd0b696fb80 1 -- 172.21.15.79:0/1646659931 wait complete. 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.349+0000 7fd0b696fb80 1 -- start start 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.349+0000 7fd0b696fb80 1 Processor -- start 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.349+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd39a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.349+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd395790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.349+0000 7fd0b696fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 0x564efd395cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.349+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564efd390840 con 0x564efd3761f0 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564efd390170 con 0x564efd37c460 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0b696fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564efd390f10 con 0x564efd37bb40 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0b696fb80 10 client.0 _ll_register_callbacks cb 0x564efd25d330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:01.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a9600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd39a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a8c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd395790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a9600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd39a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36860/0 (socket says 172.21.15.79:36860) 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a9600700 1 -- 172.21.15.79:0/376398598 learned_addr learned my addr 172.21.15.79:0/376398598 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a8c00700 1 -- 172.21.15.79:0/376398598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 msgr2=0x564efd395cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 0x564efd395cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a8c00700 1 -- 172.21.15.79:0/376398598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 msgr2=0x564efd39a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd39a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a8c00700 1 -- 172.21.15.79:0/376398598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564efd3963a0 con 0x564efd37bb40 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.350+0000 7fd0a9600700 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd39a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.351+0000 7fd0a8c00700 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd395790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0900ccdd0 tx=0x7fd09009fe90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6b32c605f84fad server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.351+0000 7fd08b400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.351+0000 7fd08b400700 1 -- 172.21.15.79:0/376398598 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09007f9b0 con 0x564efd37bb40 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.351+0000 7fd08b400700 1 -- 172.21.15.79:0/376398598 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd09007fb50 con 0x564efd37bb40 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.351+0000 7fd0b696fb80 1 -- 172.21.15.79:0/376398598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0900a4750 con 0x564efd37bb40 2024-06-03T02:18:01.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.351+0000 7fd08b400700 1 -- 172.21.15.79:0/376398598 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd09009bd40 con 0x564efd37bb40 2024-06-03T02:18:01.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.351+0000 7fd0b696fb80 10 client.24958 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:01.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.351+0000 7fd0b696fb80 1 -- 172.21.15.79:0/376398598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564efd3facd0 con 0x564efd37bb40 2024-06-03T02:18:01.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.351+0000 7fd08b400700 1 -- 172.21.15.79:0/376398598 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0900a4750 con 0x564efd37bb40 2024-06-03T02:18:01.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.352+0000 7fd08b400700 1 -- 172.21.15.79:0/376398598 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd090083040 con 0x564efd37bb40 2024-06-03T02:18:01.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.352+0000 7fd0b696fb80 10 client.24958 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:01.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.352+0000 7fd0b696fb80 10 client.24958 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.352+0000 7fd0b696fb80 20 client.24958 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.352+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x564efd38f550 0x564efd3fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.352+0000 7fd0b696fb80 4 client.24958 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.352+0000 7fd0b696fb80 1 -- 172.21.15.79:0/376398598 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "status"}) -- 0x564efd2cf530 con 0x564efd38f550 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.352+0000 7fd0a9600700 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x564efd38f550 0x564efd3fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0a9600700 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x564efd38f550 0x564efd3fe0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96591f2c10b8ec7b server_cookie=8e45e138e275a2f5 in_seq=0 out_seq=0 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd08b400700 10 client.24958 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd08b400700 1 -- 172.21.15.79:0/376398598 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x564efd2cf530 con 0x564efd38f550 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd08b400700 10 client.24958 handle_command_reply: tid=0 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0b696fb80 1 client.24958 shutdown 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0b696fb80 1 -- 172.21.15.79:0/376398598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 msgr2=0x564efd395790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd395790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0900ccdd0 tx=0x7fd09009fe90 comp rx=0 tx=0).stop 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0b696fb80 1 -- 172.21.15.79:0/376398598 shutdown_connections 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x564efd38f550 0x564efd3fe0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564efd37c460 0x564efd395cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564efd3761f0 0x564efd39a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0b696fb80 1 --2- 172.21.15.79:0/376398598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564efd37bb40 0x564efd395790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd08b400700 0 client.24958 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:18:01.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.353+0000 7fd0b696fb80 1 -- 172.21.15.79:0/376398598 >> 172.21.15.79:0/376398598 conn(0x564efd2a2120 msgr2=0x564efd37ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:01.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.354+0000 7fd0b696fb80 1 -- 172.21.15.79:0/376398598 shutdown_connections 2024-06-03T02:18:01.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.354+0000 7fd0b696fb80 1 -- 172.21.15.79:0/376398598 wait complete. 2024-06-03T02:18:01.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.354+0000 7fd0b696fb80 20 client.24958 trim_cache size 0 max 16384 2024-06-03T02:18:01.359 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:18:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 4, 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24482, 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 146.05822598399999, 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 276.01115699000002 2024-06-03T02:18:01.360 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:18:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 -- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd08c0a2c20 msgr2=0x7fd08c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd08c0a2c20 0x7fd08c0a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009ee0 tx=0x7fd09c009980 comp rx=0 tx=0).stop 2024-06-03T02:18:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 -- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a406ab30 msgr2=0x7fd0a419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a406ab30 0x7fd0a419b3a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e9a0 tx=0x7fd09000ecb0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 -- 172.21.15.79:0/1821195773 shutdown_connections 2024-06-03T02:18:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0a4108500 0x7fd0a419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0a406a170 0x7fd0a419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd08c0a2c20 0x7fd08c0a50e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 --2- 172.21.15.79:0/1821195773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0a406ab30 0x7fd0a419b3a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:01.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.355+0000 7fd0ac000700 1 -- 172.21.15.79:0/1821195773 >> 172.21.15.79:0/1821195773 conn(0x7fd0a4078340 msgr2=0x7fd0a410d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:01.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.356+0000 7fd0ac000700 1 -- 172.21.15.79:0/1821195773 shutdown_connections 2024-06-03T02:18:01.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:01.356+0000 7fd0ac000700 1 -- 172.21.15.79:0/1821195773 wait complete. 2024-06-03T02:18:02.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-03T02:18:02.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:02 smithi079 ceph-mon[31819]: pgmap v456: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:02 smithi135 ceph-mon[36653]: pgmap v456: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:02.691 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:02.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:02 smithi123 ceph-mon[36640]: pgmap v456: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:04.580 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:04 smithi079 ceph-mon[31819]: pgmap v457: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:04.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:04 smithi135 ceph-mon[36653]: pgmap v457: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:04 smithi123 ceph-mon[36640]: pgmap v457: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:04.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.906+0000 7f5d1e800700 1 -- 172.21.15.79:0/2761171970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d18104f80 msgr2=0x7f5d18111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:04.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.906+0000 7f5d1e800700 1 --2- 172.21.15.79:0/2761171970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d18104f80 0x7f5d18111490 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000b0a0 tx=0x7f5d1000b3b0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.907+0000 7f5d1e800700 1 -- 172.21.15.79:0/2761171970 shutdown_connections 2024-06-03T02:18:04.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.907+0000 7f5d1e800700 1 --2- 172.21.15.79:0/2761171970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d18104f80 0x7f5d18111490 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.907+0000 7f5d1e800700 1 --2- 172.21.15.79:0/2761171970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1810a650 0x7f5d1810aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.907+0000 7f5d1e800700 1 --2- 172.21.15.79:0/2761171970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d181045c0 0x7f5d18104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.907+0000 7f5d1e800700 1 -- 172.21.15.79:0/2761171970 >> 172.21.15.79:0/2761171970 conn(0x7f5d181000b0 msgr2=0x7f5d181024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:04.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.907+0000 7f5d1e800700 1 -- 172.21.15.79:0/2761171970 shutdown_connections 2024-06-03T02:18:04.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.907+0000 7f5d1e800700 1 -- 172.21.15.79:0/2761171970 wait complete. 2024-06-03T02:18:04.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.908+0000 7f5d1e800700 1 Processor -- start 2024-06-03T02:18:04.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.908+0000 7f5d1e800700 1 -- start start 2024-06-03T02:18:04.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d181045c0 0x7f5d180769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d18104f80 0x7f5d18076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1e800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1810a650 0x7f5d18077420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1e800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18114960 con 0x7f5d181045c0 2024-06-03T02:18:04.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1e800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d181147e0 con 0x7f5d1810a650 2024-06-03T02:18:04.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1e800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18114ae0 con 0x7f5d18104f80 2024-06-03T02:18:04.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d181045c0 0x7f5d180769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d181045c0 0x7f5d180769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57374/0 (socket says 172.21.15.79:57374) 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1d400700 1 -- 172.21.15.79:0/4210100367 learned_addr learned my addr 172.21.15.79:0/4210100367 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1de00700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1810a650 0x7f5d18077420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.909+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d18104f80 0x7f5d18076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d1d400700 1 -- 172.21.15.79:0/4210100367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d18104f80 msgr2=0x7f5d18076ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d1d400700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d18104f80 0x7f5d18076ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d1d400700 1 -- 172.21.15.79:0/4210100367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1810a650 msgr2=0x7f5d18077420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d1d400700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1810a650 0x7f5d18077420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d1d400700 1 -- 172.21.15.79:0/4210100367 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d181ab320 con 0x7f5d181045c0 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d1d400700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d181045c0 0x7f5d180769a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00e950 tx=0x7f5d0c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d16000700 1 -- 172.21.15.79:0/4210100367 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c004230 con 0x7f5d181045c0 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d16000700 1 -- 172.21.15.79:0/4210100367 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0c0043d0 con 0x7f5d181045c0 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d16000700 1 -- 172.21.15.79:0/4210100367 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c003dc0 con 0x7f5d181045c0 2024-06-03T02:18:04.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.910+0000 7f5d1e800700 1 -- 172.21.15.79:0/4210100367 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d181ab610 con 0x7f5d181045c0 2024-06-03T02:18:04.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.911+0000 7f5d1e800700 1 -- 172.21.15.79:0/4210100367 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d181abb00 con 0x7f5d181045c0 2024-06-03T02:18:04.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.914+0000 7f5d16000700 1 -- 172.21.15.79:0/4210100367 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d0c00c7e0 con 0x7f5d181045c0 2024-06-03T02:18:04.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.915+0000 7f5d16000700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d000a2c20 0x7f5d000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.915+0000 7f5d2a74eb80 1 Processor -- start 2024-06-03T02:18:04.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.915+0000 7f5d16000700 1 -- 172.21.15.79:0/4210100367 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5d0c0cf6d0 con 0x7f5d181045c0 2024-06-03T02:18:04.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d2a74eb80 1 -- start start 2024-06-03T02:18:04.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d000a2c20 0x7f5d000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba0d10 0x55e300ba10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 0x55e300babfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300bac4e0 0x55e300bae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e300ab09c0 con 0x55e300bac4e0 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e300af3ab0 con 0x55e300ba1630 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e300aae8a0 con 0x55e300ba0d10 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba0d10 0x55e300ba10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba0d10 0x55e300ba10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42552/0 (socket says 172.21.15.79:42552) 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d1d400700 1 -- 172.21.15.79:0/4068620351 learned_addr learned my addr 172.21.15.79:0/4068620351 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.916+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/4068620351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 0x55e300babfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d000a2c20 0x7f5d000a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d18078160 tx=0x7f5d04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d1de00700 1 --2- 172.21.15.79:0/4068620351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300bac4e0 0x55e300bae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d1d400700 1 -- 172.21.15.79:0/4068620351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 msgr2=0x55e300babfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d1d400700 1 --2- 172.21.15.79:0/4068620351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 0x55e300babfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d1d400700 1 -- 172.21.15.79:0/4068620351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300bac4e0 msgr2=0x55e300bae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:04.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d1d400700 1 --2- 172.21.15.79:0/4068620351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300bac4e0 0x55e300bae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d1d400700 1 -- 172.21.15.79:0/4068620351 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e300a9bf40 con 0x55e300ba0d10 2024-06-03T02:18:04.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d1d400700 1 --2- 172.21.15.79:0/4068620351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba0d10 0x55e300ba10f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0037d0 tx=0x7f5d0c09b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b746dc183d4296f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:04.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d0aa00700 1 -- 172.21.15.79:0/4068620351 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c09c220 con 0x55e300ba0d10 2024-06-03T02:18:04.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d0aa00700 1 -- 172.21.15.79:0/4068620351 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0c09ce50 con 0x55e300ba0d10 2024-06-03T02:18:04.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.917+0000 7f5d0aa00700 1 -- 172.21.15.79:0/4068620351 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c00cad0 con 0x55e300ba0d10 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.918+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/4068620351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba0d10 msgr2=0x55e300ba10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.918+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/4068620351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba0d10 0x55e300ba10f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0037d0 tx=0x7f5d0c09b8f0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.918+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/4068620351 shutdown_connections 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.918+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/4068620351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300bac4e0 0x55e300bae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.918+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/4068620351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba0d10 0x55e300ba10f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.918+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/4068620351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 0x55e300babfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.918+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/4068620351 >> 172.21.15.79:0/4068620351 conn(0x55e300ace120 msgr2=0x55e300b9d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.918+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/4068620351 shutdown_connections 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.919+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/4068620351 wait complete. 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.919+0000 7f5d2a74eb80 1 -- start start 2024-06-03T02:18:04.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.919+0000 7f5d2a74eb80 1 Processor -- start 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300babce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba1630 0x55e300ba8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300bac4e0 0x55e300ba9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e300a8ff10 con 0x55e300ba0d10 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e300aefa70 con 0x55e300bac4e0 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e300baef70 con 0x55e300ba1630 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d2a74eb80 10 client.0 _ll_register_callbacks cb 0x55e300a89330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300babce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300babce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57390/0 (socket says 172.21.15.79:57390) 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d1d400700 1 -- 172.21.15.79:0/251124647 learned_addr learned my addr 172.21.15.79:0/251124647 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba1630 0x55e300ba8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.920+0000 7f5d1de00700 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300bac4e0 0x55e300ba9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5d1d400700 1 -- 172.21.15.79:0/251124647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba1630 msgr2=0x55e300ba8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:04.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5d1d400700 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba1630 0x55e300ba8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5d1d400700 1 -- 172.21.15.79:0/251124647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300bac4e0 msgr2=0x55e300ba9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5d1d400700 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300bac4e0 0x55e300ba9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5d1d400700 1 -- 172.21.15.79:0/251124647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e300a42880 con 0x55e300ba0d10 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5d1d400700 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300babce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0046c0 tx=0x7f5d0c00c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59174f12ae34f67a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5cff400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5cff400700 1 -- 172.21.15.79:0/251124647 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c09bf80 con 0x55e300ba0d10 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5cff400700 1 -- 172.21.15.79:0/251124647 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5d0c0a4300 con 0x55e300ba0d10 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.921+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/251124647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0c09ce50 con 0x55e300ba0d10 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5cff400700 1 -- 172.21.15.79:0/251124647 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d0c082050 con 0x55e300ba0d10 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5d2a74eb80 10 client.14871 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:04.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/251124647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e300bb96e0 con 0x55e300ba0d10 2024-06-03T02:18:04.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5cff400700 1 -- 172.21.15.79:0/251124647 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0c09ce50 con 0x55e300ba0d10 2024-06-03T02:18:04.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5cff400700 1 -- 172.21.15.79:0/251124647 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5d0c0cf070 con 0x55e300ba0d10 2024-06-03T02:18:04.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5d2a74eb80 10 client.14871 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:04.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5d2a74eb80 10 client.14871 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T02:18:04.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5d2a74eb80 20 client.14871 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55e300bbb820 0x55e300bbcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5d2a74eb80 4 client.14871 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.922+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/251124647 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e300b00fb0 con 0x55e300bbb820 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.923+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55e300bbb820 0x55e300bbcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.923+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55e300bbb820 0x55e300bbcc90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a10d0b243a817e6 server_cookie=25f532ca71def845 in_seq=0 out_seq=0 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.923+0000 7f5cff400700 10 client.14871 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.924+0000 7f5cff400700 1 -- 172.21.15.79:0/251124647 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e300b00fb0 con 0x55e300bbb820 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.924+0000 7f5cff400700 10 client.14871 handle_command_reply: tid=0 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.924+0000 7f5d2a74eb80 1 client.14871 shutdown 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.924+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/251124647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 msgr2=0x55e300babce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.924+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300babce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0046c0 tx=0x7f5d0c00c4a0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/251124647 shutdown_connections 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300bac4e0 0x55e300ba9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55e300bbb820 0x55e300bbcc90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300babce0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/251124647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300ba1630 0x55e300ba8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/251124647 >> 172.21.15.79:0/251124647 conn(0x55e300ace120 msgr2=0x55e300b9cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:04.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5cff400700 0 client.14871 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:18:04.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/251124647 shutdown_connections 2024-06-03T02:18:04.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/251124647 wait complete. 2024-06-03T02:18:04.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.925+0000 7f5d2a74eb80 20 client.14871 trim_cache size 0 max 16384 2024-06-03T02:18:04.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.948+0000 7f5d2a74eb80 1 Processor -- start 2024-06-03T02:18:04.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d2a74eb80 1 -- start start 2024-06-03T02:18:04.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300ba7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 0x55e300bab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 0x55e300ba6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e300bbafd0 con 0x55e300ba0d10 2024-06-03T02:18:04.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e300bbb6a0 con 0x55e300ba1630 2024-06-03T02:18:04.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e300bbbd70 con 0x55e300bac4e0 2024-06-03T02:18:04.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300ba7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300ba7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57400/0 (socket says 172.21.15.79:57400) 2024-06-03T02:18:04.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d1d400700 1 -- 172.21.15.79:0/2210882463 learned_addr learned my addr 172.21.15.79:0/2210882463 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:04.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 0x55e300bab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.949+0000 7f5d1de00700 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 0x55e300ba6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d1d400700 1 -- 172.21.15.79:0/2210882463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 msgr2=0x55e300ba6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:04.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d1d400700 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 0x55e300ba6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d1d400700 1 -- 172.21.15.79:0/2210882463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 msgr2=0x55e300bab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:04.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d1d400700 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 0x55e300bab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d1d400700 1 -- 172.21.15.79:0/2210882463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e300ba1c80 con 0x55e300ba0d10 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 0x55e300bab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d1d400700 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300ba7fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0cf360 tx=0x7f5d0c09b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8d8910180e873e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d0a000700 1 -- 172.21.15.79:0/2210882463 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c0ba530 con 0x55e300ba0d10 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d0a000700 1 -- 172.21.15.79:0/2210882463 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0c09ce50 con 0x55e300ba0d10 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.950+0000 7f5d0a000700 1 -- 172.21.15.79:0/2210882463 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c0a4300 con 0x55e300ba0d10 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.951+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2210882463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 msgr2=0x55e300ba7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.951+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300ba7fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0cf360 tx=0x7f5d0c09b8f0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.951+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2210882463 shutdown_connections 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.951+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 0x55e300ba6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.951+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba0d10 0x55e300ba7fe0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.951+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2210882463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba1630 0x55e300bab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.951+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2210882463 >> 172.21.15.79:0/2210882463 conn(0x55e300ace120 msgr2=0x55e300ba78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:04.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.951+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2210882463 shutdown_connections 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.951+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2210882463 wait complete. 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.952+0000 7f5d2a74eb80 1 -- start start 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.952+0000 7f5d2a74eb80 1 Processor -- start 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.952+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba0d10 0x55e300bc6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba1630 0x55e300bc67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d2a74eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 0x55e300bc0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e300bbcb10 con 0x55e300ba1630 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e300bbc440 con 0x55e300ba0d10 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d2a74eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e300bbd1e0 con 0x55e300bac4e0 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d2a74eb80 10 client.0 _ll_register_callbacks cb 0x55e300a89330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba0d10 0x55e300bc6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba0d10 0x55e300bc6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60910/0 (socket says 172.21.15.79:60910) 2024-06-03T02:18:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1d400700 1 -- 172.21.15.79:0/2747174864 learned_addr learned my addr 172.21.15.79:0/2747174864 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba1630 0x55e300bc67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1de00700 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 0x55e300bc0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1d400700 1 -- 172.21.15.79:0/2747174864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 msgr2=0x55e300bc0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1d400700 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 0x55e300bc0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1d400700 1 -- 172.21.15.79:0/2747174864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba1630 msgr2=0x55e300bc67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1d400700 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba1630 0x55e300bc67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.953+0000 7f5d1d400700 1 -- 172.21.15.79:0/2747174864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e300bc0e10 con 0x55e300ba0d10 2024-06-03T02:18:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.954+0000 7f5d1d400700 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba0d10 0x55e300bc6290 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c004000 tx=0x7f5d0c00c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2b46b5dfb2e650 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.954+0000 7f5cff400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.954+0000 7f5cff400700 1 -- 172.21.15.79:0/2747174864 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c0a8070 con 0x55e300ba0d10 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.954+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2747174864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0c09ce50 con 0x55e300ba0d10 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.954+0000 7f5cff400700 1 -- 172.21.15.79:0/2747174864 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5d0c09bd80 con 0x55e300ba0d10 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.954+0000 7f5cff400700 1 -- 172.21.15.79:0/2747174864 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d0c0a4790 con 0x55e300ba0d10 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.954+0000 7f5d2a74eb80 10 client.24995 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.954+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2747174864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e300bc0e10 con 0x55e300ba0d10 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.955+0000 7f5cff400700 1 -- 172.21.15.79:0/2747174864 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0c09ce50 con 0x55e300ba0d10 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.955+0000 7f5cff400700 1 -- 172.21.15.79:0/2747174864 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5d0c0ab030 con 0x55e300ba0d10 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.955+0000 7f5d2a74eb80 10 client.24995 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.955+0000 7f5d2a74eb80 10 client.24995 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.955+0000 7f5d2a74eb80 20 client.24995 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.955+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55e300c2bf50 0x55e300c2ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.955+0000 7f5d2a74eb80 4 client.24995 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.955+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2747174864 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "status"}) -- 0x55e300afb530 con 0x55e300c2bf50 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.955+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55e300c2bf50 0x55e300c2ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.956+0000 7f5d1ca00700 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55e300c2bf50 0x55e300c2ac60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=42b2ee2e579bae4f server_cookie=68ffc518bbcc3687 in_seq=0 out_seq=0 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.956+0000 7f5cff400700 10 client.24995 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.956+0000 7f5cff400700 1 -- 172.21.15.79:0/2747174864 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55e300afb530 con 0x55e300c2bf50 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.956+0000 7f5cff400700 10 client.24995 handle_command_reply: tid=0 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.956+0000 7f5d2a74eb80 1 client.24995 shutdown 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.956+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2747174864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba0d10 msgr2=0x55e300bc6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.956+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba0d10 0x55e300bc6290 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c004000 tx=0x7f5d0c00c4a0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2747174864 shutdown_connections 2024-06-03T02:18:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e300bac4e0 0x55e300bc0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e300ba0d10 0x55e300bc6290 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55e300c2bf50 0x55e300c2ac60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5d2a74eb80 1 --2- 172.21.15.79:0/2747174864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e300ba1630 0x55e300bc67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5cff400700 0 client.24995 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2747174864 >> 172.21.15.79:0/2747174864 conn(0x55e300ace120 msgr2=0x55e300ba4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2747174864 shutdown_connections 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5d2a74eb80 1 -- 172.21.15.79:0/2747174864 wait complete. 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.957+0000 7f5d2a74eb80 20 client.24995 trim_cache size 0 max 16384 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:18:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 3, 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24490, 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 150.66665827400001, 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 267.91321931599998 2024-06-03T02:18:04.960 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 -- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d000a2c20 msgr2=0x7f5d000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d000a2c20 0x7f5d000a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d18078160 tx=0x7f5d04009260 comp rx=0 tx=0).stop 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 -- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d181045c0 msgr2=0x7f5d180769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d181045c0 0x7f5d180769a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00e950 tx=0x7f5d0c00ec60 comp rx=0 tx=0).stop 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 -- 172.21.15.79:0/4210100367 shutdown_connections 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1810a650 0x7f5d18077420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d000a2c20 0x7f5d000a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d181045c0 0x7f5d180769a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 --2- 172.21.15.79:0/4210100367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d18104f80 0x7f5d18076ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 -- 172.21.15.79:0/4210100367 >> 172.21.15.79:0/4210100367 conn(0x7f5d181000b0 msgr2=0x7f5d18101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 -- 172.21.15.79:0/4210100367 shutdown_connections 2024-06-03T02:18:04.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:04.958+0000 7f5d1e800700 1 -- 172.21.15.79:0/4210100367 wait complete. 2024-06-03T02:18:05.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-03T02:18:06.318 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:06 smithi079 ceph-mon[31819]: pgmap v458: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:06 smithi135 ceph-mon[36653]: pgmap v458: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:06.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:06 smithi123 ceph-mon[36640]: pgmap v458: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:08.418 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:08 smithi079 ceph-mon[31819]: pgmap v459: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:08.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.580+0000 7fe92f800700 1 -- 172.21.15.79:0/3478361915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9281030f0 msgr2=0x7fe928107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:08.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.580+0000 7fe92f800700 1 --2- 172.21.15.79:0/3478361915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9281030f0 0x7fe928107110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe9140099c0 tx=0x7fe914009cd0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.580+0000 7fe92f800700 1 -- 172.21.15.79:0/3478361915 shutdown_connections 2024-06-03T02:18:08.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.580+0000 7fe92f800700 1 --2- 172.21.15.79:0/3478361915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe928107840 0x7fe92810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.580+0000 7fe92f800700 1 --2- 172.21.15.79:0/3478361915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe928102740 0x7fe928102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.580+0000 7fe92f800700 1 --2- 172.21.15.79:0/3478361915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9281030f0 0x7fe928107110 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.580+0000 7fe92f800700 1 -- 172.21.15.79:0/3478361915 >> 172.21.15.79:0/3478361915 conn(0x7fe9280795f0 msgr2=0x7fe928079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:08.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.580+0000 7fe92f800700 1 -- 172.21.15.79:0/3478361915 shutdown_connections 2024-06-03T02:18:08.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.580+0000 7fe92f800700 1 -- 172.21.15.79:0/3478361915 wait complete. 2024-06-03T02:18:08.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.582+0000 7fe92f800700 1 Processor -- start 2024-06-03T02:18:08.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.582+0000 7fe92f800700 1 -- start start 2024-06-03T02:18:08.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.582+0000 7fe92f800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe928102740 0x7fe92819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.582+0000 7fe92f800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9281030f0 0x7fe92819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe92f800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe928107840 0x7fe928198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe92f800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281127a0 con 0x7fe9281030f0 2024-06-03T02:18:08.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe92f800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe928112620 con 0x7fe928102740 2024-06-03T02:18:08.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe92f800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe928112920 con 0x7fe928107840 2024-06-03T02:18:08.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe927e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9281030f0 0x7fe92819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe927e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9281030f0 0x7fe92819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57428/0 (socket says 172.21.15.79:57428) 2024-06-03T02:18:08.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe927e00700 1 -- 172.21.15.79:0/2636448094 learned_addr learned my addr 172.21.15.79:0/2636448094 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:08.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe92ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe928102740 0x7fe92819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe92d800700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe928107840 0x7fe928198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe927e00700 1 -- 172.21.15.79:0/2636448094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe928107840 msgr2=0x7fe928198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe927e00700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe928107840 0x7fe928198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe927e00700 1 -- 172.21.15.79:0/2636448094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe928102740 msgr2=0x7fe92819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe927e00700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe928102740 0x7fe92819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe927e00700 1 -- 172.21.15.79:0/2636448094 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe928199160 con 0x7fe9281030f0 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.583+0000 7fe92ce00700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe928102740 0x7fe92819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.584+0000 7fe927e00700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9281030f0 0x7fe92819ed80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe914005910 tx=0x7fe91400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.584+0000 7fe925600700 1 -- 172.21.15.79:0/2636448094 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914027070 con 0x7fe9281030f0 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.584+0000 7fe92f800700 1 -- 172.21.15.79:0/2636448094 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9281993f0 con 0x7fe9281030f0 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.584+0000 7fe92f800700 1 -- 172.21.15.79:0/2636448094 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9281a5710 con 0x7fe9281030f0 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.584+0000 7fe925600700 1 -- 172.21.15.79:0/2636448094 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe914003680 con 0x7fe9281030f0 2024-06-03T02:18:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.584+0000 7fe925600700 1 -- 172.21.15.79:0/2636448094 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9140173f0 con 0x7fe9281030f0 2024-06-03T02:18:08.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.587+0000 7fe925600700 1 -- 172.21.15.79:0/2636448094 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe914022590 con 0x7fe9281030f0 2024-06-03T02:18:08.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.589+0000 7fe925600700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe9100a2ca0 0x7fe9100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.589+0000 7fe925600700 1 -- 172.21.15.79:0/2636448094 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe9140cfde0 con 0x7fe9281030f0 2024-06-03T02:18:08.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.589+0000 7fe93a09cb80 1 Processor -- start 2024-06-03T02:18:08.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.589+0000 7fe92ce00700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe9100a2ca0 0x7fe9100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe93a09cb80 1 -- start start 2024-06-03T02:18:08.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f83995d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995d870 0x55f8399570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 0x55f839957a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f8398619c0 con 0x55f83995cf50 2024-06-03T02:18:08.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f8398a4ab0 con 0x55f83995d870 2024-06-03T02:18:08.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f83985f8a0 con 0x55f839957600 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe92ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f83995d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe92ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f83995d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57438/0 (socket says 172.21.15.79:57438) 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe92ce00700 1 -- 172.21.15.79:0/1093553552 learned_addr learned my addr 172.21.15.79:0/1093553552 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.590+0000 7fe92d800700 1 --2- 172.21.15.79:0/1093553552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 0x55f839957a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.591+0000 7fe92ce00700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe9100a2ca0 0x7fe9100a5160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe928199b40 tx=0x7fe91c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.591+0000 7fe927e00700 1 --2- 172.21.15.79:0/1093553552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995d870 0x55f8399570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.591+0000 7fe92ce00700 1 -- 172.21.15.79:0/1093553552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 msgr2=0x55f839957a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.591+0000 7fe92ce00700 1 --2- 172.21.15.79:0/1093553552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 0x55f839957a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.591+0000 7fe92ce00700 1 -- 172.21.15.79:0/1093553552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995d870 msgr2=0x55f8399570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.591+0000 7fe92ce00700 1 --2- 172.21.15.79:0/1093553552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995d870 0x55f8399570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.591+0000 7fe92ce00700 1 -- 172.21.15.79:0/1093553552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83984cf40 con 0x55f83995cf50 2024-06-03T02:18:08.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.591+0000 7fe92ce00700 1 --2- 172.21.15.79:0/1093553552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f83995d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe91c004390 tx=0x7fe91c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44cfccd43a1a629 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:08.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.591+0000 7fe91a000700 1 -- 172.21.15.79:0/1093553552 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c003d10 con 0x55f83995cf50 2024-06-03T02:18:08.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.592+0000 7fe91a000700 1 -- 172.21.15.79:0/1093553552 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c01e900 con 0x55f83995cf50 2024-06-03T02:18:08.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.592+0000 7fe91a000700 1 -- 172.21.15.79:0/1093553552 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c026910 con 0x55f83995cf50 2024-06-03T02:18:08.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.592+0000 7fe93a09cb80 1 -- 172.21.15.79:0/1093553552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 msgr2=0x55f83995d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:08.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.592+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/1093553552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f83995d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe91c004390 tx=0x7fe91c014000 comp rx=0 tx=0).stop 2024-06-03T02:18:08.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.592+0000 7fe93a09cb80 1 -- 172.21.15.79:0/1093553552 shutdown_connections 2024-06-03T02:18:08.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.592+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/1093553552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 0x55f839957a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.592+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/1093553552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f83995d330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.592+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/1093553552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995d870 0x55f8399570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.592+0000 7fe93a09cb80 1 -- 172.21.15.79:0/1093553552 >> 172.21.15.79:0/1093553552 conn(0x55f83987f120 msgr2=0x55f83994dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:08.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.593+0000 7fe93a09cb80 1 -- 172.21.15.79:0/1093553552 shutdown_connections 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.593+0000 7fe93a09cb80 1 -- 172.21.15.79:0/1093553552 wait complete. 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.593+0000 7fe93a09cb80 1 -- start start 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.593+0000 7fe93a09cb80 1 Processor -- start 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 0x55f8399635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995d870 0x55f839960e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f839840f10 con 0x55f83995d870 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f8398a0a70 con 0x55f83995cf50 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f839964250 con 0x55f839957600 2024-06-03T02:18:08.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe93a09cb80 10 client.0 _ll_register_callbacks cb 0x55f83983a330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:08.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:08 smithi135 ceph-mon[36653]: pgmap v459: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:08.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe92d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995d870 0x55f839960e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe92d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995d870 0x55f839960e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57454/0 (socket says 172.21.15.79:57454) 2024-06-03T02:18:08.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe92d800700 1 -- 172.21.15.79:0/3714431732 learned_addr learned my addr 172.21.15.79:0/3714431732 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:08.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe92ce00700 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 0x55f8399635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.594+0000 7fe927e00700 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe92d800700 1 -- 172.21.15.79:0/3714431732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 msgr2=0x55f8399635c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:08.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe92d800700 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 0x55f8399635c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe92d800700 1 -- 172.21.15.79:0/3714431732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 msgr2=0x55f8399608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe92d800700 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe92d800700 1 -- 172.21.15.79:0/3714431732 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8397f3880 con 0x55f83995d870 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe92d800700 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995d870 0x55f839960e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe920011f60 tx=0x7fe920014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aaaff4db61520be server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe90ea00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe90ea00700 1 -- 172.21.15.79:0/3714431732 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920014ba0 con 0x55f83995d870 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe90ea00700 1 -- 172.21.15.79:0/3714431732 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe920014d40 con 0x55f83995d870 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.595+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3714431732 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe91c01e900 con 0x55f83995d870 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe90ea00700 1 -- 172.21.15.79:0/3714431732 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe920027050 con 0x55f83995d870 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe93a09cb80 10 client.14889 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3714431732 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f83996ea40 con 0x55f83995d870 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe90ea00700 1 -- 172.21.15.79:0/3714431732 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c01e900 con 0x55f83995d870 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe90ea00700 1 -- 172.21.15.79:0/3714431732 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe920007500 con 0x55f83995d870 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe93a09cb80 10 client.14889 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe93a09cb80 10 client.14889 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-03T02:18:08.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe93a09cb80 20 client.14889 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:08.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55f839971070 0x55f8399724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe93a09cb80 4 client.14889 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:18:08.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.596+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3714431732 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8398b1fb0 con 0x55f839971070 2024-06-03T02:18:08.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.597+0000 7fe92ce00700 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55f839971070 0x55f8399724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.597+0000 7fe92ce00700 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55f839971070 0x55f8399724e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cdbfb3ac926e3d8 server_cookie=d8092b0da3b20a23 in_seq=0 out_seq=0 2024-06-03T02:18:08.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.597+0000 7fe90ea00700 10 client.14889 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:18:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.598+0000 7fe90ea00700 1 -- 172.21.15.79:0/3714431732 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f8398b1fb0 con 0x55f839971070 2024-06-03T02:18:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.598+0000 7fe90ea00700 10 client.14889 handle_command_reply: tid=0 2024-06-03T02:18:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.598+0000 7fe93a09cb80 1 client.14889 shutdown 2024-06-03T02:18:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3714431732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995d870 msgr2=0x55f839960e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995d870 0x55f839960e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe920011f60 tx=0x7fe920014600 comp rx=0 tx=0).stop 2024-06-03T02:18:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3714431732 shutdown_connections 2024-06-03T02:18:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995d870 0x55f839960e00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f839957600 0x55f8399635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55f839971070 0x55f8399724e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3714431732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3714431732 >> 172.21.15.79:0/3714431732 conn(0x55f83987f120 msgr2=0x55f839947740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe90ea00700 0 client.14889 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:18:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3714431732 shutdown_connections 2024-06-03T02:18:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3714431732 wait complete. 2024-06-03T02:18:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.599+0000 7fe93a09cb80 20 client.14889 trim_cache size 0 max 16384 2024-06-03T02:18:08.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.622+0000 7fe93a09cb80 1 Processor -- start 2024-06-03T02:18:08.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.622+0000 7fe93a09cb80 1 -- start start 2024-06-03T02:18:08.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f839957600 0x55f8399570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 0x55f8399601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f839970820 con 0x55f839957600 2024-06-03T02:18:08.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f839970ef0 con 0x55f83995cf50 2024-06-03T02:18:08.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f8399715c0 con 0x55f83995d870 2024-06-03T02:18:08.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe927e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe927e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60962/0 (socket says 172.21.15.79:60962) 2024-06-03T02:18:08.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe927e00700 1 -- 172.21.15.79:0/2781497013 learned_addr learned my addr 172.21.15.79:0/2781497013 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:08.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe92ce00700 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f839957600 0x55f8399570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.623+0000 7fe92d800700 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 0x55f8399601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe927e00700 1 -- 172.21.15.79:0/2781497013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 msgr2=0x55f8399601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe927e00700 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 0x55f8399601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe927e00700 1 -- 172.21.15.79:0/2781497013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f839957600 msgr2=0x55f8399570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe927e00700 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f839957600 0x55f8399570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe927e00700 1 -- 172.21.15.79:0/2781497013 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f839973440 con 0x55f83995cf50 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe92d800700 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 0x55f8399601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe927e00700 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399480e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe9140cd470 tx=0x7fe914005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=239a5fc649a2a90d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe919600700 1 -- 172.21.15.79:0/2781497013 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91409c2c0 con 0x55f83995cf50 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe919600700 1 -- 172.21.15.79:0/2781497013 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9140a0670 con 0x55f83995cf50 2024-06-03T02:18:08.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.624+0000 7fe919600700 1 -- 172.21.15.79:0/2781497013 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91409c520 con 0x55f83995cf50 2024-06-03T02:18:08.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.625+0000 7fe93a09cb80 1 -- 172.21.15.79:0/2781497013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 msgr2=0x55f8399480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:08.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.625+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399480e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe9140cd470 tx=0x7fe914005040 comp rx=0 tx=0).stop 2024-06-03T02:18:08.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.625+0000 7fe93a09cb80 1 -- 172.21.15.79:0/2781497013 shutdown_connections 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.625+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 0x55f8399601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.625+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f839957600 0x55f8399570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.625+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/2781497013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f83995cf50 0x55f8399480e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.625+0000 7fe93a09cb80 1 -- 172.21.15.79:0/2781497013 >> 172.21.15.79:0/2781497013 conn(0x55f83987f120 msgr2=0x55f83995a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.625+0000 7fe93a09cb80 1 -- 172.21.15.79:0/2781497013 shutdown_connections 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.625+0000 7fe93a09cb80 1 -- 172.21.15.79:0/2781497013 wait complete. 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.626+0000 7fe93a09cb80 1 -- start start 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.626+0000 7fe93a09cb80 1 Processor -- start 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.626+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f839957600 0x55f839962210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.626+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f839962750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe93a09cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 0x55f83997a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f839972360 con 0x55f83995cf50 2024-06-03T02:18:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f839971c90 con 0x55f839957600 2024-06-03T02:18:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe93a09cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f839972a30 con 0x55f83995d870 2024-06-03T02:18:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe93a09cb80 10 client.0 _ll_register_callbacks cb 0x55f83983a330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe927e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f839962750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe927e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f839962750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57470/0 (socket says 172.21.15.79:57470) 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe92ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f839957600 0x55f839962210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe927e00700 1 -- 172.21.15.79:0/3061391617 learned_addr learned my addr 172.21.15.79:0/3061391617 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe92d800700 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 0x55f83997a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe927e00700 1 -- 172.21.15.79:0/3061391617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 msgr2=0x55f83997a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe927e00700 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 0x55f83997a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe927e00700 1 -- 172.21.15.79:0/3061391617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f839957600 msgr2=0x55f839962210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe927e00700 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f839957600 0x55f839962210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe927e00700 1 -- 172.21.15.79:0/3061391617 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83997ae80 con 0x55f83995cf50 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.627+0000 7fe92ce00700 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f839957600 0x55f839962210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.628+0000 7fe927e00700 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f839962750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe9140228e0 tx=0x7fe914098610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09ff67c29621a54 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.628+0000 7fe90ea00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.628+0000 7fe90ea00700 1 -- 172.21.15.79:0/3061391617 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9140b4070 con 0x55f83995cf50 2024-06-03T02:18:08.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.628+0000 7fe90ea00700 1 -- 172.21.15.79:0/3061391617 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe91409c520 con 0x55f83995cf50 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.628+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3061391617 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9140a0670 con 0x55f83995cf50 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.628+0000 7fe90ea00700 1 -- 172.21.15.79:0/3061391617 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe91409c720 con 0x55f83995cf50 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.628+0000 7fe93a09cb80 10 client.14895 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.628+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3061391617 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f83997c180 con 0x55f83995cf50 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.628+0000 7fe90ea00700 1 -- 172.21.15.79:0/3061391617 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9140a0670 con 0x55f83995cf50 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.629+0000 7fe90ea00700 1 -- 172.21.15.79:0/3061391617 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe91409b710 con 0x55f83995cf50 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.629+0000 7fe93a09cb80 10 client.14895 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.629+0000 7fe93a09cb80 10 client.14895 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.629+0000 7fe93a09cb80 20 client.14895 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.629+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55f8399e5170 0x55f8399e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.629+0000 7fe93a09cb80 4 client.14895 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.629+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3061391617 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "status"}) -- 0x55f8398ac530 con 0x55f8399e5170 2024-06-03T02:18:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.629+0000 7fe92ce00700 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55f8399e5170 0x55f8399e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:08.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe90ea00700 10 client.14895 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe92ce00700 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55f8399e5170 0x55f8399e3e80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b20d08e2a3533ccc server_cookie=1688283faff4bb44 in_seq=0 out_seq=0 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe90ea00700 1 -- 172.21.15.79:0/3061391617 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55f8398ac530 con 0x55f8399e5170 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe90ea00700 10 client.14895 handle_command_reply: tid=0 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 client.14895 shutdown 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3061391617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 msgr2=0x55f839962750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f839962750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe9140228e0 tx=0x7fe914098610 comp rx=0 tx=0).stop 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3061391617 shutdown_connections 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f83995d870 0x55f83997a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55f8399e5170 0x55f8399e3e80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f839957600 0x55f839962210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 --2- 172.21.15.79:0/3061391617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f83995cf50 0x55f839962750 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3061391617 >> 172.21.15.79:0/3061391617 conn(0x55f83987f120 msgr2=0x55f83995dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe90ea00700 0 client.14895 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:18:08.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3061391617 shutdown_connections 2024-06-03T02:18:08.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.630+0000 7fe93a09cb80 1 -- 172.21.15.79:0/3061391617 wait complete. 2024-06-03T02:18:08.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.631+0000 7fe93a09cb80 20 client.14895 trim_cache size 0 max 16384 2024-06-03T02:18:08.634 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:18:08.634 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:18:08.634 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 2, 2024-06-03T02:18:08.634 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24500, 2024-06-03T02:18:08.634 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:18:08.634 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:18:08.634 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:18:08.635 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 155.35137850999999, 2024-06-03T02:18:08.635 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:18:08.635 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:18:08.635 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:18:08.635 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 274.51571116399998 2024-06-03T02:18:08.635 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:18:08.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.631+0000 7fe92f800700 1 -- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe9100a2ca0 msgr2=0x7fe9100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.631+0000 7fe92f800700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe9100a2ca0 0x7fe9100a5160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe928199b40 tx=0x7fe91c009260 comp rx=0 tx=0).stop 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.631+0000 7fe92f800700 1 -- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9281030f0 msgr2=0x7fe92819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.631+0000 7fe92f800700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9281030f0 0x7fe92819ed80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe914005910 tx=0x7fe91400f690 comp rx=0 tx=0).stop 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.632+0000 7fe92f800700 1 -- 172.21.15.79:0/2636448094 shutdown_connections 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.632+0000 7fe92f800700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe928107840 0x7fe928198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.632+0000 7fe92f800700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe928102740 0x7fe92819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.632+0000 7fe92f800700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe9100a2ca0 0x7fe9100a5160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.632+0000 7fe92f800700 1 --2- 172.21.15.79:0/2636448094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9281030f0 0x7fe92819ed80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.632+0000 7fe92f800700 1 -- 172.21.15.79:0/2636448094 >> 172.21.15.79:0/2636448094 conn(0x7fe9280795f0 msgr2=0x7fe928108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.632+0000 7fe92f800700 1 -- 172.21.15.79:0/2636448094 shutdown_connections 2024-06-03T02:18:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:08.632+0000 7fe92f800700 1 -- 172.21.15.79:0/2636448094 wait complete. 2024-06-03T02:18:08.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:08 smithi123 ceph-mon[36640]: pgmap v459: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:09.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.g status 2024-06-03T02:18:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:10.525 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:10 smithi079 ceph-mon[31819]: pgmap v460: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:10 smithi135 ceph-mon[36653]: pgmap v460: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:10.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:10 smithi123 ceph-mon[36640]: pgmap v460: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.223+0000 7fe130c00700 1 -- 172.21.15.79:0/1593133241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe12c109480 msgr2=0x7fe12c115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.223+0000 7fe130c00700 1 --2- 172.21.15.79:0/1593133241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe12c109480 0x7fe12c115990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe12400b0a0 tx=0x7fe12400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.223+0000 7fe130c00700 1 -- 172.21.15.79:0/1593133241 shutdown_connections 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.223+0000 7fe130c00700 1 --2- 172.21.15.79:0/1593133241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe12c109480 0x7fe12c115990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.223+0000 7fe130c00700 1 --2- 172.21.15.79:0/1593133241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe12c10eac0 0x7fe12c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.223+0000 7fe130c00700 1 --2- 172.21.15.79:0/1593133241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe12c108ac0 0x7fe12c108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.223+0000 7fe130c00700 1 -- 172.21.15.79:0/1593133241 >> 172.21.15.79:0/1593133241 conn(0x7fe12c075c60 msgr2=0x7fe12c1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.224+0000 7fe130c00700 1 -- 172.21.15.79:0/1593133241 shutdown_connections 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.224+0000 7fe130c00700 1 -- 172.21.15.79:0/1593133241 wait complete. 2024-06-03T02:18:12.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.225+0000 7fe130c00700 1 Processor -- start 2024-06-03T02:18:12.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.225+0000 7fe130c00700 1 -- start start 2024-06-03T02:18:12.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.225+0000 7fe130c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe12c108ac0 0x7fe12c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.225+0000 7fe130c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe12c109480 0x7fe12c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.225+0000 7fe130c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe12c10eac0 0x7fe12c1a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.225+0000 7fe130c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c118e50 con 0x7fe12c10eac0 2024-06-03T02:18:12.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.225+0000 7fe130c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c118cd0 con 0x7fe12c108ac0 2024-06-03T02:18:12.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.225+0000 7fe130c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c118fd0 con 0x7fe12c109480 2024-06-03T02:18:12.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe12c109480 0x7fe12c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe12c108ac0 0x7fe12c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe12c109480 0x7fe12c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42664/0 (socket says 172.21.15.79:42664) 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129400700 1 -- 172.21.15.79:0/712514869 learned_addr learned my addr 172.21.15.79:0/712514869 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe12a800700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe12c10eac0 0x7fe12c1a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129e00700 1 -- 172.21.15.79:0/712514869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe12c109480 msgr2=0x7fe12c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129e00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe12c109480 0x7fe12c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129e00700 1 -- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe12c10eac0 msgr2=0x7fe12c1a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129e00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe12c10eac0 0x7fe12c1a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129e00700 1 -- 172.21.15.79:0/712514869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c1a41d0 con 0x7fe12c108ac0 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.226+0000 7fe129e00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe12c108ac0 0x7fe12c19f1c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00e9f0 tx=0x7fe11c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.227+0000 7fe122a00700 1 -- 172.21.15.79:0/712514869 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c004220 con 0x7fe12c108ac0 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.227+0000 7fe122a00700 1 -- 172.21.15.79:0/712514869 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe11c004d10 con 0x7fe12c108ac0 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.227+0000 7fe130c00700 1 -- 172.21.15.79:0/712514869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c1a44c0 con 0x7fe12c108ac0 2024-06-03T02:18:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.227+0000 7fe122a00700 1 -- 172.21.15.79:0/712514869 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c005020 con 0x7fe12c108ac0 2024-06-03T02:18:12.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.227+0000 7fe130c00700 1 -- 172.21.15.79:0/712514869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c1abd40 con 0x7fe12c108ac0 2024-06-03T02:18:12.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.231+0000 7fe122a00700 1 -- 172.21.15.79:0/712514869 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe11c004750 con 0x7fe12c108ac0 2024-06-03T02:18:12.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.233+0000 7fe13b509b80 1 Processor -- start 2024-06-03T02:18:12.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.233+0000 7fe13b509b80 1 -- start start 2024-06-03T02:18:12.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.233+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa762f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.233+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 0x5640aa75ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.233+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.233+0000 7fe13b509b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5640aa66b9c0 con 0x5640aa762b40 2024-06-03T02:18:12.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.233+0000 7fe13b509b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5640aa6aeab0 con 0x5640aa75d1f0 2024-06-03T02:18:12.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.233+0000 7fe13b509b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5640aa6698a0 con 0x5640aa763460 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.233+0000 7fe122a00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe1100a2c20 0x7fe1100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe129400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe129400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:32770/0 (socket says 172.21.15.79:32770) 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe129400700 1 -- 172.21.15.79:0/1857768267 learned_addr learned my addr 172.21.15.79:0/1857768267 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe122a00700 1 -- 172.21.15.79:0/712514869 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe11c0cf290 con 0x7fe12c108ac0 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe12a800700 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa762f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe129e00700 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 0x5640aa75ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe12a800700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe1100a2c20 0x7fe1100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe129400700 1 -- 172.21.15.79:0/1857768267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 msgr2=0x5640aa75ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe129400700 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 0x5640aa75ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe129400700 1 -- 172.21.15.79:0/1857768267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 msgr2=0x5640aa762f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:12.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe129400700 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa762f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe129400700 1 -- 172.21.15.79:0/1857768267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640aa656f40 con 0x5640aa75d1f0 2024-06-03T02:18:12.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.234+0000 7fe12a800700 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa762f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:18:12.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.235+0000 7fe129400700 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75d630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe11800be60 tx=0x7fe1180106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a208533498fe9e12 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.235+0000 7fe117400700 1 -- 172.21.15.79:0/1857768267 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118011440 con 0x5640aa75d1f0 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.235+0000 7fe117400700 1 -- 172.21.15.79:0/1857768267 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe118011ac0 con 0x5640aa75d1f0 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.235+0000 7fe117400700 1 -- 172.21.15.79:0/1857768267 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118019a90 con 0x5640aa75d1f0 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.235+0000 7fe12a800700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe1100a2c20 0x7fe1100a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe124006130 tx=0x7fe12400b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.236+0000 7fe13b509b80 1 -- 172.21.15.79:0/1857768267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 msgr2=0x5640aa75d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.236+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75d630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe11800be60 tx=0x7fe1180106b0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.236+0000 7fe13b509b80 1 -- 172.21.15.79:0/1857768267 shutdown_connections 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.236+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75d630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.236+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa762f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.236+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1857768267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 0x5640aa75ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.236+0000 7fe13b509b80 1 -- 172.21.15.79:0/1857768267 >> 172.21.15.79:0/1857768267 conn(0x5640aa689120 msgr2=0x5640aa757fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:12.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.236+0000 7fe13b509b80 1 -- 172.21.15.79:0/1857768267 shutdown_connections 2024-06-03T02:18:12.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.236+0000 7fe13b509b80 1 -- 172.21.15.79:0/1857768267 wait complete. 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.237+0000 7fe13b509b80 1 -- start start 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.237+0000 7fe13b509b80 1 Processor -- start 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa75d1f0 0x5640aa7691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa7664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 0x5640aa766a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe13b509b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5640aa64af10 con 0x5640aa762b40 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe13b509b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5640aa6aaa70 con 0x5640aa763460 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe13b509b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5640aa769e40 con 0x5640aa75d1f0 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe13b509b80 10 client.0 _ll_register_callbacks cb 0x5640aa644330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:12.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe12a800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 0x5640aa766a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe129400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa7664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe12a800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 0x5640aa766a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:32780/0 (socket says 172.21.15.79:32780) 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe12a800700 1 -- 172.21.15.79:0/1419500138 learned_addr learned my addr 172.21.15.79:0/1419500138 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.238+0000 7fe129e00700 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa75d1f0 0x5640aa7691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe129e00700 1 -- 172.21.15.79:0/1419500138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 msgr2=0x5640aa766a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe129e00700 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 0x5640aa766a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe129e00700 1 -- 172.21.15.79:0/1419500138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 msgr2=0x5640aa7664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe129e00700 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa7664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe129e00700 1 -- 172.21.15.79:0/1419500138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640aa5fd880 con 0x5640aa75d1f0 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe129e00700 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa75d1f0 0x5640aa7691c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe11c09b3e0 tx=0x7fe11c09bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c9131fc0e8e0e0a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:12.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe10be00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe10be00700 1 -- 172.21.15.79:0/1419500138 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0b4060 con 0x5640aa75d1f0 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe10be00700 1 -- 172.21.15.79:0/1419500138 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe11c0cca90 con 0x5640aa75d1f0 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe13b509b80 1 -- 172.21.15.79:0/1419500138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe118011ac0 con 0x5640aa75d1f0 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe10be00700 1 -- 172.21.15.79:0/1419500138 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe11c01c040 con 0x5640aa75d1f0 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe13b509b80 10 client.25006 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.239+0000 7fe13b509b80 1 -- 172.21.15.79:0/1419500138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640aa774620 con 0x5640aa75d1f0 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.240+0000 7fe10be00700 1 -- 172.21.15.79:0/1419500138 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe118011ac0 con 0x5640aa75d1f0 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.240+0000 7fe10be00700 1 -- 172.21.15.79:0/1419500138 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe11c09ab30 con 0x5640aa75d1f0 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.240+0000 7fe13b509b80 10 client.25006 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.240+0000 7fe13b509b80 10 client.25006 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-03T02:18:12.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.240+0000 7fe13b509b80 20 client.25006 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:12.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.240+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5640aa776b90 0x5640aa778000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.240+0000 7fe13b509b80 4 client.25006 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:18:12.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.240+0000 7fe13b509b80 1 -- 172.21.15.79:0/1419500138 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640aa6bbfb0 con 0x5640aa776b90 2024-06-03T02:18:12.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.240+0000 7fe129400700 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5640aa776b90 0x5640aa778000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.241+0000 7fe129400700 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5640aa776b90 0x5640aa778000 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=322ca0ae572074b5 server_cookie=95b209596041e577 in_seq=0 out_seq=0 2024-06-03T02:18:12.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.241+0000 7fe10be00700 10 client.25006 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:18:12.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe10be00700 1 -- 172.21.15.79:0/1419500138 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5640aa6bbfb0 con 0x5640aa776b90 2024-06-03T02:18:12.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe10be00700 10 client.25006 handle_command_reply: tid=0 2024-06-03T02:18:12.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe13b509b80 1 client.25006 shutdown 2024-06-03T02:18:12.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe13b509b80 1 -- 172.21.15.79:0/1419500138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa75d1f0 msgr2=0x5640aa7691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:12.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa75d1f0 0x5640aa7691c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe11c09b3e0 tx=0x7fe11c09bc20 comp rx=0 tx=0).stop 2024-06-03T02:18:12.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe13b509b80 1 -- 172.21.15.79:0/1419500138 shutdown_connections 2024-06-03T02:18:12.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 0x5640aa766a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5640aa776b90 0x5640aa778000 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa75d1f0 0x5640aa7691c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1419500138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa7664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe10be00700 0 client.25006 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:18:12.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.242+0000 7fe13b509b80 1 -- 172.21.15.79:0/1419500138 >> 172.21.15.79:0/1419500138 conn(0x5640aa689120 msgr2=0x5640aa751740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:12.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.243+0000 7fe13b509b80 1 -- 172.21.15.79:0/1419500138 shutdown_connections 2024-06-03T02:18:12.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.243+0000 7fe13b509b80 1 -- 172.21.15.79:0/1419500138 wait complete. 2024-06-03T02:18:12.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.243+0000 7fe13b509b80 20 client.25006 trim_cache size 0 max 16384 2024-06-03T02:18:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.267+0000 7fe13b509b80 1 Processor -- start 2024-06-03T02:18:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.267+0000 7fe13b509b80 1 -- start start 2024-06-03T02:18:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.267+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.267+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa758150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.267+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 0x5640aa7520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.267+0000 7fe13b509b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5640aa776340 con 0x5640aa762b40 2024-06-03T02:18:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.267+0000 7fe13b509b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5640aa776a10 con 0x5640aa75d1f0 2024-06-03T02:18:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.267+0000 7fe13b509b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5640aa7770e0 con 0x5640aa763460 2024-06-03T02:18:12.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe129400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa758150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe12a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 0x5640aa7520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe129400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa758150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57502/0 (socket says 172.21.15.79:57502) 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe129400700 1 -- 172.21.15.79:0/1279368104 learned_addr learned my addr 172.21.15.79:0/1279368104 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe129e00700 1 --2- 172.21.15.79:0/1279368104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe129400700 1 -- 172.21.15.79:0/1279368104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 msgr2=0x5640aa7520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe129400700 1 --2- 172.21.15.79:0/1279368104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 0x5640aa7520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe129400700 1 -- 172.21.15.79:0/1279368104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 msgr2=0x5640aa75ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe129400700 1 --2- 172.21.15.79:0/1279368104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.268+0000 7fe129400700 1 -- 172.21.15.79:0/1279368104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640aa764a30 con 0x5640aa762b40 2024-06-03T02:18:12.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.269+0000 7fe129400700 1 --2- 172.21.15.79:0/1279368104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa758150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe1180195c0 tx=0x7fe1180195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cc6f16dff3a84b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.269+0000 7fe116a00700 1 -- 172.21.15.79:0/1279368104 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11801f040 con 0x5640aa762b40 2024-06-03T02:18:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.269+0000 7fe116a00700 1 -- 172.21.15.79:0/1279368104 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe11800b860 con 0x5640aa762b40 2024-06-03T02:18:12.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.269+0000 7fe116a00700 1 -- 172.21.15.79:0/1279368104 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118003be0 con 0x5640aa762b40 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 -- 172.21.15.79:0/1279368104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 msgr2=0x5640aa758150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1279368104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa758150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe1180195c0 tx=0x7fe1180195f0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 -- 172.21.15.79:0/1279368104 shutdown_connections 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1279368104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa763460 0x5640aa7520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1279368104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa75d1f0 0x5640aa75ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 --2- 172.21.15.79:0/1279368104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa762b40 0x5640aa758150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 -- 172.21.15.79:0/1279368104 >> 172.21.15.79:0/1279368104 conn(0x5640aa689120 msgr2=0x5640aa757a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 -- 172.21.15.79:0/1279368104 shutdown_connections 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 -- 172.21.15.79:0/1279368104 wait complete. 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.270+0000 7fe13b509b80 1 -- start start 2024-06-03T02:18:12.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe13b509b80 1 Processor -- start 2024-06-03T02:18:12.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa75d1f0 0x5640aa781480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa762b40 0x5640aa7819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe13b509b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 0x5640aa77c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe13b509b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5640aa777e80 con 0x5640aa75d1f0 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe13b509b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5640aa7777b0 con 0x5640aa763460 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe13b509b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5640aa778550 con 0x5640aa762b40 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe13b509b80 10 client.0 _ll_register_callbacks cb 0x5640aa644330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe129e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa75d1f0 0x5640aa781480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe129e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa75d1f0 0x5640aa781480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57508/0 (socket says 172.21.15.79:57508) 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe129e00700 1 -- 172.21.15.79:0/2507116458 learned_addr learned my addr 172.21.15.79:0/2507116458 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe129400700 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa762b40 0x5640aa7819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.271+0000 7fe12a800700 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 0x5640aa77c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe129e00700 1 -- 172.21.15.79:0/2507116458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa762b40 msgr2=0x5640aa7819c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe129e00700 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa762b40 0x5640aa7819c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe129e00700 1 -- 172.21.15.79:0/2507116458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 msgr2=0x5640aa77c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:12.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe129e00700 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 0x5640aa77c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe129e00700 1 -- 172.21.15.79:0/2507116458 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640aa77c970 con 0x5640aa75d1f0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe129e00700 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa75d1f0 0x5640aa781480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00c5b0 tx=0x7fe11c00c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a2ba7ebc7c5050 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe10be00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe10be00700 1 -- 172.21.15.79:0/2507116458 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c09bde0 con 0x5640aa75d1f0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe10be00700 1 -- 172.21.15.79:0/2507116458 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe11c097430 con 0x5640aa75d1f0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.272+0000 7fe13b509b80 1 -- 172.21.15.79:0/2507116458 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe11800b860 con 0x5640aa75d1f0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe10be00700 1 -- 172.21.15.79:0/2507116458 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe11c00c850 con 0x5640aa75d1f0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe13b509b80 10 client.14913 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe13b509b80 1 -- 172.21.15.79:0/2507116458 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640aa7e2b80 con 0x5640aa75d1f0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe10be00700 1 -- 172.21.15.79:0/2507116458 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe11800b860 con 0x5640aa75d1f0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe10be00700 1 -- 172.21.15.79:0/2507116458 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe11c097630 con 0x5640aa75d1f0 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe13b509b80 10 client.14913 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:12.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe13b509b80 10 client.14913 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-03T02:18:12.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe13b509b80 20 client.14913 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:12.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe13b509b80 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5640aa7e7560 0x5640aa7e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:12.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe13b509b80 4 client.14913 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:18:12.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.273+0000 7fe13b509b80 1 -- 172.21.15.79:0/2507116458 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "status"}) -- 0x5640aa6b6530 con 0x5640aa7e7560 2024-06-03T02:18:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.274+0000 7fe129400700 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5640aa7e7560 0x5640aa7e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.274+0000 7fe129400700 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5640aa7e7560 0x5640aa7e6270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8c32015e1d8775 server_cookie=97a7f5f3496b19fe in_seq=0 out_seq=0 2024-06-03T02:18:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.274+0000 7fe10be00700 10 client.14913 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:18:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.274+0000 7fe10be00700 1 -- 172.21.15.79:0/2507116458 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5640aa6b6530 con 0x5640aa7e7560 2024-06-03T02:18:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe10be00700 10 client.14913 handle_command_reply: tid=0 2024-06-03T02:18:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 client.14913 shutdown 2024-06-03T02:18:12.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 -- 172.21.15.79:0/2507116458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa75d1f0 msgr2=0x5640aa781480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa75d1f0 0x5640aa781480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00c5b0 tx=0x7fe11c00c6e0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 -- 172.21.15.79:0/2507116458 shutdown_connections 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640aa763460 0x5640aa77c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5640aa75d1f0 0x5640aa781480 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5640aa7e7560 0x5640aa7e6270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 --2- 172.21.15.79:0/2507116458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640aa762b40 0x5640aa7819c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 -- 172.21.15.79:0/2507116458 >> 172.21.15.79:0/2507116458 conn(0x5640aa689120 msgr2=0x5640aa763840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe10be00700 0 client.14913 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 -- 172.21.15.79:0/2507116458 shutdown_connections 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 1 -- 172.21.15.79:0/2507116458 wait complete. 2024-06-03T02:18:12.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.275+0000 7fe13b509b80 20 client.14913 trim_cache size 0 max 16384 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 1, 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24515, 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 160.00866377299999, 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 270.05767697200002 2024-06-03T02:18:12.279 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:18:12.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 -- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe1100a2c20 msgr2=0x7fe1100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:12.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe1100a2c20 0x7fe1100a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe124006130 tx=0x7fe12400b720 comp rx=0 tx=0).stop 2024-06-03T02:18:12.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 -- 172.21.15.79:0/712514869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe12c108ac0 msgr2=0x7fe12c19f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:12.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe12c108ac0 0x7fe12c19f1c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00e9f0 tx=0x7fe11c00ed00 comp rx=0 tx=0).stop 2024-06-03T02:18:12.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 -- 172.21.15.79:0/712514869 shutdown_connections 2024-06-03T02:18:12.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe1100a2c20 0x7fe1100a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe12c10eac0 0x7fe12c1a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe12c108ac0 0x7fe12c19f1c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 --2- 172.21.15.79:0/712514869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe12c109480 0x7fe12c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:12.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.276+0000 7fe130c00700 1 -- 172.21.15.79:0/712514869 >> 172.21.15.79:0/712514869 conn(0x7fe12c075c60 msgr2=0x7fe12c0761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:12.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.277+0000 7fe130c00700 1 -- 172.21.15.79:0/712514869 shutdown_connections 2024-06-03T02:18:12.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:12.277+0000 7fe130c00700 1 -- 172.21.15.79:0/712514869 wait complete. 2024-06-03T02:18:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:12 smithi079 ceph-mon[31819]: pgmap v461: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:12 smithi135 ceph-mon[36653]: pgmap v461: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:12 smithi123 ceph-mon[36640]: pgmap v461: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:13.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-03T02:18:13.648 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:13.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:13 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:18:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:13 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:18:14.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:13 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:18:14.887 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:14 smithi079 ceph-mon[31819]: pgmap v462: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:14.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:14 smithi123 ceph-mon[36640]: pgmap v462: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:15.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:14 smithi135 ceph-mon[36653]: pgmap v462: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:15.729 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:18:15.729 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:18:15.729 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:15 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:18:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.967+0000 7ffb73200700 1 -- 172.21.15.79:0/3867303474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 msgr2=0x7ffb6c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.967+0000 7ffb73200700 1 --2- 172.21.15.79:0/3867303474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 0x7ffb6c102b20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0099c0 tx=0x7ffb5c009cd0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.967+0000 7ffb73200700 1 -- 172.21.15.79:0/3867303474 shutdown_connections 2024-06-03T02:18:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.967+0000 7ffb73200700 1 --2- 172.21.15.79:0/3867303474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c107840 0x7ffb6c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.967+0000 7ffb73200700 1 --2- 172.21.15.79:0/3867303474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 0x7ffb6c102b20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.967+0000 7ffb73200700 1 --2- 172.21.15.79:0/3867303474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6c1030f0 0x7ffb6c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.967+0000 7ffb73200700 1 -- 172.21.15.79:0/3867303474 >> 172.21.15.79:0/3867303474 conn(0x7ffb6c0795f0 msgr2=0x7ffb6c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.967+0000 7ffb73200700 1 -- 172.21.15.79:0/3867303474 shutdown_connections 2024-06-03T02:18:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.967+0000 7ffb73200700 1 -- 172.21.15.79:0/3867303474 wait complete. 2024-06-03T02:18:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.968+0000 7ffb73200700 1 Processor -- start 2024-06-03T02:18:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.968+0000 7ffb73200700 1 -- start start 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb73200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 0x7ffb6c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb73200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c1030f0 0x7ffb6c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb73200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6c107840 0x7ffb6c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb73200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c1127d0 con 0x7ffb6c102740 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb73200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c112650 con 0x7ffb6c107840 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb73200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c112950 con 0x7ffb6c1030f0 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb6be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 0x7ffb6c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb6be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 0x7ffb6c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60452/0 (socket says 172.21.15.79:60452) 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb6be00700 1 -- 172.21.15.79:0/2070409125 learned_addr learned my addr 172.21.15.79:0/2070409125 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb6b400700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c1030f0 0x7ffb6c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb71200700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6c107840 0x7ffb6c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb6be00700 1 -- 172.21.15.79:0/2070409125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c1030f0 msgr2=0x7ffb6c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb6be00700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c1030f0 0x7ffb6c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb6be00700 1 -- 172.21.15.79:0/2070409125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6c107840 msgr2=0x7ffb6c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb6be00700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6c107840 0x7ffb6c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.969+0000 7ffb6be00700 1 -- 172.21.15.79:0/2070409125 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c199190 con 0x7ffb6c102740 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.970+0000 7ffb6be00700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 0x7ffb6c19e870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c006010 tx=0x7ffb5c003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.970+0000 7ffb68c00700 1 -- 172.21.15.79:0/2070409125 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c003e50 con 0x7ffb6c102740 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.970+0000 7ffb68c00700 1 -- 172.21.15.79:0/2070409125 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c0055a0 con 0x7ffb6c102740 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.970+0000 7ffb68c00700 1 -- 172.21.15.79:0/2070409125 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00fe30 con 0x7ffb6c102740 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.970+0000 7ffb73200700 1 -- 172.21.15.79:0/2070409125 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6c199420 con 0x7ffb6c102740 2024-06-03T02:18:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.970+0000 7ffb73200700 1 -- 172.21.15.79:0/2070409125 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6c1a5740 con 0x7ffb6c102740 2024-06-03T02:18:15.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:18:15.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:18:15.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:15 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:18:15.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.973+0000 7ffb68c00700 1 -- 172.21.15.79:0/2070409125 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb5c00f4f0 con 0x7ffb6c102740 2024-06-03T02:18:15.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.975+0000 7ffb68c00700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb500a2e40 0x7ffb500a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.975+0000 7ffb68c00700 1 -- 172.21.15.79:0/2070409125 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ffb5c0d02a0 con 0x7ffb6c102740 2024-06-03T02:18:15.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.975+0000 7ffb7da82b80 1 Processor -- start 2024-06-03T02:18:15.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.975+0000 7ffb6b400700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb500a2e40 0x7ffb500a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.975+0000 7ffb7da82b80 1 -- start start 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.975+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f34009f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.975+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f34003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f34004630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560f33f129c0 con 0x560f3400a460 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560f33f55ab0 con 0x560f340041f0 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560f33f108a0 con 0x560f34009b40 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f34009f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6b400700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb500a2e40 0x7ffb500a5300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c102030 tx=0x7ffb58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f34009f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56574/0 (socket says 172.21.15.79:56574) 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6be00700 1 -- 172.21.15.79:0/3390522129 learned_addr learned my addr 172.21.15.79:0/3390522129 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb71200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f34004630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6b400700 1 --2- 172.21.15.79:0/3390522129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f34003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6be00700 1 -- 172.21.15.79:0/3390522129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 msgr2=0x560f34004630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:15.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6be00700 1 --2- 172.21.15.79:0/3390522129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f34004630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6be00700 1 -- 172.21.15.79:0/3390522129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 msgr2=0x560f34003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:15.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6be00700 1 --2- 172.21.15.79:0/3390522129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f34003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6be00700 1 -- 172.21.15.79:0/3390522129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f33efdf40 con 0x560f34009b40 2024-06-03T02:18:15.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.976+0000 7ffb6b400700 1 --2- 172.21.15.79:0/3390522129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f34003cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:18:15.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.977+0000 7ffb6be00700 1 --2- 172.21.15.79:0/3390522129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f34009f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c098be0 tx=0x7ffb5c00f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4772947c49dd8e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:15.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.977+0000 7ffb61600700 1 -- 172.21.15.79:0/3390522129 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0a3070 con 0x560f34009b40 2024-06-03T02:18:15.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.977+0000 7ffb61600700 1 -- 172.21.15.79:0/3390522129 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c0cde50 con 0x560f34009b40 2024-06-03T02:18:15.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.977+0000 7ffb61600700 1 -- 172.21.15.79:0/3390522129 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c09aa80 con 0x560f34009b40 2024-06-03T02:18:15.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.978+0000 7ffb7da82b80 1 -- 172.21.15.79:0/3390522129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 msgr2=0x560f34009f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:15.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.978+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/3390522129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f34009f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c098be0 tx=0x7ffb5c00f460 comp rx=0 tx=0).stop 2024-06-03T02:18:15.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.978+0000 7ffb7da82b80 1 -- 172.21.15.79:0/3390522129 shutdown_connections 2024-06-03T02:18:15.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.978+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/3390522129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f34004630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.978+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/3390522129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f34009f20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.978+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/3390522129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f34003cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.978+0000 7ffb7da82b80 1 -- 172.21.15.79:0/3390522129 >> 172.21.15.79:0/3390522129 conn(0x560f33f30120 msgr2=0x560f33ffefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:15.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.978+0000 7ffb7da82b80 1 -- 172.21.15.79:0/3390522129 shutdown_connections 2024-06-03T02:18:15.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.978+0000 7ffb7da82b80 1 -- 172.21.15.79:0/3390522129 wait complete. 2024-06-03T02:18:15.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.979+0000 7ffb7da82b80 1 -- start start 2024-06-03T02:18:15.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.979+0000 7ffb7da82b80 1 Processor -- start 2024-06-03T02:18:15.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.979+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f340101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.979+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 0x560f3400d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.979+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f3400dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.979+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560f33ef1f10 con 0x560f3400a460 2024-06-03T02:18:15.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.979+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560f33f51a70 con 0x560f34009b40 2024-06-03T02:18:15.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.979+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560f34010bf0 con 0x560f340041f0 2024-06-03T02:18:15.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.979+0000 7ffb7da82b80 10 client.0 _ll_register_callbacks cb 0x560f33eeb330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:15.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f340101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 0x560f3400d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f340101d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56578/0 (socket says 172.21.15.79:56578) 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6be00700 1 -- 172.21.15.79:0/2965194884 learned_addr learned my addr 172.21.15.79:0/2965194884 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb71200700 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f3400dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6be00700 1 -- 172.21.15.79:0/2965194884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 msgr2=0x560f3400d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6be00700 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 0x560f3400d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6be00700 1 -- 172.21.15.79:0/2965194884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 msgr2=0x560f3400dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6be00700 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f3400dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6be00700 1 -- 172.21.15.79:0/2965194884 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f33ea4880 con 0x560f340041f0 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.980+0000 7ffb6be00700 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f340101d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c099150 tx=0x7ffb5c09c1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de5770f1041b94d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.981+0000 7ffb56000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.981+0000 7ffb56000700 1 -- 172.21.15.79:0/2965194884 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0b4020 con 0x560f340041f0 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.981+0000 7ffb56000700 1 -- 172.21.15.79:0/2965194884 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ffb5c0870d0 con 0x560f340041f0 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.981+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2965194884 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb5c0cde50 con 0x560f340041f0 2024-06-03T02:18:15.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.981+0000 7ffb56000700 1 -- 172.21.15.79:0/2965194884 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb5c09c780 con 0x560f340041f0 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.981+0000 7ffb7da82b80 10 client.25024 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.981+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2965194884 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f3401b300 con 0x560f340041f0 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.981+0000 7ffb56000700 1 -- 172.21.15.79:0/2965194884 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c0cde50 con 0x560f340041f0 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.982+0000 7ffb56000700 1 -- 172.21.15.79:0/2965194884 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffb5c098040 con 0x560f340041f0 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.982+0000 7ffb7da82b80 10 client.25024 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.982+0000 7ffb7da82b80 10 client.25024 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.982+0000 7ffb7da82b80 20 client.25024 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.982+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560f3401d6a0 0x560f3401eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.982+0000 7ffb7da82b80 4 client.25024 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.982+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2965194884 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f33f62fb0 con 0x560f3401d6a0 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.982+0000 7ffb6b400700 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560f3401d6a0 0x560f3401eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.983+0000 7ffb6b400700 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560f3401d6a0 0x560f3401eb10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91d18d6ae1f3030c server_cookie=d558f5153124440c in_seq=0 out_seq=0 2024-06-03T02:18:15.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.983+0000 7ffb56000700 10 client.25024 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:18:15.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb56000700 1 -- 172.21.15.79:0/2965194884 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560f33f62fb0 con 0x560f3401d6a0 2024-06-03T02:18:15.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb56000700 10 client.25024 handle_command_reply: tid=0 2024-06-03T02:18:15.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 client.25024 shutdown 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2965194884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 msgr2=0x560f340101d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f340101d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c099150 tx=0x7ffb5c09c1a0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2965194884 shutdown_connections 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560f3401d6a0 0x560f3401eb10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f3400dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f340101d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2965194884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 0x560f3400d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2965194884 >> 172.21.15.79:0/2965194884 conn(0x560f33f30120 msgr2=0x560f33ff8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb56000700 0 client.25024 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.984+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2965194884 shutdown_connections 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.985+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2965194884 wait complete. 2024-06-03T02:18:15.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:15.985+0000 7ffb7da82b80 20 client.25024 trim_cache size 0 max 16384 2024-06-03T02:18:16.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.008+0000 7ffb7da82b80 1 Processor -- start 2024-06-03T02:18:16.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.008+0000 7ffb7da82b80 1 -- start start 2024-06-03T02:18:16.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.008+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f34003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:16.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.008+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 0x560f33fff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:16.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.008+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f33ff90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:16.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.008+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560f3401ce50 con 0x560f3400a460 2024-06-03T02:18:16.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.008+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560f3401d520 con 0x560f34009b40 2024-06-03T02:18:16.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.008+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560f3401dbf0 con 0x560f340041f0 2024-06-03T02:18:16.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb71200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f33ff90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb6b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 0x560f33fff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb71200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f33ff90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60478/0 (socket says 172.21.15.79:60478) 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb71200700 1 -- 172.21.15.79:0/396583397 learned_addr learned my addr 172.21.15.79:0/396583397 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb6be00700 1 --2- 172.21.15.79:0/396583397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f34003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb6b400700 1 -- 172.21.15.79:0/396583397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 msgr2=0x560f34003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb6b400700 1 --2- 172.21.15.79:0/396583397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f34003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb6b400700 1 -- 172.21.15.79:0/396583397 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 msgr2=0x560f33ff90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb6b400700 1 --2- 172.21.15.79:0/396583397 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f33ff90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb6b400700 1 -- 172.21.15.79:0/396583397 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3401fa70 con 0x560f34009b40 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.009+0000 7ffb6b400700 1 --2- 172.21.15.79:0/396583397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 0x560f33fff150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x560f33f51bc0 tx=0x7ffb58014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91b96d6e829b8e76 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb60c00700 1 -- 172.21.15.79:0/396583397 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5801e9e0 con 0x560f34009b40 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb60c00700 1 -- 172.21.15.79:0/396583397 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb58004500 con 0x560f34009b40 2024-06-03T02:18:16.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb60c00700 1 -- 172.21.15.79:0/396583397 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58004dc0 con 0x560f34009b40 2024-06-03T02:18:16.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb7da82b80 1 -- 172.21.15.79:0/396583397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 msgr2=0x560f33fff150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:16.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/396583397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 0x560f33fff150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x560f33f51bc0 tx=0x7ffb58014000 comp rx=0 tx=0).stop 2024-06-03T02:18:16.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb7da82b80 1 -- 172.21.15.79:0/396583397 shutdown_connections 2024-06-03T02:18:16.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/396583397 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f33ff90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/396583397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f340041f0 0x560f34003cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/396583397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f34009b40 0x560f33fff150 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb7da82b80 1 -- 172.21.15.79:0/396583397 >> 172.21.15.79:0/396583397 conn(0x560f33f30120 msgr2=0x560f33ffea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:16.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb7da82b80 1 -- 172.21.15.79:0/396583397 shutdown_connections 2024-06-03T02:18:16.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.010+0000 7ffb7da82b80 1 -- 172.21.15.79:0/396583397 wait complete. 2024-06-03T02:18:16.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.011+0000 7ffb7da82b80 1 -- start start 2024-06-03T02:18:16.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.011+0000 7ffb7da82b80 1 Processor -- start 2024-06-03T02:18:16.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.011+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f340283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:16.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.011+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f3400f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:16.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.011+0000 7ffb7da82b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f3400f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:16.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560f3401e990 con 0x560f3400a460 2024-06-03T02:18:16.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560f3401e2c0 con 0x560f340041f0 2024-06-03T02:18:16.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb7da82b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560f3401f060 con 0x560f34009b40 2024-06-03T02:18:16.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb7da82b80 10 client.0 _ll_register_callbacks cb 0x560f33eeb330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:16.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb71200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f3400f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:16.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb6be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f340283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:16.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb6be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f340283e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60472/0 (socket says 172.21.15.79:60472) 2024-06-03T02:18:16.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb6be00700 1 -- 172.21.15.79:0/2760591623 learned_addr learned my addr 172.21.15.79:0/2760591623 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:16.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb6b400700 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f3400f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:16.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb6be00700 1 -- 172.21.15.79:0/2760591623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 msgr2=0x560f3400f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:16.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb6be00700 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f3400f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb6be00700 1 -- 172.21.15.79:0/2760591623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 msgr2=0x560f3400f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:16.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb6be00700 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f3400f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.012+0000 7ffb6be00700 1 -- 172.21.15.79:0/2760591623 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3400fbc0 con 0x560f340041f0 2024-06-03T02:18:16.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb6be00700 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f340283e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c09aa30 tx=0x7ffb5c09c210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec6911d19a811fb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:16.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb56000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:18:16.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb56000700 1 -- 172.21.15.79:0/2760591623 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0b4070 con 0x560f340041f0 2024-06-03T02:18:16.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2760591623 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb58004500 con 0x560f340041f0 2024-06-03T02:18:16.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb56000700 1 -- 172.21.15.79:0/2760591623 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ffb5c09bdb0 con 0x560f340041f0 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb56000700 1 -- 172.21.15.79:0/2760591623 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb5c09c550 con 0x560f340041f0 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb7da82b80 10 client.25049 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2760591623 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f3400fbc0 con 0x560f340041f0 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb56000700 1 -- 172.21.15.79:0/2760591623 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb58004500 con 0x560f340041f0 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.013+0000 7ffb56000700 1 -- 172.21.15.79:0/2760591623 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffb5c098040 con 0x560f340041f0 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.014+0000 7ffb7da82b80 10 client.25049 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.014+0000 7ffb7da82b80 10 client.25049 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.014+0000 7ffb7da82b80 20 client.25049 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.014+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560f34095c40 0x560f34094950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.014+0000 7ffb7da82b80 4 client.25049 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.014+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2760591623 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "status"}) -- 0x560f33f5d530 con 0x560f34095c40 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.014+0000 7ffb6b400700 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560f34095c40 0x560f34094950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb56000700 10 client.25049 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb6b400700 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560f34095c40 0x560f34094950 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a4f2c2a7e9674e server_cookie=e43bee8c246854b7 in_seq=0 out_seq=0 2024-06-03T02:18:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb56000700 1 -- 172.21.15.79:0/2760591623 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x560f33f5d530 con 0x560f34095c40 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb56000700 10 client.25049 handle_command_reply: tid=0 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 client.25049 shutdown 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2760591623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 msgr2=0x560f340283e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f340283e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c09aa30 tx=0x7ffb5c09c210 comp rx=0 tx=0).stop 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2760591623 shutdown_connections 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560f34095c40 0x560f34094950 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560f3400a460 0x560f3400f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f340041f0 0x560f340283e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 --2- 172.21.15.79:0/2760591623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f34009b40 0x560f3400f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb56000700 0 client.25049 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2760591623 >> 172.21.15.79:0/2760591623 conn(0x560f33f30120 msgr2=0x560f3400a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2760591623 shutdown_connections 2024-06-03T02:18:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 1 -- 172.21.15.79:0/2760591623 wait complete. 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.015+0000 7ffb7da82b80 20 client.25049 trim_cache size 0 max 16384 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24539, 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 243.25145340700001, 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:18:16.021 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:18:16.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:18:16.022 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 266.96299878399998 2024-06-03T02:18:16.022 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:18:16.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 -- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb500a2e40 msgr2=0x7ffb500a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:16.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb500a2e40 0x7ffb500a5300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c102030 tx=0x7ffb58009260 comp rx=0 tx=0).stop 2024-06-03T02:18:16.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 -- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 msgr2=0x7ffb6c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:16.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 0x7ffb6c19e870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c006010 tx=0x7ffb5c003a90 comp rx=0 tx=0).stop 2024-06-03T02:18:16.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 -- 172.21.15.79:0/2070409125 shutdown_connections 2024-06-03T02:18:16.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6c107840 0x7ffb6c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb500a2e40 0x7ffb500a5300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb6c102740 0x7ffb6c19e870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 --2- 172.21.15.79:0/2070409125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c1030f0 0x7ffb6c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:16.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 -- 172.21.15.79:0/2070409125 >> 172.21.15.79:0/2070409125 conn(0x7ffb6c0795f0 msgr2=0x7ffb6c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:16.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 -- 172.21.15.79:0/2070409125 shutdown_connections 2024-06-03T02:18:16.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:16.016+0000 7ffb73200700 1 -- 172.21.15.79:0/2070409125 wait complete. 2024-06-03T02:18:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:18:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:18:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:15 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:18:16.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:16 smithi079 ceph-mon[31819]: pgmap v463: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:16.901 INFO:tasks.cephadm:Setup complete, yielding 2024-06-03T02:18:16.901 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T02:18:16.912 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi079.front.sepia.ceph.com 2024-06-03T02:18:16.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-03T02:18:16.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:16 smithi123 ceph-mon[36640]: pgmap v463: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:17.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:16 smithi135 ceph-mon[36653]: pgmap v463: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:17.359 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:18.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:18 smithi079 ceph-mon[31819]: pgmap v464: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:18 smithi135 ceph-mon[36653]: pgmap v464: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:18 smithi123 ceph-mon[36640]: pgmap v464: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:19.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.524+0000 7fdfe8200700 1 -- 172.21.15.79:0/1626650388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0103700 msgr2=0x7fdfe0103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:19.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.524+0000 7fdfe8200700 1 --2- 172.21.15.79:0/1626650388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0103700 0x7fdfe0103b60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0099c0 tx=0x7fdfcc009cd0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.524+0000 7fdfe8200700 1 -- 172.21.15.79:0/1626650388 shutdown_connections 2024-06-03T02:18:19.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.524+0000 7fdfe8200700 1 --2- 172.21.15.79:0/1626650388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe01040a0 0x7fdfe010a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.524+0000 7fdfe8200700 1 --2- 172.21.15.79:0/1626650388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe0102500 0x7fdfe0102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.524+0000 7fdfe8200700 1 --2- 172.21.15.79:0/1626650388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0103700 0x7fdfe0103b60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.524+0000 7fdfe8200700 1 -- 172.21.15.79:0/1626650388 >> 172.21.15.79:0/1626650388 conn(0x7fdfe00fda60 msgr2=0x7fdfe00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:19.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.524+0000 7fdfe8200700 1 -- 172.21.15.79:0/1626650388 shutdown_connections 2024-06-03T02:18:19.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.525+0000 7fdfe8200700 1 -- 172.21.15.79:0/1626650388 wait complete. 2024-06-03T02:18:19.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.526+0000 7fdfe8200700 1 Processor -- start 2024-06-03T02:18:19.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.526+0000 7fdfe8200700 1 -- start start 2024-06-03T02:18:19.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe8200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0102500 0x7fdfe0076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:19.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe8200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe0103700 0x7fdfe0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:19.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe8200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe01040a0 0x7fdfe0074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:19.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe8200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe010e130 con 0x7fdfe0102500 2024-06-03T02:18:19.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe8200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe010dfb0 con 0x7fdfe0103700 2024-06-03T02:18:19.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe8200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe010e2b0 con 0x7fdfe01040a0 2024-06-03T02:18:19.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe5800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0102500 0x7fdfe0076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:19.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe5800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0102500 0x7fdfe0076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60502/0 (socket says 172.21.15.79:60502) 2024-06-03T02:18:19.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe5800700 1 -- 172.21.15.79:0/4176729640 learned_addr learned my addr 172.21.15.79:0/4176729640 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:19.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe4e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe0103700 0x7fdfe0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:19.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe6200700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe01040a0 0x7fdfe0074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe5800700 1 -- 172.21.15.79:0/4176729640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe01040a0 msgr2=0x7fdfe0074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe5800700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe01040a0 0x7fdfe0074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe5800700 1 -- 172.21.15.79:0/4176729640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe0103700 msgr2=0x7fdfe0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe5800700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe0103700 0x7fdfe0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.527+0000 7fdfe5800700 1 -- 172.21.15.79:0/4176729640 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe0075670 con 0x7fdfe0102500 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.528+0000 7fdfe6200700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe01040a0 0x7fdfe0074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.528+0000 7fdfe5800700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0102500 0x7fdfe0076230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400e950 tx=0x7fdfd400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.528+0000 7fdfde000700 1 -- 172.21.15.79:0/4176729640 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400eea0 con 0x7fdfe0102500 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.528+0000 7fdfde000700 1 -- 172.21.15.79:0/4176729640 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd4004510 con 0x7fdfe0102500 2024-06-03T02:18:19.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.528+0000 7fdfde000700 1 -- 172.21.15.79:0/4176729640 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4010430 con 0x7fdfe0102500 2024-06-03T02:18:19.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.528+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe01a28e0 con 0x7fdfe0102500 2024-06-03T02:18:19.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.528+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe01a2df0 con 0x7fdfe0102500 2024-06-03T02:18:19.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.529+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe0069180 con 0x7fdfe0102500 2024-06-03T02:18:19.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.532+0000 7fdfde000700 1 -- 172.21.15.79:0/4176729640 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdfd4003e90 con 0x7fdfe0102500 2024-06-03T02:18:19.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.534+0000 7fdfde000700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdfc80a2ca0 0x7fdfc80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:19.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.534+0000 7fdfde000700 1 -- 172.21.15.79:0/4176729640 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fdfd40cf7d0 con 0x7fdfe0102500 2024-06-03T02:18:19.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.534+0000 7fdfe4e00700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdfc80a2ca0 0x7fdfc80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:19.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.535+0000 7fdfe4e00700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdfc80a2ca0 0x7fdfc80a5160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc003790 tx=0x7fdfcc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:19.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.537+0000 7fdfde000700 1 -- 172.21.15.79:0/4176729640 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdfd409c320 con 0x7fdfe0102500 2024-06-03T02:18:19.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.776+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fdfe00628d0 con 0x7fdfc80a2ca0 2024-06-03T02:18:19.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.778+0000 7fdfde000700 1 -- 172.21.15.79:0/4176729640 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fdfe00628d0 con 0x7fdfc80a2ca0 2024-06-03T02:18:19.778 INFO:teuthology.orchestra.run.smithi079.stdout:Backend: cephadm 2024-06-03T02:18:19.778 INFO:teuthology.orchestra.run.smithi079.stdout:Available: Yes 2024-06-03T02:18:19.779 INFO:teuthology.orchestra.run.smithi079.stdout:Paused: No 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdfc80a2ca0 msgr2=0x7fdfc80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdfc80a2ca0 0x7fdfc80a5160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc003790 tx=0x7fdfcc000bc0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0102500 msgr2=0x7fdfe0076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0102500 0x7fdfe0076230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400e950 tx=0x7fdfd400ec60 comp rx=0 tx=0).stop 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 shutdown_connections 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfe01040a0 0x7fdfe0074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdfc80a2ca0 0x7fdfc80a5160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfe0102500 0x7fdfe0076230 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 --2- 172.21.15.79:0/4176729640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe0103700 0x7fdfe0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.782+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 >> 172.21.15.79:0/4176729640 conn(0x7fdfe00fda60 msgr2=0x7fdfe0106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:19.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.783+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 shutdown_connections 2024-06-03T02:18:19.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:19.783+0000 7fdfe8200700 1 -- 172.21.15.79:0/4176729640 wait complete. 2024-06-03T02:18:20.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:20 smithi079 ceph-mon[31819]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:20.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:20 smithi079 ceph-mon[31819]: pgmap v465: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:20 smithi135 ceph-mon[36653]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:20 smithi135 ceph-mon[36653]: pgmap v465: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:20.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-03T02:18:20.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:20 smithi123 ceph-mon[36640]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:20.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:20 smithi123 ceph-mon[36640]: pgmap v465: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:21.169 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:22.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:22 smithi079 ceph-mon[31819]: pgmap v466: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:22 smithi135 ceph-mon[36653]: pgmap v466: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:22.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:22 smithi123 ceph-mon[36640]: pgmap v466: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:23.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.609+0000 7f7f76e00700 1 -- 172.21.15.79:0/2745639947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f7010a6d0 msgr2=0x7f7f70110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:23.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.609+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2745639947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f7010a6d0 0x7f7f70110f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800b0d0 tx=0x7f7f6800b3e0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.609+0000 7f7f76e00700 1 -- 172.21.15.79:0/2745639947 shutdown_connections 2024-06-03T02:18:23.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.610+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2745639947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f7010a6d0 0x7f7f70110f80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.610+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2745639947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f70108b30 0x7f7f70108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.610+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2745639947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f70109d30 0x7f7f7010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.610+0000 7f7f76e00700 1 -- 172.21.15.79:0/2745639947 >> 172.21.15.79:0/2745639947 conn(0x7f7f70077b00 msgr2=0x7f7f70079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:23.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.610+0000 7f7f76e00700 1 -- 172.21.15.79:0/2745639947 shutdown_connections 2024-06-03T02:18:23.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.610+0000 7f7f76e00700 1 -- 172.21.15.79:0/2745639947 wait complete. 2024-06-03T02:18:23.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.611+0000 7f7f76e00700 1 Processor -- start 2024-06-03T02:18:23.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.611+0000 7f7f76e00700 1 -- start start 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.611+0000 7f7f76e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f70108b30 0x7f7f7019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.611+0000 7f7f76e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f70109d30 0x7f7f7019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.611+0000 7f7f76e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f7010a6d0 0x7f7f701a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.611+0000 7f7f76e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70114750 con 0x7f7f70108b30 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.611+0000 7f7f76e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701145d0 con 0x7f7f7010a6d0 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.611+0000 7f7f76e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701148d0 con 0x7f7f70109d30 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f70108b30 0x7f7f7019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f70109d30 0x7f7f7019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f70108b30 0x7f7f7019ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60524/0 (socket says 172.21.15.79:60524) 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f74e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f7010a6d0 0x7f7f701a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6fe00700 1 -- 172.21.15.79:0/2638045651 learned_addr learned my addr 172.21.15.79:0/2638045651 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6f400700 1 -- 172.21.15.79:0/2638045651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f7010a6d0 msgr2=0x7f7f701a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6f400700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f7010a6d0 0x7f7f701a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6f400700 1 -- 172.21.15.79:0/2638045651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f70108b30 msgr2=0x7f7f7019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:23.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6f400700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f70108b30 0x7f7f7019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6f400700 1 -- 172.21.15.79:0/2638045651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f701a68f0 con 0x7f7f70109d30 2024-06-03T02:18:23.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f74e00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f7010a6d0 0x7f7f701a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:23.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.612+0000 7f7f6f400700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f70109d30 0x7f7f7019f1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00e9f0 tx=0x7f7f5c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.613+0000 7f7f6cc00700 1 -- 172.21.15.79:0/2638045651 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00c9c0 con 0x7f7f70109d30 2024-06-03T02:18:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.613+0000 7f7f6cc00700 1 -- 172.21.15.79:0/2638045651 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c00cb60 con 0x7f7f70109d30 2024-06-03T02:18:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.613+0000 7f7f6cc00700 1 -- 172.21.15.79:0/2638045651 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c018680 con 0x7f7f70109d30 2024-06-03T02:18:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.613+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f701a6be0 con 0x7f7f70109d30 2024-06-03T02:18:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.613+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f701a7170 con 0x7f7f70109d30 2024-06-03T02:18:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.614+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f70069180 con 0x7f7f70109d30 2024-06-03T02:18:23.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.617+0000 7f7f6cc00700 1 -- 172.21.15.79:0/2638045651 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f5c010430 con 0x7f7f70109d30 2024-06-03T02:18:23.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.618+0000 7f7f6cc00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7f540a2ca0 0x7f7f540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:23.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.618+0000 7f7f6cc00700 1 -- 172.21.15.79:0/2638045651 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7f5c0cf710 con 0x7f7f70109d30 2024-06-03T02:18:23.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.618+0000 7f7f6fe00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7f540a2ca0 0x7f7f540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:23.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.619+0000 7f7f6fe00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7f540a2ca0 0x7f7f540a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7f7019fb50 tx=0x7f7f60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:23.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.620+0000 7f7f6cc00700 1 -- 172.21.15.79:0/2638045651 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f5c09c260 con 0x7f7f70109d30 2024-06-03T02:18:23.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.853+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7f70062990 con 0x7f7f540a2ca0 2024-06-03T02:18:23.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.894+0000 7f7f6cc00700 1 -- 172.21.15.79:0/2638045651 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f7f70062990 con 0x7f7f540a2ca0 2024-06-03T02:18:23.895 INFO:teuthology.orchestra.run.smithi079.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T02:18:23.895 INFO:teuthology.orchestra.run.smithi079.stdout:mds.a smithi079 running (4m) 2m ago 4m 12.7M - 19.0.0-2946-g3d89accf b045ac545f82 f1472a61b4ec 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.b smithi123 running (4m) 2m ago 4m 26.5M - 19.0.0-2946-g3d89accf b045ac545f82 a6f06cdcd13f 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.c smithi135 running (4m) 2m ago 4m 16.2M - 19.0.0-2946-g3d89accf b045ac545f82 e93d986aac4d 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.d smithi079 running (5m) 2m ago 5m 14.8M - 19.0.0-2946-g3d89accf b045ac545f82 4f6fb36aef59 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.e smithi123 running (4m) 2m ago 4m 17.7M - 19.0.0-2946-g3d89accf b045ac545f82 6b4da3e35fdd 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.f smithi135 running (4m) 2m ago 4m 15.9M - 19.0.0-2946-g3d89accf b045ac545f82 db117653c628 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.g smithi079 running (4m) 2m ago 4m 16.2M - 19.0.0-2946-g3d89accf b045ac545f82 64f029655276 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.h smithi123 running (4m) 2m ago 4m 19.1M - 19.0.0-2946-g3d89accf b045ac545f82 47d4e72909ab 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.i smithi135 running (4m) 2m ago 4m 16.5M - 19.0.0-2946-g3d89accf b045ac545f82 2232e148aa25 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.j smithi079 running (5m) 2m ago 5m 14.9M - 19.0.0-2946-g3d89accf b045ac545f82 305331830f1e 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.k smithi123 running (5m) 2m ago 5m 16.4M - 19.0.0-2946-g3d89accf b045ac545f82 cd3843bbaf1d 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mds.l smithi135 running (5m) 2m ago 5m 14.5M - 19.0.0-2946-g3d89accf b045ac545f82 037a71e766e0 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mgr.x smithi079 *:9283,8765 running (13m) 2m ago 13m 516M - 19.0.0-2946-g3d89accf b045ac545f82 35cbaa7fc85a 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mgr.y smithi123 *:8443,8765 running (11m) 2m ago 11m 457M - 19.0.0-2946-g3d89accf b045ac545f82 a4a1e043d3c4 2024-06-03T02:18:23.896 INFO:teuthology.orchestra.run.smithi079.stdout:mgr.z smithi135 *:8443,8765 running (11m) 2m ago 11m 455M - 19.0.0-2946-g3d89accf b045ac545f82 5f6977c5c179 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:mon.a smithi079 running (13m) 2m ago 13m 50.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 5de779843158 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:mon.b smithi123 running (11m) 2m ago 11m 41.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 0144ff284336 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:mon.c smithi135 running (11m) 2m ago 11m 43.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 abe230c9a7e3 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0 smithi079 running (10m) 2m ago 10m 45.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 865659a55367 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1 smithi123 running (10m) 2m ago 10m 47.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e18da9247846 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2 smithi135 running (9m) 2m ago 9m 47.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 4dfd9dee0a85 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3 smithi079 running (9m) 2m ago 9m 43.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 20cc42a16e0a 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4 smithi123 running (8m) 2m ago 8m 44.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 fb8d3392ad36 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5 smithi135 running (8m) 2m ago 8m 53.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 5d8758198dd2 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6 smithi079 running (7m) 2m ago 7m 41.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 aeb61145c801 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7 smithi123 running (7m) 2m ago 7m 47.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 15fbbe69ba48 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.8 smithi135 running (7m) 2m ago 7m 45.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d3d103cd354d 2024-06-03T02:18:23.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.9 smithi079 running (6m) 2m ago 6m 41.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 6951ae1c31e2 2024-06-03T02:18:23.898 INFO:teuthology.orchestra.run.smithi079.stdout:osd.10 smithi123 running (6m) 2m ago 6m 55.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 59b5ae5c94f0 2024-06-03T02:18:23.898 INFO:teuthology.orchestra.run.smithi079.stdout:osd.11 smithi135 running (5m) 2m ago 5m 52.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 f3518e6805d0 2024-06-03T02:18:23.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7f540a2ca0 msgr2=0x7f7f540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7f540a2ca0 0x7f7f540a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7f7019fb50 tx=0x7f7f60009260 comp rx=0 tx=0).stop 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f70109d30 msgr2=0x7f7f7019f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f70109d30 0x7f7f7019f1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00e9f0 tx=0x7f7f5c00ed00 comp rx=0 tx=0).stop 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 shutdown_connections 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f7010a6d0 0x7f7f701a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7f540a2ca0 0x7f7f540a5160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f70108b30 0x7f7f7019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 --2- 172.21.15.79:0/2638045651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f70109d30 0x7f7f7019f1b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 >> 172.21.15.79:0/2638045651 conn(0x7f7f70077b00 msgr2=0x7f7f7010cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 shutdown_connections 2024-06-03T02:18:23.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:23.898+0000 7f7f76e00700 1 -- 172.21.15.79:0/2638045651 wait complete. 2024-06-03T02:18:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:24 smithi079 ceph-mon[31819]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:24 smithi079 ceph-mon[31819]: pgmap v467: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:24 smithi135 ceph-mon[36653]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:24 smithi135 ceph-mon[36653]: pgmap v467: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:24 smithi123 ceph-mon[36640]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:24 smithi123 ceph-mon[36640]: pgmap v467: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:24.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-03T02:18:25.305 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:26 smithi079 ceph-mon[31819]: pgmap v468: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:26 smithi135 ceph-mon[36653]: pgmap v468: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:26 smithi123 ceph-mon[36640]: pgmap v468: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:27.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.523+0000 7f1857a00700 1 -- 172.21.15.79:0/3067808553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18501024f0 msgr2=0x7f1850102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:27.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.523+0000 7f1857a00700 1 --2- 172.21.15.79:0/3067808553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18501024f0 0x7f1850102910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1844009990 tx=0x7f1844009ca0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.524+0000 7f1857a00700 1 -- 172.21.15.79:0/3067808553 shutdown_connections 2024-06-03T02:18:27.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.524+0000 7f1857a00700 1 --2- 172.21.15.79:0/3067808553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1850104090 0x7f185010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.524+0000 7f1857a00700 1 --2- 172.21.15.79:0/3067808553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18501024f0 0x7f1850102910 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.524+0000 7f1857a00700 1 --2- 172.21.15.79:0/3067808553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501036f0 0x7f1850103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.524+0000 7f1857a00700 1 -- 172.21.15.79:0/3067808553 >> 172.21.15.79:0/3067808553 conn(0x7f18500fda90 msgr2=0x7f18500ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:27.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.524+0000 7f1857a00700 1 -- 172.21.15.79:0/3067808553 shutdown_connections 2024-06-03T02:18:27.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.524+0000 7f1857a00700 1 -- 172.21.15.79:0/3067808553 wait complete. 2024-06-03T02:18:27.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.525+0000 7f1857a00700 1 Processor -- start 2024-06-03T02:18:27.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.525+0000 7f1857a00700 1 -- start start 2024-06-03T02:18:27.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.525+0000 7f1857a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18501024f0 0x7f1850198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:27.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.525+0000 7f1857a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501036f0 0x7f1850198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:27.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.525+0000 7f1857a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1850104090 0x7f185019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:27.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.525+0000 7f1857a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f185010e0f0 con 0x7f18501024f0 2024-06-03T02:18:27.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.525+0000 7f1857a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f185010df70 con 0x7f18501036f0 2024-06-03T02:18:27.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.525+0000 7f1857a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f185010e270 con 0x7f1850104090 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.525+0000 7f1855000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18501024f0 0x7f1850198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f184fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501036f0 0x7f1850198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f184fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501036f0 0x7f1850198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49182/0 (socket says 172.21.15.79:49182) 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f184fe00700 1 -- 172.21.15.79:0/4213070137 learned_addr learned my addr 172.21.15.79:0/4213070137 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f1855a00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1850104090 0x7f185019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f184fe00700 1 -- 172.21.15.79:0/4213070137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1850104090 msgr2=0x7f185019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f184fe00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1850104090 0x7f185019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f184fe00700 1 -- 172.21.15.79:0/4213070137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18501024f0 msgr2=0x7f1850198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f184fe00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18501024f0 0x7f1850198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f184fe00700 1 -- 172.21.15.79:0/4213070137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18501a03d0 con 0x7f18501036f0 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.526+0000 7f184fe00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501036f0 0x7f1850198cb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e9c0 tx=0x7f183c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.527+0000 7f184d600700 1 -- 172.21.15.79:0/4213070137 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c00cbd0 con 0x7f18501036f0 2024-06-03T02:18:27.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.527+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18501a06c0 con 0x7f18501036f0 2024-06-03T02:18:27.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.527+0000 7f184d600700 1 -- 172.21.15.79:0/4213070137 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183c00ee60 con 0x7f18501036f0 2024-06-03T02:18:27.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.527+0000 7f184d600700 1 -- 172.21.15.79:0/4213070137 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c005680 con 0x7f18501036f0 2024-06-03T02:18:27.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.527+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501a0c00 con 0x7f18501036f0 2024-06-03T02:18:27.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.528+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1850069180 con 0x7f18501036f0 2024-06-03T02:18:27.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.530+0000 7f184d600700 1 -- 172.21.15.79:0/4213070137 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f183c004750 con 0x7f18501036f0 2024-06-03T02:18:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.531+0000 7f184d600700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18380a2ca0 0x7f18380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.531+0000 7f184d600700 1 -- 172.21.15.79:0/4213070137 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f183c0cf620 con 0x7f18501036f0 2024-06-03T02:18:27.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.532+0000 7f1855000700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18380a2ca0 0x7f18380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:27.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.532+0000 7f1855000700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18380a2ca0 0x7f18380a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f184400b820 tx=0x7f184400b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:27.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.536+0000 7f184d600700 1 -- 172.21.15.79:0/4213070137 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f183c09c170 con 0x7f18501036f0 2024-06-03T02:18:27.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.771+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f18500628d0 con 0x7f18380a2ca0 2024-06-03T02:18:27.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.781+0000 7f184d600700 1 -- 172.21.15.79:0/4213070137 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f18500628d0 con 0x7f18380a2ca0 2024-06-03T02:18:27.780 INFO:teuthology.orchestra.run.smithi079.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-03T02:18:27.781 INFO:teuthology.orchestra.run.smithi079.stdout:mds.all 12/12 2m ago 5m smithi079=a;smithi079=d;smithi079=g;smithi079=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-06-03T02:18:27.781 INFO:teuthology.orchestra.run.smithi079.stdout:mgr 3/3 2m ago 11m smithi079=x;smithi123=y;smithi135=z;count:3 2024-06-03T02:18:27.781 INFO:teuthology.orchestra.run.smithi079.stdout:mon 3/3 2m ago 11m smithi079:172.21.15.79=a;smithi123:172.21.15.123=b;smithi135:172.21.15.135=c;count:3 2024-06-03T02:18:27.781 INFO:teuthology.orchestra.run.smithi079.stdout:osd 12 2m ago - 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18380a2ca0 msgr2=0x7f18380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18380a2ca0 0x7f18380a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f184400b820 tx=0x7f184400b8f0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501036f0 msgr2=0x7f1850198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501036f0 0x7f1850198cb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e9c0 tx=0x7f183c00ecd0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 shutdown_connections 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1850104090 0x7f185019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f18380a2ca0 0x7f18380a5160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18501024f0 0x7f1850198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 --2- 172.21.15.79:0/4213070137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501036f0 0x7f1850198cb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 >> 172.21.15.79:0/4213070137 conn(0x7f18500fda90 msgr2=0x7f1850106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 shutdown_connections 2024-06-03T02:18:27.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:27.784+0000 7f1857a00700 1 -- 172.21.15.79:0/4213070137 wait complete. 2024-06-03T02:18:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:28 smithi079 ceph-mon[31819]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:28 smithi079 ceph-mon[31819]: pgmap v469: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:28 smithi135 ceph-mon[36653]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:28 smithi135 ceph-mon[36653]: pgmap v469: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:28.691 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-03T02:18:28.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:28 smithi123 ceph-mon[36640]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:28 smithi123 ceph-mon[36640]: pgmap v469: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:30 smithi079 ceph-mon[31819]: pgmap v470: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:30.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:30 smithi135 ceph-mon[36653]: pgmap v470: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:30 smithi123 ceph-mon[36640]: pgmap v470: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:31.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.359+0000 7fb1f2c00700 1 -- 172.21.15.79:0/691872531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1ec103700 msgr2=0x7fb1ec103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:31.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.359+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/691872531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1ec103700 0x7fb1ec103b60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80099c0 tx=0x7fb1d8009cd0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.359+0000 7fb1f2c00700 1 -- 172.21.15.79:0/691872531 shutdown_connections 2024-06-03T02:18:31.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.359+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/691872531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1ec1040a0 0x7fb1ec10a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.359+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/691872531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec102500 0x7fb1ec102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.359+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/691872531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1ec103700 0x7fb1ec103b60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.359+0000 7fb1f2c00700 1 -- 172.21.15.79:0/691872531 >> 172.21.15.79:0/691872531 conn(0x7fb1ec0fda60 msgr2=0x7fb1ec0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:31.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.359+0000 7fb1f2c00700 1 -- 172.21.15.79:0/691872531 shutdown_connections 2024-06-03T02:18:31.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.359+0000 7fb1f2c00700 1 -- 172.21.15.79:0/691872531 wait complete. 2024-06-03T02:18:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.361+0000 7fb1f2c00700 1 Processor -- start 2024-06-03T02:18:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.361+0000 7fb1f2c00700 1 -- start start 2024-06-03T02:18:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.361+0000 7fb1f2c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec102500 0x7fb1ec19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.361+0000 7fb1f2c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1ec103700 0x7fb1ec19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.361+0000 7fb1f2c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1ec1040a0 0x7fb1ec1a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:31.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.361+0000 7fb1f2c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10e140 con 0x7fb1ec1040a0 2024-06-03T02:18:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.361+0000 7fb1f2c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10dfc0 con 0x7fb1ec102500 2024-06-03T02:18:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.361+0000 7fb1f2c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10e2c0 con 0x7fb1ec103700 2024-06-03T02:18:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.361+0000 7fb1eb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1ec103700 0x7fb1ec19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1eb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1ec103700 0x7fb1ec19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:34952/0 (socket says 172.21.15.79:34952) 2024-06-03T02:18:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1eb400700 1 -- 172.21.15.79:0/414330076 learned_addr learned my addr 172.21.15.79:0/414330076 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1ebe00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec102500 0x7fb1ec19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:31.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1f0c00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1ec1040a0 0x7fb1ec1a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1ebe00700 1 -- 172.21.15.79:0/414330076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1ec103700 msgr2=0x7fb1ec19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1ebe00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1ec103700 0x7fb1ec19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1ebe00700 1 -- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1ec1040a0 msgr2=0x7fb1ec1a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1ebe00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1ec1040a0 0x7fb1ec1a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1ebe00700 1 -- 172.21.15.79:0/414330076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec1a25f0 con 0x7fb1ec102500 2024-06-03T02:18:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.362+0000 7fb1eb400700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1ec103700 0x7fb1ec19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.363+0000 7fb1ebe00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec102500 0x7fb1ec19a990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00e9c0 tx=0x7fb1dc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:31.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.363+0000 7fb1e8c00700 1 -- 172.21.15.79:0/414330076 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc004220 con 0x7fb1ec102500 2024-06-03T02:18:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.363+0000 7fb1e8c00700 1 -- 172.21.15.79:0/414330076 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1dc0043c0 con 0x7fb1ec102500 2024-06-03T02:18:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.363+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ec1a28e0 con 0x7fb1ec102500 2024-06-03T02:18:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.363+0000 7fb1e8c00700 1 -- 172.21.15.79:0/414330076 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc003d80 con 0x7fb1ec102500 2024-06-03T02:18:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.363+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ec1a2e20 con 0x7fb1ec102500 2024-06-03T02:18:31.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.364+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ec069180 con 0x7fb1ec102500 2024-06-03T02:18:31.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.368+0000 7fb1e8c00700 1 -- 172.21.15.79:0/414330076 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1dc00c7c0 con 0x7fb1ec102500 2024-06-03T02:18:31.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.369+0000 7fb1e8c00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1d00a2c20 0x7fb1d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:31.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.369+0000 7fb1e8c00700 1 -- 172.21.15.79:0/414330076 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb1dc0cf650 con 0x7fb1ec102500 2024-06-03T02:18:31.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.370+0000 7fb1eb400700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1d00a2c20 0x7fb1d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:31.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.370+0000 7fb1eb400700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1d00a2c20 0x7fb1d00a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8000c00 tx=0x7fb1d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:31.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.374+0000 7fb1e8c00700 1 -- 172.21.15.79:0/414330076 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb1dc09c1a0 con 0x7fb1ec102500 2024-06-03T02:18:31.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.607+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fb1ec075de0 con 0x7fb1d00a2c20 2024-06-03T02:18:31.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.610+0000 7fb1e8c00700 1 -- 172.21.15.79:0/414330076 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fb1ec075de0 con 0x7fb1d00a2c20 2024-06-03T02:18:31.610 INFO:teuthology.orchestra.run.smithi079.stdout:HOST ADDR LABELS STATUS 2024-06-03T02:18:31.610 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079 172.21.15.79 2024-06-03T02:18:31.610 INFO:teuthology.orchestra.run.smithi079.stdout:smithi123 172.21.15.123 2024-06-03T02:18:31.610 INFO:teuthology.orchestra.run.smithi079.stdout:smithi135 172.21.15.135 2024-06-03T02:18:31.610 INFO:teuthology.orchestra.run.smithi079.stdout:3 hosts in cluster 2024-06-03T02:18:31.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.612+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1d00a2c20 msgr2=0x7fb1d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:31.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.612+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1d00a2c20 0x7fb1d00a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8000c00 tx=0x7fb1d8005fd0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.612+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec102500 msgr2=0x7fb1ec19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:31.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.613+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec102500 0x7fb1ec19a990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00e9c0 tx=0x7fb1dc00ecd0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.613+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 shutdown_connections 2024-06-03T02:18:31.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.613+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1d00a2c20 0x7fb1d00a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.613+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1ec1040a0 0x7fb1ec1a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.613+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec102500 0x7fb1ec19a990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.613+0000 7fb1f2c00700 1 --2- 172.21.15.79:0/414330076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1ec103700 0x7fb1ec19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:31.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.613+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 >> 172.21.15.79:0/414330076 conn(0x7fb1ec0fda60 msgr2=0x7fb1ec0746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:31.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.613+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 shutdown_connections 2024-06-03T02:18:31.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:31.613+0000 7fb1f2c00700 1 -- 172.21.15.79:0/414330076 wait complete. 2024-06-03T02:18:32.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-03T02:18:32.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:32 smithi079 ceph-mon[31819]: from='client.25073 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:32.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:32 smithi079 ceph-mon[31819]: pgmap v471: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:32 smithi135 ceph-mon[36653]: from='client.25073 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:32 smithi135 ceph-mon[36653]: pgmap v471: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:32 smithi123 ceph-mon[36640]: from='client.25073 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:32 smithi123 ceph-mon[36640]: pgmap v471: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:32.979 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:34 smithi135 ceph-mon[36653]: pgmap v472: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:34 smithi123 ceph-mon[36640]: pgmap v472: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:34.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:34 smithi079 ceph-mon[31819]: pgmap v472: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:35.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.368+0000 7fa403200700 1 -- 172.21.15.79:0/4097742693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3fc105930 msgr2=0x7fa3fc105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:35.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.368+0000 7fa403200700 1 --2- 172.21.15.79:0/4097742693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3fc105930 0x7fa3fc105d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80099c0 tx=0x7fa3e8009cd0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.368+0000 7fa403200700 1 -- 172.21.15.79:0/4097742693 shutdown_connections 2024-06-03T02:18:35.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.368+0000 7fa403200700 1 --2- 172.21.15.79:0/4097742693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3fc1062d0 0x7fa3fc10cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.368+0000 7fa403200700 1 --2- 172.21.15.79:0/4097742693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3fc104730 0x7fa3fc104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.368+0000 7fa403200700 1 --2- 172.21.15.79:0/4097742693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3fc105930 0x7fa3fc105d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.368+0000 7fa403200700 1 -- 172.21.15.79:0/4097742693 >> 172.21.15.79:0/4097742693 conn(0x7fa3fc0ffcf0 msgr2=0x7fa3fc102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:35.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.368+0000 7fa403200700 1 -- 172.21.15.79:0/4097742693 shutdown_connections 2024-06-03T02:18:35.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.368+0000 7fa403200700 1 -- 172.21.15.79:0/4097742693 wait complete. 2024-06-03T02:18:35.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.369+0000 7fa403200700 1 Processor -- start 2024-06-03T02:18:35.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa403200700 1 -- start start 2024-06-03T02:18:35.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa403200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3fc104730 0x7fa3fc19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:35.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa403200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3fc105930 0x7fa3fc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:35.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa403200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3fc1062d0 0x7fa3fc1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:35.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa403200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc110320 con 0x7fa3fc1062d0 2024-06-03T02:18:35.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa403200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc1101a0 con 0x7fa3fc105930 2024-06-03T02:18:35.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa403200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc1104a0 con 0x7fa3fc104730 2024-06-03T02:18:35.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa401200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3fc1062d0 0x7fa3fc1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:35.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa3fbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3fc104730 0x7fa3fc19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:35.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa401200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3fc1062d0 0x7fa3fc1a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:48004/0 (socket says 172.21.15.79:48004) 2024-06-03T02:18:35.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.370+0000 7fa401200700 1 -- 172.21.15.79:0/2803319759 learned_addr learned my addr 172.21.15.79:0/2803319759 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:35.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.371+0000 7fa3fb400700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3fc105930 0x7fa3fc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:35.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.371+0000 7fa3fbe00700 1 -- 172.21.15.79:0/2803319759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3fc105930 msgr2=0x7fa3fc19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:35.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.371+0000 7fa3fbe00700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3fc105930 0x7fa3fc19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.371+0000 7fa3fbe00700 1 -- 172.21.15.79:0/2803319759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3fc1062d0 msgr2=0x7fa3fc1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:35.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.371+0000 7fa3fbe00700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3fc1062d0 0x7fa3fc1a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.371+0000 7fa3fbe00700 1 -- 172.21.15.79:0/2803319759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc1a2510 con 0x7fa3fc104730 2024-06-03T02:18:35.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.371+0000 7fa3fbe00700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3fc104730 0x7fa3fc19a8b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00e9f0 tx=0x7fa3ec00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:35.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.372+0000 7fa3f8c00700 1 -- 172.21.15.79:0/2803319759 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec004220 con 0x7fa3fc104730 2024-06-03T02:18:35.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.372+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3fc1a2800 con 0x7fa3fc104730 2024-06-03T02:18:35.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.372+0000 7fa3f8c00700 1 -- 172.21.15.79:0/2803319759 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3ec0043c0 con 0x7fa3fc104730 2024-06-03T02:18:35.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.372+0000 7fa3f8c00700 1 -- 172.21.15.79:0/2803319759 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec003cb0 con 0x7fa3fc104730 2024-06-03T02:18:35.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.372+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3fc1a2d40 con 0x7fa3fc104730 2024-06-03T02:18:35.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.373+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3fc069180 con 0x7fa3fc104730 2024-06-03T02:18:35.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.375+0000 7fa3f8c00700 1 -- 172.21.15.79:0/2803319759 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa3ec043ae0 con 0x7fa3fc104730 2024-06-03T02:18:35.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.376+0000 7fa3f8c00700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa3e00a2c20 0x7fa3e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:35.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.376+0000 7fa3f8c00700 1 -- 172.21.15.79:0/2803319759 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa3ec0cf630 con 0x7fa3fc104730 2024-06-03T02:18:35.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.377+0000 7fa3fb400700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa3e00a2c20 0x7fa3e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:35.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.377+0000 7fa3fb400700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa3e00a2c20 0x7fa3e00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000c00 tx=0x7fa3e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:35.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.379+0000 7fa3f8c00700 1 -- 172.21.15.79:0/2803319759 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa3ec09c180 con 0x7fa3fc104730 2024-06-03T02:18:35.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.612+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa3fc0628d0 con 0x7fa3e00a2c20 2024-06-03T02:18:35.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.616+0000 7fa3f8c00700 1 -- 172.21.15.79:0/2803319759 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa3fc0628d0 con 0x7fa3e00a2c20 2024-06-03T02:18:35.616 INFO:teuthology.orchestra.run.smithi079.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-03T02:18:35.616 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013G400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-06-03T02:18:35.616 INFO:teuthology.orchestra.run.smithi079.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-06-03T02:18:35.616 INFO:teuthology.orchestra.run.smithi079.stdout:smithi135 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401BA400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-03T02:18:35.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa3e00a2c20 msgr2=0x7fa3e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:35.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa3e00a2c20 0x7fa3e00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000c00 tx=0x7fa3e8005fb0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3fc104730 msgr2=0x7fa3fc19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:35.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3fc104730 0x7fa3fc19a8b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00e9f0 tx=0x7fa3ec00ed00 comp rx=0 tx=0).stop 2024-06-03T02:18:35.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 shutdown_connections 2024-06-03T02:18:35.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa3e00a2c20 0x7fa3e00a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3fc1062d0 0x7fa3fc1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3fc104730 0x7fa3fc19a8b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 --2- 172.21.15.79:0/2803319759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3fc105930 0x7fa3fc19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:35.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.619+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 >> 172.21.15.79:0/2803319759 conn(0x7fa3fc0ffcf0 msgr2=0x7fa3fc108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:35.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.620+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 shutdown_connections 2024-06-03T02:18:35.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:35.620+0000 7fa403200700 1 -- 172.21.15.79:0/2803319759 wait complete. 2024-06-03T02:18:36.378 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T02:18:36.390 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi079.front.sepia.ceph.com 2024-06-03T02:18:36.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-03T02:18:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:36 smithi079 ceph-mon[31819]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:36 smithi079 ceph-mon[31819]: pgmap v473: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:36 smithi135 ceph-mon[36653]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:36 smithi135 ceph-mon[36653]: pgmap v473: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:36 smithi123 ceph-mon[36640]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:18:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:36 smithi123 ceph-mon[36640]: pgmap v473: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:36.867 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:38.471 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:38 smithi079 ceph-mon[31819]: pgmap v474: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:38 smithi135 ceph-mon[36653]: pgmap v474: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:38.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:38 smithi123 ceph-mon[36640]: pgmap v474: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:38.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:38.998+0000 7fa815e00700 1 -- 172.21.15.79:0/508175085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa810102f70 msgr2=0x7fa810105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:38.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:38.998+0000 7fa815e00700 1 --2- 172.21.15.79:0/508175085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa810102f70 0x7fa810105360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa80000b0a0 tx=0x7fa80000b3b0 comp rx=0 tx=0).stop 2024-06-03T02:18:38.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:38.998+0000 7fa815e00700 1 -- 172.21.15.79:0/508175085 shutdown_connections 2024-06-03T02:18:38.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:38.998+0000 7fa815e00700 1 --2- 172.21.15.79:0/508175085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8101081d0 0x7fa81010a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:38.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:38.998+0000 7fa815e00700 1 --2- 172.21.15.79:0/508175085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa810102f70 0x7fa810105360 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:38.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:38.998+0000 7fa815e00700 1 --2- 172.21.15.79:0/508175085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8101058a0 0x7fa810107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:38.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:38.998+0000 7fa815e00700 1 -- 172.21.15.79:0/508175085 >> 172.21.15.79:0/508175085 conn(0x7fa8100fc980 msgr2=0x7fa8100fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:38.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:38.999+0000 7fa815e00700 1 -- 172.21.15.79:0/508175085 shutdown_connections 2024-06-03T02:18:39.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:38.999+0000 7fa815e00700 1 -- 172.21.15.79:0/508175085 wait complete. 2024-06-03T02:18:39.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa815e00700 1 Processor -- start 2024-06-03T02:18:39.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa815e00700 1 -- start start 2024-06-03T02:18:39.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa815e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa810102f70 0x7fa810198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:39.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa815e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8101058a0 0x7fa810198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:39.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa815e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8101081d0 0x7fa81019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:39.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa815e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa81010e0b0 con 0x7fa8101081d0 2024-06-03T02:18:39.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa815e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa81010df30 con 0x7fa810102f70 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa815e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa81010e230 con 0x7fa8101058a0 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa806000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8101058a0 0x7fa810198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa80f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8101081d0 0x7fa81019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa80ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa810102f70 0x7fa810198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa806000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8101058a0 0x7fa810198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:43234/0 (socket says 172.21.15.79:43234) 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.000+0000 7fa806000700 1 -- 172.21.15.79:0/2790185703 learned_addr learned my addr 172.21.15.79:0/2790185703 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.001+0000 7fa806000700 1 -- 172.21.15.79:0/2790185703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa810102f70 msgr2=0x7fa810198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.001+0000 7fa806000700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa810102f70 0x7fa810198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.001+0000 7fa806000700 1 -- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8101081d0 msgr2=0x7fa81019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.001+0000 7fa806000700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8101081d0 0x7fa81019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.001+0000 7fa806000700 1 -- 172.21.15.79:0/2790185703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8101a0330 con 0x7fa8101058a0 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.001+0000 7fa80ea00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa810102f70 0x7fa810198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:39.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.001+0000 7fa80f400700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8101081d0 0x7fa81019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:39.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.001+0000 7fa806000700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8101058a0 0x7fa810198c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e9d0 tx=0x7fa7f800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:39.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.002+0000 7fa814a00700 1 -- 172.21.15.79:0/2790185703 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800cbd0 con 0x7fa8101058a0 2024-06-03T02:18:39.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.002+0000 7fa814a00700 1 -- 172.21.15.79:0/2790185703 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f800ee70 con 0x7fa8101058a0 2024-06-03T02:18:39.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.002+0000 7fa814a00700 1 -- 172.21.15.79:0/2790185703 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8018660 con 0x7fa8101058a0 2024-06-03T02:18:39.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.002+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8101a0620 con 0x7fa8101058a0 2024-06-03T02:18:39.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.002+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8101a0b10 con 0x7fa8101058a0 2024-06-03T02:18:39.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.003+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa810069180 con 0x7fa8101058a0 2024-06-03T02:18:39.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.005+0000 7fa814a00700 1 -- 172.21.15.79:0/2790185703 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa7f8010430 con 0x7fa8101058a0 2024-06-03T02:18:39.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.006+0000 7fa814a00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7f40a2ca0 0x7fa7f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:39.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.006+0000 7fa814a00700 1 -- 172.21.15.79:0/2790185703 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa7f80cf7f0 con 0x7fa8101058a0 2024-06-03T02:18:39.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.007+0000 7fa80ea00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7f40a2ca0 0x7fa7f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:39.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.007+0000 7fa80ea00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7f40a2ca0 0x7fa7f40a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa80000b780 tx=0x7fa8000060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:39.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.009+0000 7fa814a00700 1 -- 172.21.15.79:0/2790185703 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa7f809c340 con 0x7fa8101058a0 2024-06-03T02:18:39.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.275+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa8100630a0 con 0x7fa8101058a0 2024-06-03T02:18:39.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.277+0000 7fa814a00700 1 -- 172.21.15.79:0/2790185703 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3092 (secure 0 0 0) 0x7fa7f800cd70 con 0x7fa8101058a0 2024-06-03T02:18:39.278 INFO:teuthology.orchestra.run.smithi079.stdout:e29 2024-06-03T02:18:39.278 INFO:teuthology.orchestra.run.smithi079.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T02:18:39.278 INFO:teuthology.orchestra.run.smithi079.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T02:18:39.278 INFO:teuthology.orchestra.run.smithi079.stdout:legacy client fscid: 1 2024-06-03T02:18:39.278 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:Filesystem 'cephfs' (1) 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:fs_name cephfs 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 29 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:created 2024-06-03T02:14:12.550646+0000 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:modified 2024-06-03T02:15:40.029293+0000 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:tableserver 0 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:root 0 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:session_timeout 300 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:session_autoclose 300 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:max_file_size 1099511627776 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:max_xattr_size 65536 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:required_client_features {} 2024-06-03T02:18:39.279 INFO:teuthology.orchestra.run.smithi079.stdout:last_failure 0 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:last_failure_osd_epoch 0 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:max_mds 5 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:in 0,1,2,3,4 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:up {0=24539,1=24515,2=24500,3=24490,4=24482} 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:failed 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:damaged 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:stopped 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:data_pools [3,4] 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:metadata_pool 2 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:inline_data disabled 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:balancer 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:bal_rank_mask -1 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:standby_count_wanted 0 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:qdb_cluster leader: 24539 members: 24539,24490,24482,24500,24515 2024-06-03T02:18:39.280 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b{0:24539} state up:active seq 7 addr [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.c{0:24478} state up:standby-replay seq 1 addr [v2:172.21.15.135:6836/755294241,v1:172.21.15.135:6837/755294241] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.g{1:24515} state up:active seq 29 addr [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.d{1:24470} state up:standby-replay seq 1 addr [v2:172.21.15.79:6836/476881413,v1:172.21.15.79:6837/476881413] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.h{2:24500} state up:active seq 31 addr [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.f{2:24466} state up:standby-replay seq 1 addr [v2:172.21.15.135:6834/3846200406,v1:172.21.15.135:6835/3846200406] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.i{3:24490} state up:active seq 31 addr [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.k{3:24452} state up:standby-replay seq 1 addr [v2:172.21.15.123:6832/296204924,v1:172.21.15.123:6833/296204924] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.e{4:24482} state up:active seq 34 addr [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.j{4:24448} state up:standby-replay seq 1 addr [v2:172.21.15.79:6834/859282294,v1:172.21.15.79:6835/859282294] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:Standby daemons: 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.a{-1:14409} state up:standby seq 1 addr [v2:172.21.15.79:6840/1411940491,v1:172.21.15.79:6841/1411940491] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.281 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.l{-1:24430} state up:standby seq 1 addr [v2:172.21.15.135:6832/1231185643,v1:172.21.15.135:6833/1231185643] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T02:18:39.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.282+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7f40a2ca0 msgr2=0x7fa7f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:39.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.282+0000 7fa815e00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7f40a2ca0 0x7fa7f40a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa80000b780 tx=0x7fa8000060c0 comp rx=0 tx=0).stop 2024-06-03T02:18:39.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.282+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8101058a0 msgr2=0x7fa810198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:39.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.282+0000 7fa815e00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8101058a0 0x7fa810198c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e9d0 tx=0x7fa7f800ece0 comp rx=0 tx=0).stop 2024-06-03T02:18:39.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.283+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 shutdown_connections 2024-06-03T02:18:39.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.283+0000 7fa815e00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa7f40a2ca0 0x7fa7f40a5160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:39.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.283+0000 7fa815e00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8101081d0 0x7fa81019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:39.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.283+0000 7fa815e00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa810102f70 0x7fa810198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:39.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.283+0000 7fa815e00700 1 --2- 172.21.15.79:0/2790185703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8101058a0 0x7fa810198c60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:39.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.283+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 >> 172.21.15.79:0/2790185703 conn(0x7fa8100fc980 msgr2=0x7fa8101039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:39.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.283+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 shutdown_connections 2024-06-03T02:18:39.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:39.283+0000 7fa815e00700 1 -- 172.21.15.79:0/2790185703 wait complete. 2024-06-03T02:18:39.285 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 29 2024-06-03T02:18:39.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:39 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2790185703' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T02:18:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:39 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2790185703' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T02:18:39.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:39 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2790185703' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T02:18:40.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-03T02:18:40.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:40 smithi079 ceph-mon[31819]: pgmap v475: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:40.640 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:40.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:40 smithi123 ceph-mon[36640]: pgmap v475: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:40.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:40 smithi135 ceph-mon[36653]: pgmap v475: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:42 smithi135 ceph-mon[36653]: pgmap v476: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:42.687 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:42 smithi079 ceph-mon[31819]: pgmap v476: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:42.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:42 smithi123 ceph-mon[36640]: pgmap v476: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:42.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.996+0000 7fe53f200700 1 -- 172.21.15.79:0/2608126390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe538109d30 msgr2=0x7fe53810a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:42.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.996+0000 7fe53f200700 1 --2- 172.21.15.79:0/2608126390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe538109d30 0x7fe53810a190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe5240099c0 tx=0x7fe524009cd0 comp rx=0 tx=0).stop 2024-06-03T02:18:42.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.996+0000 7fe53f200700 1 -- 172.21.15.79:0/2608126390 shutdown_connections 2024-06-03T02:18:42.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.996+0000 7fe53f200700 1 --2- 172.21.15.79:0/2608126390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe53810a6d0 0x7fe538110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:42.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.996+0000 7fe53f200700 1 --2- 172.21.15.79:0/2608126390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe538108b30 0x7fe538108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:42.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.996+0000 7fe53f200700 1 --2- 172.21.15.79:0/2608126390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe538109d30 0x7fe53810a190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:42.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.996+0000 7fe53f200700 1 -- 172.21.15.79:0/2608126390 >> 172.21.15.79:0/2608126390 conn(0x7fe538077b00 msgr2=0x7fe538079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:42.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.996+0000 7fe53f200700 1 -- 172.21.15.79:0/2608126390 shutdown_connections 2024-06-03T02:18:42.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.996+0000 7fe53f200700 1 -- 172.21.15.79:0/2608126390 wait complete. 2024-06-03T02:18:42.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.997+0000 7fe53f200700 1 Processor -- start 2024-06-03T02:18:42.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.997+0000 7fe53f200700 1 -- start start 2024-06-03T02:18:42.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe53f200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe538108b30 0x7fe53819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:42.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe53f200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe538109d30 0x7fe53819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:42.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe53f200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53810a6d0 0x7fe5381a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:42.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe53f200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe538114740 con 0x7fe538108b30 2024-06-03T02:18:42.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe53f200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381145c0 con 0x7fe53810a6d0 2024-06-03T02:18:42.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe53f200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381148c0 con 0x7fe538109d30 2024-06-03T02:18:42.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe537e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe538108b30 0x7fe53819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe537e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe538108b30 0x7fe53819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:48044/0 (socket says 172.21.15.79:48044) 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe537e00700 1 -- 172.21.15.79:0/3715379173 learned_addr learned my addr 172.21.15.79:0/3715379173 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe53d200700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53810a6d0 0x7fe5381a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.998+0000 7fe537400700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe538109d30 0x7fe53819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe537e00700 1 -- 172.21.15.79:0/3715379173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe538109d30 msgr2=0x7fe53819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe537e00700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe538109d30 0x7fe53819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe537e00700 1 -- 172.21.15.79:0/3715379173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53810a6d0 msgr2=0x7fe5381a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe537e00700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53810a6d0 0x7fe5381a6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe537e00700 1 -- 172.21.15.79:0/3715379173 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5381a69b0 con 0x7fe538108b30 2024-06-03T02:18:43.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe537e00700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe538108b30 0x7fe53819ed50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe52800e9a0 tx=0x7fe52800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:43.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe534c00700 1 -- 172.21.15.79:0/3715379173 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528004220 con 0x7fe538108b30 2024-06-03T02:18:43.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5381a6ca0 con 0x7fe538108b30 2024-06-03T02:18:43.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe534c00700 1 -- 172.21.15.79:0/3715379173 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5280043c0 con 0x7fe538108b30 2024-06-03T02:18:43.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:42.999+0000 7fe534c00700 1 -- 172.21.15.79:0/3715379173 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528003df0 con 0x7fe538108b30 2024-06-03T02:18:43.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.000+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5381a7180 con 0x7fe538108b30 2024-06-03T02:18:43.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.001+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53802dca0 con 0x7fe538108b30 2024-06-03T02:18:43.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.003+0000 7fe534c00700 1 -- 172.21.15.79:0/3715379173 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe52800c810 con 0x7fe538108b30 2024-06-03T02:18:43.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.004+0000 7fe534c00700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe51c0a2c20 0x7fe51c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:43.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.004+0000 7fe534c00700 1 -- 172.21.15.79:0/3715379173 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe5280cf620 con 0x7fe538108b30 2024-06-03T02:18:43.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.004+0000 7fe537400700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe51c0a2c20 0x7fe51c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:43.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.005+0000 7fe537400700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe51c0a2c20 0x7fe51c0a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe524000c00 tx=0x7fe524005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:43.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.009+0000 7fe534c00700 1 -- 172.21.15.79:0/3715379173 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe52809c170 con 0x7fe538108b30 2024-06-03T02:18:43.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.232+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe53802de70 con 0x7fe538108b30 2024-06-03T02:18:43.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.234+0000 7fe534c00700 1 -- 172.21.15.79:0/3715379173 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v101) ==== 56+0+5681 (secure 0 0 0) 0x7fe52809ba70 con 0x7fe538108b30 2024-06-03T02:18:43.235 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 101 2024-06-03T02:18:43.235 INFO:teuthology.orchestra.run.smithi079.stdout:fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:18:43.235 INFO:teuthology.orchestra.run.smithi079.stdout:created 2024-06-03T02:04:43.162039+0000 2024-06-03T02:18:43.235 INFO:teuthology.orchestra.run.smithi079.stdout:modified 2024-06-03T02:15:54.257434+0000 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:crush_version 27 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:full_ratio 0.95 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:backfillfull_ratio 0.9 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:nearfull_ratio 0.85 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:require_min_compat_client luminous 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:min_compat_client luminous 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:require_osd_release squid 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:stretch_mode_enabled false 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 97 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-03T02:18:43.236 INFO:teuthology.orchestra.run.smithi079.stdout:max_osd 12 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0 up in weight 1 up_from 8 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6802/1006585875,v1:172.21.15.79:6803/1006585875] [v2:172.21.15.79:6804/1006585875,v1:172.21.15.79:6805/1006585875] exists,up 7ff0fc4b-c2d4-4408-b6cf-827bba928e4b 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1 up in weight 1 up_from 13 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/1025664412,v1:172.21.15.123:6801/1025664412] [v2:172.21.15.123:6802/1025664412,v1:172.21.15.123:6803/1025664412] exists,up dc15c092-e416-48ac-8568-c7f12ae5a56d 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2 up in weight 1 up_from 18 up_thru 100 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/3876209625,v1:172.21.15.135:6801/3876209625] [v2:172.21.15.135:6802/3876209625,v1:172.21.15.135:6803/3876209625] exists,up bf1db324-1336-4e1b-950d-3b07e158b5eb 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3 up in weight 1 up_from 26 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6810/1694165026,v1:172.21.15.79:6811/1694165026] [v2:172.21.15.79:6812/1694165026,v1:172.21.15.79:6813/1694165026] exists,up f1394880-b33f-4511-a8df-b80c7d0273fd 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4 up in weight 1 up_from 34 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/819552045,v1:172.21.15.123:6809/819552045] [v2:172.21.15.123:6810/819552045,v1:172.21.15.123:6811/819552045] exists,up c407e697-6fbf-4447-b7ae-6a2079134052 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5 up in weight 1 up_from 40 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/3110965280,v1:172.21.15.135:6809/3110965280] [v2:172.21.15.135:6810/3110965280,v1:172.21.15.135:6811/3110965280] exists,up d3e5d988-4303-49db-9740-c3e87c443c76 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6 up in weight 1 up_from 48 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6818/2064944325,v1:172.21.15.79:6819/2064944325] [v2:172.21.15.79:6820/2064944325,v1:172.21.15.79:6821/2064944325] exists,up eabb74df-81e5-4e78-81b7-0d925a8317e5 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7 up in weight 1 up_from 53 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/283771362,v1:172.21.15.123:6817/283771362] [v2:172.21.15.123:6818/283771362,v1:172.21.15.123:6819/283771362] exists,up eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.8 up in weight 1 up_from 61 up_thru 100 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/2566672579,v1:172.21.15.135:6817/2566672579] [v2:172.21.15.135:6818/2566672579,v1:172.21.15.135:6819/2566672579] exists,up fd056a07-1f3b-450a-8da4-9a8c8edf22e2 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.9 up in weight 1 up_from 69 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6826/1463813812,v1:172.21.15.79:6827/1463813812] [v2:172.21.15.79:6828/1463813812,v1:172.21.15.79:6829/1463813812] exists,up cbad302f-a304-4c5b-96c0-3e619ba42f9d 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.10 up in weight 1 up_from 74 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/1137082874,v1:172.21.15.123:6825/1137082874] [v2:172.21.15.123:6826/1137082874,v1:172.21.15.123:6827/1137082874] exists,up 66fbdf38-f92c-4cb6-8724-4b1df4d472f8 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.11 up in weight 1 up_from 82 up_thru 100 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6824/2083189453,v1:172.21.15.135:6825/2083189453] [v2:172.21.15.135:6826/2083189453,v1:172.21.15.135:6827/2083189453] exists,up 8fd600f8-5366-44e9-8827-e7f9fc41e2ba 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 2.5 [8,11] 2024-06-03T02:18:43.237 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 2.b [2,4] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 2.10 [2,1] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 2.1c [8,11] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 3.1 [4,9] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 3.1b [4,6] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 3.23 [4,8] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 3.25 [5,9] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 3.27 [3,9] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 4.5 [9,10] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 4.f [9,8] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 4.1d [5,8] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 4.1f [9,8] 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6801/54480245 expires 2024-06-04T02:05:52.008734+0000 2024-06-03T02:18:43.238 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6800/54480245 expires 2024-06-04T02:05:52.008734+0000 2024-06-03T02:18:43.239 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/3960540784 expires 2024-06-04T02:05:52.008734+0000 2024-06-03T02:18:43.239 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/3085681251 expires 2024-06-04T02:05:16.591415+0000 2024-06-03T02:18:43.239 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6800/1845575623 expires 2024-06-04T02:05:16.591415+0000 2024-06-03T02:18:43.239 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/37254677 expires 2024-06-04T02:05:16.591415+0000 2024-06-03T02:18:43.239 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6801/1845575623 expires 2024-06-04T02:05:16.591415+0000 2024-06-03T02:18:43.239 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/603700731 expires 2024-06-04T02:05:16.591415+0000 2024-06-03T02:18:43.239 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/161718363 expires 2024-06-04T02:05:52.008734+0000 2024-06-03T02:18:43.239 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/3870624303 expires 2024-06-04T02:05:52.008734+0000 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.237+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe51c0a2c20 msgr2=0x7fe51c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.237+0000 7fe53f200700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe51c0a2c20 0x7fe51c0a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe524000c00 tx=0x7fe524005fb0 comp rx=0 tx=0).stop 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.237+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe538108b30 msgr2=0x7fe53819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.237+0000 7fe53f200700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe538108b30 0x7fe53819ed50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe52800e9a0 tx=0x7fe52800ecb0 comp rx=0 tx=0).stop 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.238+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 shutdown_connections 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.238+0000 7fe53f200700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53810a6d0 0x7fe5381a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.238+0000 7fe53f200700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe51c0a2c20 0x7fe51c0a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.238+0000 7fe53f200700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe538108b30 0x7fe53819ed50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.238+0000 7fe53f200700 1 --2- 172.21.15.79:0/3715379173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe538109d30 0x7fe53819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.238+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 >> 172.21.15.79:0/3715379173 conn(0x7fe538077b00 msgr2=0x7fe53810d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.238+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 shutdown_connections 2024-06-03T02:18:43.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:43.238+0000 7fe53f200700 1 -- 172.21.15.79:0/3715379173 wait complete. 2024-06-03T02:18:43.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:43 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3715379173' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-03T02:18:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:43 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3715379173' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-03T02:18:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:43 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3715379173' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-03T02:18:44.200 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-03T02:18:44.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:18:44.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:44 smithi079 ceph-mon[31819]: pgmap v477: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:44 smithi135 ceph-mon[36653]: pgmap v477: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:44.640 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:44 smithi123 ceph-mon[36640]: pgmap v477: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:46.697 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:46 smithi079 ceph-mon[31819]: pgmap v478: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:46 smithi123 ceph-mon[36640]: pgmap v478: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:46.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:46 smithi135 ceph-mon[36653]: pgmap v478: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.863+0000 7f435fe00700 1 -- 172.21.15.79:0/1986957349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 msgr2=0x7f4360115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.863+0000 7f435fe00700 1 --2- 172.21.15.79:0/1986957349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 0x7f4360115830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f435800b0a0 tx=0x7f435800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:18:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.863+0000 7f435fe00700 1 -- 172.21.15.79:0/1986957349 shutdown_connections 2024-06-03T02:18:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.863+0000 7f435fe00700 1 --2- 172.21.15.79:0/1986957349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 0x7f4360115830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.863+0000 7f435fe00700 1 --2- 172.21.15.79:0/1986957349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436010d8a0 0x7f436010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.863+0000 7f435fe00700 1 --2- 172.21.15.79:0/1986957349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43601089d0 0x7f4360108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.863+0000 7f435fe00700 1 -- 172.21.15.79:0/1986957349 >> 172.21.15.79:0/1986957349 conn(0x7f4360078220 msgr2=0x7f43601072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.863+0000 7f435fe00700 1 -- 172.21.15.79:0/1986957349 shutdown_connections 2024-06-03T02:18:46.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.863+0000 7f435fe00700 1 -- 172.21.15.79:0/1986957349 wait complete. 2024-06-03T02:18:46.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.864+0000 7f435fe00700 1 Processor -- start 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f435fe00700 1 -- start start 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f435fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43601089d0 0x7f436019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f435fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 0x7f436019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f435fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436010d8a0 0x7f43601a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f435fe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4360118d20 con 0x7f4360109390 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f435fe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4360118ba0 con 0x7f43601089d0 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f435fe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4360118ea0 con 0x7f436010d8a0 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f4356000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 0x7f436019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f435ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43601089d0 0x7f436019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f4356000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 0x7f436019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44418/0 (socket says 172.21.15.79:44418) 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f4356000700 1 -- 172.21.15.79:0/1218214955 learned_addr learned my addr 172.21.15.79:0/1218214955 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.865+0000 7f435f400700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436010d8a0 0x7f43601a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:46.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.866+0000 7f4356000700 1 -- 172.21.15.79:0/1218214955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436010d8a0 msgr2=0x7f43601a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:46.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.866+0000 7f4356000700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436010d8a0 0x7f43601a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:46.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.866+0000 7f4356000700 1 -- 172.21.15.79:0/1218214955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43601089d0 msgr2=0x7f436019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.866+0000 7f4356000700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43601089d0 0x7f436019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.866+0000 7f4356000700 1 -- 172.21.15.79:0/1218214955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43601a4130 con 0x7f4360109390 2024-06-03T02:18:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.866+0000 7f435ea00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43601089d0 0x7f436019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.866+0000 7f4356000700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 0x7f436019f6b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f434800d980 tx=0x7f434800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.866+0000 7f4357e00700 1 -- 172.21.15.79:0/1218214955 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348016070 con 0x7f4360109390 2024-06-03T02:18:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.867+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43601a4420 con 0x7f4360109390 2024-06-03T02:18:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.867+0000 7f4357e00700 1 -- 172.21.15.79:0/1218214955 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4348004d10 con 0x7f4360109390 2024-06-03T02:18:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.867+0000 7f4357e00700 1 -- 172.21.15.79:0/1218214955 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348005670 con 0x7f4360109390 2024-06-03T02:18:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.867+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43601abca0 con 0x7f4360109390 2024-06-03T02:18:46.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.868+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c005330 con 0x7f4360109390 2024-06-03T02:18:46.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.870+0000 7f4357e00700 1 -- 172.21.15.79:0/1218214955 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4348004220 con 0x7f4360109390 2024-06-03T02:18:46.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.871+0000 7f4357e00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f43440a2c20 0x7f43440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:46.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.871+0000 7f4357e00700 1 -- 172.21.15.79:0/1218214955 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f43480cf0d0 con 0x7f4360109390 2024-06-03T02:18:46.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.872+0000 7f435ea00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f43440a2c20 0x7f43440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:46.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.872+0000 7f435ea00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f43440a2c20 0x7f43440a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4350009fd0 tx=0x7f4350009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:46.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:46.876+0000 7f4357e00700 1 -- 172.21.15.79:0/1218214955 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f434801f020 con 0x7f4360109390 2024-06-03T02:18:47.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.148+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f433c005620 con 0x7f4360109390 2024-06-03T02:18:47.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.150+0000 7f4357e00700 1 -- 172.21.15.79:0/1218214955 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7f4348022350 con 0x7f4360109390 2024-06-03T02:18:47.150 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:18:47.150 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:15:40.029293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24490,24482,24500,24515]},"id":1}]} 2024-06-03T02:18:47.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.152+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f43440a2c20 msgr2=0x7f43440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:47.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.152+0000 7f435fe00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f43440a2c20 0x7f43440a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4350009fd0 tx=0x7f4350009260 comp rx=0 tx=0).stop 2024-06-03T02:18:47.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 msgr2=0x7f436019f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:47.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 0x7f436019f6b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f434800d980 tx=0x7f434800dc90 comp rx=0 tx=0).stop 2024-06-03T02:18:47.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 shutdown_connections 2024-06-03T02:18:47.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436010d8a0 0x7f43601a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:47.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43601089d0 0x7f436019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:47.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f43440a2c20 0x7f43440a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:47.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 --2- 172.21.15.79:0/1218214955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4360109390 0x7f436019f6b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:47.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 >> 172.21.15.79:0/1218214955 conn(0x7f4360078220 msgr2=0x7f4360078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:47.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 shutdown_connections 2024-06-03T02:18:47.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:47.153+0000 7f435fe00700 1 -- 172.21.15.79:0/1218214955 wait complete. 2024-06-03T02:18:47.154 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 29 2024-06-03T02:18:47.547 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:47 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1218214955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:18:47.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:47 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1218214955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:18:47.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:47 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1218214955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:18:48.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:18:48.525 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:48.675 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:48 smithi079 ceph-mon[31819]: pgmap v479: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:48 smithi123 ceph-mon[36640]: pgmap v479: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:48.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:48 smithi135 ceph-mon[36653]: pgmap v479: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:50.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.816+0000 7fe322800700 1 -- 172.21.15.79:0/1710806311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe31c06b620 msgr2=0x7fe31c10f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:50.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.816+0000 7fe322800700 1 --2- 172.21.15.79:0/1710806311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe31c06b620 0x7fe31c10f350 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe31400b0a0 tx=0x7fe31400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:18:50.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.817+0000 7fe322800700 1 -- 172.21.15.79:0/1710806311 shutdown_connections 2024-06-03T02:18:50.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.817+0000 7fe322800700 1 --2- 172.21.15.79:0/1710806311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe31c06b620 0x7fe31c10f350 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:50.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.817+0000 7fe322800700 1 --2- 172.21.15.79:0/1710806311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31c108580 0x7fe31c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:50.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.817+0000 7fe322800700 1 --2- 172.21.15.79:0/1710806311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31c06ac60 0x7fe31c06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:50.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.817+0000 7fe322800700 1 -- 172.21.15.79:0/1710806311 >> 172.21.15.79:0/1710806311 conn(0x7fe31c078430 msgr2=0x7fe31c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:50.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.817+0000 7fe322800700 1 -- 172.21.15.79:0/1710806311 shutdown_connections 2024-06-03T02:18:50.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.817+0000 7fe322800700 1 -- 172.21.15.79:0/1710806311 wait complete. 2024-06-03T02:18:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.818+0000 7fe322800700 1 Processor -- start 2024-06-03T02:18:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.818+0000 7fe322800700 1 -- start start 2024-06-03T02:18:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.818+0000 7fe322800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe31c06ac60 0x7fe31c101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.818+0000 7fe322800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31c06b620 0x7fe31c101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.818+0000 7fe322800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31c108580 0x7fe31c101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.818+0000 7fe322800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c1127c0 con 0x7fe31c108580 2024-06-03T02:18:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.818+0000 7fe322800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c112640 con 0x7fe31c06b620 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.818+0000 7fe322800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c112940 con 0x7fe31c06ac60 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe31c06ac60 0x7fe31c101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31c06b620 0x7fe31c101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe31c06ac60 0x7fe31c101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33204/0 (socket says 172.21.15.79:33204) 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31b400700 1 -- 172.21.15.79:0/988857601 learned_addr learned my addr 172.21.15.79:0/988857601 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31be00700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31c108580 0x7fe31c101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31aa00700 1 -- 172.21.15.79:0/988857601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe31c06ac60 msgr2=0x7fe31c101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31aa00700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe31c06ac60 0x7fe31c101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31aa00700 1 -- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31c108580 msgr2=0x7fe31c101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31aa00700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31c108580 0x7fe31c101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.819+0000 7fe31aa00700 1 -- 172.21.15.79:0/988857601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c1a7090 con 0x7fe31c06b620 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.820+0000 7fe31aa00700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31c06b620 0x7fe31c101940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe30800d750 tx=0x7fe30800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.820+0000 7fe313e00700 1 -- 172.21.15.79:0/988857601 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308014030 con 0x7fe31c06b620 2024-06-03T02:18:50.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.820+0000 7fe313e00700 1 -- 172.21.15.79:0/988857601 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe308004500 con 0x7fe31c06b620 2024-06-03T02:18:50.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.820+0000 7fe313e00700 1 -- 172.21.15.79:0/988857601 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308005420 con 0x7fe31c06b620 2024-06-03T02:18:50.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.820+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31c1a7380 con 0x7fe31c06b620 2024-06-03T02:18:50.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.820+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31c1a7870 con 0x7fe31c06b620 2024-06-03T02:18:50.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.821+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31c069180 con 0x7fe31c06b620 2024-06-03T02:18:50.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.824+0000 7fe313e00700 1 -- 172.21.15.79:0/988857601 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe308004750 con 0x7fe31c06b620 2024-06-03T02:18:50.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.825+0000 7fe313e00700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3000a2c20 0x7fe3000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:50.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.825+0000 7fe313e00700 1 -- 172.21.15.79:0/988857601 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe3080cf100 con 0x7fe31c06b620 2024-06-03T02:18:50.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.825+0000 7fe31b400700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3000a2c20 0x7fe3000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:50.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.826+0000 7fe31b400700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3000a2c20 0x7fe3000a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009ca0 tx=0x7fe30c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:50.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:50.827+0000 7fe313e00700 1 -- 172.21.15.79:0/988857601 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe30809bc50 con 0x7fe31c06b620 2024-06-03T02:18:51.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.097+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe31c02dca0 con 0x7fe31c06b620 2024-06-03T02:18:51.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.098+0000 7fe313e00700 1 -- 172.21.15.79:0/988857601 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7fe30809b550 con 0x7fe31c06b620 2024-06-03T02:18:51.099 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:18:51.099 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:15:40.029293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24490,24482,24500,24515]},"id":1}]} 2024-06-03T02:18:51.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3000a2c20 msgr2=0x7fe3000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3000a2c20 0x7fe3000a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009ca0 tx=0x7fe30c009260 comp rx=0 tx=0).stop 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31c06b620 msgr2=0x7fe31c101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31c06b620 0x7fe31c101940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe30800d750 tx=0x7fe30800da60 comp rx=0 tx=0).stop 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 shutdown_connections 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3000a2c20 0x7fe3000a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe31c108580 0x7fe31c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe31c06ac60 0x7fe31c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 --2- 172.21.15.79:0/988857601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31c06b620 0x7fe31c101940 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 >> 172.21.15.79:0/988857601 conn(0x7fe31c078430 msgr2=0x7fe31c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 shutdown_connections 2024-06-03T02:18:51.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:51.101+0000 7fe322800700 1 -- 172.21.15.79:0/988857601 wait complete. 2024-06-03T02:18:51.103 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 29 2024-06-03T02:18:51.575 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:51 smithi079 ceph-mon[31819]: pgmap v480: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:51.575 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:51 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/988857601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:18:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:51 smithi135 ceph-mon[36653]: pgmap v480: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:51.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:51 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/988857601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:18:51.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:51 smithi123 ceph-mon[36640]: pgmap v480: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:18:51.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:51 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/988857601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:18:52.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:18:52.437 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:53 smithi135 ceph-mon[36653]: pgmap v481: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:53.616 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:53 smithi079 ceph-mon[31819]: pgmap v481: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:53 smithi123 ceph-mon[36640]: pgmap v481: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:18:54.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.778+0000 7f7ed1400700 1 -- 172.21.15.79:0/639327937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc10a670 msgr2=0x7f7ecc10aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:54.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.778+0000 7f7ed1400700 1 --2- 172.21.15.79:0/639327937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc10a670 0x7f7ecc10aa50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0099c0 tx=0x7f7ebc009cd0 comp rx=0 tx=0).stop 2024-06-03T02:18:54.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.778+0000 7f7ed1400700 1 -- 172.21.15.79:0/639327937 shutdown_connections 2024-06-03T02:18:54.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.778+0000 7f7ed1400700 1 --2- 172.21.15.79:0/639327937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ecc104f80 0x7f7ecc111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:54.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.778+0000 7f7ed1400700 1 --2- 172.21.15.79:0/639327937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc10a670 0x7f7ecc10aa50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:54.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.778+0000 7f7ed1400700 1 --2- 172.21.15.79:0/639327937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc1045c0 0x7f7ecc104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:54.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.778+0000 7f7ed1400700 1 -- 172.21.15.79:0/639327937 >> 172.21.15.79:0/639327937 conn(0x7f7ecc1000d0 msgr2=0x7f7ecc1024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:54.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.778+0000 7f7ed1400700 1 -- 172.21.15.79:0/639327937 shutdown_connections 2024-06-03T02:18:54.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.778+0000 7f7ed1400700 1 -- 172.21.15.79:0/639327937 wait complete. 2024-06-03T02:18:54.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.779+0000 7f7ed1400700 1 Processor -- start 2024-06-03T02:18:54.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.779+0000 7f7ed1400700 1 -- start start 2024-06-03T02:18:54.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.779+0000 7f7ed1400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc1045c0 0x7f7ecc1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:54.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.779+0000 7f7ed1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ecc104f80 0x7f7ecc1a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:54.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.779+0000 7f7ed1400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc10a670 0x7f7ecc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.779+0000 7f7ed1400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1148e0 con 0x7f7ecc1045c0 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.779+0000 7f7ed1400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc114760 con 0x7f7ecc10a670 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.779+0000 7f7ed1400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc114a60 con 0x7f7ecc104f80 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecbe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc1045c0 0x7f7ecc1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ecc104f80 0x7f7ecc1a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ecc104f80 0x7f7ecc1a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33214/0 (socket says 172.21.15.79:33214) 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecbe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc1045c0 0x7f7ecc1a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44458/0 (socket says 172.21.15.79:44458) 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecb400700 1 -- 172.21.15.79:0/626075175 learned_addr learned my addr 172.21.15.79:0/626075175 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ed0a00700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc10a670 0x7f7ecc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecbe00700 1 -- 172.21.15.79:0/626075175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ecc104f80 msgr2=0x7f7ecc1a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:54.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecbe00700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ecc104f80 0x7f7ecc1a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecbe00700 1 -- 172.21.15.79:0/626075175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc10a670 msgr2=0x7f7ecc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecbe00700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc10a670 0x7f7ecc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ecbe00700 1 -- 172.21.15.79:0/626075175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc19b260 con 0x7f7ecc1045c0 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.780+0000 7f7ed0a00700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc10a670 0x7f7ecc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.781+0000 7f7ecbe00700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc1045c0 0x7f7ecc1a08f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00b580 tx=0x7f7ebc0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.781+0000 7f7ec8c00700 1 -- 172.21.15.79:0/626075175 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc026070 con 0x7f7ecc1045c0 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.781+0000 7f7ec8c00700 1 -- 172.21.15.79:0/626075175 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc003960 con 0x7f7ecc1045c0 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.781+0000 7f7ec8c00700 1 -- 172.21.15.79:0/626075175 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0047e0 con 0x7f7ecc1045c0 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.781+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc19b4f0 con 0x7f7ecc1045c0 2024-06-03T02:18:54.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.781+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a77c0 con 0x7f7ecc1045c0 2024-06-03T02:18:54.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.783+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc10cb00 con 0x7f7ecc1045c0 2024-06-03T02:18:54.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.784+0000 7f7ec8c00700 1 -- 172.21.15.79:0/626075175 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ebc029070 con 0x7f7ecc1045c0 2024-06-03T02:18:54.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.787+0000 7f7ec8c00700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7eb00a2ca0 0x7f7eb00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:54.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.787+0000 7f7ec8c00700 1 -- 172.21.15.79:0/626075175 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7ebc0cfc30 con 0x7f7ecc1045c0 2024-06-03T02:18:54.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.787+0000 7f7ecb400700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7eb00a2ca0 0x7f7eb00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:54.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.788+0000 7f7ecb400700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7eb00a2ca0 0x7f7eb00a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc109ca0 tx=0x7f7eb8007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:54.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:54.791+0000 7f7ec8c00700 1 -- 172.21.15.79:0/626075175 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7ebc05d850 con 0x7f7ecc1045c0 2024-06-03T02:18:55.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.017+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ecc19bfe0 con 0x7f7ecc1045c0 2024-06-03T02:18:55.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.022+0000 7f7ec8c00700 1 -- 172.21.15.79:0/626075175 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+24805 (secure 0 0 0) 0x7f7ebc013070 con 0x7f7ecc1045c0 2024-06-03T02:18:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:18:55.024 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":101,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:15:54.257434+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:18:55.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7eb00a2ca0 msgr2=0x7f7eb00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:55.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7eb00a2ca0 0x7f7eb00a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc109ca0 tx=0x7f7eb8007480 comp rx=0 tx=0).stop 2024-06-03T02:18:55.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc1045c0 msgr2=0x7f7ecc1a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:55.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc1045c0 0x7f7ecc1a08f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00b580 tx=0x7f7ebc0038b0 comp rx=0 tx=0).stop 2024-06-03T02:18:55.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 shutdown_connections 2024-06-03T02:18:55.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc10a670 0x7f7ecc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:55.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7eb00a2ca0 0x7f7eb00a5160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:55.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ecc1045c0 0x7f7ecc1a08f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:55.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 --2- 172.21.15.79:0/626075175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ecc104f80 0x7f7ecc1a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:55.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.025+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 >> 172.21.15.79:0/626075175 conn(0x7f7ecc1000d0 msgr2=0x7f7ecc101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:55.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.026+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 shutdown_connections 2024-06-03T02:18:55.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:55.026+0000 7f7ed1400700 1 -- 172.21.15.79:0/626075175 wait complete. 2024-06-03T02:18:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:55 smithi123 ceph-mon[36640]: pgmap v482: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:55 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/626075175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:18:55.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:55 smithi079 ceph-mon[31819]: pgmap v482: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:55.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:55 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/626075175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:18:55.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:55 smithi135 ceph-mon[36653]: pgmap v482: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:18:55.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:55 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/626075175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:18:56.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-03T02:14:12.550646+0000', 'modified': '2024-06-03T02:15:40.029293+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24515, 'mds_2': 24500, 'mds_3': 24490, 'mds_4': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24448': {'gid': 24448, 'name': 'j', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/859282294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 859282294}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 859282294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/296204924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 296204924}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 296204924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6835/3846200406', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 3846200406}, {'type': 'v1', 'addr': '172.21.15.135:6835', 'nonce': 3846200406}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6837/476881413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 476881413}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 476881413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6837/755294241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6836', 'nonce': 755294241}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 755294241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 34, 'addr': '172.21.15.123:6835/4173566972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 4173566972}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 4173566972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.135:6839/414076797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 414076797}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 414076797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.123:6837/963974438', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 963974438}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 963974438}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.79:6839/2483452059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 2483452059}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 2483452059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.123:6839/1178082108', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 1178082108}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 1178082108}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24539, 24490, 24482, 24500, 24515]} 2024-06-03T02:18:56.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-03T02:18:56.024 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-06-03T02:18:56.462 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:18:57.703 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:57 smithi079 ceph-mon[31819]: pgmap v483: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:57.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:57 smithi123 ceph-mon[36640]: pgmap v483: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:57.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:57 smithi135 ceph-mon[36653]: pgmap v483: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:18:58.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.792+0000 7f6b35800700 1 -- 172.21.15.79:0/4003970124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b301076f0 msgr2=0x7f6b3010f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:58.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.792+0000 7f6b35800700 1 --2- 172.21.15.79:0/4003970124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b301076f0 0x7f6b3010f2c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009f90 tx=0x7f6b2800b330 comp rx=0 tx=0).stop 2024-06-03T02:18:58.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.792+0000 7f6b35800700 1 -- 172.21.15.79:0/4003970124 shutdown_connections 2024-06-03T02:18:58.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.792+0000 7f6b35800700 1 --2- 172.21.15.79:0/4003970124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b301076f0 0x7f6b3010f2c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.792+0000 7f6b35800700 1 --2- 172.21.15.79:0/4003970124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b30102610 0x7f6b301029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.792+0000 7f6b35800700 1 --2- 172.21.15.79:0/4003970124 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b30102fc0 0x7f6b30107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.792+0000 7f6b35800700 1 -- 172.21.15.79:0/4003970124 >> 172.21.15.79:0/4003970124 conn(0x7f6b300fdea0 msgr2=0x7f6b301002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:58.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.792+0000 7f6b35800700 1 -- 172.21.15.79:0/4003970124 shutdown_connections 2024-06-03T02:18:58.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.792+0000 7f6b35800700 1 -- 172.21.15.79:0/4003970124 wait complete. 2024-06-03T02:18:58.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b35800700 1 Processor -- start 2024-06-03T02:18:58.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b35800700 1 -- start start 2024-06-03T02:18:58.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b35800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b30102610 0x7f6b3019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b35800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b30102fc0 0x7f6b3019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b35800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b301076f0 0x7f6b3019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b35800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6b30112750 con 0x7f6b30102fc0 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b35800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301125d0 con 0x7f6b30102610 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b35800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301128d0 con 0x7f6b301076f0 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b2f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b30102fc0 0x7f6b3019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b2f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b30102fc0 0x7f6b3019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33760/0 (socket says 172.21.15.79:33760) 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.794+0000 7f6b2f400700 1 -- 172.21.15.79:0/55475161 learned_addr learned my addr 172.21.15.79:0/55475161 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.795+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b30102610 0x7f6b3019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.795+0000 7f6b34e00700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b301076f0 0x7f6b3019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.795+0000 7f6b2f400700 1 -- 172.21.15.79:0/55475161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b301076f0 msgr2=0x7f6b3019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.795+0000 7f6b2f400700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b301076f0 0x7f6b3019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.795+0000 7f6b2f400700 1 -- 172.21.15.79:0/55475161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b30102610 msgr2=0x7f6b3019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.795+0000 7f6b2f400700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b30102610 0x7f6b3019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.795+0000 7f6b2f400700 1 -- 172.21.15.79:0/55475161 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3019fdb0 con 0x7f6b30102fc0 2024-06-03T02:18:58.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.795+0000 7f6b2f400700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b30102fc0 0x7f6b3019b2e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e9f0 tx=0x7f6b1800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:58.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.796+0000 7f6b2cc00700 1 -- 172.21.15.79:0/55475161 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800c9c0 con 0x7f6b30102fc0 2024-06-03T02:18:58.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.796+0000 7f6b2cc00700 1 -- 172.21.15.79:0/55475161 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b1800cb60 con 0x7f6b30102fc0 2024-06-03T02:18:58.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.796+0000 7f6b35800700 1 -- 172.21.15.79:0/55475161 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b301a00a0 con 0x7f6b30102fc0 2024-06-03T02:18:58.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.796+0000 7f6b2cc00700 1 -- 172.21.15.79:0/55475161 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18018680 con 0x7f6b30102fc0 2024-06-03T02:18:58.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.796+0000 7f6b35800700 1 -- 172.21.15.79:0/55475161 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b301a7920 con 0x7f6b30102fc0 2024-06-03T02:18:58.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.799+0000 7f6b2cc00700 1 -- 172.21.15.79:0/55475161 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b1800cd10 con 0x7f6b30102fc0 2024-06-03T02:18:58.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.800+0000 7f6b2cc00700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6b140a2c20 0x7f6b140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.800+0000 7f6b416ebb80 1 Processor -- start 2024-06-03T02:18:58.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b416ebb80 1 -- start start 2024-06-03T02:18:58.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b2cc00700 1 -- 172.21.15.79:0/55475161 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6b180cf4e0 con 0x7f6b30102fc0 2024-06-03T02:18:58.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6b140a2c20 0x7f6b140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e38a2df0 0x55c0e39283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 0x55c0e3928d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3931520 0x55c0e3933910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c0e38359c0 con 0x55c0e3931520 2024-06-03T02:18:58.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0e3878ab0 con 0x55c0e38a2df0 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c0e38338a0 con 0x55c0e3928910 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b34e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3931520 0x55c0e3933910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b2f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 0x55c0e3928d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b34e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3931520 0x55c0e3933910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33772/0 (socket says 172.21.15.79:33772) 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.801+0000 7f6b34e00700 1 -- 172.21.15.79:0/1922379772 learned_addr learned my addr 172.21.15.79:0/1922379772 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/1922379772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e38a2df0 0x55c0e39283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6b140a2c20 0x7f6b140a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009cb0 tx=0x7f6b20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b2f400700 1 -- 172.21.15.79:0/1922379772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e38a2df0 msgr2=0x55c0e39283d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b2f400700 1 --2- 172.21.15.79:0/1922379772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e38a2df0 0x55c0e39283d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b2f400700 1 -- 172.21.15.79:0/1922379772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3931520 msgr2=0x55c0e3933910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:58.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b2f400700 1 --2- 172.21.15.79:0/1922379772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3931520 0x55c0e3933910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b2f400700 1 -- 172.21.15.79:0/1922379772 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e3820f40 con 0x55c0e3928910 2024-06-03T02:18:58.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/1922379772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e38a2df0 0x55c0e39283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:18:58.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b2f400700 1 --2- 172.21.15.79:0/1922379772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 0x55c0e3928d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c5b0 tx=0x7f6b1809c0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbd0a2b64d95de38 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:58.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.802+0000 7f6b25600700 1 -- 172.21.15.79:0/1922379772 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1809fce0 con 0x55c0e3928910 2024-06-03T02:18:58.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.803+0000 7f6b25600700 1 -- 172.21.15.79:0/1922379772 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b180afe50 con 0x55c0e3928910 2024-06-03T02:18:58.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.803+0000 7f6b25600700 1 -- 172.21.15.79:0/1922379772 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b180860b0 con 0x55c0e3928910 2024-06-03T02:18:58.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.803+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1922379772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 msgr2=0x55c0e3928d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:58.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.803+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/1922379772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 0x55c0e3928d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c5b0 tx=0x7f6b1809c0b0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.803+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1922379772 shutdown_connections 2024-06-03T02:18:58.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.803+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/1922379772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3931520 0x55c0e3933910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.803+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/1922379772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e38a2df0 0x55c0e39283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.803+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/1922379772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 0x55c0e3928d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.803+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1922379772 >> 172.21.15.79:0/1922379772 conn(0x55c0e3853120 msgr2=0x55c0e39227d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:58.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.804+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1922379772 shutdown_connections 2024-06-03T02:18:58.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.804+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1922379772 wait complete. 2024-06-03T02:18:58.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.804+0000 7f6b416ebb80 1 -- start start 2024-06-03T02:18:58.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b416ebb80 1 Processor -- start 2024-06-03T02:18:58.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e3930d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 0x55c0e392e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e392e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c0e3814f10 con 0x55c0e38a2df0 2024-06-03T02:18:58.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0e3874a70 con 0x55c0e3931520 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c0e39340d0 con 0x55c0e3928910 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b416ebb80 10 client.0 _ll_register_callbacks cb 0x55c0e38126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b2fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e3930d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b2fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e3930d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33774/0 (socket says 172.21.15.79:33774) 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b2fe00700 1 -- 172.21.15.79:0/3059734712 learned_addr learned my addr 172.21.15.79:0/3059734712 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b34e00700 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e392e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.805+0000 7f6b2f400700 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 0x55c0e392e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.806+0000 7f6b2fe00700 1 -- 172.21.15.79:0/3059734712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 msgr2=0x55c0e392e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.806+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 0x55c0e392e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.806+0000 7f6b2fe00700 1 -- 172.21.15.79:0/3059734712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 msgr2=0x55c0e392e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.806+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e392e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.806+0000 7f6b2fe00700 1 -- 172.21.15.79:0/3059734712 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e37c7880 con 0x55c0e38a2df0 2024-06-03T02:18:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.806+0000 7f6b34e00700 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e392e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.806+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e3930d00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b20004090 tx=0x7f6b200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10a70e404cd174a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.806+0000 7f6b1e000700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:18:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.807+0000 7f6b1e000700 1 -- 172.21.15.79:0/3059734712 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20004830 con 0x55c0e38a2df0 2024-06-03T02:18:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.807+0000 7f6b416ebb80 1 -- 172.21.15.79:0/3059734712 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b180afe50 con 0x55c0e38a2df0 2024-06-03T02:18:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.807+0000 7f6b1e000700 1 -- 172.21.15.79:0/3059734712 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6b200049d0 con 0x55c0e38a2df0 2024-06-03T02:18:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.807+0000 7f6b1e000700 1 -- 172.21.15.79:0/3059734712 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b2002f050 con 0x55c0e38a2df0 2024-06-03T02:18:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.807+0000 7f6b416ebb80 10 client.15003 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.807+0000 7f6b416ebb80 1 -- 172.21.15.79:0/3059734712 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0e393e940 con 0x55c0e38a2df0 2024-06-03T02:18:58.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.807+0000 7f6b1e000700 1 -- 172.21.15.79:0/3059734712 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b180afe50 con 0x55c0e38a2df0 2024-06-03T02:18:58.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.807+0000 7f6b1e000700 1 -- 172.21.15.79:0/3059734712 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6b200052b0 con 0x55c0e38a2df0 2024-06-03T02:18:58.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.808+0000 7f6b416ebb80 10 client.15003 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:58.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.808+0000 7f6b416ebb80 10 client.15003 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-03T02:18:58.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.808+0000 7f6b416ebb80 20 client.15003 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:58.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.808+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55c0e3940f50 0x55c0e39423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.808+0000 7f6b416ebb80 4 client.15003 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.808+0000 7f6b416ebb80 1 -- 172.21.15.79:0/3059734712 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0e3885fb0 con 0x55c0e3940f50 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.808+0000 7f6b2f400700 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55c0e3940f50 0x55c0e39423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.808+0000 7f6b2f400700 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55c0e3940f50 0x55c0e39423c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e819368c7e376a53 server_cookie=ea50a3367ed7d3bd in_seq=0 out_seq=0 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.808+0000 7f6b1e000700 10 client.15003 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.809+0000 7f6b1e000700 1 -- 172.21.15.79:0/3059734712 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c0e3885fb0 con 0x55c0e3940f50 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.809+0000 7f6b1e000700 10 client.15003 handle_command_reply: tid=0 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.809+0000 7f6b416ebb80 1 client.15003 shutdown 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 -- 172.21.15.79:0/3059734712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 msgr2=0x55c0e3930d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e3930d00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b20004090 tx=0x7f6b200047c0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 -- 172.21.15.79:0/3059734712 shutdown_connections 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55c0e3940f50 0x55c0e39423c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e392e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e3930d00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/3059734712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3928910 0x55c0e392e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b1e000700 0 client.15003 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:18:58.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 -- 172.21.15.79:0/3059734712 >> 172.21.15.79:0/3059734712 conn(0x55c0e3853120 msgr2=0x55c0e39223c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:58.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 -- 172.21.15.79:0/3059734712 shutdown_connections 2024-06-03T02:18:58.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 1 -- 172.21.15.79:0/3059734712 wait complete. 2024-06-03T02:18:58.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.810+0000 7f6b416ebb80 20 client.15003 trim_cache size 0 max 16384 2024-06-03T02:18:58.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.833+0000 7f6b416ebb80 1 Processor -- start 2024-06-03T02:18:58.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.833+0000 7f6b416ebb80 1 -- start start 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.833+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e392e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.833+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3928910 0x55c0e3927030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.833+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3931520 0x55c0e3929380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.833+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c0e3940700 con 0x55c0e38a2df0 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.833+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0e3940dd0 con 0x55c0e3928910 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.833+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c0e39414a0 con 0x55c0e3931520 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3928910 0x55c0e3927030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e392e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3928910 0x55c0e3927030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:44968/0 (socket says 172.21.15.79:44968) 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e392e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33786/0 (socket says 172.21.15.79:33786) 2024-06-03T02:18:58.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2f400700 1 -- 172.21.15.79:0/1342759748 learned_addr learned my addr 172.21.15.79:0/1342759748 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b34e00700 1 --2- 172.21.15.79:0/1342759748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3931520 0x55c0e3929380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2f400700 1 -- 172.21.15.79:0/1342759748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3931520 msgr2=0x55c0e3929380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2f400700 1 --2- 172.21.15.79:0/1342759748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3931520 0x55c0e3929380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2f400700 1 -- 172.21.15.79:0/1342759748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 msgr2=0x55c0e392e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2f400700 1 --2- 172.21.15.79:0/1342759748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e392e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2f400700 1 -- 172.21.15.79:0/1342759748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e392c5e0 con 0x55c0e3928910 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.834+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/1342759748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e392e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.835+0000 7f6b2f400700 1 --2- 172.21.15.79:0/1342759748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3928910 0x55c0e3927030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b18010ba0 tx=0x7f6b1809c1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e68e7e0b8fcf3f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.835+0000 7f6b24c00700 1 -- 172.21.15.79:0/1342759748 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b180b9070 con 0x55c0e3928910 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.835+0000 7f6b24c00700 1 -- 172.21.15.79:0/1342759748 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b180b3070 con 0x55c0e3928910 2024-06-03T02:18:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.835+0000 7f6b24c00700 1 -- 172.21.15.79:0/1342759748 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b180a4370 con 0x55c0e3928910 2024-06-03T02:18:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1342759748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3928910 msgr2=0x55c0e3927030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/1342759748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3928910 0x55c0e3927030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b18010ba0 tx=0x7f6b1809c1a0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1342759748 shutdown_connections 2024-06-03T02:18:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/1342759748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e3931520 0x55c0e3929380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/1342759748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e38a2df0 0x55c0e392e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/1342759748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3928910 0x55c0e3927030 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1342759748 >> 172.21.15.79:0/1342759748 conn(0x55c0e3853120 msgr2=0x55c0e3921900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:58.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1342759748 shutdown_connections 2024-06-03T02:18:58.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 -- 172.21.15.79:0/1342759748 wait complete. 2024-06-03T02:18:58.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.836+0000 7f6b416ebb80 1 -- start start 2024-06-03T02:18:58.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.837+0000 7f6b416ebb80 1 Processor -- start 2024-06-03T02:18:58.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.837+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e38a2df0 0x55c0e3948580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.837+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3928910 0x55c0e3948ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.837+0000 7f6b416ebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e3944dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.837+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c0e3942240 con 0x55c0e3928910 2024-06-03T02:18:58.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.837+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0e3941b70 con 0x55c0e3931520 2024-06-03T02:18:58.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.837+0000 7f6b416ebb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c0e3942910 con 0x55c0e38a2df0 2024-06-03T02:18:58.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.837+0000 7f6b416ebb80 10 client.0 _ll_register_callbacks cb 0x55c0e38126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:18:58.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b34e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e3944dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b2f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3928910 0x55c0e3948ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b2f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3928910 0x55c0e3948ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33790/0 (socket says 172.21.15.79:33790) 2024-06-03T02:18:58.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b34e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e3944dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:44972/0 (socket says 172.21.15.79:44972) 2024-06-03T02:18:58.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b2f400700 1 -- 172.21.15.79:0/2711104276 learned_addr learned my addr 172.21.15.79:0/2711104276 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:18:58.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e38a2df0 0x55c0e3948580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b34e00700 1 -- 172.21.15.79:0/2711104276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e38a2df0 msgr2=0x55c0e3948580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:58.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b34e00700 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e38a2df0 0x55c0e3948580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b34e00700 1 -- 172.21.15.79:0/2711104276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3928910 msgr2=0x55c0e3948ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:18:58.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b34e00700 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3928910 0x55c0e3948ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b34e00700 1 -- 172.21.15.79:0/2711104276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e3945430 con 0x55c0e3931520 2024-06-03T02:18:58.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b2f400700 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3928910 0x55c0e3948ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:18:58.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b34e00700 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e3944dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009f90 tx=0x7f6b28003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a7d9a814e938ab server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:18:58.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.838+0000 7f6b1e000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:18:58.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.839+0000 7f6b1e000700 1 -- 172.21.15.79:0/2711104276 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2802a020 con 0x55c0e3931520 2024-06-03T02:18:58.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.839+0000 7f6b416ebb80 1 -- 172.21.15.79:0/2711104276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b180b3070 con 0x55c0e3931520 2024-06-03T02:18:58.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.839+0000 7f6b1e000700 1 -- 172.21.15.79:0/2711104276 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6b28005250 con 0x55c0e3931520 2024-06-03T02:18:58.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.839+0000 7f6b1e000700 1 -- 172.21.15.79:0/2711104276 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b28032050 con 0x55c0e3931520 2024-06-03T02:18:58.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.839+0000 7f6b416ebb80 10 client.25127 fetch_fsmap learned FSMap version 29 2024-06-03T02:18:58.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.839+0000 7f6b416ebb80 1 -- 172.21.15.79:0/2711104276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0e3949df0 con 0x55c0e3931520 2024-06-03T02:18:58.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.839+0000 7f6b1e000700 1 -- 172.21.15.79:0/2711104276 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b180b3070 con 0x55c0e3931520 2024-06-03T02:18:58.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.839+0000 7f6b1e000700 1 -- 172.21.15.79:0/2711104276 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6b28025070 con 0x55c0e3931520 2024-06-03T02:18:58.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.839+0000 7f6b416ebb80 10 client.25127 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:18:58.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.840+0000 7f6b416ebb80 10 client.25127 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-03T02:18:58.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.840+0000 7f6b416ebb80 20 client.25127 populate_metadata read hostname 'smithi079' 2024-06-03T02:18:58.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.840+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55c0e39b0da0 0x55c0e39afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:18:58.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.840+0000 7f6b416ebb80 4 client.25127 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:18:58.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.840+0000 7f6b416ebb80 1 -- 172.21.15.79:0/2711104276 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "status"}) -- 0x55c0e3880530 con 0x55c0e39b0da0 2024-06-03T02:18:58.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.840+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55c0e39b0da0 0x55c0e39afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:18:58.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.840+0000 7f6b2fe00700 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55c0e39b0da0 0x55c0e39afab0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82d7e248cb34d436 server_cookie=ecf763f617ad8c50 in_seq=0 out_seq=0 2024-06-03T02:18:58.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.840+0000 7f6b1e000700 10 client.25127 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:18:58.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b1e000700 1 -- 172.21.15.79:0/2711104276 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55c0e3880530 con 0x55c0e39b0da0 2024-06-03T02:18:58.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b1e000700 10 client.25127 handle_command_reply: tid=0 2024-06-03T02:18:58.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 client.25127 shutdown 2024-06-03T02:18:58.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 -- 172.21.15.79:0/2711104276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 msgr2=0x55c0e3944dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:58.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e3944dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009f90 tx=0x7f6b28003f90 comp rx=0 tx=0).stop 2024-06-03T02:18:58.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 -- 172.21.15.79:0/2711104276 shutdown_connections 2024-06-03T02:18:58.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55c0e39b0da0 0x55c0e39afab0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e3931520 0x55c0e3944dc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0e38a2df0 0x55c0e3948580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 --2- 172.21.15.79:0/2711104276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c0e3928910 0x55c0e3948ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 -- 172.21.15.79:0/2711104276 >> 172.21.15.79:0/2711104276 conn(0x55c0e3853120 msgr2=0x55c0e3929380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:58.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b1e000700 0 client.25127 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:18:58.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 -- 172.21.15.79:0/2711104276 shutdown_connections 2024-06-03T02:18:58.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 1 -- 172.21.15.79:0/2711104276 wait complete. 2024-06-03T02:18:58.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.841+0000 7f6b416ebb80 20 client.25127 trim_cache size 0 max 16384 2024-06-03T02:18:58.855 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:18:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:18:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 4, 2024-06-03T02:18:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24482, 2024-06-03T02:18:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:18:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:18:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:18:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 203.54588391499999, 2024-06-03T02:18:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:18:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:18:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:18:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 333.49881568000001 2024-06-03T02:18:58.857 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:18:58.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 -- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6b140a2c20 msgr2=0x7f6b140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:58.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6b140a2c20 0x7f6b140a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009cb0 tx=0x7f6b20009260 comp rx=0 tx=0).stop 2024-06-03T02:18:58.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 -- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b30102fc0 msgr2=0x7f6b3019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:18:58.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b30102fc0 0x7f6b3019b2e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e9f0 tx=0x7f6b1800ed00 comp rx=0 tx=0).stop 2024-06-03T02:18:58.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 -- 172.21.15.79:0/55475161 shutdown_connections 2024-06-03T02:18:58.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b301076f0 0x7f6b3019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b30102610 0x7f6b3019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6b140a2c20 0x7f6b140a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 --2- 172.21.15.79:0/55475161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6b30102fc0 0x7f6b3019b2e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:18:58.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 -- 172.21.15.79:0/55475161 >> 172.21.15.79:0/55475161 conn(0x7f6b300fdea0 msgr2=0x7f6b3010d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:18:58.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 -- 172.21.15.79:0/55475161 shutdown_connections 2024-06-03T02:18:58.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:18:58.842+0000 7f6b35800700 1 -- 172.21.15.79:0/55475161 wait complete. 2024-06-03T02:18:59.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:18:59 smithi079 ceph-mon[31819]: pgmap v484: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:59.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:18:59 smithi123 ceph-mon[36640]: pgmap v484: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:18:59.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-03T02:18:59.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:18:59 smithi135 ceph-mon[36653]: pgmap v484: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:19:00.215 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:01.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:01 smithi079 ceph-mon[31819]: pgmap v485: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:19:01.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:01 smithi123 ceph-mon[36640]: pgmap v485: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:19:01.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:01 smithi135 ceph-mon[36653]: pgmap v485: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:19:02.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.582+0000 7f8f33c00700 1 -- 172.21.15.79:0/4119008810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f2c06ab30 msgr2=0x7f8f2c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:02.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.582+0000 7f8f33c00700 1 --2- 172.21.15.79:0/4119008810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f2c06ab30 0x7f8f2c10f320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400b0a0 tx=0x7f8f2400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.582+0000 7f8f33c00700 1 -- 172.21.15.79:0/4119008810 shutdown_connections 2024-06-03T02:19:02.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.582+0000 7f8f33c00700 1 --2- 172.21.15.79:0/4119008810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f2c06ab30 0x7f8f2c10f320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.582+0000 7f8f33c00700 1 --2- 172.21.15.79:0/4119008810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f2c108500 0x7f8f2c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.582+0000 7f8f33c00700 1 --2- 172.21.15.79:0/4119008810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f2c06a170 0x7f8f2c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.582+0000 7f8f33c00700 1 -- 172.21.15.79:0/4119008810 >> 172.21.15.79:0/4119008810 conn(0x7f8f2c078340 msgr2=0x7f8f2c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:02.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.582+0000 7f8f33c00700 1 -- 172.21.15.79:0/4119008810 shutdown_connections 2024-06-03T02:19:02.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.582+0000 7f8f33c00700 1 -- 172.21.15.79:0/4119008810 wait complete. 2024-06-03T02:19:02.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.583+0000 7f8f33c00700 1 Processor -- start 2024-06-03T02:19:02.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.584+0000 7f8f33c00700 1 -- start start 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.584+0000 7f8f33c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f2c06a170 0x7f8f2c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.584+0000 7f8f33c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f2c06ab30 0x7f8f2c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.584+0000 7f8f33c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f2c108500 0x7f8f2c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.584+0000 7f8f33c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1127b0 con 0x7f8f2c06ab30 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.584+0000 7f8f33c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c112630 con 0x7f8f2c108500 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.584+0000 7f8f33c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c112930 con 0x7f8f2c06a170 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.584+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f2c06ab30 0x7f8f2c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f2c06ab30 0x7f8f2c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33800/0 (socket says 172.21.15.79:33800) 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f31200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f2c06a170 0x7f8f2c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f31c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f2c108500 0x7f8f2c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f2be00700 1 -- 172.21.15.79:0/1583585453 learned_addr learned my addr 172.21.15.79:0/1583585453 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f31c00700 1 -- 172.21.15.79:0/1583585453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f2c06a170 msgr2=0x7f8f2c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:02.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f31c00700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f2c06a170 0x7f8f2c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f31c00700 1 -- 172.21.15.79:0/1583585453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f2c06ab30 msgr2=0x7f8f2c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:02.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f31c00700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f2c06ab30 0x7f8f2c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f31c00700 1 -- 172.21.15.79:0/1583585453 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c199130 con 0x7f8f2c108500 2024-06-03T02:19:02.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f31200700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f2c06a170 0x7f8f2c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:02.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.585+0000 7f8f31c00700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f2c108500 0x7f8f2c1989d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400b070 tx=0x7f8f24004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:02.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.586+0000 7f8f29600700 1 -- 172.21.15.79:0/1583585453 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24021070 con 0x7f8f2c108500 2024-06-03T02:19:02.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.586+0000 7f8f29600700 1 -- 172.21.15.79:0/1583585453 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f24007b80 con 0x7f8f2c108500 2024-06-03T02:19:02.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.586+0000 7f8f33c00700 1 -- 172.21.15.79:0/1583585453 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2c1993c0 con 0x7f8f2c108500 2024-06-03T02:19:02.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.586+0000 7f8f29600700 1 -- 172.21.15.79:0/1583585453 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24005040 con 0x7f8f2c108500 2024-06-03T02:19:02.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.586+0000 7f8f33c00700 1 -- 172.21.15.79:0/1583585453 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2c1a56e0 con 0x7f8f2c108500 2024-06-03T02:19:02.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.589+0000 7f8f29600700 1 -- 172.21.15.79:0/1583585453 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f24024070 con 0x7f8f2c108500 2024-06-03T02:19:02.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.590+0000 7f8f29600700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8f140a2c20 0x7f8f140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.590+0000 7f8f3e50eb80 1 Processor -- start 2024-06-03T02:19:02.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.590+0000 7f8f29600700 1 -- 172.21.15.79:0/1583585453 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8f240d02f0 con 0x7f8f2c108500 2024-06-03T02:19:02.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.591+0000 7f8f31200700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8f140a2c20 0x7f8f140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.591+0000 7f8f3e50eb80 1 -- start start 2024-06-03T02:19:02.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.591+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f70020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.591+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 0x563705f709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.591+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f7b2b0 0x563705f7d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.591+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563705e7f9c0 con 0x563705f7b2b0 2024-06-03T02:19:02.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.591+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563705ec2ab0 con 0x563705eecdf0 2024-06-03T02:19:02.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.591+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563705e7d8a0 con 0x563705f70560 2024-06-03T02:19:02.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.591+0000 7f8f31200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f70020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f70020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:44990/0 (socket says 172.21.15.79:44990) 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 0x563705f709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31200700 1 -- 172.21.15.79:0/294194870 learned_addr learned my addr 172.21.15.79:0/294194870 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f7b2b0 0x563705f7d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31200700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8f140a2c20 0x7f8f140a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c199b10 tx=0x7f8f20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31200700 1 -- 172.21.15.79:0/294194870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 msgr2=0x563705f709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31200700 1 --2- 172.21.15.79:0/294194870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 0x563705f709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31200700 1 -- 172.21.15.79:0/294194870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f7b2b0 msgr2=0x563705f7d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31200700 1 --2- 172.21.15.79:0/294194870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f7b2b0 0x563705f7d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31200700 1 -- 172.21.15.79:0/294194870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563705e6af40 con 0x563705eecdf0 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.592+0000 7f8f31c00700 1 --2- 172.21.15.79:0/294194870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f7b2b0 0x563705f7d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:19:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.593+0000 7f8f31200700 1 --2- 172.21.15.79:0/294194870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f70020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f20004200 tx=0x7f8f20014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba260978c5da741c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:02.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.593+0000 7f8f1e000700 1 -- 172.21.15.79:0/294194870 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20003d10 con 0x563705eecdf0 2024-06-03T02:19:02.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.593+0000 7f8f1e000700 1 -- 172.21.15.79:0/294194870 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f20021070 con 0x563705eecdf0 2024-06-03T02:19:02.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.593+0000 7f8f1e000700 1 -- 172.21.15.79:0/294194870 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2001b750 con 0x563705eecdf0 2024-06-03T02:19:02.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.594+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/294194870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 msgr2=0x563705f70020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.594+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/294194870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f70020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f20004200 tx=0x7f8f20014000 comp rx=0 tx=0).stop 2024-06-03T02:19:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.594+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/294194870 shutdown_connections 2024-06-03T02:19:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.594+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/294194870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f7b2b0 0x563705f7d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.594+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/294194870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f70020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.594+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/294194870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 0x563705f709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.594+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/294194870 >> 172.21.15.79:0/294194870 conn(0x563705e9d120 msgr2=0x563705f6c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.594+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/294194870 shutdown_connections 2024-06-03T02:19:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.594+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/294194870 wait complete. 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.595+0000 7f8f3e50eb80 1 -- start start 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.595+0000 7f8f3e50eb80 1 Processor -- start 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.595+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f7aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.595+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f70560 0x563705f77dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.595+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f7b2b0 0x563705f78310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.595+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563705e5ef10 con 0x563705f70560 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.595+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563705ebea70 con 0x563705eecdf0 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.595+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563705f7df70 con 0x563705f7b2b0 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f3e50eb80 10 client.0 _ll_register_callbacks cb 0x563705e5c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:02.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f70560 0x563705f77dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f31200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f7aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f70560 0x563705f77dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33830/0 (socket says 172.21.15.79:33830) 2024-06-03T02:19:02.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f2be00700 1 -- 172.21.15.79:0/4200481033 learned_addr learned my addr 172.21.15.79:0/4200481033 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:02.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f31200700 1 -- 172.21.15.79:0/4200481033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f7b2b0 msgr2=0x563705f78310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:19:02.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f31200700 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f7b2b0 0x563705f78310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f31200700 1 -- 172.21.15.79:0/4200481033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f70560 msgr2=0x563705f77dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f31200700 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f70560 0x563705f77dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f31200700 1 -- 172.21.15.79:0/4200481033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563705e11880 con 0x563705eecdf0 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.596+0000 7f8f2be00700 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f70560 0x563705f77dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.597+0000 7f8f31200700 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f7aad0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009fd0 tx=0x7f8f2001dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f7c25aaa12bd391 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.597+0000 7f8f12a00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.597+0000 7f8f12a00700 1 -- 172.21.15.79:0/4200481033 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20034060 con 0x563705eecdf0 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.597+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/4200481033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f20021070 con 0x563705eecdf0 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.597+0000 7f8f12a00700 1 -- 172.21.15.79:0/4200481033 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8f2001d830 con 0x563705eecdf0 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.597+0000 7f8f12a00700 1 -- 172.21.15.79:0/4200481033 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f2003b050 con 0x563705eecdf0 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.597+0000 7f8f3e50eb80 10 client.25139 fetch_fsmap learned FSMap version 29 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.597+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/4200481033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563705f8cb50 con 0x563705eecdf0 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.598+0000 7f8f12a00700 1 -- 172.21.15.79:0/4200481033 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f20021070 con 0x563705eecdf0 2024-06-03T02:19:02.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.598+0000 7f8f12a00700 1 -- 172.21.15.79:0/4200481033 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8f20026630 con 0x563705eecdf0 2024-06-03T02:19:02.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.598+0000 7f8f3e50eb80 10 client.25139 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:19:02.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.598+0000 7f8f3e50eb80 10 client.25139 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T02:19:02.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.598+0000 7f8f3e50eb80 20 client.25139 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:02.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.598+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x563705f8eda0 0x563705f90210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.598+0000 7f8f3e50eb80 4 client.25139 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:19:02.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.598+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/4200481033 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563705ecffb0 con 0x563705f8eda0 2024-06-03T02:19:02.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.599+0000 7f8f2be00700 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x563705f8eda0 0x563705f90210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.599+0000 7f8f2be00700 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x563705f8eda0 0x563705f90210 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9637dc2301cdf0bf server_cookie=6edfb72a07672a1d in_seq=0 out_seq=0 2024-06-03T02:19:02.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.599+0000 7f8f12a00700 10 client.25139 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:19:02.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f12a00700 1 -- 172.21.15.79:0/4200481033 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563705ecffb0 con 0x563705f8eda0 2024-06-03T02:19:02.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f12a00700 10 client.25139 handle_command_reply: tid=0 2024-06-03T02:19:02.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f3e50eb80 1 client.25139 shutdown 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/4200481033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 msgr2=0x563705f7aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f7aad0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009fd0 tx=0x7f8f2001dc90 comp rx=0 tx=0).stop 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/4200481033 shutdown_connections 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f7b2b0 0x563705f78310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705eecdf0 0x563705f7aad0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x563705f8eda0 0x563705f90210 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/4200481033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705f70560 0x563705f77dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f12a00700 0 client.25139 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.600+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/4200481033 >> 172.21.15.79:0/4200481033 conn(0x563705e9d120 msgr2=0x563705f6b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.601+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/4200481033 shutdown_connections 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.601+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/4200481033 wait complete. 2024-06-03T02:19:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.601+0000 7f8f3e50eb80 20 client.25139 trim_cache size 0 max 16384 2024-06-03T02:19:02.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.624+0000 7f8f3e50eb80 1 Processor -- start 2024-06-03T02:19:02.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.624+0000 7f8f3e50eb80 1 -- start start 2024-06-03T02:19:02.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.624+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705e9d120 0x563705f76d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705eecdf0 0x563705f7a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705f70560 0x563705f7c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563705f8e550 con 0x563705e9d120 2024-06-03T02:19:02.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563705f8ec20 con 0x563705f70560 2024-06-03T02:19:02.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563705f8f2f0 con 0x563705eecdf0 2024-06-03T02:19:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f31200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705e9d120 0x563705f76d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705eecdf0 0x563705f7a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f31c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705f70560 0x563705f7c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705eecdf0 0x563705f7a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60700/0 (socket says 172.21.15.79:60700) 2024-06-03T02:19:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f31200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705e9d120 0x563705f76d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33834/0 (socket says 172.21.15.79:33834) 2024-06-03T02:19:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f2be00700 1 -- 172.21.15.79:0/626355935 learned_addr learned my addr 172.21.15.79:0/626355935 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f2be00700 1 -- 172.21.15.79:0/626355935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705f70560 msgr2=0x563705f7c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f2be00700 1 --2- 172.21.15.79:0/626355935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705f70560 0x563705f7c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f2be00700 1 -- 172.21.15.79:0/626355935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705e9d120 msgr2=0x563705f76d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:02.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f2be00700 1 --2- 172.21.15.79:0/626355935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705e9d120 0x563705f76d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f2be00700 1 -- 172.21.15.79:0/626355935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563705f76340 con 0x563705eecdf0 2024-06-03T02:19:02.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f31c00700 1 --2- 172.21.15.79:0/626355935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705f70560 0x563705f7c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:02.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.625+0000 7f8f31200700 1 --2- 172.21.15.79:0/626355935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705e9d120 0x563705f76d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:02.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.626+0000 7f8f2be00700 1 --2- 172.21.15.79:0/626355935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705eecdf0 0x563705f7a610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8f180096b0 tx=0x7f8f18009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace80a97b88154d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:02.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.626+0000 7f8f1d600700 1 -- 172.21.15.79:0/626355935 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18012110 con 0x563705eecdf0 2024-06-03T02:19:02.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.626+0000 7f8f1d600700 1 -- 172.21.15.79:0/626355935 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f18004510 con 0x563705eecdf0 2024-06-03T02:19:02.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.626+0000 7f8f1d600700 1 -- 172.21.15.79:0/626355935 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18005280 con 0x563705eecdf0 2024-06-03T02:19:02.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.627+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/626355935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705eecdf0 msgr2=0x563705f7a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:02.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.627+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/626355935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705eecdf0 0x563705f7a610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8f180096b0 tx=0x7f8f18009260 comp rx=0 tx=0).stop 2024-06-03T02:19:02.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.627+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/626355935 shutdown_connections 2024-06-03T02:19:02.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.627+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/626355935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705f70560 0x563705f7c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.627+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/626355935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705e9d120 0x563705f76d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.627+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/626355935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705eecdf0 0x563705f7a610 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.627+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/626355935 >> 172.21.15.79:0/626355935 conn(0x563705f684b0 msgr2=0x563705f86a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:02.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.627+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/626355935 shutdown_connections 2024-06-03T02:19:02.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.627+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/626355935 wait complete. 2024-06-03T02:19:02.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.628+0000 7f8f3e50eb80 1 -- start start 2024-06-03T02:19:02.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.628+0000 7f8f3e50eb80 1 Processor -- start 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.628+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705e9d120 0x563705f8afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.628+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705eecdf0 0x563705f86320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.628+0000 7f8f3e50eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 0x563705f86860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.628+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563705f90090 con 0x563705eecdf0 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.628+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563705f8f9c0 con 0x563705e9d120 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.628+0000 7f8f3e50eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563705f90760 con 0x563705f70560 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.628+0000 7f8f3e50eb80 10 client.0 _ll_register_callbacks cb 0x563705e5c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705e9d120 0x563705f8afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705eecdf0 0x563705f86320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705e9d120 0x563705f8afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:45004/0 (socket says 172.21.15.79:45004) 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 0x563705f86860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31200700 1 -- 172.21.15.79:0/2784040065 learned_addr learned my addr 172.21.15.79:0/2784040065 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31c00700 1 -- 172.21.15.79:0/2784040065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705e9d120 msgr2=0x563705f8afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31c00700 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705e9d120 0x563705f8afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31c00700 1 -- 172.21.15.79:0/2784040065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705eecdf0 msgr2=0x563705f86320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31c00700 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705eecdf0 0x563705f86320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31c00700 1 -- 172.21.15.79:0/2784040065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563705f87070 con 0x563705f70560 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31200700 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705e9d120 0x563705f8afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f2be00700 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705eecdf0 0x563705f86320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f31c00700 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 0x563705f86860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009fd0 tx=0x7f8f240a30b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=846c5b9fba376dfb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.629+0000 7f8f12a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.630+0000 7f8f12a00700 1 -- 172.21.15.79:0/2784040065 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2402b070 con 0x563705f70560 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.630+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/2784040065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f18004510 con 0x563705f70560 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.630+0000 7f8f12a00700 1 -- 172.21.15.79:0/2784040065 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8f240074e0 con 0x563705f70560 2024-06-03T02:19:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.630+0000 7f8f12a00700 1 -- 172.21.15.79:0/2784040065 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f2409c7d0 con 0x563705f70560 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.630+0000 7f8f3e50eb80 10 client.25123 fetch_fsmap learned FSMap version 29 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.630+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/2784040065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563705fff030 con 0x563705f70560 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.630+0000 7f8f12a00700 1 -- 172.21.15.79:0/2784040065 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f18004510 con 0x563705f70560 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.630+0000 7f8f12a00700 1 -- 172.21.15.79:0/2784040065 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8f24011070 con 0x563705f70560 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.630+0000 7f8f3e50eb80 10 client.25123 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.631+0000 7f8f3e50eb80 10 client.25123 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.631+0000 7f8f3e50eb80 20 client.25123 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.631+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x563705f7b2b0 0x563706002940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.631+0000 7f8f3e50eb80 4 client.25123 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.631+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/2784040065 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "status"}) -- 0x563705eca530 con 0x563705f7b2b0 2024-06-03T02:19:02.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.631+0000 7f8f31200700 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x563705f7b2b0 0x563706002940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:02.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.631+0000 7f8f12a00700 10 client.25123 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.631+0000 7f8f31200700 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x563705f7b2b0 0x563706002940 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9862d00b9d086dad server_cookie=3f4a8da9720b73b7 in_seq=0 out_seq=0 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f12a00700 1 -- 172.21.15.79:0/2784040065 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x563705eca530 con 0x563705f7b2b0 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f12a00700 10 client.25123 handle_command_reply: tid=0 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 client.25123 shutdown 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/2784040065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 msgr2=0x563705f86860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 0x563705f86860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009fd0 tx=0x7f8f240a30b0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/2784040065 shutdown_connections 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563705f70560 0x563705f86860 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563705e9d120 0x563705f8afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x563705f7b2b0 0x563706002940 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 --2- 172.21.15.79:0/2784040065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563705eecdf0 0x563705f86320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/2784040065 >> 172.21.15.79:0/2784040065 conn(0x563705f684b0 msgr2=0x563705f77650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f12a00700 0 client.25123 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:19:02.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/2784040065 shutdown_connections 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 1 -- 172.21.15.79:0/2784040065 wait complete. 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.632+0000 7f8f3e50eb80 20 client.25123 trim_cache size 0 max 16384 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 3, 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24490, 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:19:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 208.342199368, 2024-06-03T02:19:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:19:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:19:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:19:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 325.58876012000002 2024-06-03T02:19:02.637 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:19:02.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 -- 172.21.15.79:0/1583585453 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8f140a2c20 msgr2=0x7f8f140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:02.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8f140a2c20 0x7f8f140a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c199b10 tx=0x7f8f20009260 comp rx=0 tx=0).stop 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 -- 172.21.15.79:0/1583585453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f2c108500 msgr2=0x7f8f2c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f2c108500 0x7f8f2c1989d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400b070 tx=0x7f8f24004c80 comp rx=0 tx=0).stop 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 -- 172.21.15.79:0/1583585453 shutdown_connections 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f2c108500 0x7f8f2c1989d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f2c06a170 0x7f8f2c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8f140a2c20 0x7f8f140a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 --2- 172.21.15.79:0/1583585453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f2c06ab30 0x7f8f2c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 -- 172.21.15.79:0/1583585453 >> 172.21.15.79:0/1583585453 conn(0x7f8f2c078340 msgr2=0x7f8f2c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.633+0000 7f8f33c00700 1 -- 172.21.15.79:0/1583585453 shutdown_connections 2024-06-03T02:19:02.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:02.634+0000 7f8f33c00700 1 -- 172.21.15.79:0/1583585453 wait complete. 2024-06-03T02:19:03.445 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-03T02:19:03.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:03 smithi135 ceph-mon[36653]: pgmap v486: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:19:03.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:03 smithi123 ceph-mon[36640]: pgmap v486: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:19:03.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:03 smithi079 ceph-mon[31819]: pgmap v486: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:19:03.891 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:05.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:05 smithi079 ceph-mon[31819]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:19:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:05 smithi123 ceph-mon[36640]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:19:05.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:05 smithi135 ceph-mon[36653]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:19:06.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.067+0000 7f9472400700 1 -- 172.21.15.79:0/2454372660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f946c105070 msgr2=0x7f946c111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:06.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.067+0000 7f9472400700 1 --2- 172.21.15.79:0/2454372660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f946c105070 0x7f946c111580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f946400b0a0 tx=0x7f946400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.067+0000 7f9472400700 1 -- 172.21.15.79:0/2454372660 shutdown_connections 2024-06-03T02:19:06.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.067+0000 7f9472400700 1 --2- 172.21.15.79:0/2454372660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f946c105070 0x7f946c111580 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.067+0000 7f9472400700 1 --2- 172.21.15.79:0/2454372660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f946c10a6b0 0x7f946c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.067+0000 7f9472400700 1 --2- 172.21.15.79:0/2454372660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946c1046b0 0x7f946c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.067+0000 7f9472400700 1 -- 172.21.15.79:0/2454372660 >> 172.21.15.79:0/2454372660 conn(0x7f946c1001a0 msgr2=0x7f946c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:06.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.067+0000 7f9472400700 1 -- 172.21.15.79:0/2454372660 shutdown_connections 2024-06-03T02:19:06.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.067+0000 7f9472400700 1 -- 172.21.15.79:0/2454372660 wait complete. 2024-06-03T02:19:06.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.068+0000 7f9472400700 1 Processor -- start 2024-06-03T02:19:06.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f9472400700 1 -- start start 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f9472400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946c1046b0 0x7f946c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f9472400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f946c105070 0x7f946c19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f9472400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f946c10a6b0 0x7f946c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f9472400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f946c114a40 con 0x7f946c10a6b0 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f9472400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1148c0 con 0x7f946c1046b0 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f9472400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f946c114bc0 con 0x7f946c105070 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f946b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946c1046b0 0x7f946c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f946be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f946c10a6b0 0x7f946c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f946b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946c1046b0 0x7f946c19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36698/0 (socket says 172.21.15.79:36698) 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.069+0000 7f946b400700 1 -- 172.21.15.79:0/2077703814 learned_addr learned my addr 172.21.15.79:0/2077703814 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.070+0000 7f946aa00700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f946c105070 0x7f946c19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.070+0000 7f946be00700 1 -- 172.21.15.79:0/2077703814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f946c105070 msgr2=0x7f946c19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:06.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.070+0000 7f946be00700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f946c105070 0x7f946c19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.070+0000 7f946be00700 1 -- 172.21.15.79:0/2077703814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946c1046b0 msgr2=0x7f946c19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.070+0000 7f946be00700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946c1046b0 0x7f946c19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.070+0000 7f946be00700 1 -- 172.21.15.79:0/2077703814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c19fda0 con 0x7f946c10a6b0 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.070+0000 7f946aa00700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f946c105070 0x7f946c19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.070+0000 7f946b400700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946c1046b0 0x7f946c19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.070+0000 7f946be00700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f946c10a6b0 0x7f946c19f660 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f946400b0a0 tx=0x7f9464004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.071+0000 7f9463e00700 1 -- 172.21.15.79:0/2077703814 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464021070 con 0x7f946c10a6b0 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.071+0000 7f9472400700 1 -- 172.21.15.79:0/2077703814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946c1a0030 con 0x7f946c10a6b0 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.071+0000 7f9463e00700 1 -- 172.21.15.79:0/2077703814 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9464007b80 con 0x7f946c10a6b0 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.071+0000 7f9463e00700 1 -- 172.21.15.79:0/2077703814 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94640052a0 con 0x7f946c10a6b0 2024-06-03T02:19:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.071+0000 7f9472400700 1 -- 172.21.15.79:0/2077703814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946c1a7910 con 0x7f946c10a6b0 2024-06-03T02:19:06.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.074+0000 7f9463e00700 1 -- 172.21.15.79:0/2077703814 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9464024070 con 0x7f946c10a6b0 2024-06-03T02:19:06.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.075+0000 7f9463e00700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94500a2c20 0x7f94500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.075+0000 7f947cd60b80 1 Processor -- start 2024-06-03T02:19:06.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.075+0000 7f9463e00700 1 -- 172.21.15.79:0/2077703814 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f94640d0390 con 0x7f946c10a6b0 2024-06-03T02:19:06.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f946b400700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94500a2c20 0x7f94500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f947cd60b80 1 -- start start 2024-06-03T02:19:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b8bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b85cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b86630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f947cd60b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560af5a949c0 con 0x560af5b8c460 2024-06-03T02:19:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f947cd60b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560af5ad7ab0 con 0x560af5b861f0 2024-06-03T02:19:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f947cd60b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560af5a928a0 con 0x560af5b8bb40 2024-06-03T02:19:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f946b400700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94500a2c20 0x7f94500a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f945c011550 tx=0x7f945c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.076+0000 7f946aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b85cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b85cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44344/0 (socket says 172.21.15.79:44344) 2024-06-03T02:19:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946aa00700 1 -- 172.21.15.79:0/758504188 learned_addr learned my addr 172.21.15.79:0/758504188 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946be00700 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b86630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946b400700 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b8bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946be00700 1 -- 172.21.15.79:0/758504188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 msgr2=0x560af5b8bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946be00700 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b8bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946be00700 1 -- 172.21.15.79:0/758504188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 msgr2=0x560af5b85cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946be00700 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b85cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946be00700 1 -- 172.21.15.79:0/758504188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af5a7ff40 con 0x560af5b861f0 2024-06-03T02:19:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946aa00700 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b85cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:06.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.077+0000 7f946be00700 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b86630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f946409c800 tx=0x7f94640060a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c4fbf94de530 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:06.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.078+0000 7f9460c00700 1 -- 172.21.15.79:0/758504188 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946402b070 con 0x560af5b861f0 2024-06-03T02:19:06.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.078+0000 7f9460c00700 1 -- 172.21.15.79:0/758504188 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946401de50 con 0x560af5b861f0 2024-06-03T02:19:06.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.078+0000 7f9460c00700 1 -- 172.21.15.79:0/758504188 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94640a30b0 con 0x560af5b861f0 2024-06-03T02:19:06.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.078+0000 7f947cd60b80 1 -- 172.21.15.79:0/758504188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 msgr2=0x560af5b86630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:06.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.078+0000 7f947cd60b80 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b86630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f946409c800 tx=0x7f94640060a0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.079+0000 7f947cd60b80 1 -- 172.21.15.79:0/758504188 shutdown_connections 2024-06-03T02:19:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.079+0000 7f947cd60b80 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b86630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.079+0000 7f947cd60b80 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b8bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.079+0000 7f947cd60b80 1 --2- 172.21.15.79:0/758504188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b85cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.079+0000 7f947cd60b80 1 -- 172.21.15.79:0/758504188 >> 172.21.15.79:0/758504188 conn(0x560af5ab2120 msgr2=0x560af5b80fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:06.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.079+0000 7f947cd60b80 1 -- 172.21.15.79:0/758504188 shutdown_connections 2024-06-03T02:19:06.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.079+0000 7f947cd60b80 1 -- 172.21.15.79:0/758504188 wait complete. 2024-06-03T02:19:06.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.079+0000 7f947cd60b80 1 -- start start 2024-06-03T02:19:06.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.079+0000 7f947cd60b80 1 Processor -- start 2024-06-03T02:19:06.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b92240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b8f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b8fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f947cd60b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560af5a73f10 con 0x560af5b8c460 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f947cd60b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560af5ad3a70 con 0x560af5b861f0 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f947cd60b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560af5b92bf0 con 0x560af5b8bb40 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f947cd60b80 10 client.0 _ll_register_callbacks cb 0x560af5a6d330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f946b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b92240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f946be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b8fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f946be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b8fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44352/0 (socket says 172.21.15.79:44352) 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f946be00700 1 -- 172.21.15.79:0/3301010656 learned_addr learned my addr 172.21.15.79:0/3301010656 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f946b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b92240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36720/0 (socket says 172.21.15.79:36720) 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.080+0000 7f946aa00700 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b8f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.081+0000 7f946b400700 1 -- 172.21.15.79:0/3301010656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 msgr2=0x560af5b8f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.081+0000 7f946b400700 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b8f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.081+0000 7f946b400700 1 -- 172.21.15.79:0/3301010656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 msgr2=0x560af5b8fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.081+0000 7f946b400700 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b8fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.081+0000 7f946b400700 1 -- 172.21.15.79:0/3301010656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af5a26880 con 0x560af5b861f0 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.081+0000 7f946b400700 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b92240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f945c009f50 tx=0x7f945c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4598a14938a84816 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.081+0000 7f9455600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.081+0000 7f9455600700 1 -- 172.21.15.79:0/3301010656 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c01fb70 con 0x560af5b861f0 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.082+0000 7f947cd60b80 1 -- 172.21.15.79:0/3301010656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f946401de50 con 0x560af5b861f0 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.082+0000 7f9455600700 1 -- 172.21.15.79:0/3301010656 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f945c01fd10 con 0x560af5b861f0 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.082+0000 7f9455600700 1 -- 172.21.15.79:0/3301010656 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f945c004830 con 0x560af5b861f0 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.082+0000 7f947cd60b80 10 client.25157 fetch_fsmap learned FSMap version 29 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.082+0000 7f947cd60b80 1 -- 172.21.15.79:0/3301010656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560af5b9d3f0 con 0x560af5b861f0 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.082+0000 7f9455600700 1 -- 172.21.15.79:0/3301010656 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946401de50 con 0x560af5b861f0 2024-06-03T02:19:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.082+0000 7f9455600700 1 -- 172.21.15.79:0/3301010656 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f945c026070 con 0x560af5b861f0 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.082+0000 7f947cd60b80 10 client.25157 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.083+0000 7f947cd60b80 10 client.25157 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.083+0000 7f947cd60b80 20 client.25157 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.083+0000 7f947cd60b80 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x560af5b9fa10 0x560af5ba0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.083+0000 7f947cd60b80 4 client.25157 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.083+0000 7f947cd60b80 1 -- 172.21.15.79:0/3301010656 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560af5ae4fb0 con 0x560af5b9fa10 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.083+0000 7f946aa00700 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x560af5b9fa10 0x560af5ba0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.084+0000 7f946aa00700 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x560af5b9fa10 0x560af5ba0e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc074485522d9146 server_cookie=1b9767f2f3f5c332 in_seq=0 out_seq=0 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.084+0000 7f9455600700 10 client.25157 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.084+0000 7f9455600700 1 -- 172.21.15.79:0/3301010656 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560af5ae4fb0 con 0x560af5b9fa10 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.084+0000 7f9455600700 10 client.25157 handle_command_reply: tid=0 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 client.25157 shutdown 2024-06-03T02:19:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 -- 172.21.15.79:0/3301010656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 msgr2=0x560af5b92240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b92240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f945c009f50 tx=0x7f945c0047c0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 -- 172.21.15.79:0/3301010656 shutdown_connections 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b8fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x560af5b9fa10 0x560af5ba0e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b92240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 --2- 172.21.15.79:0/3301010656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b8f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 -- 172.21.15.79:0/3301010656 >> 172.21.15.79:0/3301010656 conn(0x560af5ab2120 msgr2=0x560af5b7a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f9455600700 0 client.25157 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 -- 172.21.15.79:0/3301010656 shutdown_connections 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 1 -- 172.21.15.79:0/3301010656 wait complete. 2024-06-03T02:19:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.085+0000 7f947cd60b80 20 client.25157 trim_cache size 0 max 16384 2024-06-03T02:19:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.108+0000 7f947cd60b80 1 Processor -- start 2024-06-03T02:19:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.108+0000 7f947cd60b80 1 -- start start 2024-06-03T02:19:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b85cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b81150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b8e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f947cd60b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560af5b9f1c0 con 0x560af5b8c460 2024-06-03T02:19:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f947cd60b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560af5b9f890 con 0x560af5b861f0 2024-06-03T02:19:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f947cd60b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560af5b9ff60 con 0x560af5b8bb40 2024-06-03T02:19:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b8e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b85cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b81150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b81150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:51588/0 (socket says 172.21.15.79:51588) 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946aa00700 1 -- 172.21.15.79:0/2114110570 learned_addr learned my addr 172.21.15.79:0/2114110570 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946b400700 1 -- 172.21.15.79:0/2114110570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 msgr2=0x560af5b81150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946b400700 1 --2- 172.21.15.79:0/2114110570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b81150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946b400700 1 -- 172.21.15.79:0/2114110570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 msgr2=0x560af5b8e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946b400700 1 --2- 172.21.15.79:0/2114110570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b8e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946b400700 1 -- 172.21.15.79:0/2114110570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af5b81cb0 con 0x560af5b861f0 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.109+0000 7f946aa00700 1 --2- 172.21.15.79:0/2114110570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b81150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.110+0000 7f946b400700 1 --2- 172.21.15.79:0/2114110570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b85cb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f945c027440 tx=0x7f945c009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a396fdf78d9dc509 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.110+0000 7f9457e00700 1 -- 172.21.15.79:0/2114110570 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c01f9f0 con 0x560af5b861f0 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.110+0000 7f9457e00700 1 -- 172.21.15.79:0/2114110570 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f945c027b90 con 0x560af5b861f0 2024-06-03T02:19:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.110+0000 7f9457e00700 1 -- 172.21.15.79:0/2114110570 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0347d0 con 0x560af5b861f0 2024-06-03T02:19:06.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 -- 172.21.15.79:0/2114110570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 msgr2=0x560af5b85cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:06.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 --2- 172.21.15.79:0/2114110570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b85cb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f945c027440 tx=0x7f945c009350 comp rx=0 tx=0).stop 2024-06-03T02:19:06.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 -- 172.21.15.79:0/2114110570 shutdown_connections 2024-06-03T02:19:06.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 --2- 172.21.15.79:0/2114110570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8c460 0x560af5b8e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 --2- 172.21.15.79:0/2114110570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b861f0 0x560af5b85cb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 --2- 172.21.15.79:0/2114110570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b8bb40 0x560af5b81150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 -- 172.21.15.79:0/2114110570 >> 172.21.15.79:0/2114110570 conn(0x560af5ab2120 msgr2=0x560af5b80a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:06.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 -- 172.21.15.79:0/2114110570 shutdown_connections 2024-06-03T02:19:06.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 -- 172.21.15.79:0/2114110570 wait complete. 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.111+0000 7f947cd60b80 1 -- start start 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.112+0000 7f947cd60b80 1 Processor -- start 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.112+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b861f0 0x560af5baa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.112+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8bb40 0x560af5b91350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.112+0000 7f947cd60b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b8c460 0x560af5b91890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.112+0000 7f947cd60b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560af5ba0d00 con 0x560af5b8bb40 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.112+0000 7f947cd60b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560af5ba0630 con 0x560af5b8c460 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.112+0000 7f947cd60b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560af5ba13d0 con 0x560af5b861f0 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.112+0000 7f947cd60b80 10 client.0 _ll_register_callbacks cb 0x560af5a6d330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.112+0000 7f946aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8bb40 0x560af5b91350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b861f0 0x560af5baa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b8c460 0x560af5b91890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b861f0 0x560af5baa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:51594/0 (socket says 172.21.15.79:51594) 2024-06-03T02:19:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946b400700 1 -- 172.21.15.79:0/4057809307 learned_addr learned my addr 172.21.15.79:0/4057809307 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946be00700 1 -- 172.21.15.79:0/4057809307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b861f0 msgr2=0x560af5baa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946be00700 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b861f0 0x560af5baa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946be00700 1 -- 172.21.15.79:0/4057809307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8bb40 msgr2=0x560af5b91350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946be00700 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8bb40 0x560af5b91350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946be00700 1 -- 172.21.15.79:0/4057809307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af5b920d0 con 0x560af5b8c460 2024-06-03T02:19:06.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f946be00700 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b8c460 0x560af5b91890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f946400b3f0 tx=0x7f9464098450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc8c7f7da5ddc724 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f9455600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f9455600700 1 -- 172.21.15.79:0/4057809307 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946402b040 con 0x560af5b8c460 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f947cd60b80 1 -- 172.21.15.79:0/4057809307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945c027b90 con 0x560af5b8c460 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.113+0000 7f9455600700 1 -- 172.21.15.79:0/4057809307 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f94640d0130 con 0x560af5b8c460 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f9455600700 1 -- 172.21.15.79:0/4057809307 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f946409cf70 con 0x560af5b8c460 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f947cd60b80 10 client.25163 fetch_fsmap learned FSMap version 29 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f947cd60b80 1 -- 172.21.15.79:0/4057809307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560af5c13ab0 con 0x560af5b8c460 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f9455600700 1 -- 172.21.15.79:0/4057809307 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f945c027b90 con 0x560af5b8c460 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f9455600700 1 -- 172.21.15.79:0/4057809307 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9464011070 con 0x560af5b8c460 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f947cd60b80 10 client.25163 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f947cd60b80 10 client.25163 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-03T02:19:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f947cd60b80 20 client.25163 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f947cd60b80 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x560af5c18400 0x560af5c17110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f947cd60b80 4 client.25163 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.114+0000 7f947cd60b80 1 -- 172.21.15.79:0/4057809307 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "status"}) -- 0x560af5adf530 con 0x560af5c18400 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.115+0000 7f946b400700 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x560af5c18400 0x560af5c17110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.115+0000 7f946b400700 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x560af5c18400 0x560af5c17110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bf561ba65f17036 server_cookie=d1185d1068adbf3d in_seq=0 out_seq=0 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.115+0000 7f9455600700 10 client.25163 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.115+0000 7f9455600700 1 -- 172.21.15.79:0/4057809307 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x560af5adf530 con 0x560af5c18400 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.115+0000 7f9455600700 10 client.25163 handle_command_reply: tid=0 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.115+0000 7f947cd60b80 1 client.25163 shutdown 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.115+0000 7f947cd60b80 1 -- 172.21.15.79:0/4057809307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b8c460 msgr2=0x560af5b91890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b8c460 0x560af5b91890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f946400b3f0 tx=0x7f9464098450 comp rx=0 tx=0).stop 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 1 -- 172.21.15.79:0/4057809307 shutdown_connections 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x560af5c18400 0x560af5c17110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560af5b8c460 0x560af5b91890 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560af5b861f0 0x560af5baa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 1 --2- 172.21.15.79:0/4057809307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560af5b8bb40 0x560af5b91350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f9455600700 0 client.25163 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 1 -- 172.21.15.79:0/4057809307 >> 172.21.15.79:0/4057809307 conn(0x560af5ab2120 msgr2=0x560af5b8c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 1 -- 172.21.15.79:0/4057809307 shutdown_connections 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 1 -- 172.21.15.79:0/4057809307 wait complete. 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.116+0000 7f947cd60b80 20 client.25163 trim_cache size 0 max 16384 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 2, 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24500, 2024-06-03T02:19:06.120 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:19:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:19:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:19:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 212.83680992800001, 2024-06-03T02:19:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:19:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:19:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:19:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 332.00114251600002 2024-06-03T02:19:06.121 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:19:06.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 -- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94500a2c20 msgr2=0x7f94500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:06.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94500a2c20 0x7f94500a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f945c011550 tx=0x7f945c009260 comp rx=0 tx=0).stop 2024-06-03T02:19:06.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 -- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f946c10a6b0 msgr2=0x7f946c19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:06.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f946c10a6b0 0x7f946c19f660 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f946400b0a0 tx=0x7f9464004d00 comp rx=0 tx=0).stop 2024-06-03T02:19:06.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 -- 172.21.15.79:0/2077703814 shutdown_connections 2024-06-03T02:19:06.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94500a2c20 0x7f94500a50e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f946c10a6b0 0x7f946c19f660 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f946c1046b0 0x7f946c19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 --2- 172.21.15.79:0/2077703814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f946c105070 0x7f946c19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:06.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 -- 172.21.15.79:0/2077703814 >> 172.21.15.79:0/2077703814 conn(0x7f946c1001a0 msgr2=0x7f946c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:06.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 -- 172.21.15.79:0/2077703814 shutdown_connections 2024-06-03T02:19:06.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:06.117+0000 7f9472400700 1 -- 172.21.15.79:0/2077703814 wait complete. 2024-06-03T02:19:07.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.g status 2024-06-03T02:19:07.548 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:07.697 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:07 smithi079 ceph-mon[31819]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:19:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:07 smithi123 ceph-mon[36640]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:19:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:07 smithi135 ceph-mon[36653]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:19:09.467 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:09 smithi079 ceph-mon[31819]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:19:09.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:09 smithi123 ceph-mon[36640]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:19:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.735+0000 7f8ff4e00700 1 -- 172.21.15.79:0/3512439118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 msgr2=0x7f8ff0111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.735+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/3512439118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 0x7f8ff0111440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800b0d0 tx=0x7f8fe800b3e0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.735+0000 7f8ff4e00700 1 -- 172.21.15.79:0/3512439118 shutdown_connections 2024-06-03T02:19:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.735+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/3512439118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 0x7f8ff0111440 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.735+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/3512439118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff010a670 0x7f8ff010aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.735+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/3512439118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff01045c0 0x7f8ff0104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.736+0000 7f8ff4e00700 1 -- 172.21.15.79:0/3512439118 >> 172.21.15.79:0/3512439118 conn(0x7f8ff01000d0 msgr2=0x7f8ff01024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.736+0000 7f8ff4e00700 1 -- 172.21.15.79:0/3512439118 shutdown_connections 2024-06-03T02:19:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.736+0000 7f8ff4e00700 1 -- 172.21.15.79:0/3512439118 wait complete. 2024-06-03T02:19:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.737+0000 7f8ff4e00700 1 Processor -- start 2024-06-03T02:19:09.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.737+0000 7f8ff4e00700 1 -- start start 2024-06-03T02:19:09.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.737+0000 7f8ff4e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff01045c0 0x7f8ff01a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.737+0000 7f8ff4e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 0x7f8ff01a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.737+0000 7f8ff4e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff010a670 0x7f8ff019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.737+0000 7f8ff4e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0114920 con 0x7f8ff0104f80 2024-06-03T02:19:09.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.737+0000 7f8ff4e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01147a0 con 0x7f8ff01045c0 2024-06-03T02:19:09.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.737+0000 7f8ff4e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0114aa0 con 0x7f8ff010a670 2024-06-03T02:19:09.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8feea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 0x7f8ff01a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8fef400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff01045c0 0x7f8ff01a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8feea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 0x7f8ff01a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44388/0 (socket says 172.21.15.79:44388) 2024-06-03T02:19:09.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8feea00700 1 -- 172.21.15.79:0/386573613 learned_addr learned my addr 172.21.15.79:0/386573613 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8fefe00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff010a670 0x7f8ff019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8fef400700 1 -- 172.21.15.79:0/386573613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff010a670 msgr2=0x7f8ff019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8fef400700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff010a670 0x7f8ff019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8fef400700 1 -- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 msgr2=0x7f8ff01a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8fef400700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 0x7f8ff01a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8fef400700 1 -- 172.21.15.79:0/386573613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff019b1c0 con 0x7f8ff01045c0 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.738+0000 7f8feea00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 0x7f8ff01a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.739+0000 7f8fef400700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff01045c0 0x7f8ff01a08a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e9f0 tx=0x7f8fe000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.739+0000 7f8fe7e00700 1 -- 172.21.15.79:0/386573613 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000c9c0 con 0x7f8ff01045c0 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.739+0000 7f8fe7e00700 1 -- 172.21.15.79:0/386573613 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe000cb60 con 0x7f8ff01045c0 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.739+0000 7f8ff4e00700 1 -- 172.21.15.79:0/386573613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff019b4b0 con 0x7f8ff01045c0 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.739+0000 7f8fe7e00700 1 -- 172.21.15.79:0/386573613 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0018680 con 0x7f8ff01045c0 2024-06-03T02:19:09.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.739+0000 7f8ff4e00700 1 -- 172.21.15.79:0/386573613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01a7770 con 0x7f8ff01045c0 2024-06-03T02:19:09.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.742+0000 7f8fe7e00700 1 -- 172.21.15.79:0/386573613 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8fe0010430 con 0x7f8ff01045c0 2024-06-03T02:19:09.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.744+0000 7f8fe7e00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd40a2ca0 0x7f8fd40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.744+0000 7f9000c84b80 1 Processor -- start 2024-06-03T02:19:09.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.744+0000 7f8fe7e00700 1 -- 172.21.15.79:0/386573613 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8fe00cf4d0 con 0x7f8ff01045c0 2024-06-03T02:19:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.744+0000 7f9000c84b80 1 -- start start 2024-06-03T02:19:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.744+0000 7f8feea00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd40a2ca0 0x7f8fd40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.744+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca826013d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.744+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 0x55ca82601d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.744+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca8260c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f9000c84b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ca8250e9c0 con 0x55ca8260a520 2024-06-03T02:19:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f9000c84b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca82551ab0 con 0x55ca82601910 2024-06-03T02:19:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f9000c84b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca8250c8a0 con 0x55ca8257bdf0 2024-06-03T02:19:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca8260c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca8260c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44392/0 (socket says 172.21.15.79:44392) 2024-06-03T02:19:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fef400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca826013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fefe00700 1 -- 172.21.15.79:0/3597986947 learned_addr learned my addr 172.21.15.79:0/3597986947 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8feea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 0x55ca82601d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8feea00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd40a2ca0 0x7f8fd40a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ff01036f0 tx=0x7f8fdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:09.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fef400700 1 -- 172.21.15.79:0/3597986947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 msgr2=0x55ca82601d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fef400700 1 --2- 172.21.15.79:0/3597986947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 0x55ca82601d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fef400700 1 -- 172.21.15.79:0/3597986947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 msgr2=0x55ca8260c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fef400700 1 --2- 172.21.15.79:0/3597986947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca8260c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fef400700 1 -- 172.21.15.79:0/3597986947 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca824f9f40 con 0x55ca8257bdf0 2024-06-03T02:19:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.745+0000 7f8fefe00700 1 --2- 172.21.15.79:0/3597986947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca8260c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.746+0000 7f8fef400700 1 --2- 172.21.15.79:0/3597986947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca826013d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8fe009b620 tx=0x7f8fe000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad63341fe25c304b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.746+0000 7f8fe4c00700 1 -- 172.21.15.79:0/3597986947 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe009b920 con 0x55ca8257bdf0 2024-06-03T02:19:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.746+0000 7f8fe4c00700 1 -- 172.21.15.79:0/3597986947 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe009fcd0 con 0x55ca8257bdf0 2024-06-03T02:19:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.746+0000 7f8fe4c00700 1 -- 172.21.15.79:0/3597986947 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0010720 con 0x55ca8257bdf0 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 -- 172.21.15.79:0/3597986947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 msgr2=0x55ca826013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 --2- 172.21.15.79:0/3597986947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca826013d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8fe009b620 tx=0x7f8fe000cd10 comp rx=0 tx=0).stop 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 -- 172.21.15.79:0/3597986947 shutdown_connections 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 --2- 172.21.15.79:0/3597986947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca8260c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 --2- 172.21.15.79:0/3597986947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca826013d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 --2- 172.21.15.79:0/3597986947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 0x55ca82601d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 -- 172.21.15.79:0/3597986947 >> 172.21.15.79:0/3597986947 conn(0x55ca8252c120 msgr2=0x55ca825fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 -- 172.21.15.79:0/3597986947 shutdown_connections 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 -- 172.21.15.79:0/3597986947 wait complete. 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.747+0000 7f9000c84b80 1 -- start start 2024-06-03T02:19:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.748+0000 7f9000c84b80 1 Processor -- start 2024-06-03T02:19:09.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.748+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82609d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.748+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82607020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.748+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 0x55ca82607560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.748+0000 7f9000c84b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ca824edf10 con 0x55ca8257bdf0 2024-06-03T02:19:09.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.748+0000 7f9000c84b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca8254da70 con 0x55ca8260a520 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.748+0000 7f9000c84b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca8260d070 con 0x55ca82601910 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.748+0000 7f9000c84b80 10 client.0 _ll_register_callbacks cb 0x55ca824eb6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8fef400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82609d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8feea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82607020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8feea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82607020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:51616/0 (socket says 172.21.15.79:51616) 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8feea00700 1 -- 172.21.15.79:0/588829445 learned_addr learned my addr 172.21.15.79:0/588829445 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8fef400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82609d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44406/0 (socket says 172.21.15.79:44406) 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8fefe00700 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 0x55ca82607560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8feea00700 1 -- 172.21.15.79:0/588829445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 msgr2=0x55ca82607560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8feea00700 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 0x55ca82607560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8feea00700 1 -- 172.21.15.79:0/588829445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 msgr2=0x55ca82609d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8feea00700 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82609d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8feea00700 1 -- 172.21.15.79:0/588829445 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca824a0880 con 0x55ca82601910 2024-06-03T02:19:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8fef400700 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82609d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.749+0000 7f8feea00700 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82607020 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc01cab0 tx=0x7f8fdc013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbacd89432cd6620 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.750+0000 7f8fd9600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.750+0000 7f8fd9600700 1 -- 172.21.15.79:0/588829445 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc02c060 con 0x55ca82601910 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.750+0000 7f8fd9600700 1 -- 172.21.15.79:0/588829445 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8fdc01a820 con 0x55ca82601910 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.750+0000 7f9000c84b80 1 -- 172.21.15.79:0/588829445 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe009fcd0 con 0x55ca82601910 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.750+0000 7f8fd9600700 1 -- 172.21.15.79:0/588829445 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8fdc033050 con 0x55ca82601910 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.750+0000 7f9000c84b80 10 client.25153 fetch_fsmap learned FSMap version 29 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.750+0000 7f9000c84b80 1 -- 172.21.15.79:0/588829445 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca82617880 con 0x55ca82601910 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.751+0000 7f8fd9600700 1 -- 172.21.15.79:0/588829445 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe009fcd0 con 0x55ca82601910 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.751+0000 7f8fd9600700 1 -- 172.21.15.79:0/588829445 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8fdc026070 con 0x55ca82601910 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.751+0000 7f9000c84b80 10 client.25153 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.751+0000 7f9000c84b80 10 client.25153 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.751+0000 7f9000c84b80 20 client.25153 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.751+0000 7f9000c84b80 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55ca82619eb0 0x55ca8261b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.751+0000 7f9000c84b80 4 client.25153 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:19:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.751+0000 7f9000c84b80 1 -- 172.21.15.79:0/588829445 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca8255efb0 con 0x55ca82619eb0 2024-06-03T02:19:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.751+0000 7f8fef400700 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55ca82619eb0 0x55ca8261b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.752+0000 7f8fef400700 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55ca82619eb0 0x55ca8261b320 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=beb66084f41ec642 server_cookie=95319acd82904c7b in_seq=0 out_seq=0 2024-06-03T02:19:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.752+0000 7f8fd9600700 10 client.25153 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f8fd9600700 1 -- 172.21.15.79:0/588829445 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ca8255efb0 con 0x55ca82619eb0 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f8fd9600700 10 client.25153 handle_command_reply: tid=0 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 client.25153 shutdown 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 -- 172.21.15.79:0/588829445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 msgr2=0x55ca82607020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82607020 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc01cab0 tx=0x7f8fdc013040 comp rx=0 tx=0).stop 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 -- 172.21.15.79:0/588829445 shutdown_connections 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 0x55ca82607560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82609d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55ca82619eb0 0x55ca8261b320 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 --2- 172.21.15.79:0/588829445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82607020 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f8fd9600700 0 client.25153 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:19:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 -- 172.21.15.79:0/588829445 >> 172.21.15.79:0/588829445 conn(0x55ca8252c120 msgr2=0x55ca825fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 -- 172.21.15.79:0/588829445 shutdown_connections 2024-06-03T02:19:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 1 -- 172.21.15.79:0/588829445 wait complete. 2024-06-03T02:19:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.753+0000 7f9000c84b80 20 client.25153 trim_cache size 0 max 16384 2024-06-03T02:19:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f9000c84b80 1 Processor -- start 2024-06-03T02:19:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f9000c84b80 1 -- start start 2024-06-03T02:19:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca82601280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 0x55ca82609860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca825fb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f9000c84b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ca82619660 con 0x55ca8260a520 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f9000c84b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca82619d30 con 0x55ca82601910 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f9000c84b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca8261a400 con 0x55ca8257bdf0 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca825fb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca825fb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44408/0 (socket says 172.21.15.79:44408) 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f8fefe00700 1 -- 172.21.15.79:0/1517626046 learned_addr learned my addr 172.21.15.79:0/1517626046 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f8feea00700 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 0x55ca82609860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.778+0000 7f8fef400700 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca82601280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.779+0000 7f8fefe00700 1 -- 172.21.15.79:0/1517626046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 msgr2=0x55ca82601280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.779+0000 7f8fefe00700 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca82601280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.779+0000 7f8fefe00700 1 -- 172.21.15.79:0/1517626046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 msgr2=0x55ca82609860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.779+0000 7f8fefe00700 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 0x55ca82609860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.779+0000 7f8fefe00700 1 -- 172.21.15.79:0/1517626046 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca826004c0 con 0x55ca8260a520 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.779+0000 7f8feea00700 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 0x55ca82609860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.779+0000 7f8fef400700 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca82601280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:09.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.779+0000 7f8fefe00700 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca825fb300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009570 tx=0x7f8fe800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa92e2ab989c8bf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:09.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f8fdbe00700 1 -- 172.21.15.79:0/1517626046 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8029070 con 0x55ca8260a520 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f8fdbe00700 1 -- 172.21.15.79:0/1517626046 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe800bda0 con 0x55ca8260a520 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f8fdbe00700 1 -- 172.21.15.79:0/1517626046 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80074e0 con 0x55ca8260a520 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f9000c84b80 1 -- 172.21.15.79:0/1517626046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 msgr2=0x55ca825fb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f9000c84b80 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca825fb300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009570 tx=0x7f8fe800b040 comp rx=0 tx=0).stop 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f9000c84b80 1 -- 172.21.15.79:0/1517626046 shutdown_connections 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f9000c84b80 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8260a520 0x55ca825fb300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f9000c84b80 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8257bdf0 0x55ca82601280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f9000c84b80 1 --2- 172.21.15.79:0/1517626046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca82601910 0x55ca82609860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.780+0000 7f9000c84b80 1 -- 172.21.15.79:0/1517626046 >> 172.21.15.79:0/1517626046 conn(0x55ca8252c120 msgr2=0x55ca82602380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.781+0000 7f9000c84b80 1 -- 172.21.15.79:0/1517626046 shutdown_connections 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.781+0000 7f9000c84b80 1 -- 172.21.15.79:0/1517626046 wait complete. 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.781+0000 7f9000c84b80 1 -- start start 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.781+0000 7f9000c84b80 1 Processor -- start 2024-06-03T02:19:09.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82603a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82603fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f9000c84b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 0x55ca82623790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f9000c84b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ca8261b1a0 con 0x55ca8257bdf0 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f9000c84b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca8261aad0 con 0x55ca8260a520 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f9000c84b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca8261b870 con 0x55ca82601910 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f9000c84b80 10 client.0 _ll_register_callbacks cb 0x55ca824eb6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f8fef400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82603a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f8fefe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 0x55ca82623790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f8fef400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82603a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44424/0 (socket says 172.21.15.79:44424) 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f8fef400700 1 -- 172.21.15.79:0/413580337 learned_addr learned my addr 172.21.15.79:0/413580337 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f8feea00700 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82603fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f8fefe00700 1 -- 172.21.15.79:0/413580337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 msgr2=0x55ca82603fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f8fefe00700 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82603fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f8fefe00700 1 -- 172.21.15.79:0/413580337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 msgr2=0x55ca82603a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:09.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.782+0000 7f8fefe00700 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82603a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f8fefe00700 1 -- 172.21.15.79:0/413580337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca82623cd0 con 0x55ca8260a520 2024-06-03T02:19:09.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f8fef400700 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82603a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f8fefe00700 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 0x55ca82623790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009540 tx=0x7f8fe8005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777750cb16d6fe77 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f8fd9600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f8fd9600700 1 -- 172.21.15.79:0/413580337 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8029040 con 0x55ca8260a520 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f8fd9600700 1 -- 172.21.15.79:0/413580337 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8fe8003a00 con 0x55ca8260a520 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f9000c84b80 1 -- 172.21.15.79:0/413580337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe800bda0 con 0x55ca8260a520 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f8fd9600700 1 -- 172.21.15.79:0/413580337 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8fe80092e0 con 0x55ca8260a520 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f9000c84b80 10 client.25181 fetch_fsmap learned FSMap version 29 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.783+0000 7f9000c84b80 1 -- 172.21.15.79:0/413580337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca82624fd0 con 0x55ca8260a520 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.784+0000 7f8fd9600700 1 -- 172.21.15.79:0/413580337 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe800bda0 con 0x55ca8260a520 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.784+0000 7f8fd9600700 1 -- 172.21.15.79:0/413580337 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8fe8011070 con 0x55ca8260a520 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.784+0000 7f9000c84b80 10 client.25181 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:19:09.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.784+0000 7f9000c84b80 10 client.25181 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.784+0000 7f9000c84b80 20 client.25181 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.784+0000 7f9000c84b80 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55ca82689720 0x55ca82688430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.784+0000 7f9000c84b80 4 client.25181 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.784+0000 7f9000c84b80 1 -- 172.21.15.79:0/413580337 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "status"}) -- 0x55ca82559530 con 0x55ca82689720 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.784+0000 7f8fef400700 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55ca82689720 0x55ca82688430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f8fef400700 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55ca82689720 0x55ca82688430 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34b8138eef388db6 server_cookie=ae59523e1aea3ff3 in_seq=0 out_seq=0 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f8fd9600700 10 client.25181 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f8fd9600700 1 -- 172.21.15.79:0/413580337 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55ca82559530 con 0x55ca82689720 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f8fd9600700 10 client.25181 handle_command_reply: tid=0 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f9000c84b80 1 client.25181 shutdown 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f9000c84b80 1 -- 172.21.15.79:0/413580337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 msgr2=0x55ca82623790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f9000c84b80 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 0x55ca82623790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009540 tx=0x7f8fe8005300 comp rx=0 tx=0).stop 2024-06-03T02:19:09.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f9000c84b80 1 -- 172.21.15.79:0/413580337 shutdown_connections 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f9000c84b80 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca8260a520 0x55ca82623790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f9000c84b80 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca8257bdf0 0x55ca82603a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f9000c84b80 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55ca82689720 0x55ca82688430 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f9000c84b80 1 --2- 172.21.15.79:0/413580337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca82601910 0x55ca82603fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f9000c84b80 1 -- 172.21.15.79:0/413580337 >> 172.21.15.79:0/413580337 conn(0x55ca8252c120 msgr2=0x55ca825fb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.785+0000 7f8fd9600700 0 client.25181 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.786+0000 7f9000c84b80 1 -- 172.21.15.79:0/413580337 shutdown_connections 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.786+0000 7f9000c84b80 1 -- 172.21.15.79:0/413580337 wait complete. 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.786+0000 7f9000c84b80 20 client.25181 trim_cache size 0 max 16384 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:19:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 1, 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24515, 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 217.519271256, 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 327.568287247 2024-06-03T02:19:09.790 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.786+0000 7f8ff4e00700 1 -- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd40a2ca0 msgr2=0x7f8fd40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.786+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd40a2ca0 0x7f8fd40a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ff01036f0 tx=0x7f8fdc009260 comp rx=0 tx=0).stop 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 -- 172.21.15.79:0/386573613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff01045c0 msgr2=0x7f8ff01a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff01045c0 0x7f8ff01a08a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e9f0 tx=0x7f8fe000ed00 comp rx=0 tx=0).stop 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 -- 172.21.15.79:0/386573613 shutdown_connections 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff010a670 0x7f8ff019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff01045c0 0x7f8ff01a08a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd40a2ca0 0x7f8fd40a5160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 --2- 172.21.15.79:0/386573613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ff0104f80 0x7f8ff01a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 -- 172.21.15.79:0/386573613 >> 172.21.15.79:0/386573613 conn(0x7f8ff01000d0 msgr2=0x7f8ff0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:09.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 -- 172.21.15.79:0/386573613 shutdown_connections 2024-06-03T02:19:09.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:09.787+0000 7f8ff4e00700 1 -- 172.21.15.79:0/386573613 wait complete. 2024-06-03T02:19:09.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:09 smithi135 ceph-mon[36653]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:19:10.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-03T02:19:11.266 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:11.652 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:11 smithi079 ceph-mon[31819]: pgmap v490: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:19:11.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:11 smithi123 ceph-mon[36640]: pgmap v490: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:19:11.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:11 smithi135 ceph-mon[36653]: pgmap v490: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:19:13.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.536+0000 7f1a30200700 1 -- 172.21.15.79:0/1015496847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 msgr2=0x7f1a28101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.536+0000 7f1a30200700 1 --2- 172.21.15.79:0/1015496847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 0x7f1a28101810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009990 tx=0x7f1a14009ca0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.536+0000 7f1a30200700 1 -- 172.21.15.79:0/1015496847 shutdown_connections 2024-06-03T02:19:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.536+0000 7f1a30200700 1 --2- 172.21.15.79:0/1015496847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a28101d50 0x7f1a2810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.536+0000 7f1a30200700 1 --2- 172.21.15.79:0/1015496847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a28107400 0x7f1a281077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.536+0000 7f1a30200700 1 --2- 172.21.15.79:0/1015496847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 0x7f1a28101810 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.536+0000 7f1a30200700 1 -- 172.21.15.79:0/1015496847 >> 172.21.15.79:0/1015496847 conn(0x7f1a28078340 msgr2=0x7f1a28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.537+0000 7f1a30200700 1 -- 172.21.15.79:0/1015496847 shutdown_connections 2024-06-03T02:19:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.537+0000 7f1a30200700 1 -- 172.21.15.79:0/1015496847 wait complete. 2024-06-03T02:19:13.538 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:13 smithi079 ceph-mon[31819]: pgmap v491: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:19:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.538+0000 7f1a30200700 1 Processor -- start 2024-06-03T02:19:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.538+0000 7f1a30200700 1 -- start start 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.538+0000 7f1a30200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 0x7f1a28103300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.538+0000 7f1a30200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a28101d50 0x7f1a28103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.538+0000 7f1a30200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a28107400 0x7f1a28103d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.538+0000 7f1a30200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281116c0 con 0x7f1a28107400 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.538+0000 7f1a30200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28111540 con 0x7f1a28101d50 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.538+0000 7f1a30200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28111840 con 0x7f1a28101390 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 0x7f1a28103300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2e200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a28107400 0x7f1a28103d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 0x7f1a28103300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:51644/0 (socket says 172.21.15.79:51644) 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2e200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a28107400 0x7f1a28103d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44452/0 (socket says 172.21.15.79:44452) 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2d800700 1 -- 172.21.15.79:0/1571621804 learned_addr learned my addr 172.21.15.79:0/1571621804 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2ce00700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a28101d50 0x7f1a28103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2d800700 1 -- 172.21.15.79:0/1571621804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a28101d50 msgr2=0x7f1a28103840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2d800700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a28101d50 0x7f1a28103840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2d800700 1 -- 172.21.15.79:0/1571621804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a28107400 msgr2=0x7f1a28103d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:13.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2d800700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a28107400 0x7f1a28103d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.539+0000 7f1a2d800700 1 -- 172.21.15.79:0/1571621804 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a281a71f0 con 0x7f1a28101390 2024-06-03T02:19:13.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.540+0000 7f1a2d800700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 0x7f1a28103300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00e9a0 tx=0x7f1a1c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:13.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.540+0000 7f1a26000700 1 -- 172.21.15.79:0/1571621804 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00cbd0 con 0x7f1a28101390 2024-06-03T02:19:13.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.540+0000 7f1a26000700 1 -- 172.21.15.79:0/1571621804 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c00cd70 con 0x7f1a28101390 2024-06-03T02:19:13.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.540+0000 7f1a30200700 1 -- 172.21.15.79:0/1571621804 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a281a74e0 con 0x7f1a28101390 2024-06-03T02:19:13.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.540+0000 7f1a26000700 1 -- 172.21.15.79:0/1571621804 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c018750 con 0x7f1a28101390 2024-06-03T02:19:13.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.540+0000 7f1a30200700 1 -- 172.21.15.79:0/1571621804 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a281a7a70 con 0x7f1a28101390 2024-06-03T02:19:13.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.543+0000 7f1a26000700 1 -- 172.21.15.79:0/1571621804 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a1c010430 con 0x7f1a28101390 2024-06-03T02:19:13.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.544+0000 7f1a3ac76b80 1 Processor -- start 2024-06-03T02:19:13.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a26000700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1a100a2ca0 0x7f1a100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a3ac76b80 1 -- start start 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a26000700 1 -- 172.21.15.79:0/1571621804 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1a1c0cf640 con 0x7f1a28101390 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498edfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498ee0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564498dee9c0 con 0x564498ee5b40 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564498e31ab0 con 0x564498ee6460 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564498dec8a0 con 0x564498ee01f0 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a2e200700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1a100a2ca0 0x7f1a100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.545+0000 7f1a2d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498edfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498edfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36818/0 (socket says 172.21.15.79:36818) 2024-06-03T02:19:13.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2ce00700 1 -- 172.21.15.79:0/494543530 learned_addr learned my addr 172.21.15.79:0/494543530 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:13.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44462/0 (socket says 172.21.15.79:44462) 2024-06-03T02:19:13.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2e200700 1 --2- 172.21.15.79:0/494543530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498ee0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2ce00700 1 -- 172.21.15.79:0/494543530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 msgr2=0x564498ee0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:13.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2ce00700 1 --2- 172.21.15.79:0/494543530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498ee0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2ce00700 1 -- 172.21.15.79:0/494543530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 msgr2=0x564498ee5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:13.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2ce00700 1 --2- 172.21.15.79:0/494543530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2ce00700 1 -- 172.21.15.79:0/494543530 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564498dd9f40 con 0x564498ee6460 2024-06-03T02:19:13.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2d800700 1 --2- 172.21.15.79:0/494543530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:13.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2e200700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1a100a2ca0 0x7f1a100a5160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009c90 tx=0x7f1a20004320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:13.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.546+0000 7f1a2ce00700 1 --2- 172.21.15.79:0/494543530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498edfcb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009990 tx=0x7f1a14005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5249e55f9a2d98 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.547+0000 7f1a1aa00700 1 -- 172.21.15.79:0/494543530 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14009510 con 0x564498ee6460 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.547+0000 7f1a1aa00700 1 -- 172.21.15.79:0/494543530 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1400bbe0 con 0x564498ee6460 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.547+0000 7f1a1aa00700 1 -- 172.21.15.79:0/494543530 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14003c50 con 0x564498ee6460 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.548+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/494543530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 msgr2=0x564498edfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.548+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/494543530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498edfcb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009990 tx=0x7f1a14005c00 comp rx=0 tx=0).stop 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.548+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/494543530 shutdown_connections 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.548+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/494543530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498ee0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.548+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/494543530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.548+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/494543530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498edfcb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.548+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/494543530 >> 172.21.15.79:0/494543530 conn(0x564498e0c120 msgr2=0x564498edafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.548+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/494543530 shutdown_connections 2024-06-03T02:19:13.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.548+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/494543530 wait complete. 2024-06-03T02:19:13.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.549+0000 7f1a3ac76b80 1 -- start start 2024-06-03T02:19:13.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.549+0000 7f1a3ac76b80 1 Processor -- start 2024-06-03T02:19:13.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.549+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.549+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.549+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498ee9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.549+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564498dcdf10 con 0x564498ee5b40 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.549+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564498e2da70 con 0x564498ee6460 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.549+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564498eecd80 con 0x564498ee01f0 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.549+0000 7f1a3ac76b80 10 client.0 _ll_register_callbacks cb 0x564498dc7330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2e200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498ee9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44472/0 (socket says 172.21.15.79:44472) 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2ce00700 1 -- 172.21.15.79:0/1699705709 learned_addr learned my addr 172.21.15.79:0/1699705709 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2e200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498ee9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36824/0 (socket says 172.21.15.79:36824) 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2e200700 1 -- 172.21.15.79:0/1699705709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 msgr2=0x564498eec1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:13.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2e200700 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eec1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2e200700 1 -- 172.21.15.79:0/1699705709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 msgr2=0x564498ee94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2e200700 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2e200700 1 -- 172.21.15.79:0/1699705709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564498d80880 con 0x564498ee6460 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a2e200700 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498ee9a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009550 tx=0x7f1a20004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7c5e3a4e2b50cfc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.550+0000 7f1a0f400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.551+0000 7f1a0f400700 1 -- 172.21.15.79:0/1699705709 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20007830 con 0x564498ee6460 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.551+0000 7f1a0f400700 1 -- 172.21.15.79:0/1699705709 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1a200079d0 con 0x564498ee6460 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.551+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1699705709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a1400bbe0 con 0x564498ee6460 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.551+0000 7f1a0f400700 1 -- 172.21.15.79:0/1699705709 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a2001be40 con 0x564498ee6460 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.551+0000 7f1a3ac76b80 10 client.25193 fetch_fsmap learned FSMap version 29 2024-06-03T02:19:13.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.551+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1699705709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564498ef7550 con 0x564498ee6460 2024-06-03T02:19:13.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.552+0000 7f1a0f400700 1 -- 172.21.15.79:0/1699705709 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1400bbe0 con 0x564498ee6460 2024-06-03T02:19:13.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.552+0000 7f1a0f400700 1 -- 172.21.15.79:0/1699705709 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1a20018040 con 0x564498ee6460 2024-06-03T02:19:13.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.552+0000 7f1a3ac76b80 10 client.25193 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:19:13.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.552+0000 7f1a3ac76b80 10 client.25193 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T02:19:13.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.552+0000 7f1a3ac76b80 20 client.25193 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:13.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.552+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564498ef9bd0 0x564498efb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.552+0000 7f1a3ac76b80 4 client.25193 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:19:13.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.552+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1699705709 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564498e3efb0 con 0x564498ef9bd0 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.552+0000 7f1a2d800700 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564498ef9bd0 0x564498efb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.553+0000 7f1a2d800700 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564498ef9bd0 0x564498efb040 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe55eb8efc641b5b server_cookie=418df02414eb203f in_seq=0 out_seq=0 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.553+0000 7f1a0f400700 10 client.25193 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a0f400700 1 -- 172.21.15.79:0/1699705709 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564498e3efb0 con 0x564498ef9bd0 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a0f400700 10 client.25193 handle_command_reply: tid=0 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 client.25193 shutdown 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1699705709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 msgr2=0x564498ee9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498ee9a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009550 tx=0x7f1a20004b40 comp rx=0 tx=0).stop 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1699705709 shutdown_connections 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498ee9a00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eec1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564498ef9bd0 0x564498efb040 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1699705709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498ee94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a0f400700 0 client.25193 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:13.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1699705709 >> 172.21.15.79:0/1699705709 conn(0x564498e0c120 msgr2=0x564498ed4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:13.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1699705709 shutdown_connections 2024-06-03T02:19:13.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.554+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1699705709 wait complete. 2024-06-03T02:19:13.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.555+0000 7f1a3ac76b80 20 client.25193 trim_cache size 0 max 16384 2024-06-03T02:19:13.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.577+0000 7f1a3ac76b80 1 Processor -- start 2024-06-03T02:19:13.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a3ac76b80 1 -- start start 2024-06-03T02:19:13.821 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:13 smithi123 ceph-mon[36640]: pgmap v491: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee01f0 0x564498edfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee5b40 0x564498ee9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee6460 0x564498edb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564498ef9380 con 0x564498ee01f0 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564498ef9a50 con 0x564498ee5b40 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564498efa120 con 0x564498ee6460 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a2d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee01f0 0x564498edfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a2d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee01f0 0x564498edfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44474/0 (socket says 172.21.15.79:44474) 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a2d800700 1 -- 172.21.15.79:0/1329005533 learned_addr learned my addr 172.21.15.79:0/1329005533 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a2ce00700 1 --2- 172.21.15.79:0/1329005533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee5b40 0x564498ee9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.578+0000 7f1a2e200700 1 --2- 172.21.15.79:0/1329005533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee6460 0x564498edb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.579+0000 7f1a2d800700 1 -- 172.21.15.79:0/1329005533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee6460 msgr2=0x564498edb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.579+0000 7f1a2d800700 1 --2- 172.21.15.79:0/1329005533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee6460 0x564498edb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.579+0000 7f1a2d800700 1 -- 172.21.15.79:0/1329005533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee5b40 msgr2=0x564498ee9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.579+0000 7f1a2d800700 1 --2- 172.21.15.79:0/1329005533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee5b40 0x564498ee9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.579+0000 7f1a2d800700 1 -- 172.21.15.79:0/1329005533 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564498eecd80 con 0x564498ee01f0 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.579+0000 7f1a2d800700 1 --2- 172.21.15.79:0/1329005533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee01f0 0x564498edfcb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c097ef0 tx=0x7f1a1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0749ace0d18e9b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.579+0000 7f1a1a000700 1 -- 172.21.15.79:0/1329005533 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c09fe40 con 0x564498ee01f0 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.579+0000 7f1a1a000700 1 -- 172.21.15.79:0/1329005533 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c084070 con 0x564498ee01f0 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.579+0000 7f1a1a000700 1 -- 172.21.15.79:0/1329005533 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c09bcf0 con 0x564498ee01f0 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.580+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1329005533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee01f0 msgr2=0x564498edfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.580+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1329005533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee01f0 0x564498edfcb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c097ef0 tx=0x7f1a1c009840 comp rx=0 tx=0).stop 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.580+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1329005533 shutdown_connections 2024-06-03T02:19:13.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.580+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1329005533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee6460 0x564498edb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.580+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1329005533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee01f0 0x564498edfcb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.580+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/1329005533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee5b40 0x564498ee9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.580+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1329005533 >> 172.21.15.79:0/1329005533 conn(0x564498e0c120 msgr2=0x564498edaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.580+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1329005533 shutdown_connections 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.580+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/1329005533 wait complete. 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.581+0000 7f1a3ac76b80 1 -- start start 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.581+0000 7f1a3ac76b80 1 Processor -- start 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.581+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eeb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.581+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498eebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.581+0000 7f1a3ac76b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498eec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.581+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564498efaec0 con 0x564498ee5b40 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.581+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564498efa7f0 con 0x564498ee6460 2024-06-03T02:19:13.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.581+0000 7f1a3ac76b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564498efb590 con 0x564498ee01f0 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.581+0000 7f1a3ac76b80 10 client.0 _ll_register_callbacks cb 0x564498dc7330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eeb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eeb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:51688/0 (socket says 172.21.15.79:51688) 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2d800700 1 -- 172.21.15.79:0/2969910516 learned_addr learned my addr 172.21.15.79:0/2969910516 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2e200700 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498eec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2ce00700 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498eebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2e200700 1 -- 172.21.15.79:0/2969910516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 msgr2=0x564498eeb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2e200700 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eeb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2e200700 1 -- 172.21.15.79:0/2969910516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 msgr2=0x564498eebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2e200700 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498eebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2e200700 1 -- 172.21.15.79:0/2969910516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564498f02930 con 0x564498ee6460 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2d800700 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eeb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a2e200700 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498eec030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a200049b0 tx=0x7f1a20004850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7be153c61d2e2a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.582+0000 7f1a0f400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.583+0000 7f1a0f400700 1 -- 172.21.15.79:0/2969910516 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20019b90 con 0x564498ee6460 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.583+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/2969910516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a1c084070 con 0x564498ee6460 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.583+0000 7f1a0f400700 1 -- 172.21.15.79:0/2969910516 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1a20019d30 con 0x564498ee6460 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.583+0000 7f1a0f400700 1 -- 172.21.15.79:0/2969910516 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a20044050 con 0x564498ee6460 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.583+0000 7f1a3ac76b80 10 client.25199 fetch_fsmap learned FSMap version 29 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.583+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/2969910516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564498f6ded0 con 0x564498ee6460 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.583+0000 7f1a0f400700 1 -- 172.21.15.79:0/2969910516 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c084070 con 0x564498ee6460 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.584+0000 7f1a0f400700 1 -- 172.21.15.79:0/2969910516 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1a2003f030 con 0x564498ee6460 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.584+0000 7f1a3ac76b80 10 client.25199 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.584+0000 7f1a3ac76b80 10 client.25199 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.584+0000 7f1a3ac76b80 20 client.25199 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.584+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564498f72a90 0x564498f717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.584+0000 7f1a3ac76b80 4 client.25199 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T02:19:13.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.584+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/2969910516 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "status"}) -- 0x564498e39530 con 0x564498f72a90 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.584+0000 7f1a2d800700 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564498f72a90 0x564498f717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a2d800700 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564498f72a90 0x564498f717a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca893c70c5cff5a server_cookie=b5c9dc5ad81f8e5 in_seq=0 out_seq=0 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a0f400700 10 client.25199 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a0f400700 1 -- 172.21.15.79:0/2969910516 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x564498e39530 con 0x564498f72a90 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a0f400700 10 client.25199 handle_command_reply: tid=0 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 client.25199 shutdown 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/2969910516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 msgr2=0x564498eec030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498eec030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a200049b0 tx=0x7f1a20004850 comp rx=0 tx=0).stop 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/2969910516 shutdown_connections 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564498ee6460 0x564498eec030 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564498ee01f0 0x564498eeb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564498f72a90 0x564498f717a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 --2- 172.21.15.79:0/2969910516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564498ee5b40 0x564498eebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a0f400700 0 client.25199 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:13.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/2969910516 >> 172.21.15.79:0/2969910516 conn(0x564498e0c120 msgr2=0x564498ee6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:13.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.585+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/2969910516 shutdown_connections 2024-06-03T02:19:13.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a3ac76b80 1 -- 172.21.15.79:0/2969910516 wait complete. 2024-06-03T02:19:13.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a3ac76b80 20 client.25199 trim_cache size 0 max 16384 2024-06-03T02:19:13.831 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:19:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "7a1e9ad6-214d-11ef-bc9b-c7b262605968", 2024-06-03T02:19:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2024-06-03T02:19:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 24539, 2024-06-03T02:19:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-06-03T02:19:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-06-03T02:19:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-06-03T02:19:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 300.821617667, 2024-06-03T02:19:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 29, 2024-06-03T02:19:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 101, 2024-06-03T02:19:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 99, 2024-06-03T02:19:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 324.53316286099999 2024-06-03T02:19:13.833 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:19:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 -- 172.21.15.79:0/1571621804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1a100a2ca0 msgr2=0x7f1a100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1a100a2ca0 0x7f1a100a5160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009c90 tx=0x7f1a20004320 comp rx=0 tx=0).stop 2024-06-03T02:19:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 -- 172.21.15.79:0/1571621804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 msgr2=0x7f1a28103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 0x7f1a28103300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00e9a0 tx=0x7f1a1c00ecb0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 -- 172.21.15.79:0/1571621804 shutdown_connections 2024-06-03T02:19:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1a100a2ca0 0x7f1a100a5160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a28107400 0x7f1a28103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a28101390 0x7f1a28103300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 --2- 172.21.15.79:0/1571621804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a28101d50 0x7f1a28103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:13.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.586+0000 7f1a30200700 1 -- 172.21.15.79:0/1571621804 >> 172.21.15.79:0/1571621804 conn(0x7f1a28078340 msgr2=0x7f1a281055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:13.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.587+0000 7f1a30200700 1 -- 172.21.15.79:0/1571621804 shutdown_connections 2024-06-03T02:19:13.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:13.587+0000 7f1a30200700 1 -- 172.21.15.79:0/1571621804 wait complete. 2024-06-03T02:19:13.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:13 smithi135 ceph-mon[36653]: pgmap v491: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:19:14.512 INFO:teuthology.run_tasks:Running task sequential... 2024-06-03T02:19:14.523 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-03T02:19:14.523 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-03T02:19:14.524 INFO:teuthology.task.print:Enabling mgr modules 2024-06-03T02:19:14.524 INFO:teuthology.run_tasks:Running task kclient... 2024-06-03T02:19:14.544 INFO:tasks.kclient:Mounting kernel clients... 2024-06-03T02:19:14.544 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-03T02:19:14.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:14.545 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-06-03T02:19:14.604 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-03T02:19:14.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:14.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T02:19:14.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:14.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2024-06-03T02:19:14.807 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-06-03T02:19:14.807 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-03T02:19:14.808 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-06-03T02:19:14.808 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-06-03T02:19:14.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-03T02:19:15.226 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:15.686 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:15 smithi079 ceph-mon[31819]: pgmap v492: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:19:15.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:15 smithi123 ceph-mon[36640]: pgmap v492: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:19:15.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:15 smithi135 ceph-mon[36653]: pgmap v492: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T02:19:16.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:16 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:19:16.830 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:16 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:19:16.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:16 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:19:17.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.369+0000 7f52bf200700 1 -- 172.21.15.79:0/3933249578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b8102740 msgr2=0x7f52b8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:17.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.369+0000 7f52bf200700 1 --2- 172.21.15.79:0/3933249578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b8102740 0x7f52b8102b20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f52a80099c0 tx=0x7f52a8009cd0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.369+0000 7f52bf200700 1 -- 172.21.15.79:0/3933249578 shutdown_connections 2024-06-03T02:19:17.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.369+0000 7f52bf200700 1 --2- 172.21.15.79:0/3933249578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b8107840 0x7f52b810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.369+0000 7f52bf200700 1 --2- 172.21.15.79:0/3933249578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b8102740 0x7f52b8102b20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.369+0000 7f52bf200700 1 --2- 172.21.15.79:0/3933249578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52b81030f0 0x7f52b8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.370+0000 7f52bf200700 1 -- 172.21.15.79:0/3933249578 >> 172.21.15.79:0/3933249578 conn(0x7f52b80795f0 msgr2=0x7f52b8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:17.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.370+0000 7f52bf200700 1 -- 172.21.15.79:0/3933249578 shutdown_connections 2024-06-03T02:19:17.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.370+0000 7f52bf200700 1 -- 172.21.15.79:0/3933249578 wait complete. 2024-06-03T02:19:17.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.371+0000 7f52bf200700 1 Processor -- start 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.371+0000 7f52bf200700 1 -- start start 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.371+0000 7f52bf200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52b8102740 0x7f52b8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.371+0000 7f52bf200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b81030f0 0x7f52b8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.371+0000 7f52bf200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b8107840 0x7f52b8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.371+0000 7f52bf200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81127b0 con 0x7f52b8107840 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.371+0000 7f52bf200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8112630 con 0x7f52b81030f0 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.371+0000 7f52bf200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8112930 con 0x7f52b8102740 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52b7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52b8102740 0x7f52b8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52b7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52b8102740 0x7f52b8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41234/0 (socket says 172.21.15.79:41234) 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52b7e00700 1 -- 172.21.15.79:0/1268957985 learned_addr learned my addr 172.21.15.79:0/1268957985 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52af400700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b81030f0 0x7f52b8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52bd200700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b8107840 0x7f52b8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52b7e00700 1 -- 172.21.15.79:0/1268957985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b81030f0 msgr2=0x7f52b8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52b7e00700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b81030f0 0x7f52b8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52b7e00700 1 -- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b8107840 msgr2=0x7f52b8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:17.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52b7e00700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b8107840 0x7f52b8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.372+0000 7f52b7e00700 1 -- 172.21.15.79:0/1268957985 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b81a7120 con 0x7f52b8102740 2024-06-03T02:19:17.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.373+0000 7f52b7e00700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52b8102740 0x7f52b8074790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52a8006010 tx=0x7f52a8003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:17.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.373+0000 7f52b5600700 1 -- 172.21.15.79:0/1268957985 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8003e50 con 0x7f52b8102740 2024-06-03T02:19:17.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.373+0000 7f52b5600700 1 -- 172.21.15.79:0/1268957985 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a80055a0 con 0x7f52b8102740 2024-06-03T02:19:17.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.373+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b81a73b0 con 0x7f52b8102740 2024-06-03T02:19:17.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.373+0000 7f52b5600700 1 -- 172.21.15.79:0/1268957985 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a800fe30 con 0x7f52b8102740 2024-06-03T02:19:17.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.373+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81a7800 con 0x7f52b8102740 2024-06-03T02:19:17.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.374+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b810aa80 con 0x7f52b8102740 2024-06-03T02:19:17.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.376+0000 7f52b5600700 1 -- 172.21.15.79:0/1268957985 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f52a800f4f0 con 0x7f52b8102740 2024-06-03T02:19:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.377+0000 7f52b5600700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f529c0a2dc0 0x7f529c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.378+0000 7f52b5600700 1 -- 172.21.15.79:0/1268957985 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f52a80d0280 con 0x7f52b8102740 2024-06-03T02:19:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.378+0000 7f52af400700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f529c0a2dc0 0x7f529c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.379+0000 7f52af400700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f529c0a2dc0 0x7f529c0a5280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f52b8100f40 tx=0x7f52a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:17.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.380+0000 7f52b5600700 1 -- 172.21.15.79:0/1268957985 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f52a8013070 con 0x7f52b8102740 2024-06-03T02:19:17.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.652+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f52b802dca0 con 0x7f52b8102740 2024-06-03T02:19:17.653 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:17 smithi079 ceph-mon[31819]: pgmap v493: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:19:17.653 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:19:17.653 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:19:17.653 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:19:17.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.653+0000 7f52b5600700 1 -- 172.21.15.79:0/1268957985 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f52a809ce60 con 0x7f52b8102740 2024-06-03T02:19:17.654 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-03T02:19:17.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f529c0a2dc0 msgr2=0x7f529c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:17.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f529c0a2dc0 0x7f529c0a5280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f52b8100f40 tx=0x7f52a0009260 comp rx=0 tx=0).stop 2024-06-03T02:19:17.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52b8102740 msgr2=0x7f52b8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:17.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52b8102740 0x7f52b8074790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52a8006010 tx=0x7f52a8003a90 comp rx=0 tx=0).stop 2024-06-03T02:19:17.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 shutdown_connections 2024-06-03T02:19:17.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f529c0a2dc0 0x7f529c0a5280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b8107840 0x7f52b8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52b8102740 0x7f52b8074790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 --2- 172.21.15.79:0/1268957985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b81030f0 0x7f52b8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:17.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 >> 172.21.15.79:0/1268957985 conn(0x7f52b80795f0 msgr2=0x7f52b8101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:17.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.656+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 shutdown_connections 2024-06-03T02:19:17.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:17.657+0000 7f52bf200700 1 -- 172.21.15.79:0/1268957985 wait complete. 2024-06-03T02:19:17.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:17 smithi123 ceph-mon[36640]: pgmap v493: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:19:17.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:19:17.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:19:17.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:19:17.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:17 smithi135 ceph-mon[36653]: pgmap v493: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-03T02:19:17.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:19:17.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:19:17.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:19:18.636 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-03T02:19:18.636 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-03T02:19:18.636 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2024-06-03T02:19:18.636 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-03T02:19:18.636 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-06-03T02:19:18.636 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-03T02:19:18.636 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-03T02:19:18.636 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-03T02:19:18.636 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-06-03T02:19:18.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:18.637 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2024-06-03T02:19:18.689 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f0 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2024-06-03T02:19:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f1 2024-06-03T02:19:18.691 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T02:19:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2024-06-03T02:19:18.691 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-03T02:19:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2024-06-03T02:19:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-06-03T02:19:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 41707sec preferred_lft 41707sec 2024-06-03T02:19:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::ec4:7aff:fe8f:ce87/64 scope link 2024-06-03T02:19:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-06-03T02:19:18.692 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-03T02:19:18.692 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:19:18.692 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:19:18.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add name ceph-brx type bridge 2024-06-03T02:19:18.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr flush dev ceph-brx 2024-06-03T02:19:18.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set ceph-brx up 2024-06-03T02:19:18.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-03T02:19:18.693 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:19:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:18 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1268957985' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T02:19:18.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:19:18.814 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:18 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1268957985' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T02:19:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:18 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1268957985' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T02:19:18.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:19:18.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:18.928 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-03T02:19:18.992 INFO:teuthology.orchestra.run.smithi079.stdout:1 2024-06-03T02:19:18.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:18.996 DEBUG:teuthology.orchestra.run.smithi079:> route 2024-06-03T02:19:19.045 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2024-06-03T02:19:19.045 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-03T02:19:19.045 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-03T02:19:19.045 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-03T02:19:19.046 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-03T02:19:19.046 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:19:19.046 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:19:19.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-03T02:19:19.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-03T02:19:19.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-03T02:19:19.046 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:19:19.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:19:19.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:19:19.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:19.258 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-06-03T02:19:19.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:19.308 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2024-06-03T02:19:19.353 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:19:19.353 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:19:19.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--tmp-cephfs 2024-06-03T02:19:19.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-06-03T02:19:19.353 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:19:19.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:19:19.526 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:19 smithi079 ceph-mon[31819]: pgmap v494: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:19:19.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:19:19.583 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-06-03T02:19:19.583 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:19:19.583 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:19:19.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-06-03T02:19:19.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-03T02:19:19.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-06-03T02:19:19.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-06-03T02:19:19.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-06-03T02:19:19.583 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:19:19.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:19:19.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:19 smithi123 ceph-mon[36640]: pgmap v494: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:19:19.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:19 smithi135 ceph-mon[36653]: pgmap v494: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T02:19:19.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:19:19.855 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T02:19:19.855 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-06-03T02:19:19.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2024-06-03T02:19:19.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.0 master ceph-brx 2024-06-03T02:19:19.855 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-06-03T02:19:19.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:19:20.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:19:20.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T02:19:20.054 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /tmp/cephfs 2024-06-03T02:19:20.098 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/tmp/cephfs' 2024-06-03T02:19:20.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T02:19:20.098 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /tmp/cephfs 2024-06-03T02:19:20.190 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-03T02:19:20.191 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-03T02:19:20.191 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/384b230e-7fb1-414a-9276-a11ddd986ce8 2024-06-03T02:19:20.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:20.191 DEBUG:teuthology.orchestra.run.smithi079:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/384b230e-7fb1-414a-9276-a11ddd986ce8 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-06-03T02:19:20.316 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-06-03T02:19:20.316 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-06-03T02:19:21.052 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-06-03T02:19:21.052 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-06-03T02:19:21.052 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.123:3300,172.21.15.135:3300" 2024-06-03T02:19:21.052 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: trying mount with new device syntax: 0@7a1e9ad6-214d-11ef-bc9b-c7b262605968.cephfs=/volumes/qa/sv_1/384b230e-7fb1-414a-9276-a11ddd986ce8 2024-06-03T02:19:21.052 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.79:3300/172.21.15.123:3300/172.21.15.135:3300" will pass to kernel 2024-06-03T02:19:21.064 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-03T02:19:21.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:19:21.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /tmp/cephfs 2024-06-03T02:19:21.147 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T02:19:21.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T02:19:21.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:19:21.274 INFO:teuthology.orchestra.run.smithi079.stdout:client25186 2024-06-03T02:19:21.274 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:19:21.274 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T02:19:21.327 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T02:19:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T02:19:21.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:19:21.451 INFO:teuthology.orchestra.run.smithi079.stdout:client25186 2024-06-03T02:19:21.451 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T02:19:21.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T02:19:21.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:19:21.569 INFO:teuthology.orchestra.run.smithi079.stdout:client25186 2024-06-03T02:19:21.569 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968.client25186/status') 2024-06-03T02:19:21.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T02:19:21.693 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:21 smithi079 ceph-mon[31819]: pgmap v495: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:19:21.701 INFO:teuthology.orchestra.run.smithi079.stdout:instance: client.25186 (3)172.21.15.79:0/957687897 2024-06-03T02:19:21.701 INFO:teuthology.orchestra.run.smithi079.stdout:blocklisted: false 2024-06-03T02:19:21.702 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2024-06-03T02:19:21.702 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2024-06-03T02:19:21.702 INFO:teuthology.orchestra.run.smithi079.stderr:70 bytes copied, 8.0777e-05 s, 867 kB/s 2024-06-03T02:19:21.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T02:19:21.715 INFO:teuthology.run_tasks:Running task exec... 2024-06-03T02:19:21.724 INFO:teuthology.task.exec:Executing custom commands... 2024-06-03T02:19:21.725 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi079.front.sepia.ceph.com 2024-06-03T02:19:21.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-06-03T02:19:21.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:21 smithi123 ceph-mon[36640]: pgmap v495: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:19:21.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:21 smithi135 ceph-mon[36653]: pgmap v495: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T02:19:21.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.927+0000 7f06a4c00640 1 -- 172.21.15.79:0/3893948370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a0068370 msgr2=0x7f06a0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:21.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.927+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3893948370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a0068370 0x7f06a0100c60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f069000a9a0 tx=0x7f069002b6b0 comp rx=0 tx=0).stop 2024-06-03T02:19:21.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.927+0000 7f06a4c00640 1 -- 172.21.15.79:0/3893948370 shutdown_connections 2024-06-03T02:19:21.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.927+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3893948370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06a0105ea0 0x7f06a0108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:21.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.927+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3893948370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06a01011a0 0x7f06a0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:21.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.927+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3893948370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a0068370 0x7f06a0100c60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:21.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.927+0000 7f06a4c00640 1 -- 172.21.15.79:0/3893948370 >> 172.21.15.79:0/3893948370 conn(0x7f06a00fb000 msgr2=0x7f06a00fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:21.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.927+0000 7f06a4c00640 1 -- 172.21.15.79:0/3893948370 shutdown_connections 2024-06-03T02:19:21.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.927+0000 7f06a4c00640 1 -- 172.21.15.79:0/3893948370 wait complete. 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.928+0000 7f06a4c00640 1 Processor -- start 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.928+0000 7f06a4c00640 1 -- start start 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.928+0000 7f06a4c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06a0068370 0x7f06a0197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.928+0000 7f06a4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a01011a0 0x7f06a0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f06a4c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06a0105ea0 0x7f06a019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f06a4c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f06a010b300 con 0x7f06a0068370 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f06a4c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06a010b180 con 0x7f06a01011a0 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f06a4c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f06a010b480 con 0x7f06a0105ea0 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f069f400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06a0068370 0x7f06a0197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:21.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f069f400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06a0068370 0x7f06a0197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47232/0 (socket says 172.21.15.79:47232) 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f069f400640 1 -- 172.21.15.79:0/3130727279 learned_addr learned my addr 172.21.15.79:0/3130727279 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f0697e00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a01011a0 0x7f06a0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f069fe00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06a0105ea0 0x7f06a019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f069f400640 1 -- 172.21.15.79:0/3130727279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06a0105ea0 msgr2=0x7f06a019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f069f400640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06a0105ea0 0x7f06a019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f069f400640 1 -- 172.21.15.79:0/3130727279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a01011a0 msgr2=0x7f06a0198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f069f400640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a01011a0 0x7f06a0198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.929+0000 7f069f400640 1 -- 172.21.15.79:0/3130727279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a019f810 con 0x7f06a0068370 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.930+0000 7f069f400640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06a0068370 0x7f06a0197bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f069002bca0 tx=0x7f06900052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.930+0000 7f069cc00640 1 -- 172.21.15.79:0/3130727279 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690004250 con 0x7f06a0068370 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.930+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a019faa0 con 0x7f06a0068370 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.930+0000 7f069cc00640 1 -- 172.21.15.79:0/3130727279 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06900043f0 con 0x7f06a0068370 2024-06-03T02:19:21.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.930+0000 7f069cc00640 1 -- 172.21.15.79:0/3130727279 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690004c40 con 0x7f06a0068370 2024-06-03T02:19:21.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.930+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a019ff90 con 0x7f06a0068370 2024-06-03T02:19:21.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.931+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a01a0160 con 0x7f06a0068370 2024-06-03T02:19:21.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.933+0000 7f069cc00640 1 -- 172.21.15.79:0/3130727279 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0690030510 con 0x7f06a0068370 2024-06-03T02:19:21.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.934+0000 7f069cc00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06840a2b80 0x7f06840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:21.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.935+0000 7f069cc00640 1 -- 172.21.15.79:0/3130727279 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f06900f0750 con 0x7f06a0068370 2024-06-03T02:19:21.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.935+0000 7f0697e00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06840a2b80 0x7f06840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:21.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.935+0000 7f0697e00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06840a2b80 0x7f06840a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06a0199100 tx=0x7f0688009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:21.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:21.939+0000 7f069cc00640 1 -- 172.21.15.79:0/3130727279 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0690040030 con 0x7f06a0068370 2024-06-03T02:19:22.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.185+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f06a01a0550 con 0x7f06a0068370 2024-06-03T02:19:22.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.441+0000 7f069cc00640 1 -- 172.21.15.79:0/3130727279 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7f06900bd2a0 con 0x7f06a0068370 2024-06-03T02:19:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.445+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06840a2b80 msgr2=0x7f06840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.445+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06840a2b80 0x7f06840a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06a0199100 tx=0x7f0688009260 comp rx=0 tx=0).stop 2024-06-03T02:19:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.445+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06a0068370 msgr2=0x7f06a0197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.445+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06a0068370 0x7f06a0197bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f069002bca0 tx=0x7f06900052f0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.446+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 shutdown_connections 2024-06-03T02:19:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.446+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06840a2b80 0x7f06840a5040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.446+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06a0105ea0 0x7f06a019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.446+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a01011a0 0x7f06a0198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.446+0000 7f06a4c00640 1 --2- 172.21.15.79:0/3130727279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06a0068370 0x7f06a0197bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.446+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 >> 172.21.15.79:0/3130727279 conn(0x7f06a00fb000 msgr2=0x7f06a0107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:22.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.446+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 shutdown_connections 2024-06-03T02:19:22.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.446+0000 7f06a4c00640 1 -- 172.21.15.79:0/3130727279 wait complete. 2024-06-03T02:19:22.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-06-03T02:19:22.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.665+0000 7f1edec00640 1 -- 172.21.15.79:0/480062290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed8103c20 msgr2=0x7f1ed8104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:22.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.665+0000 7f1edec00640 1 --2- 172.21.15.79:0/480062290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed8103c20 0x7f1ed8104040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009900 tx=0x7f1ec802a3f0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.665+0000 7f1edec00640 1 -- 172.21.15.79:0/480062290 shutdown_connections 2024-06-03T02:19:22.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.665+0000 7f1edec00640 1 --2- 172.21.15.79:0/480062290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ed81057d0 0x7f1ed810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.665+0000 7f1edec00640 1 --2- 172.21.15.79:0/480062290 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ed8104e10 0x7f1ed8105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.665+0000 7f1edec00640 1 --2- 172.21.15.79:0/480062290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed8103c20 0x7f1ed8104040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.665+0000 7f1edec00640 1 -- 172.21.15.79:0/480062290 >> 172.21.15.79:0/480062290 conn(0x7f1ed80ff190 msgr2=0x7f1ed81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:22.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.665+0000 7f1edec00640 1 -- 172.21.15.79:0/480062290 shutdown_connections 2024-06-03T02:19:22.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.665+0000 7f1edec00640 1 -- 172.21.15.79:0/480062290 wait complete. 2024-06-03T02:19:22.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.666+0000 7f1edec00640 1 Processor -- start 2024-06-03T02:19:22.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.666+0000 7f1edec00640 1 -- start start 2024-06-03T02:19:22.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.666+0000 7f1edec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed8103c20 0x7f1ed8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:22.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.666+0000 7f1edec00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ed8104e10 0x7f1ed819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:22.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.666+0000 7f1edec00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ed81057d0 0x7f1ed81a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:22.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.666+0000 7f1edec00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed810f8a0 con 0x7f1ed81057d0 2024-06-03T02:19:22.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.666+0000 7f1edec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed810f720 con 0x7f1ed8103c20 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.666+0000 7f1edec00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed810fa20 con 0x7f1ed8104e10 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1ed7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ed8104e10 0x7f1ed819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1edcc00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ed81057d0 0x7f1ed81a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1edcc00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ed81057d0 0x7f1ed81a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47248/0 (socket says 172.21.15.79:47248) 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1ed7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ed8104e10 0x7f1ed819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41274/0 (socket says 172.21.15.79:41274) 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1edcc00640 1 -- 172.21.15.79:0/3213263452 learned_addr learned my addr 172.21.15.79:0/3213263452 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1ed7e00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed8103c20 0x7f1ed8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1ed7400640 1 -- 172.21.15.79:0/3213263452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed8103c20 msgr2=0x7f1ed8199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1ed7400640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed8103c20 0x7f1ed8199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1ed7400640 1 -- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ed81057d0 msgr2=0x7f1ed81a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1ed7400640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ed81057d0 0x7f1ed81a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1ed7400640 1 -- 172.21.15.79:0/3213263452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed81a1b00 con 0x7f1ed8104e10 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1edcc00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ed81057d0 0x7f1ed81a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:22.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.667+0000 7f1ed7400640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ed8104e10 0x7f1ed819a420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000e9f0 tx=0x7f1ec000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:22.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.668+0000 7f1ed4c00640 1 -- 172.21.15.79:0/3213263452 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000ccc0 con 0x7f1ed8104e10 2024-06-03T02:19:22.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.668+0000 7f1ed4c00640 1 -- 172.21.15.79:0/3213263452 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec0010910 con 0x7f1ed8104e10 2024-06-03T02:19:22.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.668+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed81a1df0 con 0x7f1ed8104e10 2024-06-03T02:19:22.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.668+0000 7f1ed4c00640 1 -- 172.21.15.79:0/3213263452 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0018960 con 0x7f1ed8104e10 2024-06-03T02:19:22.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.668+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed81a22b0 con 0x7f1ed8104e10 2024-06-03T02:19:22.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.669+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed810af40 con 0x7f1ed8104e10 2024-06-03T02:19:22.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.671+0000 7f1ed4c00640 1 -- 172.21.15.79:0/3213263452 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ec0010ac0 con 0x7f1ed8104e10 2024-06-03T02:19:22.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.673+0000 7f1ed4c00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1ebc0a2c30 0x7f1ebc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:22.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.673+0000 7f1ed4c00640 1 -- 172.21.15.79:0/3213263452 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1ec00cf7a0 con 0x7f1ed8104e10 2024-06-03T02:19:22.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.673+0000 7f1ed7e00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1ebc0a2c30 0x7f1ebc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:22.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.674+0000 7f1ed7e00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1ebc0a2c30 0x7f1ebc0a50f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005ab0 tx=0x7f1ec8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:22.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.677+0000 7f1ed4c00640 1 -- 172.21.15.79:0/3213263452 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ec009bca0 con 0x7f1ed8104e10 2024-06-03T02:19:22.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:22 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3130727279' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-03T02:19:22.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:22 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3130727279' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-03T02:19:22.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:22 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3130727279' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-03T02:19:22.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.905+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1ed8061f40 con 0x7f1ebc0a2c30 2024-06-03T02:19:22.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.937+0000 7f1ed4c00640 1 -- 172.21.15.79:0/3213263452 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f1ed8061f40 con 0x7f1ebc0a2c30 2024-06-03T02:19:22.937 INFO:teuthology.orchestra.run.smithi079.stdout:{} 2024-06-03T02:19:22.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.939+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1ebc0a2c30 msgr2=0x7f1ebc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:22.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.939+0000 7f1edec00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1ebc0a2c30 0x7f1ebc0a50f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005ab0 tx=0x7f1ec8000bc0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.939+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ed8104e10 msgr2=0x7f1ed819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:22.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.939+0000 7f1edec00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ed8104e10 0x7f1ed819a420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000e9f0 tx=0x7f1ec000eeb0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.939+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 shutdown_connections 2024-06-03T02:19:22.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.939+0000 7f1edec00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ed81057d0 0x7f1ed81a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.939+0000 7f1edec00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1ebc0a2c30 0x7f1ebc0a50f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.939+0000 7f1edec00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ed8104e10 0x7f1ed819a420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.940+0000 7f1edec00640 1 --2- 172.21.15.79:0/3213263452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ed8103c20 0x7f1ed8199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:22.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.940+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 >> 172.21.15.79:0/3213263452 conn(0x7f1ed80ff190 msgr2=0x7f1ed8108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:22.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.940+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 shutdown_connections 2024-06-03T02:19:22.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:22.940+0000 7f1edec00640 1 -- 172.21.15.79:0/3213263452 wait complete. 2024-06-03T02:19:22.959 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-03T02:19:22.970 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-03T02:19:22.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-03T02:19:23.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.171+0000 7fb1ed600640 1 -- 172.21.15.79:0/913813209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e8077f40 msgr2=0x7fb1e80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.171+0000 7fb1ed600640 1 --2- 172.21.15.79:0/913813209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e8077f40 0x7fb1e80787e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009be0 tx=0x7fb1e002a6c0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.171+0000 7fb1ed600640 1 -- 172.21.15.79:0/913813209 shutdown_connections 2024-06-03T02:19:23.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.171+0000 7fb1ed600640 1 --2- 172.21.15.79:0/913813209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e8077f40 0x7fb1e80787e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.171+0000 7fb1ed600640 1 --2- 172.21.15.79:0/913813209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e80775a0 0x7fb1e8077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.171+0000 7fb1ed600640 1 --2- 172.21.15.79:0/913813209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e8076360 0x7fb1e8076780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.171+0000 7fb1ed600640 1 -- 172.21.15.79:0/913813209 >> 172.21.15.79:0/913813209 conn(0x7fb1e80ff190 msgr2=0x7fb1e81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:23.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.171+0000 7fb1ed600640 1 -- 172.21.15.79:0/913813209 shutdown_connections 2024-06-03T02:19:23.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.171+0000 7fb1ed600640 1 -- 172.21.15.79:0/913813209 wait complete. 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.172+0000 7fb1ed600640 1 Processor -- start 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.172+0000 7fb1ed600640 1 -- start start 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.172+0000 7fb1ed600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e8076360 0x7fb1e819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.172+0000 7fb1ed600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e80775a0 0x7fb1e819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.172+0000 7fb1ed600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e8077f40 0x7fb1e81a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.172+0000 7fb1ed600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8113cb0 con 0x7fb1e80775a0 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.172+0000 7fb1ed600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8113b30 con 0x7fb1e8076360 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.172+0000 7fb1ed600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8113e30 con 0x7fb1e8077f40 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.172+0000 7fb1e6000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e80775a0 0x7fb1e819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:23.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e8077f40 0x7fb1e81a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e7400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e8077f40 0x7fb1e81a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41306/0 (socket says 172.21.15.79:41306) 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e7400640 1 -- 172.21.15.79:0/3080555902 learned_addr learned my addr 172.21.15.79:0/3080555902 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e6a00640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e8076360 0x7fb1e819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e7400640 1 -- 172.21.15.79:0/3080555902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e8076360 msgr2=0x7fb1e819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e7400640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e8076360 0x7fb1e819e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e7400640 1 -- 172.21.15.79:0/3080555902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e80775a0 msgr2=0x7fb1e819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e7400640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e80775a0 0x7fb1e819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e7400640 1 -- 172.21.15.79:0/3080555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e81a5ed0 con 0x7fb1e8077f40 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.173+0000 7fb1e7400640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e8077f40 0x7fb1e81a5860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009b30 tx=0x7fb1e00305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.174+0000 7fb1df400640 1 -- 172.21.15.79:0/3080555902 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0041020 con 0x7fb1e8077f40 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.174+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e81a6160 con 0x7fb1e8077f40 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.174+0000 7fb1df400640 1 -- 172.21.15.79:0/3080555902 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e0030740 con 0x7fb1e8077f40 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.174+0000 7fb1df400640 1 -- 172.21.15.79:0/3080555902 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0036040 con 0x7fb1e8077f40 2024-06-03T02:19:23.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.174+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e81a66a0 con 0x7fb1e8077f40 2024-06-03T02:19:23.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.175+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e8110720 con 0x7fb1e8077f40 2024-06-03T02:19:23.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.177+0000 7fb1df400640 1 -- 172.21.15.79:0/3080555902 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1e0044020 con 0x7fb1e8077f40 2024-06-03T02:19:23.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.178+0000 7fb1df400640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1cc0a2b90 0x7fb1cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:23.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.179+0000 7fb1df400640 1 -- 172.21.15.79:0/3080555902 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb1e00f04f0 con 0x7fb1e8077f40 2024-06-03T02:19:23.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.179+0000 7fb1e6a00640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1cc0a2b90 0x7fb1cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:23.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.179+0000 7fb1e6a00640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1cc0a2b90 0x7fb1cc0a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8005fd0 tx=0x7fb1d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:23.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.181+0000 7fb1df400640 1 -- 172.21.15.79:0/3080555902 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb1e004b020 con 0x7fb1e8077f40 2024-06-03T02:19:23.314 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:19:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:19:22.914+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:19:23.314 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:19:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:19:22.914+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:19:23.314 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:19:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:19:22.914+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:19:23.314 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:19:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:19:22.914+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:19:23.314 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:19:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:19:22.914+0000 7f1b8ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T02:19:23.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.389+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1e80625a0 con 0x7fb1e8077f40 2024-06-03T02:19:23.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.391+0000 7fb1df400640 1 -- 172.21.15.79:0/3080555902 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+24805 (secure 0 0 0) 0x7fb1e00bd040 con 0x7fb1e8077f40 2024-06-03T02:19:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:23.392 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":101,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:15:54.257434+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.393+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1cc0a2b90 msgr2=0x7fb1cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.393+0000 7fb1ed600640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1cc0a2b90 0x7fb1cc0a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8005fd0 tx=0x7fb1d8009260 comp rx=0 tx=0).stop 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.393+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e8077f40 msgr2=0x7fb1e81a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.393+0000 7fb1ed600640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e8077f40 0x7fb1e81a5860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009b30 tx=0x7fb1e00305d0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.393+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 shutdown_connections 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.393+0000 7fb1ed600640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb1cc0a2b90 0x7fb1cc0a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.393+0000 7fb1ed600640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e8077f40 0x7fb1e81a5860 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.393+0000 7fb1ed600640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e80775a0 0x7fb1e819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.393+0000 7fb1ed600640 1 --2- 172.21.15.79:0/3080555902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e8076360 0x7fb1e819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.394+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 >> 172.21.15.79:0/3080555902 conn(0x7fb1e80ff190 msgr2=0x7fb1e8100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:23.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.394+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 shutdown_connections 2024-06-03T02:19:23.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.394+0000 7fb1ed600640 1 -- 172.21.15.79:0/3080555902 wait complete. 2024-06-03T02:19:23.415 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-03T02:08:38.699457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-03T02:14:03.801578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-03T02:14:08.648682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-03T02:14:21.759252+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-03T02:19:23.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-03T02:19:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:23 smithi079 ceph-mon[31819]: pgmap v496: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:19:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:23 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3130727279' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-03T02:19:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:23 smithi079 ceph-mon[31819]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T02:19:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:23 smithi079 ceph-mon[31819]: from='client.25198 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:19:23.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:23 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3080555902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:19:23.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.612+0000 7f5483e00640 1 -- 172.21.15.79:0/2760191568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5484109070 msgr2=0x7f54841094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.612+0000 7f5483e00640 1 --2- 172.21.15.79:0/2760191568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5484109070 0x7f54841094f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5470009900 tx=0x7f547002a3f0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.612+0000 7f5483e00640 1 -- 172.21.15.79:0/2760191568 shutdown_connections 2024-06-03T02:19:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.612+0000 7f5483e00640 1 --2- 172.21.15.79:0/2760191568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5484109a30 0x7f5484110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.612+0000 7f5483e00640 1 --2- 172.21.15.79:0/2760191568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5484109070 0x7f54841094f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.612+0000 7f5483e00640 1 --2- 172.21.15.79:0/2760191568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5484107e90 0x7f54841082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.612+0000 7f5483e00640 1 -- 172.21.15.79:0/2760191568 >> 172.21.15.79:0/2760191568 conn(0x7f5484076e70 msgr2=0x7f54840792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.612+0000 7f5483e00640 1 -- 172.21.15.79:0/2760191568 shutdown_connections 2024-06-03T02:19:23.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.612+0000 7f5483e00640 1 -- 172.21.15.79:0/2760191568 wait complete. 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.613+0000 7f5483e00640 1 Processor -- start 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.613+0000 7f5483e00640 1 -- start start 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5483e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5484107e90 0x7f548419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5483e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5484109070 0x7f548419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5483e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5484109a30 0x7f54841a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5483e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5484113bb0 con 0x7f5484109070 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5483e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5484113a30 con 0x7f5484107e90 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5483e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5484113d30 con 0x7f5484109a30 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5482000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5484109070 0x7f548419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5482000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5484109070 0x7f548419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47308/0 (socket says 172.21.15.79:47308) 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5482000640 1 -- 172.21.15.79:0/2177143390 learned_addr learned my addr 172.21.15.79:0/2177143390 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5482a00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5484107e90 0x7f548419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.614+0000 7f5483400640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5484109a30 0x7f54841a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:23.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f5482000640 1 -- 172.21.15.79:0/2177143390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5484109a30 msgr2=0x7f54841a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f5482000640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5484109a30 0x7f54841a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f5482000640 1 -- 172.21.15.79:0/2177143390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5484107e90 msgr2=0x7f548419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f5482000640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5484107e90 0x7f548419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f5482000640 1 -- 172.21.15.79:0/2177143390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54841a5dc0 con 0x7f5484109070 2024-06-03T02:19:23.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f5482a00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5484107e90 0x7f548419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:23.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f5482000640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5484109070 0x7f548419e6e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5470031440 tx=0x7f5470032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:23.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f547b400640 1 -- 172.21.15.79:0/2177143390 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547003f030 con 0x7f5484109070 2024-06-03T02:19:23.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f547b400640 1 -- 172.21.15.79:0/2177143390 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5470003ce0 con 0x7f5484109070 2024-06-03T02:19:23.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f547b400640 1 -- 172.21.15.79:0/2177143390 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470035070 con 0x7f5484109070 2024-06-03T02:19:23.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.615+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54841a6050 con 0x7f5484109070 2024-06-03T02:19:23.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.616+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54841a64f0 con 0x7f5484109070 2024-06-03T02:19:23.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.617+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548410f1d0 con 0x7f5484109070 2024-06-03T02:19:23.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.619+0000 7f547b400640 1 -- 172.21.15.79:0/2177143390 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5470042030 con 0x7f5484109070 2024-06-03T02:19:23.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.620+0000 7f547b400640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54680a2b90 0x7f54680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:23.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.620+0000 7f547b400640 1 -- 172.21.15.79:0/2177143390 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f54700ef9b0 con 0x7f5484109070 2024-06-03T02:19:23.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.620+0000 7f5482a00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54680a2b90 0x7f54680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:23.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.621+0000 7f5482a00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54680a2b90 0x7f54680a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5484108be0 tx=0x7f5474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:23.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.625+0000 7f547b400640 1 -- 172.21.15.79:0/2177143390 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f54700bbeb0 con 0x7f5484109070 2024-06-03T02:19:23.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:23 smithi123 ceph-mon[36640]: pgmap v496: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:19:23.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:23 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3130727279' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-03T02:19:23.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:23 smithi123 ceph-mon[36640]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T02:19:23.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:23 smithi123 ceph-mon[36640]: from='client.25198 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:19:23.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:23 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3080555902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:19:23.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.823+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f548402d500 con 0x7f5484109070 2024-06-03T02:19:23.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.824+0000 7f547b400640 1 -- 172.21.15.79:0/2177143390 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f54700c01b0 con 0x7f5484109070 2024-06-03T02:19:23.825 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-06-03T02:19:23.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54680a2b90 msgr2=0x7f54680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54680a2b90 0x7f54680a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5484108be0 tx=0x7f5474005f90 comp rx=0 tx=0).stop 2024-06-03T02:19:23.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5484109070 msgr2=0x7f548419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:23.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5484109070 0x7f548419e6e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5470031440 tx=0x7f5470032ce0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 shutdown_connections 2024-06-03T02:19:23.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54680a2b90 0x7f54680a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5484109a30 0x7f54841a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5484109070 0x7f548419e6e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 --2- 172.21.15.79:0/2177143390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5484107e90 0x7f548419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:23.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 >> 172.21.15.79:0/2177143390 conn(0x7f5484076e70 msgr2=0x7f548410c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:23.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 shutdown_connections 2024-06-03T02:19:23.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:23.827+0000 7f5483e00640 1 -- 172.21.15.79:0/2177143390 wait complete. 2024-06-03T02:19:23.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:23 smithi135 ceph-mon[36653]: pgmap v496: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T02:19:23.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:23 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3130727279' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-03T02:19:23.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:23 smithi135 ceph-mon[36653]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T02:19:23.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:23 smithi135 ceph-mon[36653]: from='client.25198 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T02:19:23.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:23 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3080555902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:19:23.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-03T02:19:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.050+0000 7f41ef000640 1 -- 172.21.15.79:0/2072429482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e8074400 msgr2=0x7f41e8074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.050+0000 7f41ef000640 1 --2- 172.21.15.79:0/2072429482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e8074400 0x7f41e8074880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009930 tx=0x7f41d402a440 comp rx=0 tx=0).stop 2024-06-03T02:19:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.050+0000 7f41ef000640 1 -- 172.21.15.79:0/2072429482 shutdown_connections 2024-06-03T02:19:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.050+0000 7f41ef000640 1 --2- 172.21.15.79:0/2072429482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e810df00 0x7f41e81102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.050+0000 7f41ef000640 1 --2- 172.21.15.79:0/2072429482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e8074400 0x7f41e8074880 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.050+0000 7f41ef000640 1 --2- 172.21.15.79:0/2072429482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41e80759e0 0x7f41e8073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.050+0000 7f41ef000640 1 -- 172.21.15.79:0/2072429482 >> 172.21.15.79:0/2072429482 conn(0x7f41e80fcea0 msgr2=0x7f41e80ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.050+0000 7f41ef000640 1 -- 172.21.15.79:0/2072429482 shutdown_connections 2024-06-03T02:19:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.050+0000 7f41ef000640 1 -- 172.21.15.79:0/2072429482 wait complete. 2024-06-03T02:19:24.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.051+0000 7f41ef000640 1 Processor -- start 2024-06-03T02:19:24.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ef000640 1 -- start start 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ef000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e8074400 0x7f41e819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ef000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41e80759e0 0x7f41e819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ef000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e810df00 0x7f41e81a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ef000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8113b80 con 0x7f41e80759e0 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ef000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8113a00 con 0x7f41e810df00 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ef000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8113d00 con 0x7f41e8074400 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ed200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41e80759e0 0x7f41e819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ed200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41e80759e0 0x7f41e819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47326/0 (socket says 172.21.15.79:47326) 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ed200640 1 -- 172.21.15.79:0/2590466260 learned_addr learned my addr 172.21.15.79:0/2590466260 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41edc00640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e8074400 0x7f41e819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.052+0000 7f41ee600640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e810df00 0x7f41e81a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.053+0000 7f41ed200640 1 -- 172.21.15.79:0/2590466260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e8074400 msgr2=0x7f41e819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.053+0000 7f41ed200640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e8074400 0x7f41e819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.053+0000 7f41ed200640 1 -- 172.21.15.79:0/2590466260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e810df00 msgr2=0x7f41e81a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.053+0000 7f41ed200640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e810df00 0x7f41e81a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.053+0000 7f41ed200640 1 -- 172.21.15.79:0/2590466260 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e81a5e10 con 0x7f41e80759e0 2024-06-03T02:19:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.053+0000 7f41ed200640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41e80759e0 0x7f41e819e730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f41d4031440 tx=0x7f41d4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.054+0000 7f41e6000640 1 -- 172.21.15.79:0/2590466260 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d403f030 con 0x7f41e80759e0 2024-06-03T02:19:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.054+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e81a60a0 con 0x7f41e80759e0 2024-06-03T02:19:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.054+0000 7f41e6000640 1 -- 172.21.15.79:0/2590466260 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d4003ce0 con 0x7f41e80759e0 2024-06-03T02:19:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.054+0000 7f41e6000640 1 -- 172.21.15.79:0/2590466260 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4035070 con 0x7f41e80759e0 2024-06-03T02:19:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.054+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e81a6590 con 0x7f41e80759e0 2024-06-03T02:19:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.055+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e81040d0 con 0x7f41e80759e0 2024-06-03T02:19:24.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.057+0000 7f41e6000640 1 -- 172.21.15.79:0/2590466260 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f41d4042030 con 0x7f41e80759e0 2024-06-03T02:19:24.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.058+0000 7f41e6000640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f41d00a2b90 0x7f41d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.058+0000 7f41e6000640 1 -- 172.21.15.79:0/2590466260 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f41d40ef8f0 con 0x7f41e80759e0 2024-06-03T02:19:24.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.058+0000 7f41edc00640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f41d00a2b90 0x7f41d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.059+0000 7f41edc00640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f41d00a2b90 0x7f41d00a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009bd0 tx=0x7f41dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:24.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.062+0000 7f41e6000640 1 -- 172.21.15.79:0/2590466260 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f41d40bbdf0 con 0x7f41e80759e0 2024-06-03T02:19:24.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.259+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41e80625a0 con 0x7f41e80759e0 2024-06-03T02:19:24.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.261+0000 7f41e6000640 1 -- 172.21.15.79:0/2590466260 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f41d40c00f0 con 0x7f41e80759e0 2024-06-03T02:19:24.262 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-06-03T02:19:24.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f41d00a2b90 msgr2=0x7f41d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f41d00a2b90 0x7f41d00a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009bd0 tx=0x7f41dc005f90 comp rx=0 tx=0).stop 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41e80759e0 msgr2=0x7f41e819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41e80759e0 0x7f41e819e730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f41d4031440 tx=0x7f41d4032ce0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 shutdown_connections 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e810df00 0x7f41e81a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41e80759e0 0x7f41e819e730 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f41d00a2b90 0x7f41d00a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 --2- 172.21.15.79:0/2590466260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e8074400 0x7f41e819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.263+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 >> 172.21.15.79:0/2590466260 conn(0x7f41e80fcea0 msgr2=0x7f41e81012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.264+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 shutdown_connections 2024-06-03T02:19:24.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.264+0000 7f41ef000640 1 -- 172.21.15.79:0/2590466260 wait complete. 2024-06-03T02:19:24.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-03T02:19:24.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.482+0000 7efe46800640 1 -- 172.21.15.79:0/25691698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe40102480 msgr2=0x7efe40108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.482+0000 7efe46800640 1 --2- 172.21.15.79:0/25691698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe40102480 0x7efe40108d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe38009ca0 tx=0x7efe3802a8c0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.482+0000 7efe46800640 1 -- 172.21.15.79:0/25691698 shutdown_connections 2024-06-03T02:19:24.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.482+0000 7efe46800640 1 --2- 172.21.15.79:0/25691698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe40102480 0x7efe40108d10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.482+0000 7efe46800640 1 --2- 172.21.15.79:0/25691698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe40101ac0 0x7efe40101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.482+0000 7efe46800640 1 --2- 172.21.15.79:0/25691698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe401008d0 0x7efe40100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.482+0000 7efe46800640 1 -- 172.21.15.79:0/25691698 >> 172.21.15.79:0/25691698 conn(0x7efe400fbe40 msgr2=0x7efe400fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:24.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.482+0000 7efe46800640 1 -- 172.21.15.79:0/25691698 shutdown_connections 2024-06-03T02:19:24.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.482+0000 7efe46800640 1 -- 172.21.15.79:0/25691698 wait complete. 2024-06-03T02:19:24.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.483+0000 7efe46800640 1 Processor -- start 2024-06-03T02:19:24.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.483+0000 7efe46800640 1 -- start start 2024-06-03T02:19:24.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe46800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe401008d0 0x7efe40197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe46800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe40101ac0 0x7efe401981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe46800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe40102480 0x7efe4019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe46800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efe4010c5a0 con 0x7efe401008d0 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe46800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe4010c420 con 0x7efe40102480 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe46800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efe4010c720 con 0x7efe40101ac0 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3f400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe401008d0 0x7efe40197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3f400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe401008d0 0x7efe40197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47344/0 (socket says 172.21.15.79:47344) 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3f400640 1 -- 172.21.15.79:0/4272251586 learned_addr learned my addr 172.21.15.79:0/4272251586 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3ea00640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe40101ac0 0x7efe401981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3fe00640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe40102480 0x7efe4019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3f400640 1 -- 172.21.15.79:0/4272251586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe40101ac0 msgr2=0x7efe401981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3f400640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe40101ac0 0x7efe401981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3f400640 1 -- 172.21.15.79:0/4272251586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe40102480 msgr2=0x7efe4019f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3f400640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe40102480 0x7efe4019f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3f400640 1 -- 172.21.15.79:0/4272251586 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4019f8e0 con 0x7efe401008d0 2024-06-03T02:19:24.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.484+0000 7efe3fe00640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe40102480 0x7efe4019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:19:24.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.485+0000 7efe3f400640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe401008d0 0x7efe40197ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe3000e9c0 tx=0x7efe3000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:24.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.485+0000 7efe37e00640 1 -- 172.21.15.79:0/4272251586 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3000cd80 con 0x7efe401008d0 2024-06-03T02:19:24.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.485+0000 7efe37e00640 1 -- 172.21.15.79:0/4272251586 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe300035f0 con 0x7efe401008d0 2024-06-03T02:19:24.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.485+0000 7efe37e00640 1 -- 172.21.15.79:0/4272251586 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30010690 con 0x7efe401008d0 2024-06-03T02:19:24.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.485+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4019fbd0 con 0x7efe401008d0 2024-06-03T02:19:24.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.485+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe401a00d0 con 0x7efe401008d0 2024-06-03T02:19:24.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.486+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe401a0300 con 0x7efe401008d0 2024-06-03T02:19:24.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.488+0000 7efe37e00640 1 -- 172.21.15.79:0/4272251586 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe300037a0 con 0x7efe401008d0 2024-06-03T02:19:24.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.489+0000 7efe37e00640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efe240a2b90 0x7efe240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.490+0000 7efe37e00640 1 -- 172.21.15.79:0/4272251586 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7efe300cf850 con 0x7efe401008d0 2024-06-03T02:19:24.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.490+0000 7efe3ea00640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efe240a2b90 0x7efe240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.490+0000 7efe3ea00640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efe240a2b90 0x7efe240a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efe401991a0 tx=0x7efe2c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:24.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.494+0000 7efe37e00640 1 -- 172.21.15.79:0/4272251586 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe3009c3a0 con 0x7efe401008d0 2024-06-03T02:19:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:24 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2177143390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T02:19:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:24 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2590466260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T02:19:24.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.698+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efe401a0570 con 0x7efe401008d0 2024-06-03T02:19:24.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.699+0000 7efe37e00640 1 -- 172.21.15.79:0/4272251586 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7efe3009bca0 con 0x7efe401008d0 2024-06-03T02:19:24.699 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-06-03T02:19:24.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.701+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efe240a2b90 msgr2=0x7efe240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.701+0000 7efe46800640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efe240a2b90 0x7efe240a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efe401991a0 tx=0x7efe2c00b410 comp rx=0 tx=0).stop 2024-06-03T02:19:24.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.701+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe401008d0 msgr2=0x7efe40197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.701+0000 7efe46800640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe401008d0 0x7efe40197ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe3000e9c0 tx=0x7efe3000ee80 comp rx=0 tx=0).stop 2024-06-03T02:19:24.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.702+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 shutdown_connections 2024-06-03T02:19:24.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.702+0000 7efe46800640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe40102480 0x7efe4019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.702+0000 7efe46800640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efe240a2b90 0x7efe240a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.702+0000 7efe46800640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe40101ac0 0x7efe401981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.702+0000 7efe46800640 1 --2- 172.21.15.79:0/4272251586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe401008d0 0x7efe40197ca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.702+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 >> 172.21.15.79:0/4272251586 conn(0x7efe400fbe40 msgr2=0x7efe400fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:24.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.702+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 shutdown_connections 2024-06-03T02:19:24.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.702+0000 7efe46800640 1 -- 172.21.15.79:0/4272251586 wait complete. 2024-06-03T02:19:24.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-03T02:19:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:24 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2177143390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T02:19:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:24 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2590466260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T02:19:24.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:24 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2177143390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T02:19:24.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:24 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2590466260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T02:19:24.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.929+0000 7f1ed2000640 1 -- 172.21.15.79:0/1489424993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ecc101930 msgr2=0x7f1ecc101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.929+0000 7f1ed2000640 1 --2- 172.21.15.79:0/1489424993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ecc101930 0x7f1ecc101d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009900 tx=0x7f1ec402a3f0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.929+0000 7f1ed2000640 1 -- 172.21.15.79:0/1489424993 shutdown_connections 2024-06-03T02:19:24.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.929+0000 7f1ed2000640 1 --2- 172.21.15.79:0/1489424993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc1034e0 0x7f1ecc109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.929+0000 7f1ed2000640 1 --2- 172.21.15.79:0/1489424993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc102b20 0x7f1ecc102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.929+0000 7f1ed2000640 1 --2- 172.21.15.79:0/1489424993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ecc101930 0x7f1ecc101d50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.929+0000 7f1ed2000640 1 -- 172.21.15.79:0/1489424993 >> 172.21.15.79:0/1489424993 conn(0x7f1ecc0fcea0 msgr2=0x7f1ecc0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:24.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.929+0000 7f1ed2000640 1 -- 172.21.15.79:0/1489424993 shutdown_connections 2024-06-03T02:19:24.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.929+0000 7f1ed2000640 1 -- 172.21.15.79:0/1489424993 wait complete. 2024-06-03T02:19:24.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.930+0000 7f1ed2000640 1 Processor -- start 2024-06-03T02:19:24.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.930+0000 7f1ed2000640 1 -- start start 2024-06-03T02:19:24.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.930+0000 7f1ed2000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc101930 0x7f1ecc199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.930+0000 7f1ed2000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ecc102b20 0x7f1ecc19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.930+0000 7f1ed2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc1034e0 0x7f1ecc1a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.930+0000 7f1ed2000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10d560 con 0x7f1ecc102b20 2024-06-03T02:19:24.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.930+0000 7f1ed2000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10d3e0 con 0x7f1ecc1034e0 2024-06-03T02:19:24.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.930+0000 7f1ed2000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10d6e0 con 0x7f1ecc101930 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc101930 0x7f1ecc199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed0c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc101930 0x7f1ecc199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41526/0 (socket says 172.21.15.79:41526) 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed0c00640 1 -- 172.21.15.79:0/2501919152 learned_addr learned my addr 172.21.15.79:0/2501919152 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ecbe00640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ecc102b20 0x7f1ecc19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed1600640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc1034e0 0x7f1ecc1a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed0c00640 1 -- 172.21.15.79:0/2501919152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc1034e0 msgr2=0x7f1ecc1a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed0c00640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc1034e0 0x7f1ecc1a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed0c00640 1 -- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ecc102b20 msgr2=0x7f1ecc19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed0c00640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ecc102b20 0x7f1ecc19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed0c00640 1 -- 172.21.15.79:0/2501919152 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc1a1a40 con 0x7f1ecc101930 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.931+0000 7f1ed0c00640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc101930 0x7f1ecc199e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4031440 tx=0x7f1ec4032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.932+0000 7f1ec9600640 1 -- 172.21.15.79:0/2501919152 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec403f030 con 0x7f1ecc101930 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.932+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc1a1cd0 con 0x7f1ecc101930 2024-06-03T02:19:24.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.932+0000 7f1ec9600640 1 -- 172.21.15.79:0/2501919152 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec4003ce0 con 0x7f1ecc101930 2024-06-03T02:19:24.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.932+0000 7f1ec9600640 1 -- 172.21.15.79:0/2501919152 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4035070 con 0x7f1ecc101930 2024-06-03T02:19:24.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.932+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1a21c0 con 0x7f1ecc101930 2024-06-03T02:19:24.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.933+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ecc1a2390 con 0x7f1ecc101930 2024-06-03T02:19:24.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.935+0000 7f1ec9600640 1 -- 172.21.15.79:0/2501919152 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ec4042030 con 0x7f1ecc101930 2024-06-03T02:19:24.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.936+0000 7f1ec9600640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1eb00a2b90 0x7f1eb00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:24.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.936+0000 7f1ec9600640 1 -- 172.21.15.79:0/2501919152 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1ec40ef800 con 0x7f1ecc101930 2024-06-03T02:19:24.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.936+0000 7f1ecbe00640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1eb00a2b90 0x7f1eb00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:24.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.937+0000 7f1ecbe00640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1eb00a2b90 0x7f1eb00a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc19b330 tx=0x7f1eb8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:24.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:24.939+0000 7f1ec9600640 1 -- 172.21.15.79:0/2501919152 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ec40bbd60 con 0x7f1ecc101930 2024-06-03T02:19:25.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.124+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1ecc1a2560 con 0x7f1ecc101930 2024-06-03T02:19:25.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.125+0000 7f1ec9600640 1 -- 172.21.15.79:0/2501919152 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v101) ==== 103+0+11 (secure 0 0 0) 0x7f1ec40c0060 con 0x7f1ecc101930 2024-06-03T02:19:25.126 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-06-03T02:19:25.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.128+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1eb00a2b90 msgr2=0x7f1eb00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:25.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.128+0000 7f1ed2000640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1eb00a2b90 0x7f1eb00a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc19b330 tx=0x7f1eb8009500 comp rx=0 tx=0).stop 2024-06-03T02:19:25.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.128+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc101930 msgr2=0x7f1ecc199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:25.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.128+0000 7f1ed2000640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc101930 0x7f1ecc199e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4031440 tx=0x7f1ec4032ce0 comp rx=0 tx=0).stop 2024-06-03T02:19:25.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.129+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 shutdown_connections 2024-06-03T02:19:25.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.129+0000 7f1ed2000640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc1034e0 0x7f1ecc1a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:25.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.129+0000 7f1ed2000640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ecc102b20 0x7f1ecc19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:25.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.129+0000 7f1ed2000640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1eb00a2b90 0x7f1eb00a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:25.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.129+0000 7f1ed2000640 1 --2- 172.21.15.79:0/2501919152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc101930 0x7f1ecc199e20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:25.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.129+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 >> 172.21.15.79:0/2501919152 conn(0x7f1ecc0fcea0 msgr2=0x7f1ecc073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:25.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.129+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 shutdown_connections 2024-06-03T02:19:25.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:25.129+0000 7f1ed2000640 1 -- 172.21.15.79:0/2501919152 wait complete. 2024-06-03T02:19:25.148 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-03T02:19:25.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:19:25.554 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:25.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:25 smithi079 ceph-mon[31819]: pgmap v497: 129 pgs: 129 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 852 B/s wr, 10 op/s 2024-06-03T02:19:25.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:25 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4272251586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T02:19:25.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:25 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2501919152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T02:19:25.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:25 smithi135 ceph-mon[36653]: pgmap v497: 129 pgs: 129 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 852 B/s wr, 10 op/s 2024-06-03T02:19:25.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:25 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4272251586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T02:19:25.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:25 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2501919152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T02:19:25.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:25 smithi123 ceph-mon[36640]: pgmap v497: 129 pgs: 129 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 852 B/s wr, 10 op/s 2024-06-03T02:19:25.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:25 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4272251586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T02:19:25.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:25 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2501919152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T02:19:26.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:26 smithi079 ceph-mon[31819]: mgrmap e19: x(active, since 13m), standbys: z, y 2024-06-03T02:19:26.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:26 smithi135 ceph-mon[36653]: mgrmap e19: x(active, since 13m), standbys: z, y 2024-06-03T02:19:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:26 smithi123 ceph-mon[36640]: mgrmap e19: x(active, since 13m), standbys: z, y 2024-06-03T02:19:27.707 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:27 smithi079 ceph-mon[31819]: pgmap v498: 129 pgs: 129 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-03T02:19:27.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:27 smithi135 ceph-mon[36653]: pgmap v498: 129 pgs: 129 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-03T02:19:27.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.874+0000 7f8b23200700 1 -- 172.21.15.79:0/3370254083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b1c105060 msgr2=0x7f8b1c111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:27.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.874+0000 7f8b23200700 1 --2- 172.21.15.79:0/3370254083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b1c105060 0x7f8b1c111570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8b140098f0 tx=0x7f8b14009c00 comp rx=0 tx=0).stop 2024-06-03T02:19:27.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.874+0000 7f8b23200700 1 -- 172.21.15.79:0/3370254083 shutdown_connections 2024-06-03T02:19:27.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.874+0000 7f8b23200700 1 --2- 172.21.15.79:0/3370254083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b1c105060 0x7f8b1c111570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:27.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.874+0000 7f8b23200700 1 --2- 172.21.15.79:0/3370254083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b1c10a6a0 0x7f8b1c10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:27.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.874+0000 7f8b23200700 1 --2- 172.21.15.79:0/3370254083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b1c1046a0 0x7f8b1c104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:27.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.874+0000 7f8b23200700 1 -- 172.21.15.79:0/3370254083 >> 172.21.15.79:0/3370254083 conn(0x7f8b1c1001d0 msgr2=0x7f8b1c1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:27.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.875+0000 7f8b23200700 1 -- 172.21.15.79:0/3370254083 shutdown_connections 2024-06-03T02:19:27.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.875+0000 7f8b23200700 1 -- 172.21.15.79:0/3370254083 wait complete. 2024-06-03T02:19:27.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.876+0000 7f8b23200700 1 Processor -- start 2024-06-03T02:19:27.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.876+0000 7f8b23200700 1 -- start start 2024-06-03T02:19:27.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.876+0000 7f8b23200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b1c1046a0 0x7f8b1c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:27.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.876+0000 7f8b23200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b1c105060 0x7f8b1c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.876+0000 7f8b23200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b1c10a6a0 0x7f8b1c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.876+0000 7f8b23200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c114a40 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.876+0000 7f8b23200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c1148c0 con 0x7f8b1c1046a0 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.876+0000 7f8b23200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c114bc0 con 0x7f8b1c105060 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b1b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b1c105060 0x7f8b1c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b21200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b1c10a6a0 0x7f8b1c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b1be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b1c1046a0 0x7f8b1c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b1b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b1c105060 0x7f8b1c19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41536/0 (socket says 172.21.15.79:41536) 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b1be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b1c1046a0 0x7f8b1c19aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38780/0 (socket says 172.21.15.79:38780) 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b1b400700 1 -- 172.21.15.79:0/1182678423 learned_addr learned my addr 172.21.15.79:0/1182678423 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b21200700 1 -- 172.21.15.79:0/1182678423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b1c105060 msgr2=0x7f8b1c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:27.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b21200700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b1c105060 0x7f8b1c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:27.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b21200700 1 -- 172.21.15.79:0/1182678423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b1c1046a0 msgr2=0x7f8b1c19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:27.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b21200700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b1c1046a0 0x7f8b1c19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:27.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.877+0000 7f8b21200700 1 -- 172.21.15.79:0/1182678423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c19ff00 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.878+0000 7f8b21200700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b1c10a6a0 0x7f8b1c19f7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009230 tx=0x7f8b1400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:27.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.878+0000 7f8b18c00700 1 -- 172.21.15.79:0/1182678423 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14027070 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.878+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c1a0190 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.878+0000 7f8b18c00700 1 -- 172.21.15.79:0/1182678423 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b14005520 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.878+0000 7f8b18c00700 1 -- 172.21.15.79:0/1182678423 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14004930 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.878+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c1a7a70 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.879+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1c105de0 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.881+0000 7f8b18c00700 1 -- 172.21.15.79:0/1182678423 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f8b1402a070 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.883+0000 7f8b18c00700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b000a2d30 0x7f8b000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:27.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.883+0000 7f8b18c00700 1 -- 172.21.15.79:0/1182678423 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8b140cfec0 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.883+0000 7f8b1be00700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b000a2d30 0x7f8b000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:27.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.884+0000 7f8b1be00700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b000a2d30 0x7f8b000a51f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009e60 tx=0x7f8b0c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:27.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:27.888+0000 7f8b18c00700 1 -- 172.21.15.79:0/1182678423 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8b1405e850 con 0x7f8b1c10a6a0 2024-06-03T02:19:27.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:27 smithi123 ceph-mon[36640]: pgmap v498: 129 pgs: 129 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-03T02:19:28.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.158+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b1c02dca0 con 0x7f8b1c10a6a0 2024-06-03T02:19:28.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.159+0000 7f8b18c00700 1 -- 172.21.15.79:0/1182678423 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f8b14013030 con 0x7f8b1c10a6a0 2024-06-03T02:19:28.160 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:28.160 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:19:28.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.162+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b000a2d30 msgr2=0x7f8b000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:28.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.162+0000 7f8b23200700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b000a2d30 0x7f8b000a51f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009e60 tx=0x7f8b0c00b040 comp rx=0 tx=0).stop 2024-06-03T02:19:28.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.162+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b1c10a6a0 msgr2=0x7f8b1c19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:28.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.162+0000 7f8b23200700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b1c10a6a0 0x7f8b1c19f7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009230 tx=0x7f8b1400fa70 comp rx=0 tx=0).stop 2024-06-03T02:19:28.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.163+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 shutdown_connections 2024-06-03T02:19:28.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.163+0000 7f8b23200700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b000a2d30 0x7f8b000a51f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:28.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.163+0000 7f8b23200700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b1c10a6a0 0x7f8b1c19f7c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:28.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.163+0000 7f8b23200700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b1c1046a0 0x7f8b1c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:28.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.163+0000 7f8b23200700 1 --2- 172.21.15.79:0/1182678423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b1c105060 0x7f8b1c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:28.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.163+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 >> 172.21.15.79:0/1182678423 conn(0x7f8b1c1001d0 msgr2=0x7f8b1c1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:28.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.163+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 shutdown_connections 2024-06-03T02:19:28.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:28.163+0000 7f8b23200700 1 -- 172.21.15.79:0/1182678423 wait complete. 2024-06-03T02:19:28.165 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:19:28.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:28 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1182678423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:19:28.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:28 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1182678423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:19:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:28 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1182678423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:19:29.102 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-03T02:19:29.102 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-03T02:19:29.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-03T02:19:29.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.314+0000 7f9088c00640 1 -- 172.21.15.79:0/4085150060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084109930 msgr2=0x7f9084110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:29.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.314+0000 7f9088c00640 1 --2- 172.21.15.79:0/4085150060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084109930 0x7f9084110250 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f907c009ca0 tx=0x7f907c02a8c0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.314+0000 7f9088c00640 1 -- 172.21.15.79:0/4085150060 shutdown_connections 2024-06-03T02:19:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.314+0000 7f9088c00640 1 --2- 172.21.15.79:0/4085150060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084109930 0x7f9084110250 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.314+0000 7f9088c00640 1 --2- 172.21.15.79:0/4085150060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084108f70 0x7f90841093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.314+0000 7f9088c00640 1 --2- 172.21.15.79:0/4085150060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9084107e20 0x7f9084108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.314+0000 7f9088c00640 1 -- 172.21.15.79:0/4085150060 >> 172.21.15.79:0/4085150060 conn(0x7f9084076de0 msgr2=0x7f9084079200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.314+0000 7f9088c00640 1 -- 172.21.15.79:0/4085150060 shutdown_connections 2024-06-03T02:19:29.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.314+0000 7f9088c00640 1 -- 172.21.15.79:0/4085150060 wait complete. 2024-06-03T02:19:29.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.315+0000 7f9088c00640 1 Processor -- start 2024-06-03T02:19:29.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.315+0000 7f9088c00640 1 -- start start 2024-06-03T02:19:29.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.315+0000 7f9088c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084107e20 0x7f908419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:29.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.315+0000 7f9088c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084108f70 0x7f908419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:29.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.315+0000 7f9088c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9084109930 0x7f90841a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9088c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9084113b20 con 0x7f9084109930 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9088c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90841139a0 con 0x7f9084107e20 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9088c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9084113ca0 con 0x7f9084108f70 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084107e20 0x7f908419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f907aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084108f70 0x7f908419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084107e20 0x7f908419e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38802/0 (socket says 172.21.15.79:38802) 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083400640 1 -- 172.21.15.79:0/2650199811 learned_addr learned my addr 172.21.15.79:0/2650199811 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083e00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9084109930 0x7f90841a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083400640 1 -- 172.21.15.79:0/2650199811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084108f70 msgr2=0x7f908419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083400640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084108f70 0x7f908419e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083400640 1 -- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9084109930 msgr2=0x7f90841a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083400640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9084109930 0x7f90841a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083400640 1 -- 172.21.15.79:0/2650199811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90841a5cf0 con 0x7f9084107e20 2024-06-03T02:19:29.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.316+0000 7f9083e00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9084109930 0x7f90841a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:19:29.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.317+0000 7f9083400640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084107e20 0x7f908419e0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e9f0 tx=0x7f906c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:29.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.317+0000 7f9080c00640 1 -- 172.21.15.79:0/2650199811 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c00cd80 con 0x7f9084107e20 2024-06-03T02:19:29.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.317+0000 7f9080c00640 1 -- 172.21.15.79:0/2650199811 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c010910 con 0x7f9084107e20 2024-06-03T02:19:29.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.317+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90841a5fe0 con 0x7f9084107e20 2024-06-03T02:19:29.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.317+0000 7f9080c00640 1 -- 172.21.15.79:0/2650199811 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c0188f0 con 0x7f9084107e20 2024-06-03T02:19:29.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.317+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90841a6440 con 0x7f9084107e20 2024-06-03T02:19:29.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.318+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908410f0d0 con 0x7f9084107e20 2024-06-03T02:19:29.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.320+0000 7f9080c00640 1 -- 172.21.15.79:0/2650199811 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f906c010ac0 con 0x7f9084107e20 2024-06-03T02:19:29.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.322+0000 7f9080c00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90680a2c20 0x7f90680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:29.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.322+0000 7f9080c00640 1 -- 172.21.15.79:0/2650199811 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f906c0cfe20 con 0x7f9084107e20 2024-06-03T02:19:29.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.322+0000 7f907aa00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90680a2c20 0x7f90680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:29.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.323+0000 7f907aa00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90680a2c20 0x7f90680a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f908419f5e0 tx=0x7f9074005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.324+0000 7f9080c00640 1 -- 172.21.15.79:0/2650199811 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f906c09c810 con 0x7f9084107e20 2024-06-03T02:19:29.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.522+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9084061de0 con 0x7f90680a2c20 2024-06-03T02:19:29.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.538+0000 7f9080c00640 1 -- 172.21.15.79:0/2650199811 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340748 (secure 0 0 0) 0x7f9084061de0 con 0x7f90680a2c20 2024-06-03T02:19:29.543 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:29.545 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-06-03T02:19:29.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90680a2c20 msgr2=0x7f90680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90680a2c20 0x7f90680a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f908419f5e0 tx=0x7f9074005f90 comp rx=0 tx=0).stop 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084107e20 msgr2=0x7f908419e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084107e20 0x7f908419e0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e9f0 tx=0x7f906c00eeb0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 shutdown_connections 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9084109930 0x7f90841a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90680a2c20 0x7f90680a50e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084108f70 0x7f908419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 --2- 172.21.15.79:0/2650199811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9084107e20 0x7f908419e0d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 >> 172.21.15.79:0/2650199811 conn(0x7f9084076de0 msgr2=0x7f908410c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 shutdown_connections 2024-06-03T02:19:29.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.545+0000 7f9088c00640 1 -- 172.21.15.79:0/2650199811 wait complete. 2024-06-03T02:19:29.565 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-06-03T02:19:28.301895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672584,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2473,"num_read_kb":1369,"num_write":461,"num_write_kb":3113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360840,"kb_used_data":15848,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580984,"statfs":{"total":1151940427776,"available":1151570927616,"internally_reserved":0,"allocated":16228352,"data_stored":8919395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7239,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":15,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005257"},"pg_stats":[{"pgid":"3.3f","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186895+0000","last_change":"2024-06-03T02:14:09.768431+0000","last_active":"2024-06-03T02:17:22.186895+0000","last_peered":"2024-06-03T02:17:22.186895+0000","last_clean":"2024-06-03T02:17:22.186895+0000","last_became_active":"2024-06-03T02:14:09.768059+0000","last_became_peered":"2024-06-03T02:14:09.768059+0000","last_unstale":"2024-06-03T02:17:22.186895+0000","last_undegraded":"2024-06-03T02:17:22.186895+0000","last_fullsized":"2024-06-03T02:17:22.186895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:12.265896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188670+0000","last_change":"2024-06-03T02:14:09.771842+0000","last_active":"2024-06-03T02:17:22.188670+0000","last_peered":"2024-06-03T02:17:22.188670+0000","last_clean":"2024-06-03T02:17:22.188670+0000","last_became_active":"2024-06-03T02:14:09.770961+0000","last_became_peered":"2024-06-03T02:14:09.770961+0000","last_unstale":"2024-06-03T02:17:22.188670+0000","last_undegraded":"2024-06-03T02:17:22.188670+0000","last_fullsized":"2024-06-03T02:17:22.188670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:49:02.297128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183559+0000","last_change":"2024-06-03T02:14:09.758407+0000","last_active":"2024-06-03T02:17:22.183559+0000","last_peered":"2024-06-03T02:17:22.183559+0000","last_clean":"2024-06-03T02:17:22.183559+0000","last_became_active":"2024-06-03T02:14:09.758045+0000","last_became_peered":"2024-06-03T02:14:09.758045+0000","last_unstale":"2024-06-03T02:17:22.183559+0000","last_undegraded":"2024-06-03T02:17:22.183559+0000","last_fullsized":"2024-06-03T02:17:22.183559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:01:08.166971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.152098+0000","last_change":"2024-06-03T02:14:09.747658+0000","last_active":"2024-06-03T02:17:22.152098+0000","last_peered":"2024-06-03T02:17:22.152098+0000","last_clean":"2024-06-03T02:17:22.152098+0000","last_became_active":"2024-06-03T02:14:09.747321+0000","last_became_peered":"2024-06-03T02:14:09.747321+0000","last_unstale":"2024-06-03T02:17:22.152098+0000","last_undegraded":"2024-06-03T02:17:22.152098+0000","last_fullsized":"2024-06-03T02:17:22.152098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:08.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184933+0000","last_change":"2024-06-03T02:14:09.758161+0000","last_active":"2024-06-03T02:17:22.184933+0000","last_peered":"2024-06-03T02:17:22.184933+0000","last_clean":"2024-06-03T02:17:22.184933+0000","last_became_active":"2024-06-03T02:14:09.757800+0000","last_became_peered":"2024-06-03T02:14:09.757800+0000","last_unstale":"2024-06-03T02:17:22.184933+0000","last_undegraded":"2024-06-03T02:17:22.184933+0000","last_fullsized":"2024-06-03T02:17:22.184933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:47.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.160478+0000","last_change":"2024-06-03T02:14:09.741818+0000","last_active":"2024-06-03T02:17:22.160478+0000","last_peered":"2024-06-03T02:17:22.160478+0000","last_clean":"2024-06-03T02:17:22.160478+0000","last_became_active":"2024-06-03T02:14:09.741219+0000","last_became_peered":"2024-06-03T02:14:09.741219+0000","last_unstale":"2024-06-03T02:17:22.160478+0000","last_undegraded":"2024-06-03T02:17:22.160478+0000","last_fullsized":"2024-06-03T02:17:22.160478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:16:30.585245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163369+0000","last_change":"2024-06-03T02:14:09.744863+0000","last_active":"2024-06-03T02:17:22.163369+0000","last_peered":"2024-06-03T02:17:22.163369+0000","last_clean":"2024-06-03T02:17:22.163369+0000","last_became_active":"2024-06-03T02:14:09.744514+0000","last_became_peered":"2024-06-03T02:14:09.744514+0000","last_unstale":"2024-06-03T02:17:22.163369+0000","last_undegraded":"2024-06-03T02:17:22.163369+0000","last_fullsized":"2024-06-03T02:17:22.163369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:51:42.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187625+0000","last_change":"2024-06-03T02:14:09.766897+0000","last_active":"2024-06-03T02:17:22.187625+0000","last_peered":"2024-06-03T02:17:22.187625+0000","last_clean":"2024-06-03T02:17:22.187625+0000","last_became_active":"2024-06-03T02:14:09.765461+0000","last_became_peered":"2024-06-03T02:14:09.765461+0000","last_unstale":"2024-06-03T02:17:22.187625+0000","last_undegraded":"2024-06-03T02:17:22.187625+0000","last_fullsized":"2024-06-03T02:17:22.187625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:39:28.641517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172030+0000","last_change":"2024-06-03T02:14:09.746451+0000","last_active":"2024-06-03T02:17:22.172030+0000","last_peered":"2024-06-03T02:17:22.172030+0000","last_clean":"2024-06-03T02:17:22.172030+0000","last_became_active":"2024-06-03T02:14:09.746117+0000","last_became_peered":"2024-06-03T02:14:09.746117+0000","last_unstale":"2024-06-03T02:17:22.172030+0000","last_undegraded":"2024-06-03T02:17:22.172030+0000","last_fullsized":"2024-06-03T02:17:22.172030+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:42.599993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"101'16","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.162982+0000","last_change":"2024-06-03T02:14:09.750680+0000","last_active":"2024-06-03T02:17:22.162982+0000","last_peered":"2024-06-03T02:17:22.162982+0000","last_clean":"2024-06-03T02:17:22.162982+0000","last_became_active":"2024-06-03T02:14:09.746138+0000","last_became_peered":"2024-06-03T02:14:09.746138+0000","last_unstale":"2024-06-03T02:17:22.162982+0000","last_undegraded":"2024-06-03T02:17:22.162982+0000","last_fullsized":"2024-06-03T02:17:22.162982+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:04:28.188323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172525+0000","last_change":"2024-06-03T02:14:09.743467+0000","last_active":"2024-06-03T02:17:22.172525+0000","last_peered":"2024-06-03T02:17:22.172525+0000","last_clean":"2024-06-03T02:17:22.172525+0000","last_became_active":"2024-06-03T02:14:09.743152+0000","last_became_peered":"2024-06-03T02:14:09.743152+0000","last_unstale":"2024-06-03T02:17:22.172525+0000","last_undegraded":"2024-06-03T02:17:22.172525+0000","last_fullsized":"2024-06-03T02:17:22.172525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:27.336630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.148334+0000","last_change":"2024-06-03T02:14:09.747142+0000","last_active":"2024-06-03T02:17:22.148334+0000","last_peered":"2024-06-03T02:17:22.148334+0000","last_clean":"2024-06-03T02:17:22.148334+0000","last_became_active":"2024-06-03T02:14:09.746694+0000","last_became_peered":"2024-06-03T02:14:09.746694+0000","last_unstale":"2024-06-03T02:17:22.148334+0000","last_undegraded":"2024-06-03T02:17:22.148334+0000","last_fullsized":"2024-06-03T02:17:22.148334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:04.507667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179213+0000","last_change":"2024-06-03T02:14:09.759906+0000","last_active":"2024-06-03T02:17:22.179213+0000","last_peered":"2024-06-03T02:17:22.179213+0000","last_clean":"2024-06-03T02:17:22.179213+0000","last_became_active":"2024-06-03T02:14:09.755450+0000","last_became_peered":"2024-06-03T02:14:09.755450+0000","last_unstale":"2024-06-03T02:17:22.179213+0000","last_undegraded":"2024-06-03T02:17:22.179213+0000","last_fullsized":"2024-06-03T02:17:22.179213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:23.057956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187178+0000","last_change":"2024-06-03T02:14:09.769388+0000","last_active":"2024-06-03T02:17:22.187178+0000","last_peered":"2024-06-03T02:17:22.187178+0000","last_clean":"2024-06-03T02:17:22.187178+0000","last_became_active":"2024-06-03T02:14:09.769029+0000","last_became_peered":"2024-06-03T02:14:09.769029+0000","last_unstale":"2024-06-03T02:17:22.187178+0000","last_undegraded":"2024-06-03T02:17:22.187178+0000","last_fullsized":"2024-06-03T02:17:22.187178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:24.751039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181320+0000","last_change":"2024-06-03T02:14:09.768584+0000","last_active":"2024-06-03T02:17:22.181320+0000","last_peered":"2024-06-03T02:17:22.181320+0000","last_clean":"2024-06-03T02:17:22.181320+0000","last_became_active":"2024-06-03T02:14:09.765737+0000","last_became_peered":"2024-06-03T02:14:09.765737+0000","last_unstale":"2024-06-03T02:17:22.181320+0000","last_undegraded":"2024-06-03T02:17:22.181320+0000","last_fullsized":"2024-06-03T02:17:22.181320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:36:18.807762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181401+0000","last_change":"2024-06-03T02:14:09.757828+0000","last_active":"2024-06-03T02:17:22.181401+0000","last_peered":"2024-06-03T02:17:22.181401+0000","last_clean":"2024-06-03T02:17:22.181401+0000","last_became_active":"2024-06-03T02:14:09.757040+0000","last_became_peered":"2024-06-03T02:14:09.757040+0000","last_unstale":"2024-06-03T02:17:22.181401+0000","last_undegraded":"2024-06-03T02:17:22.181401+0000","last_fullsized":"2024-06-03T02:17:22.181401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:26:28.694022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188574+0000","last_change":"2024-06-03T02:14:09.768263+0000","last_active":"2024-06-03T02:17:22.188574+0000","last_peered":"2024-06-03T02:17:22.188574+0000","last_clean":"2024-06-03T02:17:22.188574+0000","last_became_active":"2024-06-03T02:14:09.767988+0000","last_became_peered":"2024-06-03T02:14:09.767988+0000","last_unstale":"2024-06-03T02:17:22.188574+0000","last_undegraded":"2024-06-03T02:17:22.188574+0000","last_fullsized":"2024-06-03T02:17:22.188574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:28:37.432659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171811+0000","last_change":"2024-06-03T02:14:09.747118+0000","last_active":"2024-06-03T02:17:22.171811+0000","last_peered":"2024-06-03T02:17:22.171811+0000","last_clean":"2024-06-03T02:17:22.171811+0000","last_became_active":"2024-06-03T02:14:09.746580+0000","last_became_peered":"2024-06-03T02:14:09.746580+0000","last_unstale":"2024-06-03T02:17:22.171811+0000","last_undegraded":"2024-06-03T02:17:22.171811+0000","last_fullsized":"2024-06-03T02:17:22.171811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:33:04.607285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185388+0000","last_change":"2024-06-03T02:14:09.755755+0000","last_active":"2024-06-03T02:17:22.185388+0000","last_peered":"2024-06-03T02:17:22.185388+0000","last_clean":"2024-06-03T02:17:22.185388+0000","last_became_active":"2024-06-03T02:14:09.754503+0000","last_became_peered":"2024-06-03T02:14:09.754503+0000","last_unstale":"2024-06-03T02:17:22.185388+0000","last_undegraded":"2024-06-03T02:17:22.185388+0000","last_fullsized":"2024-06-03T02:17:22.185388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:30:58.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:52.213243+0000","last_change":"2024-06-03T02:14:23.260021+0000","last_active":"2024-06-03T02:16:52.213243+0000","last_peered":"2024-06-03T02:16:52.213243+0000","last_clean":"2024-06-03T02:16:52.213243+0000","last_became_active":"2024-06-03T02:14:23.259587+0000","last_became_peered":"2024-06-03T02:14:23.259587+0000","last_unstale":"2024-06-03T02:16:52.213243+0000","last_undegraded":"2024-06-03T02:16:52.213243+0000","last_fullsized":"2024-06-03T02:16:52.213243+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:09:02.403936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"101'14","reported_seq":291,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:24.835941+0000","last_change":"2024-06-03T02:14:05.239623+0000","last_active":"2024-06-03T02:19:24.835941+0000","last_peered":"2024-06-03T02:19:24.835941+0000","last_clean":"2024-06-03T02:19:24.835941+0000","last_became_active":"2024-06-03T02:14:05.239112+0000","last_became_peered":"2024-06-03T02:14:05.239112+0000","last_unstale":"2024-06-03T02:19:24.835941+0000","last_undegraded":"2024-06-03T02:19:24.835941+0000","last_fullsized":"2024-06-03T02:19:24.835941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:12:14.020068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":231,"num_read_kb":231,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185036+0000","last_change":"2024-06-03T02:14:09.764972+0000","last_active":"2024-06-03T02:17:22.185036+0000","last_peered":"2024-06-03T02:17:22.185036+0000","last_clean":"2024-06-03T02:17:22.185036+0000","last_became_active":"2024-06-03T02:14:09.764435+0000","last_became_peered":"2024-06-03T02:14:09.764435+0000","last_unstale":"2024-06-03T02:17:22.185036+0000","last_undegraded":"2024-06-03T02:17:22.185036+0000","last_fullsized":"2024-06-03T02:17:22.185036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:46:58.941554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.200287+0000","last_change":"2024-06-03T02:14:23.272076+0000","last_active":"2024-06-03T02:16:51.200287+0000","last_peered":"2024-06-03T02:16:51.200287+0000","last_clean":"2024-06-03T02:16:51.200287+0000","last_became_active":"2024-06-03T02:14:23.271578+0000","last_became_peered":"2024-06-03T02:14:23.271578+0000","last_unstale":"2024-06-03T02:16:51.200287+0000","last_undegraded":"2024-06-03T02:16:51.200287+0000","last_fullsized":"2024-06-03T02:16:51.200287+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:17:59.000291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"99'2","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:09.237250+0000","last_change":"2024-06-03T02:14:05.251952+0000","last_active":"2024-06-03T02:16:09.237250+0000","last_peered":"2024-06-03T02:16:09.237250+0000","last_clean":"2024-06-03T02:16:09.237250+0000","last_became_active":"2024-06-03T02:14:05.251152+0000","last_became_peered":"2024-06-03T02:14:05.251152+0000","last_unstale":"2024-06-03T02:16:09.237250+0000","last_undegraded":"2024-06-03T02:16:09.237250+0000","last_fullsized":"2024-06-03T02:16:09.237250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:24:32.802236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.190358+0000","last_change":"2024-06-03T02:14:09.764230+0000","last_active":"2024-06-03T02:17:22.190358+0000","last_peered":"2024-06-03T02:17:22.190358+0000","last_clean":"2024-06-03T02:17:22.190358+0000","last_became_active":"2024-06-03T02:14:09.761868+0000","last_became_peered":"2024-06-03T02:14:09.761868+0000","last_unstale":"2024-06-03T02:17:22.190358+0000","last_undegraded":"2024-06-03T02:17:22.190358+0000","last_fullsized":"2024-06-03T02:17:22.190358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:39:23.828546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:56.200917+0000","last_change":"2024-06-03T02:14:23.272169+0000","last_active":"2024-06-03T02:16:56.200917+0000","last_peered":"2024-06-03T02:16:56.200917+0000","last_clean":"2024-06-03T02:16:56.200917+0000","last_became_active":"2024-06-03T02:14:23.271636+0000","last_became_peered":"2024-06-03T02:14:23.271636+0000","last_unstale":"2024-06-03T02:16:56.200917+0000","last_undegraded":"2024-06-03T02:16:56.200917+0000","last_fullsized":"2024-06-03T02:16:56.200917+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:51:25.478028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"99'12","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:37.186203+0000","last_change":"2024-06-03T02:14:05.238590+0000","last_active":"2024-06-03T02:16:37.186203+0000","last_peered":"2024-06-03T02:16:37.186203+0000","last_clean":"2024-06-03T02:16:37.186203+0000","last_became_active":"2024-06-03T02:14:05.238128+0000","last_became_peered":"2024-06-03T02:14:05.238128+0000","last_unstale":"2024-06-03T02:16:37.186203+0000","last_undegraded":"2024-06-03T02:16:37.186203+0000","last_fullsized":"2024-06-03T02:16:37.186203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:21.298255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174715+0000","last_change":"2024-06-03T02:14:09.754489+0000","last_active":"2024-06-03T02:17:22.174715+0000","last_peered":"2024-06-03T02:17:22.174715+0000","last_clean":"2024-06-03T02:17:22.174715+0000","last_became_active":"2024-06-03T02:14:09.754155+0000","last_became_peered":"2024-06-03T02:14:09.754155+0000","last_unstale":"2024-06-03T02:17:22.174715+0000","last_undegraded":"2024-06-03T02:17:22.174715+0000","last_fullsized":"2024-06-03T02:17:22.174715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:20:43.841271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:56.200884+0000","last_change":"2024-06-03T02:14:23.261232+0000","last_active":"2024-06-03T02:16:56.200884+0000","last_peered":"2024-06-03T02:16:56.200884+0000","last_clean":"2024-06-03T02:16:56.200884+0000","last_became_active":"2024-06-03T02:14:23.260772+0000","last_became_peered":"2024-06-03T02:14:23.260772+0000","last_unstale":"2024-06-03T02:16:56.200884+0000","last_undegraded":"2024-06-03T02:16:56.200884+0000","last_fullsized":"2024-06-03T02:16:56.200884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:52.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"99'5","reported_seq":24,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272244+0000","last_change":"2024-06-03T02:14:53.274084+0000","last_active":"2024-06-03T02:15:54.272244+0000","last_peered":"2024-06-03T02:15:54.272244+0000","last_clean":"2024-06-03T02:15:54.272244+0000","last_became_active":"2024-06-03T02:14:53.265253+0000","last_became_peered":"2024-06-03T02:14:53.265253+0000","last_unstale":"2024-06-03T02:15:54.272244+0000","last_undegraded":"2024-06-03T02:15:54.272244+0000","last_fullsized":"2024-06-03T02:15:54.272244+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:25:56.189308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"101'15","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.156115+0000","last_change":"2024-06-03T02:14:09.743450+0000","last_active":"2024-06-03T02:17:22.156115+0000","last_peered":"2024-06-03T02:17:22.156115+0000","last_clean":"2024-06-03T02:17:22.156115+0000","last_became_active":"2024-06-03T02:14:09.743071+0000","last_became_peered":"2024-06-03T02:14:09.743071+0000","last_unstale":"2024-06-03T02:17:22.156115+0000","last_undegraded":"2024-06-03T02:17:22.156115+0000","last_fullsized":"2024-06-03T02:17:22.156115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:43:44.278633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:57.213954+0000","last_change":"2024-06-03T02:15:54.342978+0000","last_active":"2024-06-03T02:16:57.213954+0000","last_peered":"2024-06-03T02:16:57.213954+0000","last_clean":"2024-06-03T02:16:57.213954+0000","last_became_active":"2024-06-03T02:15:54.342477+0000","last_became_peered":"2024-06-03T02:15:54.342477+0000","last_unstale":"2024-06-03T02:16:57.213954+0000","last_undegraded":"2024-06-03T02:16:57.213954+0000","last_fullsized":"2024-06-03T02:16:57.213954+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:47.980513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"101'8","reported_seq":290,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:25.810333+0000","last_change":"2024-06-03T02:14:05.237464+0000","last_active":"2024-06-03T02:19:25.810333+0000","last_peered":"2024-06-03T02:19:25.810333+0000","last_clean":"2024-06-03T02:19:25.810333+0000","last_became_active":"2024-06-03T02:14:05.237067+0000","last_became_peered":"2024-06-03T02:14:05.237067+0000","last_unstale":"2024-06-03T02:19:25.810333+0000","last_undegraded":"2024-06-03T02:19:25.810333+0000","last_fullsized":"2024-06-03T02:19:25.810333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:06:38.850484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.150011+0000","last_change":"2024-06-03T02:14:09.748741+0000","last_active":"2024-06-03T02:17:22.150011+0000","last_peered":"2024-06-03T02:17:22.150011+0000","last_clean":"2024-06-03T02:17:22.150011+0000","last_became_active":"2024-06-03T02:14:09.748402+0000","last_became_peered":"2024-06-03T02:14:09.748402+0000","last_unstale":"2024-06-03T02:17:22.150011+0000","last_undegraded":"2024-06-03T02:17:22.150011+0000","last_fullsized":"2024-06-03T02:17:22.150011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:03:25.664415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:57.213929+0000","last_change":"2024-06-03T02:14:23.260397+0000","last_active":"2024-06-03T02:16:57.213929+0000","last_peered":"2024-06-03T02:16:57.213929+0000","last_clean":"2024-06-03T02:16:57.213929+0000","last_became_active":"2024-06-03T02:14:23.259886+0000","last_became_peered":"2024-06-03T02:14:23.259886+0000","last_unstale":"2024-06-03T02:16:57.213929+0000","last_undegraded":"2024-06-03T02:16:57.213929+0000","last_fullsized":"2024-06-03T02:16:57.213929+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:46:18.562706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"99'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:38.829788+0000","last_change":"2024-06-03T02:14:05.241536+0000","last_active":"2024-06-03T02:16:38.829788+0000","last_peered":"2024-06-03T02:16:38.829788+0000","last_clean":"2024-06-03T02:16:38.829788+0000","last_became_active":"2024-06-03T02:14:05.241203+0000","last_became_peered":"2024-06-03T02:14:05.241203+0000","last_unstale":"2024-06-03T02:16:38.829788+0000","last_undegraded":"2024-06-03T02:16:38.829788+0000","last_fullsized":"2024-06-03T02:16:38.829788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:47.052563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"101'8","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185241+0000","last_change":"2024-06-03T02:14:53.475893+0000","last_active":"2024-06-03T02:17:22.185241+0000","last_peered":"2024-06-03T02:17:22.185241+0000","last_clean":"2024-06-03T02:17:22.185241+0000","last_became_active":"2024-06-03T02:14:53.475494+0000","last_became_peered":"2024-06-03T02:14:53.475494+0000","last_unstale":"2024-06-03T02:17:22.185241+0000","last_undegraded":"2024-06-03T02:17:22.185241+0000","last_fullsized":"2024-06-03T02:17:22.185241+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:27.113842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:28.911253+0000","last_change":"2024-06-03T02:15:54.344417+0000","last_active":"2024-06-03T02:16:28.911253+0000","last_peered":"2024-06-03T02:16:28.911253+0000","last_clean":"2024-06-03T02:16:28.911253+0000","last_became_active":"2024-06-03T02:15:54.343904+0000","last_became_peered":"2024-06-03T02:15:54.343904+0000","last_unstale":"2024-06-03T02:16:28.911253+0000","last_undegraded":"2024-06-03T02:16:28.911253+0000","last_fullsized":"2024-06-03T02:16:28.911253+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:42:18.121696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"99'3","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.217749+0000","last_change":"2024-06-03T02:14:05.237743+0000","last_active":"2024-06-03T02:16:31.217749+0000","last_peered":"2024-06-03T02:16:31.217749+0000","last_clean":"2024-06-03T02:16:31.217749+0000","last_became_active":"2024-06-03T02:14:05.236944+0000","last_became_peered":"2024-06-03T02:14:05.236944+0000","last_unstale":"2024-06-03T02:16:31.217749+0000","last_undegraded":"2024-06-03T02:16:31.217749+0000","last_fullsized":"2024-06-03T02:16:31.217749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:14:09.850862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185781+0000","last_change":"2024-06-03T02:14:09.761694+0000","last_active":"2024-06-03T02:17:22.185781+0000","last_peered":"2024-06-03T02:17:22.185781+0000","last_clean":"2024-06-03T02:17:22.185781+0000","last_became_active":"2024-06-03T02:14:09.761361+0000","last_became_peered":"2024-06-03T02:14:09.761361+0000","last_unstale":"2024-06-03T02:17:22.185781+0000","last_undegraded":"2024-06-03T02:17:22.185781+0000","last_fullsized":"2024-06-03T02:17:22.185781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:19:17.803088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:22.759262+0000","last_change":"2024-06-03T02:14:23.258634+0000","last_active":"2024-06-03T02:16:22.759262+0000","last_peered":"2024-06-03T02:16:22.759262+0000","last_clean":"2024-06-03T02:16:22.759262+0000","last_became_active":"2024-06-03T02:14:23.257799+0000","last_became_peered":"2024-06-03T02:14:23.257799+0000","last_unstale":"2024-06-03T02:16:22.759262+0000","last_undegraded":"2024-06-03T02:16:22.759262+0000","last_fullsized":"2024-06-03T02:16:22.759262+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:38:33.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"99'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.849678+0000","last_change":"2024-06-03T02:14:05.236790+0000","last_active":"2024-06-03T02:16:31.849678+0000","last_peered":"2024-06-03T02:16:31.849678+0000","last_clean":"2024-06-03T02:16:31.849678+0000","last_became_active":"2024-06-03T02:14:05.236419+0000","last_became_peered":"2024-06-03T02:14:05.236419+0000","last_unstale":"2024-06-03T02:16:31.849678+0000","last_undegraded":"2024-06-03T02:16:31.849678+0000","last_fullsized":"2024-06-03T02:16:31.849678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:44:00.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183352+0000","last_change":"2024-06-03T02:14:09.744680+0000","last_active":"2024-06-03T02:17:22.183352+0000","last_peered":"2024-06-03T02:17:22.183352+0000","last_clean":"2024-06-03T02:17:22.183352+0000","last_became_active":"2024-06-03T02:14:09.744250+0000","last_became_peered":"2024-06-03T02:14:09.744250+0000","last_unstale":"2024-06-03T02:17:22.183352+0000","last_undegraded":"2024-06-03T02:17:22.183352+0000","last_fullsized":"2024-06-03T02:17:22.183352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:58.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:44.242362+0000","last_change":"2024-06-03T02:14:23.259325+0000","last_active":"2024-06-03T02:16:44.242362+0000","last_peered":"2024-06-03T02:16:44.242362+0000","last_clean":"2024-06-03T02:16:44.242362+0000","last_became_active":"2024-06-03T02:14:23.258925+0000","last_became_peered":"2024-06-03T02:14:23.258925+0000","last_unstale":"2024-06-03T02:16:44.242362+0000","last_undegraded":"2024-06-03T02:16:44.242362+0000","last_fullsized":"2024-06-03T02:16:44.242362+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:32.038564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"101'7","reported_seq":509,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:23.653249+0000","last_change":"2024-06-03T02:14:05.242690+0000","last_active":"2024-06-03T02:19:23.653249+0000","last_peered":"2024-06-03T02:19:23.653249+0000","last_clean":"2024-06-03T02:19:23.653249+0000","last_became_active":"2024-06-03T02:14:05.242320+0000","last_became_peered":"2024-06-03T02:14:05.242320+0000","last_unstale":"2024-06-03T02:19:23.653249+0000","last_undegraded":"2024-06-03T02:19:23.653249+0000","last_fullsized":"2024-06-03T02:19:23.653249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:02.793186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":456,"num_read_kb":456,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.170769+0000","last_change":"2024-06-03T02:14:09.747146+0000","last_active":"2024-06-03T02:17:22.170769+0000","last_peered":"2024-06-03T02:17:22.170769+0000","last_clean":"2024-06-03T02:17:22.170769+0000","last_became_active":"2024-06-03T02:14:09.746803+0000","last_became_peered":"2024-06-03T02:14:09.746803+0000","last_unstale":"2024-06-03T02:17:22.170769+0000","last_undegraded":"2024-06-03T02:17:22.170769+0000","last_fullsized":"2024-06-03T02:17:22.170769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:04:36.262223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:45.418229+0000","last_change":"2024-06-03T02:14:23.270328+0000","last_active":"2024-06-03T02:16:45.418229+0000","last_peered":"2024-06-03T02:16:45.418229+0000","last_clean":"2024-06-03T02:16:45.418229+0000","last_became_active":"2024-06-03T02:14:23.264189+0000","last_became_peered":"2024-06-03T02:14:23.264189+0000","last_unstale":"2024-06-03T02:16:45.418229+0000","last_undegraded":"2024-06-03T02:16:45.418229+0000","last_fullsized":"2024-06-03T02:16:45.418229+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:36.248884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"99'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:32.401595+0000","last_change":"2024-06-03T02:14:05.238898+0000","last_active":"2024-06-03T02:16:32.401595+0000","last_peered":"2024-06-03T02:16:32.401595+0000","last_clean":"2024-06-03T02:16:32.401595+0000","last_became_active":"2024-06-03T02:14:05.238552+0000","last_became_peered":"2024-06-03T02:14:05.238552+0000","last_unstale":"2024-06-03T02:16:32.401595+0000","last_undegraded":"2024-06-03T02:16:32.401595+0000","last_fullsized":"2024-06-03T02:16:32.401595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:29:33.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.190294+0000","last_change":"2024-06-03T02:14:09.769973+0000","last_active":"2024-06-03T02:17:22.190294+0000","last_peered":"2024-06-03T02:17:22.190294+0000","last_clean":"2024-06-03T02:17:22.190294+0000","last_became_active":"2024-06-03T02:14:09.768984+0000","last_became_peered":"2024-06-03T02:14:09.768984+0000","last_unstale":"2024-06-03T02:17:22.190294+0000","last_undegraded":"2024-06-03T02:17:22.190294+0000","last_fullsized":"2024-06-03T02:17:22.190294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:48:33.720441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:46.199590+0000","last_change":"2024-06-03T02:14:23.265947+0000","last_active":"2024-06-03T02:16:46.199590+0000","last_peered":"2024-06-03T02:16:46.199590+0000","last_clean":"2024-06-03T02:16:46.199590+0000","last_became_active":"2024-06-03T02:14:23.265412+0000","last_became_peered":"2024-06-03T02:14:23.265412+0000","last_unstale":"2024-06-03T02:16:46.199590+0000","last_undegraded":"2024-06-03T02:16:46.199590+0000","last_fullsized":"2024-06-03T02:16:46.199590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:35:08.664884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"99'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.849724+0000","last_change":"2024-06-03T02:14:05.252751+0000","last_active":"2024-06-03T02:16:31.849724+0000","last_peered":"2024-06-03T02:16:31.849724+0000","last_clean":"2024-06-03T02:16:31.849724+0000","last_became_active":"2024-06-03T02:14:05.252267+0000","last_became_peered":"2024-06-03T02:14:05.252267+0000","last_unstale":"2024-06-03T02:16:31.849724+0000","last_undegraded":"2024-06-03T02:16:31.849724+0000","last_fullsized":"2024-06-03T02:16:31.849724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:58.091711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171446+0000","last_change":"2024-06-03T02:14:09.750757+0000","last_active":"2024-06-03T02:17:22.171446+0000","last_peered":"2024-06-03T02:17:22.171446+0000","last_clean":"2024-06-03T02:17:22.171446+0000","last_became_active":"2024-06-03T02:14:09.750414+0000","last_became_peered":"2024-06-03T02:14:09.750414+0000","last_unstale":"2024-06-03T02:17:22.171446+0000","last_undegraded":"2024-06-03T02:17:22.171446+0000","last_fullsized":"2024-06-03T02:17:22.171446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:29:55.435408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:45.418260+0000","last_change":"2024-06-03T02:14:23.263440+0000","last_active":"2024-06-03T02:16:45.418260+0000","last_peered":"2024-06-03T02:16:45.418260+0000","last_clean":"2024-06-03T02:16:45.418260+0000","last_became_active":"2024-06-03T02:14:23.254019+0000","last_became_peered":"2024-06-03T02:14:23.254019+0000","last_unstale":"2024-06-03T02:16:45.418260+0000","last_undegraded":"2024-06-03T02:16:45.418260+0000","last_fullsized":"2024-06-03T02:16:45.418260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:48:58.745990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'42","reported_seq":557,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:23.654352+0000","last_change":"2024-06-03T02:14:05.239353+0000","last_active":"2024-06-03T02:19:23.654352+0000","last_peered":"2024-06-03T02:19:23.654352+0000","last_clean":"2024-06-03T02:19:23.654352+0000","last_became_active":"2024-06-03T02:14:05.238976+0000","last_became_peered":"2024-06-03T02:14:05.238976+0000","last_unstale":"2024-06-03T02:19:23.654352+0000","last_undegraded":"2024-06-03T02:19:23.654352+0000","last_fullsized":"2024-06-03T02:19:23.654352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:51.598533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72797,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":469,"num_read_kb":305,"num_write":44,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.145081+0000","last_change":"2024-06-03T02:14:09.746546+0000","last_active":"2024-06-03T02:17:22.145081+0000","last_peered":"2024-06-03T02:17:22.145081+0000","last_clean":"2024-06-03T02:17:22.145081+0000","last_became_active":"2024-06-03T02:14:09.746049+0000","last_became_peered":"2024-06-03T02:14:09.746049+0000","last_unstale":"2024-06-03T02:17:22.145081+0000","last_undegraded":"2024-06-03T02:17:22.145081+0000","last_fullsized":"2024-06-03T02:17:22.145081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:34.667133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.851868+0000","last_change":"2024-06-03T02:14:23.257296+0000","last_active":"2024-06-03T02:16:51.851868+0000","last_peered":"2024-06-03T02:16:51.851868+0000","last_clean":"2024-06-03T02:16:51.851868+0000","last_became_active":"2024-06-03T02:14:23.256858+0000","last_became_peered":"2024-06-03T02:14:23.256858+0000","last_unstale":"2024-06-03T02:16:51.851868+0000","last_undegraded":"2024-06-03T02:16:51.851868+0000","last_fullsized":"2024-06-03T02:16:51.851868+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:00.464902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"99'9","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:27.210855+0000","last_change":"2024-06-03T02:14:05.245163+0000","last_active":"2024-06-03T02:16:27.210855+0000","last_peered":"2024-06-03T02:16:27.210855+0000","last_clean":"2024-06-03T02:16:27.210855+0000","last_became_active":"2024-06-03T02:14:05.244813+0000","last_became_peered":"2024-06-03T02:14:05.244813+0000","last_unstale":"2024-06-03T02:16:27.210855+0000","last_undegraded":"2024-06-03T02:16:27.210855+0000","last_fullsized":"2024-06-03T02:16:27.210855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:46:47.017859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181707+0000","last_change":"2024-06-03T02:14:09.757200+0000","last_active":"2024-06-03T02:17:22.181707+0000","last_peered":"2024-06-03T02:17:22.181707+0000","last_clean":"2024-06-03T02:17:22.181707+0000","last_became_active":"2024-06-03T02:14:09.756380+0000","last_became_peered":"2024-06-03T02:14:09.756380+0000","last_unstale":"2024-06-03T02:17:22.181707+0000","last_undegraded":"2024-06-03T02:17:22.181707+0000","last_fullsized":"2024-06-03T02:17:22.181707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:56.045338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:43.913099+0000","last_change":"2024-06-03T02:14:23.258441+0000","last_active":"2024-06-03T02:16:43.913099+0000","last_peered":"2024-06-03T02:16:43.913099+0000","last_clean":"2024-06-03T02:16:43.913099+0000","last_became_active":"2024-06-03T02:14:23.258015+0000","last_became_peered":"2024-06-03T02:14:23.258015+0000","last_unstale":"2024-06-03T02:16:43.913099+0000","last_undegraded":"2024-06-03T02:16:43.913099+0000","last_fullsized":"2024-06-03T02:16:43.913099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:03.123511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"99'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:26.849234+0000","last_change":"2024-06-03T02:14:05.240456+0000","last_active":"2024-06-03T02:16:26.849234+0000","last_peered":"2024-06-03T02:16:26.849234+0000","last_clean":"2024-06-03T02:16:26.849234+0000","last_became_active":"2024-06-03T02:14:05.240078+0000","last_became_peered":"2024-06-03T02:14:05.240078+0000","last_unstale":"2024-06-03T02:16:26.849234+0000","last_undegraded":"2024-06-03T02:16:26.849234+0000","last_fullsized":"2024-06-03T02:16:26.849234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:00:26.491773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"101'15","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179670+0000","last_change":"2024-06-03T02:14:09.750876+0000","last_active":"2024-06-03T02:17:22.179670+0000","last_peered":"2024-06-03T02:17:22.179670+0000","last_clean":"2024-06-03T02:17:22.179670+0000","last_became_active":"2024-06-03T02:14:09.750547+0000","last_became_peered":"2024-06-03T02:14:09.750547+0000","last_unstale":"2024-06-03T02:17:22.179670+0000","last_undegraded":"2024-06-03T02:17:22.179670+0000","last_fullsized":"2024-06-03T02:17:22.179670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:08:41.136225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.241201+0000","last_change":"2024-06-03T02:14:23.276172+0000","last_active":"2024-06-03T02:16:51.241201+0000","last_peered":"2024-06-03T02:16:51.241201+0000","last_clean":"2024-06-03T02:16:51.241201+0000","last_became_active":"2024-06-03T02:14:23.275410+0000","last_became_peered":"2024-06-03T02:14:23.275410+0000","last_unstale":"2024-06-03T02:16:51.241201+0000","last_undegraded":"2024-06-03T02:16:51.241201+0000","last_fullsized":"2024-06-03T02:16:51.241201+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:16:30.859453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"101'8","reported_seq":289,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:24.802602+0000","last_change":"2024-06-03T02:14:05.237263+0000","last_active":"2024-06-03T02:19:24.802602+0000","last_peered":"2024-06-03T02:19:24.802602+0000","last_clean":"2024-06-03T02:19:24.802602+0000","last_became_active":"2024-06-03T02:14:05.236890+0000","last_became_peered":"2024-06-03T02:14:05.236890+0000","last_unstale":"2024-06-03T02:19:24.802602+0000","last_undegraded":"2024-06-03T02:19:24.802602+0000","last_fullsized":"2024-06-03T02:19:24.802602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:42.292545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171587+0000","last_change":"2024-06-03T02:14:09.746260+0000","last_active":"2024-06-03T02:17:22.171587+0000","last_peered":"2024-06-03T02:17:22.171587+0000","last_clean":"2024-06-03T02:17:22.171587+0000","last_became_active":"2024-06-03T02:14:09.745940+0000","last_became_peered":"2024-06-03T02:14:09.745940+0000","last_unstale":"2024-06-03T02:17:22.171587+0000","last_undegraded":"2024-06-03T02:17:22.171587+0000","last_fullsized":"2024-06-03T02:17:22.171587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:08:55.851464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.220996+0000","last_change":"2024-06-03T02:14:23.265440+0000","last_active":"2024-06-03T02:16:51.220996+0000","last_peered":"2024-06-03T02:16:51.220996+0000","last_clean":"2024-06-03T02:16:51.220996+0000","last_became_active":"2024-06-03T02:14:23.265028+0000","last_became_peered":"2024-06-03T02:14:23.265028+0000","last_unstale":"2024-06-03T02:16:51.220996+0000","last_undegraded":"2024-06-03T02:16:51.220996+0000","last_fullsized":"2024-06-03T02:16:51.220996+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:56.237500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"99'17","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:27.185001+0000","last_change":"2024-06-03T02:14:53.271821+0000","last_active":"2024-06-03T02:16:27.185001+0000","last_peered":"2024-06-03T02:16:27.185001+0000","last_clean":"2024-06-03T02:16:27.185001+0000","last_became_active":"2024-06-03T02:14:53.263047+0000","last_became_peered":"2024-06-03T02:14:53.263047+0000","last_unstale":"2024-06-03T02:16:27.185001+0000","last_undegraded":"2024-06-03T02:16:27.185001+0000","last_fullsized":"2024-06-03T02:16:27.185001+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:02:25.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.149642+0000","last_change":"2024-06-03T02:14:09.754819+0000","last_active":"2024-06-03T02:17:22.149642+0000","last_peered":"2024-06-03T02:17:22.149642+0000","last_clean":"2024-06-03T02:17:22.149642+0000","last_became_active":"2024-06-03T02:14:09.753995+0000","last_became_peered":"2024-06-03T02:14:09.753995+0000","last_unstale":"2024-06-03T02:17:22.149642+0000","last_undegraded":"2024-06-03T02:17:22.149642+0000","last_fullsized":"2024-06-03T02:17:22.149642+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:19:51.123774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.850243+0000","last_change":"2024-06-03T02:14:23.265003+0000","last_active":"2024-06-03T02:16:36.850243+0000","last_peered":"2024-06-03T02:16:36.850243+0000","last_clean":"2024-06-03T02:16:36.850243+0000","last_became_active":"2024-06-03T02:14:23.264529+0000","last_became_peered":"2024-06-03T02:14:23.264529+0000","last_unstale":"2024-06-03T02:16:36.850243+0000","last_undegraded":"2024-06-03T02:16:36.850243+0000","last_fullsized":"2024-06-03T02:16:36.850243+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:50:19.838274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:22.184389+0000","last_change":"2024-06-03T02:14:05.236901+0000","last_active":"2024-06-03T02:16:22.184389+0000","last_peered":"2024-06-03T02:16:22.184389+0000","last_clean":"2024-06-03T02:16:22.184389+0000","last_became_active":"2024-06-03T02:14:05.236460+0000","last_became_peered":"2024-06-03T02:14:05.236460+0000","last_unstale":"2024-06-03T02:16:22.184389+0000","last_undegraded":"2024-06-03T02:16:22.184389+0000","last_fullsized":"2024-06-03T02:16:22.184389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:59.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.160124+0000","last_change":"2024-06-03T02:14:09.743135+0000","last_active":"2024-06-03T02:17:22.160124+0000","last_peered":"2024-06-03T02:17:22.160124+0000","last_clean":"2024-06-03T02:17:22.160124+0000","last_became_active":"2024-06-03T02:14:09.742506+0000","last_became_peered":"2024-06-03T02:14:09.742506+0000","last_unstale":"2024-06-03T02:17:22.160124+0000","last_undegraded":"2024-06-03T02:17:22.160124+0000","last_fullsized":"2024-06-03T02:17:22.160124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:51.413537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:32.185624+0000","last_change":"2024-06-03T02:14:23.255512+0000","last_active":"2024-06-03T02:16:32.185624+0000","last_peered":"2024-06-03T02:16:32.185624+0000","last_clean":"2024-06-03T02:16:32.185624+0000","last_became_active":"2024-06-03T02:14:23.255103+0000","last_became_peered":"2024-06-03T02:14:23.255103+0000","last_unstale":"2024-06-03T02:16:32.185624+0000","last_undegraded":"2024-06-03T02:16:32.185624+0000","last_fullsized":"2024-06-03T02:16:32.185624+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:12:05.295848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"99'1","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:11.235176+0000","last_change":"2024-06-03T02:14:05.248728+0000","last_active":"2024-06-03T02:16:11.235176+0000","last_peered":"2024-06-03T02:16:11.235176+0000","last_clean":"2024-06-03T02:16:11.235176+0000","last_became_active":"2024-06-03T02:14:05.248144+0000","last_became_peered":"2024-06-03T02:14:05.248144+0000","last_unstale":"2024-06-03T02:16:11.235176+0000","last_undegraded":"2024-06-03T02:16:11.235176+0000","last_fullsized":"2024-06-03T02:16:11.235176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:53:27.814443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184912+0000","last_change":"2024-06-03T02:14:09.752029+0000","last_active":"2024-06-03T02:17:22.184912+0000","last_peered":"2024-06-03T02:17:22.184912+0000","last_clean":"2024-06-03T02:17:22.184912+0000","last_became_active":"2024-06-03T02:14:09.751638+0000","last_became_peered":"2024-06-03T02:14:09.751638+0000","last_unstale":"2024-06-03T02:17:22.184912+0000","last_undegraded":"2024-06-03T02:17:22.184912+0000","last_fullsized":"2024-06-03T02:17:22.184912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:51.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.217797+0000","last_change":"2024-06-03T02:14:23.250406+0000","last_active":"2024-06-03T02:16:31.217797+0000","last_peered":"2024-06-03T02:16:31.217797+0000","last_clean":"2024-06-03T02:16:31.217797+0000","last_became_active":"2024-06-03T02:14:23.249946+0000","last_became_peered":"2024-06-03T02:14:23.249946+0000","last_unstale":"2024-06-03T02:16:31.217797+0000","last_undegraded":"2024-06-03T02:16:31.217797+0000","last_fullsized":"2024-06-03T02:16:31.217797+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:04:47.714532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"101'7","reported_seq":287,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:25.839775+0000","last_change":"2024-06-03T02:14:05.241666+0000","last_active":"2024-06-03T02:19:25.839775+0000","last_peered":"2024-06-03T02:19:25.839775+0000","last_clean":"2024-06-03T02:19:25.839775+0000","last_became_active":"2024-06-03T02:14:05.241299+0000","last_became_peered":"2024-06-03T02:14:05.241299+0000","last_unstale":"2024-06-03T02:19:25.839775+0000","last_undegraded":"2024-06-03T02:19:25.839775+0000","last_fullsized":"2024-06-03T02:19:25.839775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:21:16.792022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":234,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.176295+0000","last_change":"2024-06-03T02:14:09.768602+0000","last_active":"2024-06-03T02:17:22.176295+0000","last_peered":"2024-06-03T02:17:22.176295+0000","last_clean":"2024-06-03T02:17:22.176295+0000","last_became_active":"2024-06-03T02:14:09.768195+0000","last_became_peered":"2024-06-03T02:14:09.768195+0000","last_unstale":"2024-06-03T02:17:22.176295+0000","last_undegraded":"2024-06-03T02:17:22.176295+0000","last_fullsized":"2024-06-03T02:17:22.176295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:45:42.361966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.218658+0000","last_change":"2024-06-03T02:14:23.252472+0000","last_active":"2024-06-03T02:16:36.218658+0000","last_peered":"2024-06-03T02:16:36.218658+0000","last_clean":"2024-06-03T02:16:36.218658+0000","last_became_active":"2024-06-03T02:14:23.252018+0000","last_became_peered":"2024-06-03T02:14:23.252018+0000","last_unstale":"2024-06-03T02:16:36.218658+0000","last_undegraded":"2024-06-03T02:16:36.218658+0000","last_fullsized":"2024-06-03T02:16:36.218658+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:28.468657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163186+0000","last_change":"2024-06-03T02:14:09.744443+0000","last_active":"2024-06-03T02:17:22.163186+0000","last_peered":"2024-06-03T02:17:22.163186+0000","last_clean":"2024-06-03T02:17:22.163186+0000","last_became_active":"2024-06-03T02:14:09.744100+0000","last_became_peered":"2024-06-03T02:14:09.744100+0000","last_unstale":"2024-06-03T02:17:22.163186+0000","last_undegraded":"2024-06-03T02:17:22.163186+0000","last_fullsized":"2024-06-03T02:17:22.163186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:44.566745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"101'7","reported_seq":284,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:24.801818+0000","last_change":"2024-06-03T02:14:05.250734+0000","last_active":"2024-06-03T02:19:24.801818+0000","last_peered":"2024-06-03T02:19:24.801818+0000","last_clean":"2024-06-03T02:19:24.801818+0000","last_became_active":"2024-06-03T02:14:05.250300+0000","last_became_peered":"2024-06-03T02:14:05.250300+0000","last_unstale":"2024-06-03T02:19:24.801818+0000","last_undegraded":"2024-06-03T02:19:24.801818+0000","last_fullsized":"2024-06-03T02:19:24.801818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:52.322895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":231,"num_read_kb":231,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:27.401054+0000","last_change":"2024-06-03T02:14:53.471397+0000","last_active":"2024-06-03T02:16:27.401054+0000","last_peered":"2024-06-03T02:16:27.401054+0000","last_clean":"2024-06-03T02:16:27.401054+0000","last_became_active":"2024-06-03T02:14:53.470868+0000","last_became_peered":"2024-06-03T02:14:53.470868+0000","last_unstale":"2024-06-03T02:16:27.401054+0000","last_undegraded":"2024-06-03T02:16:27.401054+0000","last_fullsized":"2024-06-03T02:16:27.401054+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:58.726070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188260+0000","last_change":"2024-06-03T02:14:09.765193+0000","last_active":"2024-06-03T02:17:22.188260+0000","last_peered":"2024-06-03T02:17:22.188260+0000","last_clean":"2024-06-03T02:17:22.188260+0000","last_became_active":"2024-06-03T02:14:09.764834+0000","last_became_peered":"2024-06-03T02:14:09.764834+0000","last_unstale":"2024-06-03T02:17:22.188260+0000","last_undegraded":"2024-06-03T02:17:22.188260+0000","last_fullsized":"2024-06-03T02:17:22.188260+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:20:37.464286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"99'6","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:11.195852+0000","last_change":"2024-06-03T02:14:05.240118+0000","last_active":"2024-06-03T02:16:11.195852+0000","last_peered":"2024-06-03T02:16:11.195852+0000","last_clean":"2024-06-03T02:16:11.195852+0000","last_became_active":"2024-06-03T02:14:05.239771+0000","last_became_peered":"2024-06-03T02:14:05.239771+0000","last_unstale":"2024-06-03T02:16:11.195852+0000","last_undegraded":"2024-06-03T02:16:11.195852+0000","last_fullsized":"2024-06-03T02:16:11.195852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:08.256222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":372,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:07.405163+0000","last_change":"2024-06-03T02:12:31.842613+0000","last_active":"2024-06-03T02:17:07.405163+0000","last_peered":"2024-06-03T02:17:07.405163+0000","last_clean":"2024-06-03T02:17:07.405163+0000","last_became_active":"2024-06-03T02:12:31.842051+0000","last_became_peered":"2024-06-03T02:12:31.842051+0000","last_unstale":"2024-06-03T02:17:07.405163+0000","last_undegraded":"2024-06-03T02:17:07.405163+0000","last_fullsized":"2024-06-03T02:17:07.405163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_clean_scrub_stamp":"2024-06-03T02:08:38.749702+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:40:29.748789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.850310+0000","last_change":"2024-06-03T02:14:23.255849+0000","last_active":"2024-06-03T02:16:36.850310+0000","last_peered":"2024-06-03T02:16:36.850310+0000","last_clean":"2024-06-03T02:16:36.850310+0000","last_became_active":"2024-06-03T02:14:23.255380+0000","last_became_peered":"2024-06-03T02:14:23.255380+0000","last_unstale":"2024-06-03T02:16:36.850310+0000","last_undegraded":"2024-06-03T02:16:36.850310+0000","last_fullsized":"2024-06-03T02:16:36.850310+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:41:16.051326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"101'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.169554+0000","last_change":"2024-06-03T02:14:53.270423+0000","last_active":"2024-06-03T02:17:22.169554+0000","last_peered":"2024-06-03T02:17:22.169554+0000","last_clean":"2024-06-03T02:17:22.169554+0000","last_became_active":"2024-06-03T02:14:53.269260+0000","last_became_peered":"2024-06-03T02:14:53.269260+0000","last_unstale":"2024-06-03T02:17:22.169554+0000","last_undegraded":"2024-06-03T02:17:22.169554+0000","last_fullsized":"2024-06-03T02:17:22.169554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:57.392012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"99'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:07.399193+0000","last_change":"2024-06-03T02:14:05.235982+0000","last_active":"2024-06-03T02:16:07.399193+0000","last_peered":"2024-06-03T02:16:07.399193+0000","last_clean":"2024-06-03T02:16:07.399193+0000","last_became_active":"2024-06-03T02:14:05.235638+0000","last_became_peered":"2024-06-03T02:14:05.235638+0000","last_unstale":"2024-06-03T02:16:07.399193+0000","last_undegraded":"2024-06-03T02:16:07.399193+0000","last_fullsized":"2024-06-03T02:16:07.399193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:42:07.544005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:28.911203+0000","last_change":"2024-06-03T02:14:23.257790+0000","last_active":"2024-06-03T02:16:28.911203+0000","last_peered":"2024-06-03T02:16:28.911203+0000","last_clean":"2024-06-03T02:16:28.911203+0000","last_became_active":"2024-06-03T02:14:23.257297+0000","last_became_peered":"2024-06-03T02:14:23.257297+0000","last_unstale":"2024-06-03T02:16:28.911203+0000","last_undegraded":"2024-06-03T02:16:28.911203+0000","last_fullsized":"2024-06-03T02:16:28.911203+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:13.715455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"99'4","reported_seq":22,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:58.914613+0000","last_change":"2024-06-03T02:14:53.268495+0000","last_active":"2024-06-03T02:16:58.914613+0000","last_peered":"2024-06-03T02:16:58.914613+0000","last_clean":"2024-06-03T02:16:58.914613+0000","last_became_active":"2024-06-03T02:14:53.268147+0000","last_became_peered":"2024-06-03T02:14:53.268147+0000","last_unstale":"2024-06-03T02:16:58.914613+0000","last_undegraded":"2024-06-03T02:16:58.914613+0000","last_fullsized":"2024-06-03T02:16:58.914613+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:23.076319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172914+0000","last_change":"2024-06-03T02:14:09.758244+0000","last_active":"2024-06-03T02:17:22.172914+0000","last_peered":"2024-06-03T02:17:22.172914+0000","last_clean":"2024-06-03T02:17:22.172914+0000","last_became_active":"2024-06-03T02:14:09.757812+0000","last_became_peered":"2024-06-03T02:14:09.757812+0000","last_unstale":"2024-06-03T02:17:22.172914+0000","last_undegraded":"2024-06-03T02:17:22.172914+0000","last_fullsized":"2024-06-03T02:17:22.172914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:08.243703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:24.239124+0000","last_change":"2024-06-03T02:14:23.259980+0000","last_active":"2024-06-03T02:16:24.239124+0000","last_peered":"2024-06-03T02:16:24.239124+0000","last_clean":"2024-06-03T02:16:24.239124+0000","last_became_active":"2024-06-03T02:14:23.258308+0000","last_became_peered":"2024-06-03T02:14:23.258308+0000","last_unstale":"2024-06-03T02:16:24.239124+0000","last_undegraded":"2024-06-03T02:16:24.239124+0000","last_fullsized":"2024-06-03T02:16:24.239124+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:28.833994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'19","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:21.815379+0000","last_change":"2024-06-03T02:14:05.244855+0000","last_active":"2024-06-03T02:19:21.815379+0000","last_peered":"2024-06-03T02:19:21.815379+0000","last_clean":"2024-06-03T02:19:21.815379+0000","last_became_active":"2024-06-03T02:14:05.244335+0000","last_became_peered":"2024-06-03T02:14:05.244335+0000","last_unstale":"2024-06-03T02:19:21.815379+0000","last_undegraded":"2024-06-03T02:19:21.815379+0000","last_fullsized":"2024-06-03T02:19:21.815379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:53:19.065411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185900+0000","last_change":"2024-06-03T02:14:09.750761+0000","last_active":"2024-06-03T02:17:22.185900+0000","last_peered":"2024-06-03T02:17:22.185900+0000","last_clean":"2024-06-03T02:17:22.185900+0000","last_became_active":"2024-06-03T02:14:09.750336+0000","last_became_peered":"2024-06-03T02:14:09.750336+0000","last_unstale":"2024-06-03T02:17:22.185900+0000","last_undegraded":"2024-06-03T02:17:22.185900+0000","last_fullsized":"2024-06-03T02:17:22.185900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:59:46.651149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:32.211290+0000","last_change":"2024-06-03T02:14:23.243287+0000","last_active":"2024-06-03T02:16:32.211290+0000","last_peered":"2024-06-03T02:16:32.211290+0000","last_clean":"2024-06-03T02:16:32.211290+0000","last_became_active":"2024-06-03T02:14:23.242828+0000","last_became_peered":"2024-06-03T02:14:23.242828+0000","last_unstale":"2024-06-03T02:16:32.211290+0000","last_undegraded":"2024-06-03T02:16:32.211290+0000","last_fullsized":"2024-06-03T02:16:32.211290+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:12.312438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"99'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:16.196453+0000","last_change":"2024-06-03T02:14:05.596288+0000","last_active":"2024-06-03T02:16:16.196453+0000","last_peered":"2024-06-03T02:16:16.196453+0000","last_clean":"2024-06-03T02:16:16.196453+0000","last_became_active":"2024-06-03T02:14:05.595868+0000","last_became_peered":"2024-06-03T02:14:05.595868+0000","last_unstale":"2024-06-03T02:16:16.196453+0000","last_undegraded":"2024-06-03T02:16:16.196453+0000","last_fullsized":"2024-06-03T02:16:16.196453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:50.291541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.189478+0000","last_change":"2024-06-03T02:14:09.765317+0000","last_active":"2024-06-03T02:17:22.189478+0000","last_peered":"2024-06-03T02:17:22.189478+0000","last_clean":"2024-06-03T02:17:22.189478+0000","last_became_active":"2024-06-03T02:14:09.764983+0000","last_became_peered":"2024-06-03T02:14:09.764983+0000","last_unstale":"2024-06-03T02:17:22.189478+0000","last_undegraded":"2024-06-03T02:17:22.189478+0000","last_fullsized":"2024-06-03T02:17:22.189478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:20.075276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:41.198968+0000","last_change":"2024-06-03T02:14:23.265352+0000","last_active":"2024-06-03T02:16:41.198968+0000","last_peered":"2024-06-03T02:16:41.198968+0000","last_clean":"2024-06-03T02:16:41.198968+0000","last_became_active":"2024-06-03T02:14:23.264932+0000","last_became_peered":"2024-06-03T02:14:23.264932+0000","last_unstale":"2024-06-03T02:16:41.198968+0000","last_undegraded":"2024-06-03T02:16:41.198968+0000","last_fullsized":"2024-06-03T02:16:41.198968+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:43.803170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"99'2","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:18.326424+0000","last_change":"2024-06-03T02:14:05.248563+0000","last_active":"2024-06-03T02:16:18.326424+0000","last_peered":"2024-06-03T02:16:18.326424+0000","last_clean":"2024-06-03T02:16:18.326424+0000","last_became_active":"2024-06-03T02:14:05.248306+0000","last_became_peered":"2024-06-03T02:14:05.248306+0000","last_unstale":"2024-06-03T02:16:18.326424+0000","last_undegraded":"2024-06-03T02:16:18.326424+0000","last_fullsized":"2024-06-03T02:16:18.326424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:23:58.130157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"101'2","reported_seq":42,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.149643+0000","last_change":"2024-06-03T02:14:09.745948+0000","last_active":"2024-06-03T02:17:22.149643+0000","last_peered":"2024-06-03T02:17:22.149643+0000","last_clean":"2024-06-03T02:17:22.149643+0000","last_became_active":"2024-06-03T02:14:09.745626+0000","last_became_peered":"2024-06-03T02:14:09.745626+0000","last_unstale":"2024-06-03T02:17:22.149643+0000","last_undegraded":"2024-06-03T02:17:22.149643+0000","last_fullsized":"2024-06-03T02:17:22.149643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:34.742129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:57.181841+0000","last_change":"2024-06-03T02:15:54.341824+0000","last_active":"2024-06-03T02:15:57.181841+0000","last_peered":"2024-06-03T02:15:57.181841+0000","last_clean":"2024-06-03T02:15:57.181841+0000","last_became_active":"2024-06-03T02:15:54.341267+0000","last_became_peered":"2024-06-03T02:15:54.341267+0000","last_unstale":"2024-06-03T02:15:57.181841+0000","last_undegraded":"2024-06-03T02:15:57.181841+0000","last_fullsized":"2024-06-03T02:15:57.181841+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:47.714056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"99'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:19.238284+0000","last_change":"2024-06-03T02:14:05.244806+0000","last_active":"2024-06-03T02:16:19.238284+0000","last_peered":"2024-06-03T02:16:19.238284+0000","last_clean":"2024-06-03T02:16:19.238284+0000","last_became_active":"2024-06-03T02:14:05.242719+0000","last_became_peered":"2024-06-03T02:14:05.242719+0000","last_unstale":"2024-06-03T02:16:19.238284+0000","last_undegraded":"2024-06-03T02:16:19.238284+0000","last_fullsized":"2024-06-03T02:16:19.238284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:31.314266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181636+0000","last_change":"2024-06-03T02:14:09.755102+0000","last_active":"2024-06-03T02:17:22.181636+0000","last_peered":"2024-06-03T02:17:22.181636+0000","last_clean":"2024-06-03T02:17:22.181636+0000","last_became_active":"2024-06-03T02:14:09.754748+0000","last_became_peered":"2024-06-03T02:14:09.754748+0000","last_unstale":"2024-06-03T02:17:22.181636+0000","last_undegraded":"2024-06-03T02:17:22.181636+0000","last_fullsized":"2024-06-03T02:17:22.181636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:59.920710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"101'1","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:41.198931+0000","last_change":"2024-06-03T02:14:23.271974+0000","last_active":"2024-06-03T02:16:41.198931+0000","last_peered":"2024-06-03T02:16:41.198931+0000","last_clean":"2024-06-03T02:16:41.198931+0000","last_became_active":"2024-06-03T02:14:23.271470+0000","last_became_peered":"2024-06-03T02:14:23.271470+0000","last_unstale":"2024-06-03T02:16:41.198931+0000","last_undegraded":"2024-06-03T02:16:41.198931+0000","last_fullsized":"2024-06-03T02:16:41.198931+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:03.159351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"101'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:18:27.223978+0000","last_change":"2024-06-03T02:14:05.242261+0000","last_active":"2024-06-03T02:18:27.223978+0000","last_peered":"2024-06-03T02:18:27.223978+0000","last_clean":"2024-06-03T02:18:27.223978+0000","last_became_active":"2024-06-03T02:14:05.241854+0000","last_became_peered":"2024-06-03T02:14:05.241854+0000","last_unstale":"2024-06-03T02:18:27.223978+0000","last_undegraded":"2024-06-03T02:18:27.223978+0000","last_fullsized":"2024-06-03T02:18:27.223978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:21.427703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186160+0000","last_change":"2024-06-03T02:14:09.763591+0000","last_active":"2024-06-03T02:17:22.186160+0000","last_peered":"2024-06-03T02:17:22.186160+0000","last_clean":"2024-06-03T02:17:22.186160+0000","last_became_active":"2024-06-03T02:14:09.763240+0000","last_became_peered":"2024-06-03T02:14:09.763240+0000","last_unstale":"2024-06-03T02:17:22.186160+0000","last_undegraded":"2024-06-03T02:17:22.186160+0000","last_fullsized":"2024-06-03T02:17:22.186160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:40.146898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:41.198957+0000","last_change":"2024-06-03T02:14:23.265011+0000","last_active":"2024-06-03T02:16:41.198957+0000","last_peered":"2024-06-03T02:16:41.198957+0000","last_clean":"2024-06-03T02:16:41.198957+0000","last_became_active":"2024-06-03T02:14:23.263767+0000","last_became_peered":"2024-06-03T02:14:23.263767+0000","last_unstale":"2024-06-03T02:16:41.198957+0000","last_undegraded":"2024-06-03T02:16:41.198957+0000","last_fullsized":"2024-06-03T02:16:41.198957+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:34:04.274545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"99'4","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:22.184364+0000","last_change":"2024-06-03T02:14:53.474879+0000","last_active":"2024-06-03T02:16:22.184364+0000","last_peered":"2024-06-03T02:16:22.184364+0000","last_clean":"2024-06-03T02:16:22.184364+0000","last_became_active":"2024-06-03T02:14:53.472209+0000","last_became_peered":"2024-06-03T02:14:53.472209+0000","last_unstale":"2024-06-03T02:16:22.184364+0000","last_undegraded":"2024-06-03T02:16:22.184364+0000","last_fullsized":"2024-06-03T02:16:22.184364+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:36:57.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172711+0000","last_change":"2024-06-03T02:14:09.747756+0000","last_active":"2024-06-03T02:17:22.172711+0000","last_peered":"2024-06-03T02:17:22.172711+0000","last_clean":"2024-06-03T02:17:22.172711+0000","last_became_active":"2024-06-03T02:14:09.747453+0000","last_became_peered":"2024-06-03T02:14:09.747453+0000","last_unstale":"2024-06-03T02:17:22.172711+0000","last_undegraded":"2024-06-03T02:17:22.172711+0000","last_fullsized":"2024-06-03T02:17:22.172711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:19:02.341367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:33.328239+0000","last_change":"2024-06-03T02:14:23.248594+0000","last_active":"2024-06-03T02:16:33.328239+0000","last_peered":"2024-06-03T02:16:33.328239+0000","last_clean":"2024-06-03T02:16:33.328239+0000","last_became_active":"2024-06-03T02:14:23.248080+0000","last_became_peered":"2024-06-03T02:14:23.248080+0000","last_unstale":"2024-06-03T02:16:33.328239+0000","last_undegraded":"2024-06-03T02:16:33.328239+0000","last_fullsized":"2024-06-03T02:16:33.328239+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:24:42.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"101'6","reported_seq":286,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:25.657346+0000","last_change":"2024-06-03T02:14:05.596274+0000","last_active":"2024-06-03T02:19:25.657346+0000","last_peered":"2024-06-03T02:19:25.657346+0000","last_clean":"2024-06-03T02:19:25.657346+0000","last_became_active":"2024-06-03T02:14:05.595678+0000","last_became_peered":"2024-06-03T02:14:05.595678+0000","last_unstale":"2024-06-03T02:19:25.657346+0000","last_undegraded":"2024-06-03T02:19:25.657346+0000","last_fullsized":"2024-06-03T02:19:25.657346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:13:25.436688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.173298+0000","last_change":"2024-06-03T02:14:09.744934+0000","last_active":"2024-06-03T02:17:22.173298+0000","last_peered":"2024-06-03T02:17:22.173298+0000","last_clean":"2024-06-03T02:17:22.173298+0000","last_became_active":"2024-06-03T02:14:09.744621+0000","last_became_peered":"2024-06-03T02:14:09.744621+0000","last_unstale":"2024-06-03T02:17:22.173298+0000","last_undegraded":"2024-06-03T02:17:22.173298+0000","last_fullsized":"2024-06-03T02:17:22.173298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:01:51.970160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:37.761133+0000","last_change":"2024-06-03T02:14:23.273123+0000","last_active":"2024-06-03T02:16:37.761133+0000","last_peered":"2024-06-03T02:16:37.761133+0000","last_clean":"2024-06-03T02:16:37.761133+0000","last_became_active":"2024-06-03T02:14:23.272153+0000","last_became_peered":"2024-06-03T02:14:23.272153+0000","last_unstale":"2024-06-03T02:16:37.761133+0000","last_undegraded":"2024-06-03T02:16:37.761133+0000","last_fullsized":"2024-06-03T02:16:37.761133+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:01:43.374413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"99'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:21.848723+0000","last_change":"2024-06-03T02:14:05.245094+0000","last_active":"2024-06-03T02:16:21.848723+0000","last_peered":"2024-06-03T02:16:21.848723+0000","last_clean":"2024-06-03T02:16:21.848723+0000","last_became_active":"2024-06-03T02:14:05.244678+0000","last_became_peered":"2024-06-03T02:14:05.244678+0000","last_unstale":"2024-06-03T02:16:21.848723+0000","last_undegraded":"2024-06-03T02:16:21.848723+0000","last_fullsized":"2024-06-03T02:16:21.848723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:13:26.546135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185237+0000","last_change":"2024-06-03T02:14:09.745919+0000","last_active":"2024-06-03T02:17:22.185237+0000","last_peered":"2024-06-03T02:17:22.185237+0000","last_clean":"2024-06-03T02:17:22.185237+0000","last_became_active":"2024-06-03T02:14:09.745576+0000","last_became_peered":"2024-06-03T02:14:09.745576+0000","last_unstale":"2024-06-03T02:17:22.185237+0000","last_undegraded":"2024-06-03T02:17:22.185237+0000","last_fullsized":"2024-06-03T02:17:22.185237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:29:31.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.850220+0000","last_change":"2024-06-03T02:14:23.244174+0000","last_active":"2024-06-03T02:16:36.850220+0000","last_peered":"2024-06-03T02:16:36.850220+0000","last_clean":"2024-06-03T02:16:36.850220+0000","last_became_active":"2024-06-03T02:14:23.243595+0000","last_became_peered":"2024-06-03T02:14:23.243595+0000","last_unstale":"2024-06-03T02:16:36.850220+0000","last_undegraded":"2024-06-03T02:16:36.850220+0000","last_fullsized":"2024-06-03T02:16:36.850220+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:12:36.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"99'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:21.216403+0000","last_change":"2024-06-03T02:14:05.247669+0000","last_active":"2024-06-03T02:16:21.216403+0000","last_peered":"2024-06-03T02:16:21.216403+0000","last_clean":"2024-06-03T02:16:21.216403+0000","last_became_active":"2024-06-03T02:14:05.246685+0000","last_became_peered":"2024-06-03T02:14:05.246685+0000","last_unstale":"2024-06-03T02:16:21.216403+0000","last_undegraded":"2024-06-03T02:16:21.216403+0000","last_fullsized":"2024-06-03T02:16:21.216403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:39.454422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171702+0000","last_change":"2024-06-03T02:14:09.749242+0000","last_active":"2024-06-03T02:17:22.171702+0000","last_peered":"2024-06-03T02:17:22.171702+0000","last_clean":"2024-06-03T02:17:22.171702+0000","last_became_active":"2024-06-03T02:14:09.748370+0000","last_became_peered":"2024-06-03T02:14:09.748370+0000","last_unstale":"2024-06-03T02:17:22.171702+0000","last_undegraded":"2024-06-03T02:17:22.171702+0000","last_fullsized":"2024-06-03T02:17:22.171702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:26.189960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.162116+0000","last_change":"2024-06-03T02:14:09.745966+0000","last_active":"2024-06-03T02:17:22.162116+0000","last_peered":"2024-06-03T02:17:22.162116+0000","last_clean":"2024-06-03T02:17:22.162116+0000","last_became_active":"2024-06-03T02:14:09.745590+0000","last_became_peered":"2024-06-03T02:14:09.745590+0000","last_unstale":"2024-06-03T02:17:22.162116+0000","last_undegraded":"2024-06-03T02:17:22.162116+0000","last_fullsized":"2024-06-03T02:17:22.162116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:46.744629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"101'12","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163485+0000","last_change":"2024-06-03T02:14:09.744964+0000","last_active":"2024-06-03T02:17:22.163485+0000","last_peered":"2024-06-03T02:17:22.163485+0000","last_clean":"2024-06-03T02:17:22.163485+0000","last_became_active":"2024-06-03T02:14:09.743877+0000","last_became_peered":"2024-06-03T02:14:09.743877+0000","last_unstale":"2024-06-03T02:17:22.163485+0000","last_undegraded":"2024-06-03T02:17:22.163485+0000","last_fullsized":"2024-06-03T02:17:22.163485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:05:48.020670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186843+0000","last_change":"2024-06-03T02:14:09.761736+0000","last_active":"2024-06-03T02:17:22.186843+0000","last_peered":"2024-06-03T02:17:22.186843+0000","last_clean":"2024-06-03T02:17:22.186843+0000","last_became_active":"2024-06-03T02:14:09.761477+0000","last_became_peered":"2024-06-03T02:14:09.761477+0000","last_unstale":"2024-06-03T02:17:22.186843+0000","last_undegraded":"2024-06-03T02:17:22.186843+0000","last_fullsized":"2024-06-03T02:17:22.186843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:00:04.631330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"101'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183594+0000","last_change":"2024-06-03T02:14:53.266594+0000","last_active":"2024-06-03T02:17:22.183594+0000","last_peered":"2024-06-03T02:17:22.183594+0000","last_clean":"2024-06-03T02:17:22.183594+0000","last_became_active":"2024-06-03T02:14:53.266104+0000","last_became_peered":"2024-06-03T02:14:53.266104+0000","last_unstale":"2024-06-03T02:17:22.183594+0000","last_undegraded":"2024-06-03T02:17:22.183594+0000","last_fullsized":"2024-06-03T02:17:22.183594+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:12.018082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174909+0000","last_change":"2024-06-03T02:14:09.772332+0000","last_active":"2024-06-03T02:17:22.174909+0000","last_peered":"2024-06-03T02:17:22.174909+0000","last_clean":"2024-06-03T02:17:22.174909+0000","last_became_active":"2024-06-03T02:14:09.771888+0000","last_became_peered":"2024-06-03T02:14:09.771888+0000","last_unstale":"2024-06-03T02:17:22.174909+0000","last_undegraded":"2024-06-03T02:17:22.174909+0000","last_fullsized":"2024-06-03T02:17:22.174909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:58:49.414986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"101'6","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.156930+0000","last_change":"2024-06-03T02:14:53.270828+0000","last_active":"2024-06-03T02:17:22.156930+0000","last_peered":"2024-06-03T02:17:22.156930+0000","last_clean":"2024-06-03T02:17:22.156930+0000","last_became_active":"2024-06-03T02:14:53.270365+0000","last_became_peered":"2024-06-03T02:14:53.270365+0000","last_unstale":"2024-06-03T02:17:22.156930+0000","last_undegraded":"2024-06-03T02:17:22.156930+0000","last_fullsized":"2024-06-03T02:17:22.156930+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.051756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"101'15","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.161647+0000","last_change":"2024-06-03T02:14:09.747709+0000","last_active":"2024-06-03T02:17:22.161647+0000","last_peered":"2024-06-03T02:17:22.161647+0000","last_clean":"2024-06-03T02:17:22.161647+0000","last_became_active":"2024-06-03T02:14:09.747359+0000","last_became_peered":"2024-06-03T02:14:09.747359+0000","last_unstale":"2024-06-03T02:17:22.161647+0000","last_undegraded":"2024-06-03T02:17:22.161647+0000","last_fullsized":"2024-06-03T02:17:22.161647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:37:24.873442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"101'2","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.145482+0000","last_change":"2024-06-03T02:14:53.473136+0000","last_active":"2024-06-03T02:17:22.145482+0000","last_peered":"2024-06-03T02:17:22.145482+0000","last_clean":"2024-06-03T02:17:22.145482+0000","last_became_active":"2024-06-03T02:14:53.471214+0000","last_became_peered":"2024-06-03T02:14:53.471214+0000","last_unstale":"2024-06-03T02:17:22.145482+0000","last_undegraded":"2024-06-03T02:17:22.145482+0000","last_fullsized":"2024-06-03T02:17:22.145482+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:25.822042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184932+0000","last_change":"2024-06-03T02:14:09.746463+0000","last_active":"2024-06-03T02:17:22.184932+0000","last_peered":"2024-06-03T02:17:22.184932+0000","last_clean":"2024-06-03T02:17:22.184932+0000","last_became_active":"2024-06-03T02:14:09.746115+0000","last_became_peered":"2024-06-03T02:14:09.746115+0000","last_unstale":"2024-06-03T02:17:22.184932+0000","last_undegraded":"2024-06-03T02:17:22.184932+0000","last_fullsized":"2024-06-03T02:17:22.184932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:51:23.050739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184825+0000","last_change":"2024-06-03T02:14:09.755467+0000","last_active":"2024-06-03T02:17:22.184825+0000","last_peered":"2024-06-03T02:17:22.184825+0000","last_clean":"2024-06-03T02:17:22.184825+0000","last_became_active":"2024-06-03T02:14:09.755103+0000","last_became_peered":"2024-06-03T02:14:09.755103+0000","last_unstale":"2024-06-03T02:17:22.184825+0000","last_undegraded":"2024-06-03T02:17:22.184825+0000","last_fullsized":"2024-06-03T02:17:22.184825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:22.834519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186153+0000","last_change":"2024-06-03T02:14:09.758201+0000","last_active":"2024-06-03T02:17:22.186153+0000","last_peered":"2024-06-03T02:17:22.186153+0000","last_clean":"2024-06-03T02:17:22.186153+0000","last_became_active":"2024-06-03T02:14:09.757819+0000","last_became_peered":"2024-06-03T02:14:09.757819+0000","last_unstale":"2024-06-03T02:17:22.186153+0000","last_undegraded":"2024-06-03T02:17:22.186153+0000","last_fullsized":"2024-06-03T02:17:22.186153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:44.802694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183397+0000","last_change":"2024-06-03T02:14:09.768132+0000","last_active":"2024-06-03T02:17:22.183397+0000","last_peered":"2024-06-03T02:17:22.183397+0000","last_clean":"2024-06-03T02:17:22.183397+0000","last_became_active":"2024-06-03T02:14:09.767642+0000","last_became_peered":"2024-06-03T02:14:09.767642+0000","last_unstale":"2024-06-03T02:17:22.183397+0000","last_undegraded":"2024-06-03T02:17:22.183397+0000","last_fullsized":"2024-06-03T02:17:22.183397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:32:30.455418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.159132+0000","last_change":"2024-06-03T02:14:09.745603+0000","last_active":"2024-06-03T02:17:22.159132+0000","last_peered":"2024-06-03T02:17:22.159132+0000","last_clean":"2024-06-03T02:17:22.159132+0000","last_became_active":"2024-06-03T02:14:09.745245+0000","last_became_peered":"2024-06-03T02:14:09.745245+0000","last_unstale":"2024-06-03T02:17:22.159132+0000","last_undegraded":"2024-06-03T02:17:22.159132+0000","last_fullsized":"2024-06-03T02:17:22.159132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:09:14.507101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81982,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2347,"num_read_kb":1260,"num_write":278,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":248466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318354,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":579107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579990,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37168,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707984,"statfs":{"total":95995035648,"available":95956975616,"internally_reserved":0,"allocated":1875968,"data_stored":1242653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743516,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":577953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005154,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":577637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266792,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1785856,"data_stored":1176810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430318,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":573988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1249280,"data_stored":642164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738512,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":575303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":651223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149821,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888184,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":578452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691956,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37088,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708064,"statfs":{"total":95995035648,"available":95957057536,"internally_reserved":0,"allocated":1794048,"data_stored":1170940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":75192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":78488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":75454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T02:19:29.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-03T02:19:29.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.818+0000 7f90b4800640 1 -- 172.21.15.79:0/202123884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac0759e0 msgr2=0x7f90ac073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:29.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.818+0000 7f90b4800640 1 --2- 172.21.15.79:0/202123884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac0759e0 0x7f90ac073e30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009cd0 tx=0x7f90a402a8f0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.818+0000 7f90b4800640 1 -- 172.21.15.79:0/202123884 shutdown_connections 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.818+0000 7f90b4800640 1 --2- 172.21.15.79:0/202123884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90ac10dfc0 0x7f90ac1103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.818+0000 7f90b4800640 1 --2- 172.21.15.79:0/202123884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac074400 0x7f90ac074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.818+0000 7f90b4800640 1 --2- 172.21.15.79:0/202123884 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac0759e0 0x7f90ac073e30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.818+0000 7f90b4800640 1 -- 172.21.15.79:0/202123884 >> 172.21.15.79:0/202123884 conn(0x7f90ac0fcf80 msgr2=0x7f90ac0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.818+0000 7f90b4800640 1 -- 172.21.15.79:0/202123884 shutdown_connections 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.818+0000 7f90b4800640 1 -- 172.21.15.79:0/202123884 wait complete. 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.819+0000 7f90b4800640 1 Processor -- start 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.819+0000 7f90b4800640 1 -- start start 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.819+0000 7f90b4800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90ac074400 0x7f90ac19e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.819+0000 7f90b4800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0759e0 0x7f90ac19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.819+0000 7f90b4800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac10dfc0 0x7f90ac1a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.819+0000 7f90b4800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac113c80 con 0x7f90ac10dfc0 2024-06-03T02:19:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.819+0000 7f90b4800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac113b00 con 0x7f90ac0759e0 2024-06-03T02:19:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.819+0000 7f90b4800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac113e00 con 0x7f90ac074400 2024-06-03T02:19:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b2800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac10dfc0 0x7f90ac1a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0759e0 0x7f90ac19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0759e0 0x7f90ac19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38810/0 (socket says 172.21.15.79:38810) 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b1e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90ac074400 0x7f90ac19e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b1400640 1 -- 172.21.15.79:0/1535676115 learned_addr learned my addr 172.21.15.79:0/1535676115 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b2800640 1 -- 172.21.15.79:0/1535676115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90ac074400 msgr2=0x7f90ac19e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b2800640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90ac074400 0x7f90ac19e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b2800640 1 -- 172.21.15.79:0/1535676115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0759e0 msgr2=0x7f90ac19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b2800640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0759e0 0x7f90ac19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b2800640 1 -- 172.21.15.79:0/1535676115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac1a5f70 con 0x7f90ac10dfc0 2024-06-03T02:19:29.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b1400640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0759e0 0x7f90ac19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.820+0000 7f90b2800640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac10dfc0 0x7f90ac1a5900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f90a800e970 tx=0x7f90a800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.821+0000 7f90a2a00640 1 -- 172.21.15.79:0/1535676115 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a800ccc0 con 0x7f90ac10dfc0 2024-06-03T02:19:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.821+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac1a6260 con 0x7f90ac10dfc0 2024-06-03T02:19:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.821+0000 7f90a2a00640 1 -- 172.21.15.79:0/1535676115 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a8005020 con 0x7f90ac10dfc0 2024-06-03T02:19:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.821+0000 7f90a2a00640 1 -- 172.21.15.79:0/1535676115 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8004a00 con 0x7f90ac10dfc0 2024-06-03T02:19:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.821+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac1a67b0 con 0x7f90ac10dfc0 2024-06-03T02:19:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.822+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ac104190 con 0x7f90ac10dfc0 2024-06-03T02:19:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.824+0000 7f90a2a00640 1 -- 172.21.15.79:0/1535676115 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f90a80051d0 con 0x7f90ac10dfc0 2024-06-03T02:19:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.825+0000 7f90a2a00640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90980a2c20 0x7f90980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.826+0000 7f90a2a00640 1 -- 172.21.15.79:0/1535676115 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f90a80cf510 con 0x7f90ac10dfc0 2024-06-03T02:19:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.826+0000 7f90b1e00640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90980a2c20 0x7f90980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.826+0000 7f90b1e00640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90980a2c20 0x7f90980a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006010 tx=0x7f90a4031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:29.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:29.830+0000 7f90a2a00640 1 -- 172.21.15.79:0/1535676115 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f90a809bf00 con 0x7f90ac10dfc0 2024-06-03T02:19:29.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:29 smithi079 ceph-mon[31819]: pgmap v499: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-06-03T02:19:29.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:29 smithi135 ceph-mon[36653]: pgmap v499: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-06-03T02:19:29.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:29 smithi123 ceph-mon[36640]: pgmap v499: 129 pgs: 129 active+clean; 657 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-06-03T02:19:30.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.027+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90ac061de0 con 0x7f90980a2c20 2024-06-03T02:19:30.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.056+0000 7f90a2a00640 1 -- 172.21.15.79:0/1535676115 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340748 (secure 0 0 0) 0x7f90ac061de0 con 0x7f90980a2c20 2024-06-03T02:19:30.061 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:30.063 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90980a2c20 msgr2=0x7f90980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90980a2c20 0x7f90980a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006010 tx=0x7f90a4031000 comp rx=0 tx=0).stop 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac10dfc0 msgr2=0x7f90ac1a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac10dfc0 0x7f90ac1a5900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f90a800e970 tx=0x7f90a800ee30 comp rx=0 tx=0).stop 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 shutdown_connections 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90ac10dfc0 0x7f90ac1a5900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0759e0 0x7f90ac19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f90980a2c20 0x7f90980a50e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 --2- 172.21.15.79:0/1535676115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90ac074400 0x7f90ac19e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.064+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 >> 172.21.15.79:0/1535676115 conn(0x7f90ac0fcf80 msgr2=0x7f90ac101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.065+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 shutdown_connections 2024-06-03T02:19:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.065+0000 7f90b4800640 1 -- 172.21.15.79:0/1535676115 wait complete. 2024-06-03T02:19:30.084 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-06-03T02:19:28.301895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672584,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2473,"num_read_kb":1369,"num_write":461,"num_write_kb":3113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360840,"kb_used_data":15848,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580984,"statfs":{"total":1151940427776,"available":1151570927616,"internally_reserved":0,"allocated":16228352,"data_stored":8919395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7239,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":15,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005257"},"pg_stats":[{"pgid":"3.3f","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186895+0000","last_change":"2024-06-03T02:14:09.768431+0000","last_active":"2024-06-03T02:17:22.186895+0000","last_peered":"2024-06-03T02:17:22.186895+0000","last_clean":"2024-06-03T02:17:22.186895+0000","last_became_active":"2024-06-03T02:14:09.768059+0000","last_became_peered":"2024-06-03T02:14:09.768059+0000","last_unstale":"2024-06-03T02:17:22.186895+0000","last_undegraded":"2024-06-03T02:17:22.186895+0000","last_fullsized":"2024-06-03T02:17:22.186895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:12.265896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188670+0000","last_change":"2024-06-03T02:14:09.771842+0000","last_active":"2024-06-03T02:17:22.188670+0000","last_peered":"2024-06-03T02:17:22.188670+0000","last_clean":"2024-06-03T02:17:22.188670+0000","last_became_active":"2024-06-03T02:14:09.770961+0000","last_became_peered":"2024-06-03T02:14:09.770961+0000","last_unstale":"2024-06-03T02:17:22.188670+0000","last_undegraded":"2024-06-03T02:17:22.188670+0000","last_fullsized":"2024-06-03T02:17:22.188670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:49:02.297128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183559+0000","last_change":"2024-06-03T02:14:09.758407+0000","last_active":"2024-06-03T02:17:22.183559+0000","last_peered":"2024-06-03T02:17:22.183559+0000","last_clean":"2024-06-03T02:17:22.183559+0000","last_became_active":"2024-06-03T02:14:09.758045+0000","last_became_peered":"2024-06-03T02:14:09.758045+0000","last_unstale":"2024-06-03T02:17:22.183559+0000","last_undegraded":"2024-06-03T02:17:22.183559+0000","last_fullsized":"2024-06-03T02:17:22.183559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:01:08.166971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.152098+0000","last_change":"2024-06-03T02:14:09.747658+0000","last_active":"2024-06-03T02:17:22.152098+0000","last_peered":"2024-06-03T02:17:22.152098+0000","last_clean":"2024-06-03T02:17:22.152098+0000","last_became_active":"2024-06-03T02:14:09.747321+0000","last_became_peered":"2024-06-03T02:14:09.747321+0000","last_unstale":"2024-06-03T02:17:22.152098+0000","last_undegraded":"2024-06-03T02:17:22.152098+0000","last_fullsized":"2024-06-03T02:17:22.152098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:08.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184933+0000","last_change":"2024-06-03T02:14:09.758161+0000","last_active":"2024-06-03T02:17:22.184933+0000","last_peered":"2024-06-03T02:17:22.184933+0000","last_clean":"2024-06-03T02:17:22.184933+0000","last_became_active":"2024-06-03T02:14:09.757800+0000","last_became_peered":"2024-06-03T02:14:09.757800+0000","last_unstale":"2024-06-03T02:17:22.184933+0000","last_undegraded":"2024-06-03T02:17:22.184933+0000","last_fullsized":"2024-06-03T02:17:22.184933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:47.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.160478+0000","last_change":"2024-06-03T02:14:09.741818+0000","last_active":"2024-06-03T02:17:22.160478+0000","last_peered":"2024-06-03T02:17:22.160478+0000","last_clean":"2024-06-03T02:17:22.160478+0000","last_became_active":"2024-06-03T02:14:09.741219+0000","last_became_peered":"2024-06-03T02:14:09.741219+0000","last_unstale":"2024-06-03T02:17:22.160478+0000","last_undegraded":"2024-06-03T02:17:22.160478+0000","last_fullsized":"2024-06-03T02:17:22.160478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:16:30.585245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163369+0000","last_change":"2024-06-03T02:14:09.744863+0000","last_active":"2024-06-03T02:17:22.163369+0000","last_peered":"2024-06-03T02:17:22.163369+0000","last_clean":"2024-06-03T02:17:22.163369+0000","last_became_active":"2024-06-03T02:14:09.744514+0000","last_became_peered":"2024-06-03T02:14:09.744514+0000","last_unstale":"2024-06-03T02:17:22.163369+0000","last_undegraded":"2024-06-03T02:17:22.163369+0000","last_fullsized":"2024-06-03T02:17:22.163369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:51:42.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187625+0000","last_change":"2024-06-03T02:14:09.766897+0000","last_active":"2024-06-03T02:17:22.187625+0000","last_peered":"2024-06-03T02:17:22.187625+0000","last_clean":"2024-06-03T02:17:22.187625+0000","last_became_active":"2024-06-03T02:14:09.765461+0000","last_became_peered":"2024-06-03T02:14:09.765461+0000","last_unstale":"2024-06-03T02:17:22.187625+0000","last_undegraded":"2024-06-03T02:17:22.187625+0000","last_fullsized":"2024-06-03T02:17:22.187625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:39:28.641517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172030+0000","last_change":"2024-06-03T02:14:09.746451+0000","last_active":"2024-06-03T02:17:22.172030+0000","last_peered":"2024-06-03T02:17:22.172030+0000","last_clean":"2024-06-03T02:17:22.172030+0000","last_became_active":"2024-06-03T02:14:09.746117+0000","last_became_peered":"2024-06-03T02:14:09.746117+0000","last_unstale":"2024-06-03T02:17:22.172030+0000","last_undegraded":"2024-06-03T02:17:22.172030+0000","last_fullsized":"2024-06-03T02:17:22.172030+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:42.599993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"101'16","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.162982+0000","last_change":"2024-06-03T02:14:09.750680+0000","last_active":"2024-06-03T02:17:22.162982+0000","last_peered":"2024-06-03T02:17:22.162982+0000","last_clean":"2024-06-03T02:17:22.162982+0000","last_became_active":"2024-06-03T02:14:09.746138+0000","last_became_peered":"2024-06-03T02:14:09.746138+0000","last_unstale":"2024-06-03T02:17:22.162982+0000","last_undegraded":"2024-06-03T02:17:22.162982+0000","last_fullsized":"2024-06-03T02:17:22.162982+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:04:28.188323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172525+0000","last_change":"2024-06-03T02:14:09.743467+0000","last_active":"2024-06-03T02:17:22.172525+0000","last_peered":"2024-06-03T02:17:22.172525+0000","last_clean":"2024-06-03T02:17:22.172525+0000","last_became_active":"2024-06-03T02:14:09.743152+0000","last_became_peered":"2024-06-03T02:14:09.743152+0000","last_unstale":"2024-06-03T02:17:22.172525+0000","last_undegraded":"2024-06-03T02:17:22.172525+0000","last_fullsized":"2024-06-03T02:17:22.172525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:27.336630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.148334+0000","last_change":"2024-06-03T02:14:09.747142+0000","last_active":"2024-06-03T02:17:22.148334+0000","last_peered":"2024-06-03T02:17:22.148334+0000","last_clean":"2024-06-03T02:17:22.148334+0000","last_became_active":"2024-06-03T02:14:09.746694+0000","last_became_peered":"2024-06-03T02:14:09.746694+0000","last_unstale":"2024-06-03T02:17:22.148334+0000","last_undegraded":"2024-06-03T02:17:22.148334+0000","last_fullsized":"2024-06-03T02:17:22.148334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:04.507667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179213+0000","last_change":"2024-06-03T02:14:09.759906+0000","last_active":"2024-06-03T02:17:22.179213+0000","last_peered":"2024-06-03T02:17:22.179213+0000","last_clean":"2024-06-03T02:17:22.179213+0000","last_became_active":"2024-06-03T02:14:09.755450+0000","last_became_peered":"2024-06-03T02:14:09.755450+0000","last_unstale":"2024-06-03T02:17:22.179213+0000","last_undegraded":"2024-06-03T02:17:22.179213+0000","last_fullsized":"2024-06-03T02:17:22.179213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:23.057956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187178+0000","last_change":"2024-06-03T02:14:09.769388+0000","last_active":"2024-06-03T02:17:22.187178+0000","last_peered":"2024-06-03T02:17:22.187178+0000","last_clean":"2024-06-03T02:17:22.187178+0000","last_became_active":"2024-06-03T02:14:09.769029+0000","last_became_peered":"2024-06-03T02:14:09.769029+0000","last_unstale":"2024-06-03T02:17:22.187178+0000","last_undegraded":"2024-06-03T02:17:22.187178+0000","last_fullsized":"2024-06-03T02:17:22.187178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:24.751039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181320+0000","last_change":"2024-06-03T02:14:09.768584+0000","last_active":"2024-06-03T02:17:22.181320+0000","last_peered":"2024-06-03T02:17:22.181320+0000","last_clean":"2024-06-03T02:17:22.181320+0000","last_became_active":"2024-06-03T02:14:09.765737+0000","last_became_peered":"2024-06-03T02:14:09.765737+0000","last_unstale":"2024-06-03T02:17:22.181320+0000","last_undegraded":"2024-06-03T02:17:22.181320+0000","last_fullsized":"2024-06-03T02:17:22.181320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:36:18.807762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181401+0000","last_change":"2024-06-03T02:14:09.757828+0000","last_active":"2024-06-03T02:17:22.181401+0000","last_peered":"2024-06-03T02:17:22.181401+0000","last_clean":"2024-06-03T02:17:22.181401+0000","last_became_active":"2024-06-03T02:14:09.757040+0000","last_became_peered":"2024-06-03T02:14:09.757040+0000","last_unstale":"2024-06-03T02:17:22.181401+0000","last_undegraded":"2024-06-03T02:17:22.181401+0000","last_fullsized":"2024-06-03T02:17:22.181401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:26:28.694022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188574+0000","last_change":"2024-06-03T02:14:09.768263+0000","last_active":"2024-06-03T02:17:22.188574+0000","last_peered":"2024-06-03T02:17:22.188574+0000","last_clean":"2024-06-03T02:17:22.188574+0000","last_became_active":"2024-06-03T02:14:09.767988+0000","last_became_peered":"2024-06-03T02:14:09.767988+0000","last_unstale":"2024-06-03T02:17:22.188574+0000","last_undegraded":"2024-06-03T02:17:22.188574+0000","last_fullsized":"2024-06-03T02:17:22.188574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:28:37.432659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171811+0000","last_change":"2024-06-03T02:14:09.747118+0000","last_active":"2024-06-03T02:17:22.171811+0000","last_peered":"2024-06-03T02:17:22.171811+0000","last_clean":"2024-06-03T02:17:22.171811+0000","last_became_active":"2024-06-03T02:14:09.746580+0000","last_became_peered":"2024-06-03T02:14:09.746580+0000","last_unstale":"2024-06-03T02:17:22.171811+0000","last_undegraded":"2024-06-03T02:17:22.171811+0000","last_fullsized":"2024-06-03T02:17:22.171811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:33:04.607285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185388+0000","last_change":"2024-06-03T02:14:09.755755+0000","last_active":"2024-06-03T02:17:22.185388+0000","last_peered":"2024-06-03T02:17:22.185388+0000","last_clean":"2024-06-03T02:17:22.185388+0000","last_became_active":"2024-06-03T02:14:09.754503+0000","last_became_peered":"2024-06-03T02:14:09.754503+0000","last_unstale":"2024-06-03T02:17:22.185388+0000","last_undegraded":"2024-06-03T02:17:22.185388+0000","last_fullsized":"2024-06-03T02:17:22.185388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:30:58.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:52.213243+0000","last_change":"2024-06-03T02:14:23.260021+0000","last_active":"2024-06-03T02:16:52.213243+0000","last_peered":"2024-06-03T02:16:52.213243+0000","last_clean":"2024-06-03T02:16:52.213243+0000","last_became_active":"2024-06-03T02:14:23.259587+0000","last_became_peered":"2024-06-03T02:14:23.259587+0000","last_unstale":"2024-06-03T02:16:52.213243+0000","last_undegraded":"2024-06-03T02:16:52.213243+0000","last_fullsized":"2024-06-03T02:16:52.213243+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:09:02.403936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"101'14","reported_seq":291,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:24.835941+0000","last_change":"2024-06-03T02:14:05.239623+0000","last_active":"2024-06-03T02:19:24.835941+0000","last_peered":"2024-06-03T02:19:24.835941+0000","last_clean":"2024-06-03T02:19:24.835941+0000","last_became_active":"2024-06-03T02:14:05.239112+0000","last_became_peered":"2024-06-03T02:14:05.239112+0000","last_unstale":"2024-06-03T02:19:24.835941+0000","last_undegraded":"2024-06-03T02:19:24.835941+0000","last_fullsized":"2024-06-03T02:19:24.835941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:12:14.020068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":231,"num_read_kb":231,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185036+0000","last_change":"2024-06-03T02:14:09.764972+0000","last_active":"2024-06-03T02:17:22.185036+0000","last_peered":"2024-06-03T02:17:22.185036+0000","last_clean":"2024-06-03T02:17:22.185036+0000","last_became_active":"2024-06-03T02:14:09.764435+0000","last_became_peered":"2024-06-03T02:14:09.764435+0000","last_unstale":"2024-06-03T02:17:22.185036+0000","last_undegraded":"2024-06-03T02:17:22.185036+0000","last_fullsized":"2024-06-03T02:17:22.185036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:46:58.941554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.200287+0000","last_change":"2024-06-03T02:14:23.272076+0000","last_active":"2024-06-03T02:16:51.200287+0000","last_peered":"2024-06-03T02:16:51.200287+0000","last_clean":"2024-06-03T02:16:51.200287+0000","last_became_active":"2024-06-03T02:14:23.271578+0000","last_became_peered":"2024-06-03T02:14:23.271578+0000","last_unstale":"2024-06-03T02:16:51.200287+0000","last_undegraded":"2024-06-03T02:16:51.200287+0000","last_fullsized":"2024-06-03T02:16:51.200287+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:17:59.000291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"99'2","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:09.237250+0000","last_change":"2024-06-03T02:14:05.251952+0000","last_active":"2024-06-03T02:16:09.237250+0000","last_peered":"2024-06-03T02:16:09.237250+0000","last_clean":"2024-06-03T02:16:09.237250+0000","last_became_active":"2024-06-03T02:14:05.251152+0000","last_became_peered":"2024-06-03T02:14:05.251152+0000","last_unstale":"2024-06-03T02:16:09.237250+0000","last_undegraded":"2024-06-03T02:16:09.237250+0000","last_fullsized":"2024-06-03T02:16:09.237250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:24:32.802236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.190358+0000","last_change":"2024-06-03T02:14:09.764230+0000","last_active":"2024-06-03T02:17:22.190358+0000","last_peered":"2024-06-03T02:17:22.190358+0000","last_clean":"2024-06-03T02:17:22.190358+0000","last_became_active":"2024-06-03T02:14:09.761868+0000","last_became_peered":"2024-06-03T02:14:09.761868+0000","last_unstale":"2024-06-03T02:17:22.190358+0000","last_undegraded":"2024-06-03T02:17:22.190358+0000","last_fullsized":"2024-06-03T02:17:22.190358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:39:23.828546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:56.200917+0000","last_change":"2024-06-03T02:14:23.272169+0000","last_active":"2024-06-03T02:16:56.200917+0000","last_peered":"2024-06-03T02:16:56.200917+0000","last_clean":"2024-06-03T02:16:56.200917+0000","last_became_active":"2024-06-03T02:14:23.271636+0000","last_became_peered":"2024-06-03T02:14:23.271636+0000","last_unstale":"2024-06-03T02:16:56.200917+0000","last_undegraded":"2024-06-03T02:16:56.200917+0000","last_fullsized":"2024-06-03T02:16:56.200917+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:51:25.478028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"99'12","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:37.186203+0000","last_change":"2024-06-03T02:14:05.238590+0000","last_active":"2024-06-03T02:16:37.186203+0000","last_peered":"2024-06-03T02:16:37.186203+0000","last_clean":"2024-06-03T02:16:37.186203+0000","last_became_active":"2024-06-03T02:14:05.238128+0000","last_became_peered":"2024-06-03T02:14:05.238128+0000","last_unstale":"2024-06-03T02:16:37.186203+0000","last_undegraded":"2024-06-03T02:16:37.186203+0000","last_fullsized":"2024-06-03T02:16:37.186203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:21.298255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174715+0000","last_change":"2024-06-03T02:14:09.754489+0000","last_active":"2024-06-03T02:17:22.174715+0000","last_peered":"2024-06-03T02:17:22.174715+0000","last_clean":"2024-06-03T02:17:22.174715+0000","last_became_active":"2024-06-03T02:14:09.754155+0000","last_became_peered":"2024-06-03T02:14:09.754155+0000","last_unstale":"2024-06-03T02:17:22.174715+0000","last_undegraded":"2024-06-03T02:17:22.174715+0000","last_fullsized":"2024-06-03T02:17:22.174715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:20:43.841271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:56.200884+0000","last_change":"2024-06-03T02:14:23.261232+0000","last_active":"2024-06-03T02:16:56.200884+0000","last_peered":"2024-06-03T02:16:56.200884+0000","last_clean":"2024-06-03T02:16:56.200884+0000","last_became_active":"2024-06-03T02:14:23.260772+0000","last_became_peered":"2024-06-03T02:14:23.260772+0000","last_unstale":"2024-06-03T02:16:56.200884+0000","last_undegraded":"2024-06-03T02:16:56.200884+0000","last_fullsized":"2024-06-03T02:16:56.200884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:52.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"99'5","reported_seq":24,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272244+0000","last_change":"2024-06-03T02:14:53.274084+0000","last_active":"2024-06-03T02:15:54.272244+0000","last_peered":"2024-06-03T02:15:54.272244+0000","last_clean":"2024-06-03T02:15:54.272244+0000","last_became_active":"2024-06-03T02:14:53.265253+0000","last_became_peered":"2024-06-03T02:14:53.265253+0000","last_unstale":"2024-06-03T02:15:54.272244+0000","last_undegraded":"2024-06-03T02:15:54.272244+0000","last_fullsized":"2024-06-03T02:15:54.272244+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:25:56.189308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"101'15","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.156115+0000","last_change":"2024-06-03T02:14:09.743450+0000","last_active":"2024-06-03T02:17:22.156115+0000","last_peered":"2024-06-03T02:17:22.156115+0000","last_clean":"2024-06-03T02:17:22.156115+0000","last_became_active":"2024-06-03T02:14:09.743071+0000","last_became_peered":"2024-06-03T02:14:09.743071+0000","last_unstale":"2024-06-03T02:17:22.156115+0000","last_undegraded":"2024-06-03T02:17:22.156115+0000","last_fullsized":"2024-06-03T02:17:22.156115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:43:44.278633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:57.213954+0000","last_change":"2024-06-03T02:15:54.342978+0000","last_active":"2024-06-03T02:16:57.213954+0000","last_peered":"2024-06-03T02:16:57.213954+0000","last_clean":"2024-06-03T02:16:57.213954+0000","last_became_active":"2024-06-03T02:15:54.342477+0000","last_became_peered":"2024-06-03T02:15:54.342477+0000","last_unstale":"2024-06-03T02:16:57.213954+0000","last_undegraded":"2024-06-03T02:16:57.213954+0000","last_fullsized":"2024-06-03T02:16:57.213954+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:47.980513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"101'8","reported_seq":290,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:25.810333+0000","last_change":"2024-06-03T02:14:05.237464+0000","last_active":"2024-06-03T02:19:25.810333+0000","last_peered":"2024-06-03T02:19:25.810333+0000","last_clean":"2024-06-03T02:19:25.810333+0000","last_became_active":"2024-06-03T02:14:05.237067+0000","last_became_peered":"2024-06-03T02:14:05.237067+0000","last_unstale":"2024-06-03T02:19:25.810333+0000","last_undegraded":"2024-06-03T02:19:25.810333+0000","last_fullsized":"2024-06-03T02:19:25.810333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:06:38.850484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.150011+0000","last_change":"2024-06-03T02:14:09.748741+0000","last_active":"2024-06-03T02:17:22.150011+0000","last_peered":"2024-06-03T02:17:22.150011+0000","last_clean":"2024-06-03T02:17:22.150011+0000","last_became_active":"2024-06-03T02:14:09.748402+0000","last_became_peered":"2024-06-03T02:14:09.748402+0000","last_unstale":"2024-06-03T02:17:22.150011+0000","last_undegraded":"2024-06-03T02:17:22.150011+0000","last_fullsized":"2024-06-03T02:17:22.150011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:03:25.664415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:57.213929+0000","last_change":"2024-06-03T02:14:23.260397+0000","last_active":"2024-06-03T02:16:57.213929+0000","last_peered":"2024-06-03T02:16:57.213929+0000","last_clean":"2024-06-03T02:16:57.213929+0000","last_became_active":"2024-06-03T02:14:23.259886+0000","last_became_peered":"2024-06-03T02:14:23.259886+0000","last_unstale":"2024-06-03T02:16:57.213929+0000","last_undegraded":"2024-06-03T02:16:57.213929+0000","last_fullsized":"2024-06-03T02:16:57.213929+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:46:18.562706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"99'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:38.829788+0000","last_change":"2024-06-03T02:14:05.241536+0000","last_active":"2024-06-03T02:16:38.829788+0000","last_peered":"2024-06-03T02:16:38.829788+0000","last_clean":"2024-06-03T02:16:38.829788+0000","last_became_active":"2024-06-03T02:14:05.241203+0000","last_became_peered":"2024-06-03T02:14:05.241203+0000","last_unstale":"2024-06-03T02:16:38.829788+0000","last_undegraded":"2024-06-03T02:16:38.829788+0000","last_fullsized":"2024-06-03T02:16:38.829788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:47.052563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"101'8","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185241+0000","last_change":"2024-06-03T02:14:53.475893+0000","last_active":"2024-06-03T02:17:22.185241+0000","last_peered":"2024-06-03T02:17:22.185241+0000","last_clean":"2024-06-03T02:17:22.185241+0000","last_became_active":"2024-06-03T02:14:53.475494+0000","last_became_peered":"2024-06-03T02:14:53.475494+0000","last_unstale":"2024-06-03T02:17:22.185241+0000","last_undegraded":"2024-06-03T02:17:22.185241+0000","last_fullsized":"2024-06-03T02:17:22.185241+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:27.113842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:28.911253+0000","last_change":"2024-06-03T02:15:54.344417+0000","last_active":"2024-06-03T02:16:28.911253+0000","last_peered":"2024-06-03T02:16:28.911253+0000","last_clean":"2024-06-03T02:16:28.911253+0000","last_became_active":"2024-06-03T02:15:54.343904+0000","last_became_peered":"2024-06-03T02:15:54.343904+0000","last_unstale":"2024-06-03T02:16:28.911253+0000","last_undegraded":"2024-06-03T02:16:28.911253+0000","last_fullsized":"2024-06-03T02:16:28.911253+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:42:18.121696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"99'3","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.217749+0000","last_change":"2024-06-03T02:14:05.237743+0000","last_active":"2024-06-03T02:16:31.217749+0000","last_peered":"2024-06-03T02:16:31.217749+0000","last_clean":"2024-06-03T02:16:31.217749+0000","last_became_active":"2024-06-03T02:14:05.236944+0000","last_became_peered":"2024-06-03T02:14:05.236944+0000","last_unstale":"2024-06-03T02:16:31.217749+0000","last_undegraded":"2024-06-03T02:16:31.217749+0000","last_fullsized":"2024-06-03T02:16:31.217749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:14:09.850862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185781+0000","last_change":"2024-06-03T02:14:09.761694+0000","last_active":"2024-06-03T02:17:22.185781+0000","last_peered":"2024-06-03T02:17:22.185781+0000","last_clean":"2024-06-03T02:17:22.185781+0000","last_became_active":"2024-06-03T02:14:09.761361+0000","last_became_peered":"2024-06-03T02:14:09.761361+0000","last_unstale":"2024-06-03T02:17:22.185781+0000","last_undegraded":"2024-06-03T02:17:22.185781+0000","last_fullsized":"2024-06-03T02:17:22.185781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:19:17.803088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:22.759262+0000","last_change":"2024-06-03T02:14:23.258634+0000","last_active":"2024-06-03T02:16:22.759262+0000","last_peered":"2024-06-03T02:16:22.759262+0000","last_clean":"2024-06-03T02:16:22.759262+0000","last_became_active":"2024-06-03T02:14:23.257799+0000","last_became_peered":"2024-06-03T02:14:23.257799+0000","last_unstale":"2024-06-03T02:16:22.759262+0000","last_undegraded":"2024-06-03T02:16:22.759262+0000","last_fullsized":"2024-06-03T02:16:22.759262+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:38:33.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"99'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.849678+0000","last_change":"2024-06-03T02:14:05.236790+0000","last_active":"2024-06-03T02:16:31.849678+0000","last_peered":"2024-06-03T02:16:31.849678+0000","last_clean":"2024-06-03T02:16:31.849678+0000","last_became_active":"2024-06-03T02:14:05.236419+0000","last_became_peered":"2024-06-03T02:14:05.236419+0000","last_unstale":"2024-06-03T02:16:31.849678+0000","last_undegraded":"2024-06-03T02:16:31.849678+0000","last_fullsized":"2024-06-03T02:16:31.849678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:44:00.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183352+0000","last_change":"2024-06-03T02:14:09.744680+0000","last_active":"2024-06-03T02:17:22.183352+0000","last_peered":"2024-06-03T02:17:22.183352+0000","last_clean":"2024-06-03T02:17:22.183352+0000","last_became_active":"2024-06-03T02:14:09.744250+0000","last_became_peered":"2024-06-03T02:14:09.744250+0000","last_unstale":"2024-06-03T02:17:22.183352+0000","last_undegraded":"2024-06-03T02:17:22.183352+0000","last_fullsized":"2024-06-03T02:17:22.183352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:58.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:44.242362+0000","last_change":"2024-06-03T02:14:23.259325+0000","last_active":"2024-06-03T02:16:44.242362+0000","last_peered":"2024-06-03T02:16:44.242362+0000","last_clean":"2024-06-03T02:16:44.242362+0000","last_became_active":"2024-06-03T02:14:23.258925+0000","last_became_peered":"2024-06-03T02:14:23.258925+0000","last_unstale":"2024-06-03T02:16:44.242362+0000","last_undegraded":"2024-06-03T02:16:44.242362+0000","last_fullsized":"2024-06-03T02:16:44.242362+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:32.038564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"101'7","reported_seq":509,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:23.653249+0000","last_change":"2024-06-03T02:14:05.242690+0000","last_active":"2024-06-03T02:19:23.653249+0000","last_peered":"2024-06-03T02:19:23.653249+0000","last_clean":"2024-06-03T02:19:23.653249+0000","last_became_active":"2024-06-03T02:14:05.242320+0000","last_became_peered":"2024-06-03T02:14:05.242320+0000","last_unstale":"2024-06-03T02:19:23.653249+0000","last_undegraded":"2024-06-03T02:19:23.653249+0000","last_fullsized":"2024-06-03T02:19:23.653249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:02.793186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":456,"num_read_kb":456,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.170769+0000","last_change":"2024-06-03T02:14:09.747146+0000","last_active":"2024-06-03T02:17:22.170769+0000","last_peered":"2024-06-03T02:17:22.170769+0000","last_clean":"2024-06-03T02:17:22.170769+0000","last_became_active":"2024-06-03T02:14:09.746803+0000","last_became_peered":"2024-06-03T02:14:09.746803+0000","last_unstale":"2024-06-03T02:17:22.170769+0000","last_undegraded":"2024-06-03T02:17:22.170769+0000","last_fullsized":"2024-06-03T02:17:22.170769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:04:36.262223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:45.418229+0000","last_change":"2024-06-03T02:14:23.270328+0000","last_active":"2024-06-03T02:16:45.418229+0000","last_peered":"2024-06-03T02:16:45.418229+0000","last_clean":"2024-06-03T02:16:45.418229+0000","last_became_active":"2024-06-03T02:14:23.264189+0000","last_became_peered":"2024-06-03T02:14:23.264189+0000","last_unstale":"2024-06-03T02:16:45.418229+0000","last_undegraded":"2024-06-03T02:16:45.418229+0000","last_fullsized":"2024-06-03T02:16:45.418229+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:36.248884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"99'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:32.401595+0000","last_change":"2024-06-03T02:14:05.238898+0000","last_active":"2024-06-03T02:16:32.401595+0000","last_peered":"2024-06-03T02:16:32.401595+0000","last_clean":"2024-06-03T02:16:32.401595+0000","last_became_active":"2024-06-03T02:14:05.238552+0000","last_became_peered":"2024-06-03T02:14:05.238552+0000","last_unstale":"2024-06-03T02:16:32.401595+0000","last_undegraded":"2024-06-03T02:16:32.401595+0000","last_fullsized":"2024-06-03T02:16:32.401595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:29:33.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.190294+0000","last_change":"2024-06-03T02:14:09.769973+0000","last_active":"2024-06-03T02:17:22.190294+0000","last_peered":"2024-06-03T02:17:22.190294+0000","last_clean":"2024-06-03T02:17:22.190294+0000","last_became_active":"2024-06-03T02:14:09.768984+0000","last_became_peered":"2024-06-03T02:14:09.768984+0000","last_unstale":"2024-06-03T02:17:22.190294+0000","last_undegraded":"2024-06-03T02:17:22.190294+0000","last_fullsized":"2024-06-03T02:17:22.190294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:48:33.720441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:46.199590+0000","last_change":"2024-06-03T02:14:23.265947+0000","last_active":"2024-06-03T02:16:46.199590+0000","last_peered":"2024-06-03T02:16:46.199590+0000","last_clean":"2024-06-03T02:16:46.199590+0000","last_became_active":"2024-06-03T02:14:23.265412+0000","last_became_peered":"2024-06-03T02:14:23.265412+0000","last_unstale":"2024-06-03T02:16:46.199590+0000","last_undegraded":"2024-06-03T02:16:46.199590+0000","last_fullsized":"2024-06-03T02:16:46.199590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:35:08.664884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"99'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.849724+0000","last_change":"2024-06-03T02:14:05.252751+0000","last_active":"2024-06-03T02:16:31.849724+0000","last_peered":"2024-06-03T02:16:31.849724+0000","last_clean":"2024-06-03T02:16:31.849724+0000","last_became_active":"2024-06-03T02:14:05.252267+0000","last_became_peered":"2024-06-03T02:14:05.252267+0000","last_unstale":"2024-06-03T02:16:31.849724+0000","last_undegraded":"2024-06-03T02:16:31.849724+0000","last_fullsized":"2024-06-03T02:16:31.849724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:58.091711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171446+0000","last_change":"2024-06-03T02:14:09.750757+0000","last_active":"2024-06-03T02:17:22.171446+0000","last_peered":"2024-06-03T02:17:22.171446+0000","last_clean":"2024-06-03T02:17:22.171446+0000","last_became_active":"2024-06-03T02:14:09.750414+0000","last_became_peered":"2024-06-03T02:14:09.750414+0000","last_unstale":"2024-06-03T02:17:22.171446+0000","last_undegraded":"2024-06-03T02:17:22.171446+0000","last_fullsized":"2024-06-03T02:17:22.171446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:29:55.435408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:45.418260+0000","last_change":"2024-06-03T02:14:23.263440+0000","last_active":"2024-06-03T02:16:45.418260+0000","last_peered":"2024-06-03T02:16:45.418260+0000","last_clean":"2024-06-03T02:16:45.418260+0000","last_became_active":"2024-06-03T02:14:23.254019+0000","last_became_peered":"2024-06-03T02:14:23.254019+0000","last_unstale":"2024-06-03T02:16:45.418260+0000","last_undegraded":"2024-06-03T02:16:45.418260+0000","last_fullsized":"2024-06-03T02:16:45.418260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:48:58.745990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'42","reported_seq":557,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:23.654352+0000","last_change":"2024-06-03T02:14:05.239353+0000","last_active":"2024-06-03T02:19:23.654352+0000","last_peered":"2024-06-03T02:19:23.654352+0000","last_clean":"2024-06-03T02:19:23.654352+0000","last_became_active":"2024-06-03T02:14:05.238976+0000","last_became_peered":"2024-06-03T02:14:05.238976+0000","last_unstale":"2024-06-03T02:19:23.654352+0000","last_undegraded":"2024-06-03T02:19:23.654352+0000","last_fullsized":"2024-06-03T02:19:23.654352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:51.598533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72797,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":469,"num_read_kb":305,"num_write":44,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.145081+0000","last_change":"2024-06-03T02:14:09.746546+0000","last_active":"2024-06-03T02:17:22.145081+0000","last_peered":"2024-06-03T02:17:22.145081+0000","last_clean":"2024-06-03T02:17:22.145081+0000","last_became_active":"2024-06-03T02:14:09.746049+0000","last_became_peered":"2024-06-03T02:14:09.746049+0000","last_unstale":"2024-06-03T02:17:22.145081+0000","last_undegraded":"2024-06-03T02:17:22.145081+0000","last_fullsized":"2024-06-03T02:17:22.145081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:34.667133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.851868+0000","last_change":"2024-06-03T02:14:23.257296+0000","last_active":"2024-06-03T02:16:51.851868+0000","last_peered":"2024-06-03T02:16:51.851868+0000","last_clean":"2024-06-03T02:16:51.851868+0000","last_became_active":"2024-06-03T02:14:23.256858+0000","last_became_peered":"2024-06-03T02:14:23.256858+0000","last_unstale":"2024-06-03T02:16:51.851868+0000","last_undegraded":"2024-06-03T02:16:51.851868+0000","last_fullsized":"2024-06-03T02:16:51.851868+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:00.464902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"99'9","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:27.210855+0000","last_change":"2024-06-03T02:14:05.245163+0000","last_active":"2024-06-03T02:16:27.210855+0000","last_peered":"2024-06-03T02:16:27.210855+0000","last_clean":"2024-06-03T02:16:27.210855+0000","last_became_active":"2024-06-03T02:14:05.244813+0000","last_became_peered":"2024-06-03T02:14:05.244813+0000","last_unstale":"2024-06-03T02:16:27.210855+0000","last_undegraded":"2024-06-03T02:16:27.210855+0000","last_fullsized":"2024-06-03T02:16:27.210855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:46:47.017859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181707+0000","last_change":"2024-06-03T02:14:09.757200+0000","last_active":"2024-06-03T02:17:22.181707+0000","last_peered":"2024-06-03T02:17:22.181707+0000","last_clean":"2024-06-03T02:17:22.181707+0000","last_became_active":"2024-06-03T02:14:09.756380+0000","last_became_peered":"2024-06-03T02:14:09.756380+0000","last_unstale":"2024-06-03T02:17:22.181707+0000","last_undegraded":"2024-06-03T02:17:22.181707+0000","last_fullsized":"2024-06-03T02:17:22.181707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:56.045338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:43.913099+0000","last_change":"2024-06-03T02:14:23.258441+0000","last_active":"2024-06-03T02:16:43.913099+0000","last_peered":"2024-06-03T02:16:43.913099+0000","last_clean":"2024-06-03T02:16:43.913099+0000","last_became_active":"2024-06-03T02:14:23.258015+0000","last_became_peered":"2024-06-03T02:14:23.258015+0000","last_unstale":"2024-06-03T02:16:43.913099+0000","last_undegraded":"2024-06-03T02:16:43.913099+0000","last_fullsized":"2024-06-03T02:16:43.913099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:03.123511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"99'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:26.849234+0000","last_change":"2024-06-03T02:14:05.240456+0000","last_active":"2024-06-03T02:16:26.849234+0000","last_peered":"2024-06-03T02:16:26.849234+0000","last_clean":"2024-06-03T02:16:26.849234+0000","last_became_active":"2024-06-03T02:14:05.240078+0000","last_became_peered":"2024-06-03T02:14:05.240078+0000","last_unstale":"2024-06-03T02:16:26.849234+0000","last_undegraded":"2024-06-03T02:16:26.849234+0000","last_fullsized":"2024-06-03T02:16:26.849234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:00:26.491773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"101'15","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179670+0000","last_change":"2024-06-03T02:14:09.750876+0000","last_active":"2024-06-03T02:17:22.179670+0000","last_peered":"2024-06-03T02:17:22.179670+0000","last_clean":"2024-06-03T02:17:22.179670+0000","last_became_active":"2024-06-03T02:14:09.750547+0000","last_became_peered":"2024-06-03T02:14:09.750547+0000","last_unstale":"2024-06-03T02:17:22.179670+0000","last_undegraded":"2024-06-03T02:17:22.179670+0000","last_fullsized":"2024-06-03T02:17:22.179670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:08:41.136225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.241201+0000","last_change":"2024-06-03T02:14:23.276172+0000","last_active":"2024-06-03T02:16:51.241201+0000","last_peered":"2024-06-03T02:16:51.241201+0000","last_clean":"2024-06-03T02:16:51.241201+0000","last_became_active":"2024-06-03T02:14:23.275410+0000","last_became_peered":"2024-06-03T02:14:23.275410+0000","last_unstale":"2024-06-03T02:16:51.241201+0000","last_undegraded":"2024-06-03T02:16:51.241201+0000","last_fullsized":"2024-06-03T02:16:51.241201+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:16:30.859453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"101'8","reported_seq":289,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:24.802602+0000","last_change":"2024-06-03T02:14:05.237263+0000","last_active":"2024-06-03T02:19:24.802602+0000","last_peered":"2024-06-03T02:19:24.802602+0000","last_clean":"2024-06-03T02:19:24.802602+0000","last_became_active":"2024-06-03T02:14:05.236890+0000","last_became_peered":"2024-06-03T02:14:05.236890+0000","last_unstale":"2024-06-03T02:19:24.802602+0000","last_undegraded":"2024-06-03T02:19:24.802602+0000","last_fullsized":"2024-06-03T02:19:24.802602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:42.292545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171587+0000","last_change":"2024-06-03T02:14:09.746260+0000","last_active":"2024-06-03T02:17:22.171587+0000","last_peered":"2024-06-03T02:17:22.171587+0000","last_clean":"2024-06-03T02:17:22.171587+0000","last_became_active":"2024-06-03T02:14:09.745940+0000","last_became_peered":"2024-06-03T02:14:09.745940+0000","last_unstale":"2024-06-03T02:17:22.171587+0000","last_undegraded":"2024-06-03T02:17:22.171587+0000","last_fullsized":"2024-06-03T02:17:22.171587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:08:55.851464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.220996+0000","last_change":"2024-06-03T02:14:23.265440+0000","last_active":"2024-06-03T02:16:51.220996+0000","last_peered":"2024-06-03T02:16:51.220996+0000","last_clean":"2024-06-03T02:16:51.220996+0000","last_became_active":"2024-06-03T02:14:23.265028+0000","last_became_peered":"2024-06-03T02:14:23.265028+0000","last_unstale":"2024-06-03T02:16:51.220996+0000","last_undegraded":"2024-06-03T02:16:51.220996+0000","last_fullsized":"2024-06-03T02:16:51.220996+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:56.237500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"99'17","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:27.185001+0000","last_change":"2024-06-03T02:14:53.271821+0000","last_active":"2024-06-03T02:16:27.185001+0000","last_peered":"2024-06-03T02:16:27.185001+0000","last_clean":"2024-06-03T02:16:27.185001+0000","last_became_active":"2024-06-03T02:14:53.263047+0000","last_became_peered":"2024-06-03T02:14:53.263047+0000","last_unstale":"2024-06-03T02:16:27.185001+0000","last_undegraded":"2024-06-03T02:16:27.185001+0000","last_fullsized":"2024-06-03T02:16:27.185001+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:02:25.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.149642+0000","last_change":"2024-06-03T02:14:09.754819+0000","last_active":"2024-06-03T02:17:22.149642+0000","last_peered":"2024-06-03T02:17:22.149642+0000","last_clean":"2024-06-03T02:17:22.149642+0000","last_became_active":"2024-06-03T02:14:09.753995+0000","last_became_peered":"2024-06-03T02:14:09.753995+0000","last_unstale":"2024-06-03T02:17:22.149642+0000","last_undegraded":"2024-06-03T02:17:22.149642+0000","last_fullsized":"2024-06-03T02:17:22.149642+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:19:51.123774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.850243+0000","last_change":"2024-06-03T02:14:23.265003+0000","last_active":"2024-06-03T02:16:36.850243+0000","last_peered":"2024-06-03T02:16:36.850243+0000","last_clean":"2024-06-03T02:16:36.850243+0000","last_became_active":"2024-06-03T02:14:23.264529+0000","last_became_peered":"2024-06-03T02:14:23.264529+0000","last_unstale":"2024-06-03T02:16:36.850243+0000","last_undegraded":"2024-06-03T02:16:36.850243+0000","last_fullsized":"2024-06-03T02:16:36.850243+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:50:19.838274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:22.184389+0000","last_change":"2024-06-03T02:14:05.236901+0000","last_active":"2024-06-03T02:16:22.184389+0000","last_peered":"2024-06-03T02:16:22.184389+0000","last_clean":"2024-06-03T02:16:22.184389+0000","last_became_active":"2024-06-03T02:14:05.236460+0000","last_became_peered":"2024-06-03T02:14:05.236460+0000","last_unstale":"2024-06-03T02:16:22.184389+0000","last_undegraded":"2024-06-03T02:16:22.184389+0000","last_fullsized":"2024-06-03T02:16:22.184389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:59.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.160124+0000","last_change":"2024-06-03T02:14:09.743135+0000","last_active":"2024-06-03T02:17:22.160124+0000","last_peered":"2024-06-03T02:17:22.160124+0000","last_clean":"2024-06-03T02:17:22.160124+0000","last_became_active":"2024-06-03T02:14:09.742506+0000","last_became_peered":"2024-06-03T02:14:09.742506+0000","last_unstale":"2024-06-03T02:17:22.160124+0000","last_undegraded":"2024-06-03T02:17:22.160124+0000","last_fullsized":"2024-06-03T02:17:22.160124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:51.413537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:32.185624+0000","last_change":"2024-06-03T02:14:23.255512+0000","last_active":"2024-06-03T02:16:32.185624+0000","last_peered":"2024-06-03T02:16:32.185624+0000","last_clean":"2024-06-03T02:16:32.185624+0000","last_became_active":"2024-06-03T02:14:23.255103+0000","last_became_peered":"2024-06-03T02:14:23.255103+0000","last_unstale":"2024-06-03T02:16:32.185624+0000","last_undegraded":"2024-06-03T02:16:32.185624+0000","last_fullsized":"2024-06-03T02:16:32.185624+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:12:05.295848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"99'1","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:11.235176+0000","last_change":"2024-06-03T02:14:05.248728+0000","last_active":"2024-06-03T02:16:11.235176+0000","last_peered":"2024-06-03T02:16:11.235176+0000","last_clean":"2024-06-03T02:16:11.235176+0000","last_became_active":"2024-06-03T02:14:05.248144+0000","last_became_peered":"2024-06-03T02:14:05.248144+0000","last_unstale":"2024-06-03T02:16:11.235176+0000","last_undegraded":"2024-06-03T02:16:11.235176+0000","last_fullsized":"2024-06-03T02:16:11.235176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:53:27.814443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184912+0000","last_change":"2024-06-03T02:14:09.752029+0000","last_active":"2024-06-03T02:17:22.184912+0000","last_peered":"2024-06-03T02:17:22.184912+0000","last_clean":"2024-06-03T02:17:22.184912+0000","last_became_active":"2024-06-03T02:14:09.751638+0000","last_became_peered":"2024-06-03T02:14:09.751638+0000","last_unstale":"2024-06-03T02:17:22.184912+0000","last_undegraded":"2024-06-03T02:17:22.184912+0000","last_fullsized":"2024-06-03T02:17:22.184912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:51.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.217797+0000","last_change":"2024-06-03T02:14:23.250406+0000","last_active":"2024-06-03T02:16:31.217797+0000","last_peered":"2024-06-03T02:16:31.217797+0000","last_clean":"2024-06-03T02:16:31.217797+0000","last_became_active":"2024-06-03T02:14:23.249946+0000","last_became_peered":"2024-06-03T02:14:23.249946+0000","last_unstale":"2024-06-03T02:16:31.217797+0000","last_undegraded":"2024-06-03T02:16:31.217797+0000","last_fullsized":"2024-06-03T02:16:31.217797+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:04:47.714532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"101'7","reported_seq":287,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:25.839775+0000","last_change":"2024-06-03T02:14:05.241666+0000","last_active":"2024-06-03T02:19:25.839775+0000","last_peered":"2024-06-03T02:19:25.839775+0000","last_clean":"2024-06-03T02:19:25.839775+0000","last_became_active":"2024-06-03T02:14:05.241299+0000","last_became_peered":"2024-06-03T02:14:05.241299+0000","last_unstale":"2024-06-03T02:19:25.839775+0000","last_undegraded":"2024-06-03T02:19:25.839775+0000","last_fullsized":"2024-06-03T02:19:25.839775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:21:16.792022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":234,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.176295+0000","last_change":"2024-06-03T02:14:09.768602+0000","last_active":"2024-06-03T02:17:22.176295+0000","last_peered":"2024-06-03T02:17:22.176295+0000","last_clean":"2024-06-03T02:17:22.176295+0000","last_became_active":"2024-06-03T02:14:09.768195+0000","last_became_peered":"2024-06-03T02:14:09.768195+0000","last_unstale":"2024-06-03T02:17:22.176295+0000","last_undegraded":"2024-06-03T02:17:22.176295+0000","last_fullsized":"2024-06-03T02:17:22.176295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:45:42.361966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.218658+0000","last_change":"2024-06-03T02:14:23.252472+0000","last_active":"2024-06-03T02:16:36.218658+0000","last_peered":"2024-06-03T02:16:36.218658+0000","last_clean":"2024-06-03T02:16:36.218658+0000","last_became_active":"2024-06-03T02:14:23.252018+0000","last_became_peered":"2024-06-03T02:14:23.252018+0000","last_unstale":"2024-06-03T02:16:36.218658+0000","last_undegraded":"2024-06-03T02:16:36.218658+0000","last_fullsized":"2024-06-03T02:16:36.218658+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:28.468657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163186+0000","last_change":"2024-06-03T02:14:09.744443+0000","last_active":"2024-06-03T02:17:22.163186+0000","last_peered":"2024-06-03T02:17:22.163186+0000","last_clean":"2024-06-03T02:17:22.163186+0000","last_became_active":"2024-06-03T02:14:09.744100+0000","last_became_peered":"2024-06-03T02:14:09.744100+0000","last_unstale":"2024-06-03T02:17:22.163186+0000","last_undegraded":"2024-06-03T02:17:22.163186+0000","last_fullsized":"2024-06-03T02:17:22.163186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:44.566745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"101'7","reported_seq":284,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:24.801818+0000","last_change":"2024-06-03T02:14:05.250734+0000","last_active":"2024-06-03T02:19:24.801818+0000","last_peered":"2024-06-03T02:19:24.801818+0000","last_clean":"2024-06-03T02:19:24.801818+0000","last_became_active":"2024-06-03T02:14:05.250300+0000","last_became_peered":"2024-06-03T02:14:05.250300+0000","last_unstale":"2024-06-03T02:19:24.801818+0000","last_undegraded":"2024-06-03T02:19:24.801818+0000","last_fullsized":"2024-06-03T02:19:24.801818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:52.322895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":231,"num_read_kb":231,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:27.401054+0000","last_change":"2024-06-03T02:14:53.471397+0000","last_active":"2024-06-03T02:16:27.401054+0000","last_peered":"2024-06-03T02:16:27.401054+0000","last_clean":"2024-06-03T02:16:27.401054+0000","last_became_active":"2024-06-03T02:14:53.470868+0000","last_became_peered":"2024-06-03T02:14:53.470868+0000","last_unstale":"2024-06-03T02:16:27.401054+0000","last_undegraded":"2024-06-03T02:16:27.401054+0000","last_fullsized":"2024-06-03T02:16:27.401054+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:58.726070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188260+0000","last_change":"2024-06-03T02:14:09.765193+0000","last_active":"2024-06-03T02:17:22.188260+0000","last_peered":"2024-06-03T02:17:22.188260+0000","last_clean":"2024-06-03T02:17:22.188260+0000","last_became_active":"2024-06-03T02:14:09.764834+0000","last_became_peered":"2024-06-03T02:14:09.764834+0000","last_unstale":"2024-06-03T02:17:22.188260+0000","last_undegraded":"2024-06-03T02:17:22.188260+0000","last_fullsized":"2024-06-03T02:17:22.188260+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:20:37.464286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"99'6","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:11.195852+0000","last_change":"2024-06-03T02:14:05.240118+0000","last_active":"2024-06-03T02:16:11.195852+0000","last_peered":"2024-06-03T02:16:11.195852+0000","last_clean":"2024-06-03T02:16:11.195852+0000","last_became_active":"2024-06-03T02:14:05.239771+0000","last_became_peered":"2024-06-03T02:14:05.239771+0000","last_unstale":"2024-06-03T02:16:11.195852+0000","last_undegraded":"2024-06-03T02:16:11.195852+0000","last_fullsized":"2024-06-03T02:16:11.195852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:08.256222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":372,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:07.405163+0000","last_change":"2024-06-03T02:12:31.842613+0000","last_active":"2024-06-03T02:17:07.405163+0000","last_peered":"2024-06-03T02:17:07.405163+0000","last_clean":"2024-06-03T02:17:07.405163+0000","last_became_active":"2024-06-03T02:12:31.842051+0000","last_became_peered":"2024-06-03T02:12:31.842051+0000","last_unstale":"2024-06-03T02:17:07.405163+0000","last_undegraded":"2024-06-03T02:17:07.405163+0000","last_fullsized":"2024-06-03T02:17:07.405163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_clean_scrub_stamp":"2024-06-03T02:08:38.749702+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:40:29.748789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.850310+0000","last_change":"2024-06-03T02:14:23.255849+0000","last_active":"2024-06-03T02:16:36.850310+0000","last_peered":"2024-06-03T02:16:36.850310+0000","last_clean":"2024-06-03T02:16:36.850310+0000","last_became_active":"2024-06-03T02:14:23.255380+0000","last_became_peered":"2024-06-03T02:14:23.255380+0000","last_unstale":"2024-06-03T02:16:36.850310+0000","last_undegraded":"2024-06-03T02:16:36.850310+0000","last_fullsized":"2024-06-03T02:16:36.850310+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:41:16.051326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"101'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.169554+0000","last_change":"2024-06-03T02:14:53.270423+0000","last_active":"2024-06-03T02:17:22.169554+0000","last_peered":"2024-06-03T02:17:22.169554+0000","last_clean":"2024-06-03T02:17:22.169554+0000","last_became_active":"2024-06-03T02:14:53.269260+0000","last_became_peered":"2024-06-03T02:14:53.269260+0000","last_unstale":"2024-06-03T02:17:22.169554+0000","last_undegraded":"2024-06-03T02:17:22.169554+0000","last_fullsized":"2024-06-03T02:17:22.169554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:57.392012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"99'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:07.399193+0000","last_change":"2024-06-03T02:14:05.235982+0000","last_active":"2024-06-03T02:16:07.399193+0000","last_peered":"2024-06-03T02:16:07.399193+0000","last_clean":"2024-06-03T02:16:07.399193+0000","last_became_active":"2024-06-03T02:14:05.235638+0000","last_became_peered":"2024-06-03T02:14:05.235638+0000","last_unstale":"2024-06-03T02:16:07.399193+0000","last_undegraded":"2024-06-03T02:16:07.399193+0000","last_fullsized":"2024-06-03T02:16:07.399193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:42:07.544005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:28.911203+0000","last_change":"2024-06-03T02:14:23.257790+0000","last_active":"2024-06-03T02:16:28.911203+0000","last_peered":"2024-06-03T02:16:28.911203+0000","last_clean":"2024-06-03T02:16:28.911203+0000","last_became_active":"2024-06-03T02:14:23.257297+0000","last_became_peered":"2024-06-03T02:14:23.257297+0000","last_unstale":"2024-06-03T02:16:28.911203+0000","last_undegraded":"2024-06-03T02:16:28.911203+0000","last_fullsized":"2024-06-03T02:16:28.911203+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:13.715455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"99'4","reported_seq":22,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:58.914613+0000","last_change":"2024-06-03T02:14:53.268495+0000","last_active":"2024-06-03T02:16:58.914613+0000","last_peered":"2024-06-03T02:16:58.914613+0000","last_clean":"2024-06-03T02:16:58.914613+0000","last_became_active":"2024-06-03T02:14:53.268147+0000","last_became_peered":"2024-06-03T02:14:53.268147+0000","last_unstale":"2024-06-03T02:16:58.914613+0000","last_undegraded":"2024-06-03T02:16:58.914613+0000","last_fullsized":"2024-06-03T02:16:58.914613+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:23.076319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172914+0000","last_change":"2024-06-03T02:14:09.758244+0000","last_active":"2024-06-03T02:17:22.172914+0000","last_peered":"2024-06-03T02:17:22.172914+0000","last_clean":"2024-06-03T02:17:22.172914+0000","last_became_active":"2024-06-03T02:14:09.757812+0000","last_became_peered":"2024-06-03T02:14:09.757812+0000","last_unstale":"2024-06-03T02:17:22.172914+0000","last_undegraded":"2024-06-03T02:17:22.172914+0000","last_fullsized":"2024-06-03T02:17:22.172914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:08.243703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:24.239124+0000","last_change":"2024-06-03T02:14:23.259980+0000","last_active":"2024-06-03T02:16:24.239124+0000","last_peered":"2024-06-03T02:16:24.239124+0000","last_clean":"2024-06-03T02:16:24.239124+0000","last_became_active":"2024-06-03T02:14:23.258308+0000","last_became_peered":"2024-06-03T02:14:23.258308+0000","last_unstale":"2024-06-03T02:16:24.239124+0000","last_undegraded":"2024-06-03T02:16:24.239124+0000","last_fullsized":"2024-06-03T02:16:24.239124+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:28.833994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'19","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:21.815379+0000","last_change":"2024-06-03T02:14:05.244855+0000","last_active":"2024-06-03T02:19:21.815379+0000","last_peered":"2024-06-03T02:19:21.815379+0000","last_clean":"2024-06-03T02:19:21.815379+0000","last_became_active":"2024-06-03T02:14:05.244335+0000","last_became_peered":"2024-06-03T02:14:05.244335+0000","last_unstale":"2024-06-03T02:19:21.815379+0000","last_undegraded":"2024-06-03T02:19:21.815379+0000","last_fullsized":"2024-06-03T02:19:21.815379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:53:19.065411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185900+0000","last_change":"2024-06-03T02:14:09.750761+0000","last_active":"2024-06-03T02:17:22.185900+0000","last_peered":"2024-06-03T02:17:22.185900+0000","last_clean":"2024-06-03T02:17:22.185900+0000","last_became_active":"2024-06-03T02:14:09.750336+0000","last_became_peered":"2024-06-03T02:14:09.750336+0000","last_unstale":"2024-06-03T02:17:22.185900+0000","last_undegraded":"2024-06-03T02:17:22.185900+0000","last_fullsized":"2024-06-03T02:17:22.185900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:59:46.651149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:32.211290+0000","last_change":"2024-06-03T02:14:23.243287+0000","last_active":"2024-06-03T02:16:32.211290+0000","last_peered":"2024-06-03T02:16:32.211290+0000","last_clean":"2024-06-03T02:16:32.211290+0000","last_became_active":"2024-06-03T02:14:23.242828+0000","last_became_peered":"2024-06-03T02:14:23.242828+0000","last_unstale":"2024-06-03T02:16:32.211290+0000","last_undegraded":"2024-06-03T02:16:32.211290+0000","last_fullsized":"2024-06-03T02:16:32.211290+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:12.312438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"99'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:16.196453+0000","last_change":"2024-06-03T02:14:05.596288+0000","last_active":"2024-06-03T02:16:16.196453+0000","last_peered":"2024-06-03T02:16:16.196453+0000","last_clean":"2024-06-03T02:16:16.196453+0000","last_became_active":"2024-06-03T02:14:05.595868+0000","last_became_peered":"2024-06-03T02:14:05.595868+0000","last_unstale":"2024-06-03T02:16:16.196453+0000","last_undegraded":"2024-06-03T02:16:16.196453+0000","last_fullsized":"2024-06-03T02:16:16.196453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:50.291541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.189478+0000","last_change":"2024-06-03T02:14:09.765317+0000","last_active":"2024-06-03T02:17:22.189478+0000","last_peered":"2024-06-03T02:17:22.189478+0000","last_clean":"2024-06-03T02:17:22.189478+0000","last_became_active":"2024-06-03T02:14:09.764983+0000","last_became_peered":"2024-06-03T02:14:09.764983+0000","last_unstale":"2024-06-03T02:17:22.189478+0000","last_undegraded":"2024-06-03T02:17:22.189478+0000","last_fullsized":"2024-06-03T02:17:22.189478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:20.075276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:41.198968+0000","last_change":"2024-06-03T02:14:23.265352+0000","last_active":"2024-06-03T02:16:41.198968+0000","last_peered":"2024-06-03T02:16:41.198968+0000","last_clean":"2024-06-03T02:16:41.198968+0000","last_became_active":"2024-06-03T02:14:23.264932+0000","last_became_peered":"2024-06-03T02:14:23.264932+0000","last_unstale":"2024-06-03T02:16:41.198968+0000","last_undegraded":"2024-06-03T02:16:41.198968+0000","last_fullsized":"2024-06-03T02:16:41.198968+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:43.803170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"99'2","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:18.326424+0000","last_change":"2024-06-03T02:14:05.248563+0000","last_active":"2024-06-03T02:16:18.326424+0000","last_peered":"2024-06-03T02:16:18.326424+0000","last_clean":"2024-06-03T02:16:18.326424+0000","last_became_active":"2024-06-03T02:14:05.248306+0000","last_became_peered":"2024-06-03T02:14:05.248306+0000","last_unstale":"2024-06-03T02:16:18.326424+0000","last_undegraded":"2024-06-03T02:16:18.326424+0000","last_fullsized":"2024-06-03T02:16:18.326424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:23:58.130157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"101'2","reported_seq":42,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.149643+0000","last_change":"2024-06-03T02:14:09.745948+0000","last_active":"2024-06-03T02:17:22.149643+0000","last_peered":"2024-06-03T02:17:22.149643+0000","last_clean":"2024-06-03T02:17:22.149643+0000","last_became_active":"2024-06-03T02:14:09.745626+0000","last_became_peered":"2024-06-03T02:14:09.745626+0000","last_unstale":"2024-06-03T02:17:22.149643+0000","last_undegraded":"2024-06-03T02:17:22.149643+0000","last_fullsized":"2024-06-03T02:17:22.149643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:34.742129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:57.181841+0000","last_change":"2024-06-03T02:15:54.341824+0000","last_active":"2024-06-03T02:15:57.181841+0000","last_peered":"2024-06-03T02:15:57.181841+0000","last_clean":"2024-06-03T02:15:57.181841+0000","last_became_active":"2024-06-03T02:15:54.341267+0000","last_became_peered":"2024-06-03T02:15:54.341267+0000","last_unstale":"2024-06-03T02:15:57.181841+0000","last_undegraded":"2024-06-03T02:15:57.181841+0000","last_fullsized":"2024-06-03T02:15:57.181841+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:47.714056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"99'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:19.238284+0000","last_change":"2024-06-03T02:14:05.244806+0000","last_active":"2024-06-03T02:16:19.238284+0000","last_peered":"2024-06-03T02:16:19.238284+0000","last_clean":"2024-06-03T02:16:19.238284+0000","last_became_active":"2024-06-03T02:14:05.242719+0000","last_became_peered":"2024-06-03T02:14:05.242719+0000","last_unstale":"2024-06-03T02:16:19.238284+0000","last_undegraded":"2024-06-03T02:16:19.238284+0000","last_fullsized":"2024-06-03T02:16:19.238284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:31.314266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181636+0000","last_change":"2024-06-03T02:14:09.755102+0000","last_active":"2024-06-03T02:17:22.181636+0000","last_peered":"2024-06-03T02:17:22.181636+0000","last_clean":"2024-06-03T02:17:22.181636+0000","last_became_active":"2024-06-03T02:14:09.754748+0000","last_became_peered":"2024-06-03T02:14:09.754748+0000","last_unstale":"2024-06-03T02:17:22.181636+0000","last_undegraded":"2024-06-03T02:17:22.181636+0000","last_fullsized":"2024-06-03T02:17:22.181636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:59.920710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"101'1","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:41.198931+0000","last_change":"2024-06-03T02:14:23.271974+0000","last_active":"2024-06-03T02:16:41.198931+0000","last_peered":"2024-06-03T02:16:41.198931+0000","last_clean":"2024-06-03T02:16:41.198931+0000","last_became_active":"2024-06-03T02:14:23.271470+0000","last_became_peered":"2024-06-03T02:14:23.271470+0000","last_unstale":"2024-06-03T02:16:41.198931+0000","last_undegraded":"2024-06-03T02:16:41.198931+0000","last_fullsized":"2024-06-03T02:16:41.198931+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:03.159351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"101'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:18:27.223978+0000","last_change":"2024-06-03T02:14:05.242261+0000","last_active":"2024-06-03T02:18:27.223978+0000","last_peered":"2024-06-03T02:18:27.223978+0000","last_clean":"2024-06-03T02:18:27.223978+0000","last_became_active":"2024-06-03T02:14:05.241854+0000","last_became_peered":"2024-06-03T02:14:05.241854+0000","last_unstale":"2024-06-03T02:18:27.223978+0000","last_undegraded":"2024-06-03T02:18:27.223978+0000","last_fullsized":"2024-06-03T02:18:27.223978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:21.427703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186160+0000","last_change":"2024-06-03T02:14:09.763591+0000","last_active":"2024-06-03T02:17:22.186160+0000","last_peered":"2024-06-03T02:17:22.186160+0000","last_clean":"2024-06-03T02:17:22.186160+0000","last_became_active":"2024-06-03T02:14:09.763240+0000","last_became_peered":"2024-06-03T02:14:09.763240+0000","last_unstale":"2024-06-03T02:17:22.186160+0000","last_undegraded":"2024-06-03T02:17:22.186160+0000","last_fullsized":"2024-06-03T02:17:22.186160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:40.146898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:41.198957+0000","last_change":"2024-06-03T02:14:23.265011+0000","last_active":"2024-06-03T02:16:41.198957+0000","last_peered":"2024-06-03T02:16:41.198957+0000","last_clean":"2024-06-03T02:16:41.198957+0000","last_became_active":"2024-06-03T02:14:23.263767+0000","last_became_peered":"2024-06-03T02:14:23.263767+0000","last_unstale":"2024-06-03T02:16:41.198957+0000","last_undegraded":"2024-06-03T02:16:41.198957+0000","last_fullsized":"2024-06-03T02:16:41.198957+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:34:04.274545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"99'4","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:22.184364+0000","last_change":"2024-06-03T02:14:53.474879+0000","last_active":"2024-06-03T02:16:22.184364+0000","last_peered":"2024-06-03T02:16:22.184364+0000","last_clean":"2024-06-03T02:16:22.184364+0000","last_became_active":"2024-06-03T02:14:53.472209+0000","last_became_peered":"2024-06-03T02:14:53.472209+0000","last_unstale":"2024-06-03T02:16:22.184364+0000","last_undegraded":"2024-06-03T02:16:22.184364+0000","last_fullsized":"2024-06-03T02:16:22.184364+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:36:57.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172711+0000","last_change":"2024-06-03T02:14:09.747756+0000","last_active":"2024-06-03T02:17:22.172711+0000","last_peered":"2024-06-03T02:17:22.172711+0000","last_clean":"2024-06-03T02:17:22.172711+0000","last_became_active":"2024-06-03T02:14:09.747453+0000","last_became_peered":"2024-06-03T02:14:09.747453+0000","last_unstale":"2024-06-03T02:17:22.172711+0000","last_undegraded":"2024-06-03T02:17:22.172711+0000","last_fullsized":"2024-06-03T02:17:22.172711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:19:02.341367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:33.328239+0000","last_change":"2024-06-03T02:14:23.248594+0000","last_active":"2024-06-03T02:16:33.328239+0000","last_peered":"2024-06-03T02:16:33.328239+0000","last_clean":"2024-06-03T02:16:33.328239+0000","last_became_active":"2024-06-03T02:14:23.248080+0000","last_became_peered":"2024-06-03T02:14:23.248080+0000","last_unstale":"2024-06-03T02:16:33.328239+0000","last_undegraded":"2024-06-03T02:16:33.328239+0000","last_fullsized":"2024-06-03T02:16:33.328239+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:24:42.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"101'6","reported_seq":286,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:25.657346+0000","last_change":"2024-06-03T02:14:05.596274+0000","last_active":"2024-06-03T02:19:25.657346+0000","last_peered":"2024-06-03T02:19:25.657346+0000","last_clean":"2024-06-03T02:19:25.657346+0000","last_became_active":"2024-06-03T02:14:05.595678+0000","last_became_peered":"2024-06-03T02:14:05.595678+0000","last_unstale":"2024-06-03T02:19:25.657346+0000","last_undegraded":"2024-06-03T02:19:25.657346+0000","last_fullsized":"2024-06-03T02:19:25.657346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:13:25.436688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.173298+0000","last_change":"2024-06-03T02:14:09.744934+0000","last_active":"2024-06-03T02:17:22.173298+0000","last_peered":"2024-06-03T02:17:22.173298+0000","last_clean":"2024-06-03T02:17:22.173298+0000","last_became_active":"2024-06-03T02:14:09.744621+0000","last_became_peered":"2024-06-03T02:14:09.744621+0000","last_unstale":"2024-06-03T02:17:22.173298+0000","last_undegraded":"2024-06-03T02:17:22.173298+0000","last_fullsized":"2024-06-03T02:17:22.173298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:01:51.970160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:37.761133+0000","last_change":"2024-06-03T02:14:23.273123+0000","last_active":"2024-06-03T02:16:37.761133+0000","last_peered":"2024-06-03T02:16:37.761133+0000","last_clean":"2024-06-03T02:16:37.761133+0000","last_became_active":"2024-06-03T02:14:23.272153+0000","last_became_peered":"2024-06-03T02:14:23.272153+0000","last_unstale":"2024-06-03T02:16:37.761133+0000","last_undegraded":"2024-06-03T02:16:37.761133+0000","last_fullsized":"2024-06-03T02:16:37.761133+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:01:43.374413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"99'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:21.848723+0000","last_change":"2024-06-03T02:14:05.245094+0000","last_active":"2024-06-03T02:16:21.848723+0000","last_peered":"2024-06-03T02:16:21.848723+0000","last_clean":"2024-06-03T02:16:21.848723+0000","last_became_active":"2024-06-03T02:14:05.244678+0000","last_became_peered":"2024-06-03T02:14:05.244678+0000","last_unstale":"2024-06-03T02:16:21.848723+0000","last_undegraded":"2024-06-03T02:16:21.848723+0000","last_fullsized":"2024-06-03T02:16:21.848723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:13:26.546135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185237+0000","last_change":"2024-06-03T02:14:09.745919+0000","last_active":"2024-06-03T02:17:22.185237+0000","last_peered":"2024-06-03T02:17:22.185237+0000","last_clean":"2024-06-03T02:17:22.185237+0000","last_became_active":"2024-06-03T02:14:09.745576+0000","last_became_peered":"2024-06-03T02:14:09.745576+0000","last_unstale":"2024-06-03T02:17:22.185237+0000","last_undegraded":"2024-06-03T02:17:22.185237+0000","last_fullsized":"2024-06-03T02:17:22.185237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:29:31.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.850220+0000","last_change":"2024-06-03T02:14:23.244174+0000","last_active":"2024-06-03T02:16:36.850220+0000","last_peered":"2024-06-03T02:16:36.850220+0000","last_clean":"2024-06-03T02:16:36.850220+0000","last_became_active":"2024-06-03T02:14:23.243595+0000","last_became_peered":"2024-06-03T02:14:23.243595+0000","last_unstale":"2024-06-03T02:16:36.850220+0000","last_undegraded":"2024-06-03T02:16:36.850220+0000","last_fullsized":"2024-06-03T02:16:36.850220+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:12:36.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"99'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:21.216403+0000","last_change":"2024-06-03T02:14:05.247669+0000","last_active":"2024-06-03T02:16:21.216403+0000","last_peered":"2024-06-03T02:16:21.216403+0000","last_clean":"2024-06-03T02:16:21.216403+0000","last_became_active":"2024-06-03T02:14:05.246685+0000","last_became_peered":"2024-06-03T02:14:05.246685+0000","last_unstale":"2024-06-03T02:16:21.216403+0000","last_undegraded":"2024-06-03T02:16:21.216403+0000","last_fullsized":"2024-06-03T02:16:21.216403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:39.454422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171702+0000","last_change":"2024-06-03T02:14:09.749242+0000","last_active":"2024-06-03T02:17:22.171702+0000","last_peered":"2024-06-03T02:17:22.171702+0000","last_clean":"2024-06-03T02:17:22.171702+0000","last_became_active":"2024-06-03T02:14:09.748370+0000","last_became_peered":"2024-06-03T02:14:09.748370+0000","last_unstale":"2024-06-03T02:17:22.171702+0000","last_undegraded":"2024-06-03T02:17:22.171702+0000","last_fullsized":"2024-06-03T02:17:22.171702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:26.189960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.162116+0000","last_change":"2024-06-03T02:14:09.745966+0000","last_active":"2024-06-03T02:17:22.162116+0000","last_peered":"2024-06-03T02:17:22.162116+0000","last_clean":"2024-06-03T02:17:22.162116+0000","last_became_active":"2024-06-03T02:14:09.745590+0000","last_became_peered":"2024-06-03T02:14:09.745590+0000","last_unstale":"2024-06-03T02:17:22.162116+0000","last_undegraded":"2024-06-03T02:17:22.162116+0000","last_fullsized":"2024-06-03T02:17:22.162116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:46.744629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"101'12","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163485+0000","last_change":"2024-06-03T02:14:09.744964+0000","last_active":"2024-06-03T02:17:22.163485+0000","last_peered":"2024-06-03T02:17:22.163485+0000","last_clean":"2024-06-03T02:17:22.163485+0000","last_became_active":"2024-06-03T02:14:09.743877+0000","last_became_peered":"2024-06-03T02:14:09.743877+0000","last_unstale":"2024-06-03T02:17:22.163485+0000","last_undegraded":"2024-06-03T02:17:22.163485+0000","last_fullsized":"2024-06-03T02:17:22.163485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:05:48.020670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186843+0000","last_change":"2024-06-03T02:14:09.761736+0000","last_active":"2024-06-03T02:17:22.186843+0000","last_peered":"2024-06-03T02:17:22.186843+0000","last_clean":"2024-06-03T02:17:22.186843+0000","last_became_active":"2024-06-03T02:14:09.761477+0000","last_became_peered":"2024-06-03T02:14:09.761477+0000","last_unstale":"2024-06-03T02:17:22.186843+0000","last_undegraded":"2024-06-03T02:17:22.186843+0000","last_fullsized":"2024-06-03T02:17:22.186843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:00:04.631330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"101'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183594+0000","last_change":"2024-06-03T02:14:53.266594+0000","last_active":"2024-06-03T02:17:22.183594+0000","last_peered":"2024-06-03T02:17:22.183594+0000","last_clean":"2024-06-03T02:17:22.183594+0000","last_became_active":"2024-06-03T02:14:53.266104+0000","last_became_peered":"2024-06-03T02:14:53.266104+0000","last_unstale":"2024-06-03T02:17:22.183594+0000","last_undegraded":"2024-06-03T02:17:22.183594+0000","last_fullsized":"2024-06-03T02:17:22.183594+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:12.018082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174909+0000","last_change":"2024-06-03T02:14:09.772332+0000","last_active":"2024-06-03T02:17:22.174909+0000","last_peered":"2024-06-03T02:17:22.174909+0000","last_clean":"2024-06-03T02:17:22.174909+0000","last_became_active":"2024-06-03T02:14:09.771888+0000","last_became_peered":"2024-06-03T02:14:09.771888+0000","last_unstale":"2024-06-03T02:17:22.174909+0000","last_undegraded":"2024-06-03T02:17:22.174909+0000","last_fullsized":"2024-06-03T02:17:22.174909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:58:49.414986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"101'6","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.156930+0000","last_change":"2024-06-03T02:14:53.270828+0000","last_active":"2024-06-03T02:17:22.156930+0000","last_peered":"2024-06-03T02:17:22.156930+0000","last_clean":"2024-06-03T02:17:22.156930+0000","last_became_active":"2024-06-03T02:14:53.270365+0000","last_became_peered":"2024-06-03T02:14:53.270365+0000","last_unstale":"2024-06-03T02:17:22.156930+0000","last_undegraded":"2024-06-03T02:17:22.156930+0000","last_fullsized":"2024-06-03T02:17:22.156930+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.051756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"101'15","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.161647+0000","last_change":"2024-06-03T02:14:09.747709+0000","last_active":"2024-06-03T02:17:22.161647+0000","last_peered":"2024-06-03T02:17:22.161647+0000","last_clean":"2024-06-03T02:17:22.161647+0000","last_became_active":"2024-06-03T02:14:09.747359+0000","last_became_peered":"2024-06-03T02:14:09.747359+0000","last_unstale":"2024-06-03T02:17:22.161647+0000","last_undegraded":"2024-06-03T02:17:22.161647+0000","last_fullsized":"2024-06-03T02:17:22.161647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:37:24.873442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"101'2","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.145482+0000","last_change":"2024-06-03T02:14:53.473136+0000","last_active":"2024-06-03T02:17:22.145482+0000","last_peered":"2024-06-03T02:17:22.145482+0000","last_clean":"2024-06-03T02:17:22.145482+0000","last_became_active":"2024-06-03T02:14:53.471214+0000","last_became_peered":"2024-06-03T02:14:53.471214+0000","last_unstale":"2024-06-03T02:17:22.145482+0000","last_undegraded":"2024-06-03T02:17:22.145482+0000","last_fullsized":"2024-06-03T02:17:22.145482+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:25.822042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184932+0000","last_change":"2024-06-03T02:14:09.746463+0000","last_active":"2024-06-03T02:17:22.184932+0000","last_peered":"2024-06-03T02:17:22.184932+0000","last_clean":"2024-06-03T02:17:22.184932+0000","last_became_active":"2024-06-03T02:14:09.746115+0000","last_became_peered":"2024-06-03T02:14:09.746115+0000","last_unstale":"2024-06-03T02:17:22.184932+0000","last_undegraded":"2024-06-03T02:17:22.184932+0000","last_fullsized":"2024-06-03T02:17:22.184932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:51:23.050739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184825+0000","last_change":"2024-06-03T02:14:09.755467+0000","last_active":"2024-06-03T02:17:22.184825+0000","last_peered":"2024-06-03T02:17:22.184825+0000","last_clean":"2024-06-03T02:17:22.184825+0000","last_became_active":"2024-06-03T02:14:09.755103+0000","last_became_peered":"2024-06-03T02:14:09.755103+0000","last_unstale":"2024-06-03T02:17:22.184825+0000","last_undegraded":"2024-06-03T02:17:22.184825+0000","last_fullsized":"2024-06-03T02:17:22.184825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:22.834519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186153+0000","last_change":"2024-06-03T02:14:09.758201+0000","last_active":"2024-06-03T02:17:22.186153+0000","last_peered":"2024-06-03T02:17:22.186153+0000","last_clean":"2024-06-03T02:17:22.186153+0000","last_became_active":"2024-06-03T02:14:09.757819+0000","last_became_peered":"2024-06-03T02:14:09.757819+0000","last_unstale":"2024-06-03T02:17:22.186153+0000","last_undegraded":"2024-06-03T02:17:22.186153+0000","last_fullsized":"2024-06-03T02:17:22.186153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:44.802694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183397+0000","last_change":"2024-06-03T02:14:09.768132+0000","last_active":"2024-06-03T02:17:22.183397+0000","last_peered":"2024-06-03T02:17:22.183397+0000","last_clean":"2024-06-03T02:17:22.183397+0000","last_became_active":"2024-06-03T02:14:09.767642+0000","last_became_peered":"2024-06-03T02:14:09.767642+0000","last_unstale":"2024-06-03T02:17:22.183397+0000","last_undegraded":"2024-06-03T02:17:22.183397+0000","last_fullsized":"2024-06-03T02:17:22.183397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:32:30.455418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.159132+0000","last_change":"2024-06-03T02:14:09.745603+0000","last_active":"2024-06-03T02:17:22.159132+0000","last_peered":"2024-06-03T02:17:22.159132+0000","last_clean":"2024-06-03T02:17:22.159132+0000","last_became_active":"2024-06-03T02:14:09.745245+0000","last_became_peered":"2024-06-03T02:14:09.745245+0000","last_unstale":"2024-06-03T02:17:22.159132+0000","last_undegraded":"2024-06-03T02:17:22.159132+0000","last_fullsized":"2024-06-03T02:17:22.159132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:09:14.507101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81982,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2347,"num_read_kb":1260,"num_write":278,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":248466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318354,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":579107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579990,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37168,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707984,"statfs":{"total":95995035648,"available":95956975616,"internally_reserved":0,"allocated":1875968,"data_stored":1242653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743516,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":577953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005154,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":577637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266792,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1785856,"data_stored":1176810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430318,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":573988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1249280,"data_stored":642164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738512,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":575303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":651223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149821,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888184,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":578452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691956,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37088,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708064,"statfs":{"total":95995035648,"available":95957057536,"internally_reserved":0,"allocated":1794048,"data_stored":1170940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":75192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":78488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":75454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T02:19:30.089 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-03T02:19:30.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-03T02:19:30.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.332+0000 7fae80e00640 1 -- 172.21.15.79:0/3263641187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c109930 msgr2=0x7fae7c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:30.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.332+0000 7fae80e00640 1 --2- 172.21.15.79:0/3263641187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c109930 0x7fae7c110250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae74009860 tx=0x7fae7402a340 comp rx=0 tx=0).stop 2024-06-03T02:19:30.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.332+0000 7fae80e00640 1 -- 172.21.15.79:0/3263641187 shutdown_connections 2024-06-03T02:19:30.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.332+0000 7fae80e00640 1 --2- 172.21.15.79:0/3263641187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c109930 0x7fae7c110250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.332+0000 7fae80e00640 1 --2- 172.21.15.79:0/3263641187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae7c108f70 0x7fae7c1093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.332+0000 7fae80e00640 1 --2- 172.21.15.79:0/3263641187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae7c107e20 0x7fae7c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.332+0000 7fae80e00640 1 -- 172.21.15.79:0/3263641187 >> 172.21.15.79:0/3263641187 conn(0x7fae7c076de0 msgr2=0x7fae7c079200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.332+0000 7fae80e00640 1 -- 172.21.15.79:0/3263641187 shutdown_connections 2024-06-03T02:19:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.332+0000 7fae80e00640 1 -- 172.21.15.79:0/3263641187 wait complete. 2024-06-03T02:19:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.333+0000 7fae80e00640 1 Processor -- start 2024-06-03T02:19:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.333+0000 7fae80e00640 1 -- start start 2024-06-03T02:19:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.333+0000 7fae80e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae7c107e20 0x7fae7c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.333+0000 7fae80e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c108f70 0x7fae7c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.333+0000 7fae80e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae7c109930 0x7fae7c1a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.333+0000 7fae80e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c113aa0 con 0x7fae7c108f70 2024-06-03T02:19:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.333+0000 7fae80e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c113920 con 0x7fae7c109930 2024-06-03T02:19:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.333+0000 7fae80e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c113c20 con 0x7fae7c107e20 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c108f70 0x7fae7c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c108f70 0x7fae7c19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36586/0 (socket says 172.21.15.79:36586) 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7aa00640 1 -- 172.21.15.79:0/2931958834 learned_addr learned my addr 172.21.15.79:0/2931958834 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7b400640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae7c107e20 0x7fae7c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7be00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae7c109930 0x7fae7c1a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7aa00640 1 -- 172.21.15.79:0/2931958834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae7c107e20 msgr2=0x7fae7c19e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7aa00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae7c107e20 0x7fae7c19e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7aa00640 1 -- 172.21.15.79:0/2931958834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae7c109930 msgr2=0x7fae7c1a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7aa00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae7c109930 0x7fae7c1a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7aa00640 1 -- 172.21.15.79:0/2931958834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c1a5cd0 con 0x7fae7c108f70 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.334+0000 7fae7aa00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c108f70 0x7fae7c19e5f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae6800e9a0 tx=0x7fae6800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.335+0000 7fae73e00640 1 -- 172.21.15.79:0/2931958834 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6800cd80 con 0x7fae7c108f70 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.335+0000 7fae73e00640 1 -- 172.21.15.79:0/2931958834 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae68010910 con 0x7fae7c108f70 2024-06-03T02:19:30.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.335+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7c1a5fc0 con 0x7fae7c108f70 2024-06-03T02:19:30.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.335+0000 7fae73e00640 1 -- 172.21.15.79:0/2931958834 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68018a30 con 0x7fae7c108f70 2024-06-03T02:19:30.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.335+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7c1a6480 con 0x7fae7c108f70 2024-06-03T02:19:30.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.336+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7c10f130 con 0x7fae7c108f70 2024-06-03T02:19:30.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.338+0000 7fae73e00640 1 -- 172.21.15.79:0/2931958834 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fae68010ac0 con 0x7fae7c108f70 2024-06-03T02:19:30.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.340+0000 7fae73e00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fae600a2c20 0x7fae600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:30.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.340+0000 7fae73e00640 1 -- 172.21.15.79:0/2931958834 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fae680cf8b0 con 0x7fae7c108f70 2024-06-03T02:19:30.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.340+0000 7fae7b400640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fae600a2c20 0x7fae600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:30.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.341+0000 7fae7b400640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fae600a2c20 0x7fae600a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0099a0 tx=0x7fae6c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:30.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.346+0000 7fae73e00640 1 -- 172.21.15.79:0/2931958834 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fae68053850 con 0x7fae7c108f70 2024-06-03T02:19:30.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.544+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 --> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae7c061de0 con 0x7fae600a2c20 2024-06-03T02:19:30.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.570+0000 7fae73e00640 1 -- 172.21.15.79:0/2931958834 <== mgr.14152 v2:172.21.15.79:6800/2278164 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340748 (secure 0 0 0) 0x7fae7c061de0 con 0x7fae600a2c20 2024-06-03T02:19:30.573 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:30.577 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fae600a2c20 msgr2=0x7fae600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fae600a2c20 0x7fae600a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0099a0 tx=0x7fae6c009500 comp rx=0 tx=0).stop 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c108f70 msgr2=0x7fae7c19e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c108f70 0x7fae7c19e5f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae6800e9a0 tx=0x7fae6800ee60 comp rx=0 tx=0).stop 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 shutdown_connections 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae7c109930 0x7fae7c1a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae7c108f70 0x7fae7c19e5f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fae600a2c20 0x7fae600a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 --2- 172.21.15.79:0/2931958834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae7c107e20 0x7fae7c19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:30.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.578+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 >> 172.21.15.79:0/2931958834 conn(0x7fae7c076de0 msgr2=0x7fae7c10c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:30.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.579+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 shutdown_connections 2024-06-03T02:19:30.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:30.579+0000 7fae80e00640 1 -- 172.21.15.79:0/2931958834 wait complete. 2024-06-03T02:19:30.599 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-06-03T02:19:30.302935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":675104,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2494,"num_read_kb":1387,"num_write":462,"num_write_kb":3116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":879,"ondisk_log_size":879,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360852,"kb_used_data":15860,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580972,"statfs":{"total":1151940427776,"available":1151570915328,"internally_reserved":0,"allocated":16240640,"data_stored":8930198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9759,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":78,"num_write":16,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005465"},"pg_stats":[{"pgid":"3.3f","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186895+0000","last_change":"2024-06-03T02:14:09.768431+0000","last_active":"2024-06-03T02:17:22.186895+0000","last_peered":"2024-06-03T02:17:22.186895+0000","last_clean":"2024-06-03T02:17:22.186895+0000","last_became_active":"2024-06-03T02:14:09.768059+0000","last_became_peered":"2024-06-03T02:14:09.768059+0000","last_unstale":"2024-06-03T02:17:22.186895+0000","last_undegraded":"2024-06-03T02:17:22.186895+0000","last_fullsized":"2024-06-03T02:17:22.186895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:12.265896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188670+0000","last_change":"2024-06-03T02:14:09.771842+0000","last_active":"2024-06-03T02:17:22.188670+0000","last_peered":"2024-06-03T02:17:22.188670+0000","last_clean":"2024-06-03T02:17:22.188670+0000","last_became_active":"2024-06-03T02:14:09.770961+0000","last_became_peered":"2024-06-03T02:14:09.770961+0000","last_unstale":"2024-06-03T02:17:22.188670+0000","last_undegraded":"2024-06-03T02:17:22.188670+0000","last_fullsized":"2024-06-03T02:17:22.188670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:49:02.297128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183559+0000","last_change":"2024-06-03T02:14:09.758407+0000","last_active":"2024-06-03T02:17:22.183559+0000","last_peered":"2024-06-03T02:17:22.183559+0000","last_clean":"2024-06-03T02:17:22.183559+0000","last_became_active":"2024-06-03T02:14:09.758045+0000","last_became_peered":"2024-06-03T02:14:09.758045+0000","last_unstale":"2024-06-03T02:17:22.183559+0000","last_undegraded":"2024-06-03T02:17:22.183559+0000","last_fullsized":"2024-06-03T02:17:22.183559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:01:08.166971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.152098+0000","last_change":"2024-06-03T02:14:09.747658+0000","last_active":"2024-06-03T02:17:22.152098+0000","last_peered":"2024-06-03T02:17:22.152098+0000","last_clean":"2024-06-03T02:17:22.152098+0000","last_became_active":"2024-06-03T02:14:09.747321+0000","last_became_peered":"2024-06-03T02:14:09.747321+0000","last_unstale":"2024-06-03T02:17:22.152098+0000","last_undegraded":"2024-06-03T02:17:22.152098+0000","last_fullsized":"2024-06-03T02:17:22.152098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:08.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184933+0000","last_change":"2024-06-03T02:14:09.758161+0000","last_active":"2024-06-03T02:17:22.184933+0000","last_peered":"2024-06-03T02:17:22.184933+0000","last_clean":"2024-06-03T02:17:22.184933+0000","last_became_active":"2024-06-03T02:14:09.757800+0000","last_became_peered":"2024-06-03T02:14:09.757800+0000","last_unstale":"2024-06-03T02:17:22.184933+0000","last_undegraded":"2024-06-03T02:17:22.184933+0000","last_fullsized":"2024-06-03T02:17:22.184933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:47.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.160478+0000","last_change":"2024-06-03T02:14:09.741818+0000","last_active":"2024-06-03T02:17:22.160478+0000","last_peered":"2024-06-03T02:17:22.160478+0000","last_clean":"2024-06-03T02:17:22.160478+0000","last_became_active":"2024-06-03T02:14:09.741219+0000","last_became_peered":"2024-06-03T02:14:09.741219+0000","last_unstale":"2024-06-03T02:17:22.160478+0000","last_undegraded":"2024-06-03T02:17:22.160478+0000","last_fullsized":"2024-06-03T02:17:22.160478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:16:30.585245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163369+0000","last_change":"2024-06-03T02:14:09.744863+0000","last_active":"2024-06-03T02:17:22.163369+0000","last_peered":"2024-06-03T02:17:22.163369+0000","last_clean":"2024-06-03T02:17:22.163369+0000","last_became_active":"2024-06-03T02:14:09.744514+0000","last_became_peered":"2024-06-03T02:14:09.744514+0000","last_unstale":"2024-06-03T02:17:22.163369+0000","last_undegraded":"2024-06-03T02:17:22.163369+0000","last_fullsized":"2024-06-03T02:17:22.163369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:51:42.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187625+0000","last_change":"2024-06-03T02:14:09.766897+0000","last_active":"2024-06-03T02:17:22.187625+0000","last_peered":"2024-06-03T02:17:22.187625+0000","last_clean":"2024-06-03T02:17:22.187625+0000","last_became_active":"2024-06-03T02:14:09.765461+0000","last_became_peered":"2024-06-03T02:14:09.765461+0000","last_unstale":"2024-06-03T02:17:22.187625+0000","last_undegraded":"2024-06-03T02:17:22.187625+0000","last_fullsized":"2024-06-03T02:17:22.187625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:39:28.641517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172030+0000","last_change":"2024-06-03T02:14:09.746451+0000","last_active":"2024-06-03T02:17:22.172030+0000","last_peered":"2024-06-03T02:17:22.172030+0000","last_clean":"2024-06-03T02:17:22.172030+0000","last_became_active":"2024-06-03T02:14:09.746117+0000","last_became_peered":"2024-06-03T02:14:09.746117+0000","last_unstale":"2024-06-03T02:17:22.172030+0000","last_undegraded":"2024-06-03T02:17:22.172030+0000","last_fullsized":"2024-06-03T02:17:22.172030+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:42.599993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"101'16","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.162982+0000","last_change":"2024-06-03T02:14:09.750680+0000","last_active":"2024-06-03T02:17:22.162982+0000","last_peered":"2024-06-03T02:17:22.162982+0000","last_clean":"2024-06-03T02:17:22.162982+0000","last_became_active":"2024-06-03T02:14:09.746138+0000","last_became_peered":"2024-06-03T02:14:09.746138+0000","last_unstale":"2024-06-03T02:17:22.162982+0000","last_undegraded":"2024-06-03T02:17:22.162982+0000","last_fullsized":"2024-06-03T02:17:22.162982+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:04:28.188323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172525+0000","last_change":"2024-06-03T02:14:09.743467+0000","last_active":"2024-06-03T02:17:22.172525+0000","last_peered":"2024-06-03T02:17:22.172525+0000","last_clean":"2024-06-03T02:17:22.172525+0000","last_became_active":"2024-06-03T02:14:09.743152+0000","last_became_peered":"2024-06-03T02:14:09.743152+0000","last_unstale":"2024-06-03T02:17:22.172525+0000","last_undegraded":"2024-06-03T02:17:22.172525+0000","last_fullsized":"2024-06-03T02:17:22.172525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:27.336630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.148334+0000","last_change":"2024-06-03T02:14:09.747142+0000","last_active":"2024-06-03T02:17:22.148334+0000","last_peered":"2024-06-03T02:17:22.148334+0000","last_clean":"2024-06-03T02:17:22.148334+0000","last_became_active":"2024-06-03T02:14:09.746694+0000","last_became_peered":"2024-06-03T02:14:09.746694+0000","last_unstale":"2024-06-03T02:17:22.148334+0000","last_undegraded":"2024-06-03T02:17:22.148334+0000","last_fullsized":"2024-06-03T02:17:22.148334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:04.507667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179213+0000","last_change":"2024-06-03T02:14:09.759906+0000","last_active":"2024-06-03T02:17:22.179213+0000","last_peered":"2024-06-03T02:17:22.179213+0000","last_clean":"2024-06-03T02:17:22.179213+0000","last_became_active":"2024-06-03T02:14:09.755450+0000","last_became_peered":"2024-06-03T02:14:09.755450+0000","last_unstale":"2024-06-03T02:17:22.179213+0000","last_undegraded":"2024-06-03T02:17:22.179213+0000","last_fullsized":"2024-06-03T02:17:22.179213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:23.057956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.187178+0000","last_change":"2024-06-03T02:14:09.769388+0000","last_active":"2024-06-03T02:17:22.187178+0000","last_peered":"2024-06-03T02:17:22.187178+0000","last_clean":"2024-06-03T02:17:22.187178+0000","last_became_active":"2024-06-03T02:14:09.769029+0000","last_became_peered":"2024-06-03T02:14:09.769029+0000","last_unstale":"2024-06-03T02:17:22.187178+0000","last_undegraded":"2024-06-03T02:17:22.187178+0000","last_fullsized":"2024-06-03T02:17:22.187178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:24.751039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181320+0000","last_change":"2024-06-03T02:14:09.768584+0000","last_active":"2024-06-03T02:17:22.181320+0000","last_peered":"2024-06-03T02:17:22.181320+0000","last_clean":"2024-06-03T02:17:22.181320+0000","last_became_active":"2024-06-03T02:14:09.765737+0000","last_became_peered":"2024-06-03T02:14:09.765737+0000","last_unstale":"2024-06-03T02:17:22.181320+0000","last_undegraded":"2024-06-03T02:17:22.181320+0000","last_fullsized":"2024-06-03T02:17:22.181320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:36:18.807762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181401+0000","last_change":"2024-06-03T02:14:09.757828+0000","last_active":"2024-06-03T02:17:22.181401+0000","last_peered":"2024-06-03T02:17:22.181401+0000","last_clean":"2024-06-03T02:17:22.181401+0000","last_became_active":"2024-06-03T02:14:09.757040+0000","last_became_peered":"2024-06-03T02:14:09.757040+0000","last_unstale":"2024-06-03T02:17:22.181401+0000","last_undegraded":"2024-06-03T02:17:22.181401+0000","last_fullsized":"2024-06-03T02:17:22.181401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:26:28.694022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188574+0000","last_change":"2024-06-03T02:14:09.768263+0000","last_active":"2024-06-03T02:17:22.188574+0000","last_peered":"2024-06-03T02:17:22.188574+0000","last_clean":"2024-06-03T02:17:22.188574+0000","last_became_active":"2024-06-03T02:14:09.767988+0000","last_became_peered":"2024-06-03T02:14:09.767988+0000","last_unstale":"2024-06-03T02:17:22.188574+0000","last_undegraded":"2024-06-03T02:17:22.188574+0000","last_fullsized":"2024-06-03T02:17:22.188574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:28:37.432659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171811+0000","last_change":"2024-06-03T02:14:09.747118+0000","last_active":"2024-06-03T02:17:22.171811+0000","last_peered":"2024-06-03T02:17:22.171811+0000","last_clean":"2024-06-03T02:17:22.171811+0000","last_became_active":"2024-06-03T02:14:09.746580+0000","last_became_peered":"2024-06-03T02:14:09.746580+0000","last_unstale":"2024-06-03T02:17:22.171811+0000","last_undegraded":"2024-06-03T02:17:22.171811+0000","last_fullsized":"2024-06-03T02:17:22.171811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:33:04.607285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185388+0000","last_change":"2024-06-03T02:14:09.755755+0000","last_active":"2024-06-03T02:17:22.185388+0000","last_peered":"2024-06-03T02:17:22.185388+0000","last_clean":"2024-06-03T02:17:22.185388+0000","last_became_active":"2024-06-03T02:14:09.754503+0000","last_became_peered":"2024-06-03T02:14:09.754503+0000","last_unstale":"2024-06-03T02:17:22.185388+0000","last_undegraded":"2024-06-03T02:17:22.185388+0000","last_fullsized":"2024-06-03T02:17:22.185388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:30:58.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:52.213243+0000","last_change":"2024-06-03T02:14:23.260021+0000","last_active":"2024-06-03T02:16:52.213243+0000","last_peered":"2024-06-03T02:16:52.213243+0000","last_clean":"2024-06-03T02:16:52.213243+0000","last_became_active":"2024-06-03T02:14:23.259587+0000","last_became_peered":"2024-06-03T02:14:23.259587+0000","last_unstale":"2024-06-03T02:16:52.213243+0000","last_undegraded":"2024-06-03T02:16:52.213243+0000","last_fullsized":"2024-06-03T02:16:52.213243+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:09:02.403936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"101'14","reported_seq":291,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:24.835941+0000","last_change":"2024-06-03T02:14:05.239623+0000","last_active":"2024-06-03T02:19:24.835941+0000","last_peered":"2024-06-03T02:19:24.835941+0000","last_clean":"2024-06-03T02:19:24.835941+0000","last_became_active":"2024-06-03T02:14:05.239112+0000","last_became_peered":"2024-06-03T02:14:05.239112+0000","last_unstale":"2024-06-03T02:19:24.835941+0000","last_undegraded":"2024-06-03T02:19:24.835941+0000","last_fullsized":"2024-06-03T02:19:24.835941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:12:14.020068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":231,"num_read_kb":231,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185036+0000","last_change":"2024-06-03T02:14:09.764972+0000","last_active":"2024-06-03T02:17:22.185036+0000","last_peered":"2024-06-03T02:17:22.185036+0000","last_clean":"2024-06-03T02:17:22.185036+0000","last_became_active":"2024-06-03T02:14:09.764435+0000","last_became_peered":"2024-06-03T02:14:09.764435+0000","last_unstale":"2024-06-03T02:17:22.185036+0000","last_undegraded":"2024-06-03T02:17:22.185036+0000","last_fullsized":"2024-06-03T02:17:22.185036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:46:58.941554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.200287+0000","last_change":"2024-06-03T02:14:23.272076+0000","last_active":"2024-06-03T02:16:51.200287+0000","last_peered":"2024-06-03T02:16:51.200287+0000","last_clean":"2024-06-03T02:16:51.200287+0000","last_became_active":"2024-06-03T02:14:23.271578+0000","last_became_peered":"2024-06-03T02:14:23.271578+0000","last_unstale":"2024-06-03T02:16:51.200287+0000","last_undegraded":"2024-06-03T02:16:51.200287+0000","last_fullsized":"2024-06-03T02:16:51.200287+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:17:59.000291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"99'2","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:09.237250+0000","last_change":"2024-06-03T02:14:05.251952+0000","last_active":"2024-06-03T02:16:09.237250+0000","last_peered":"2024-06-03T02:16:09.237250+0000","last_clean":"2024-06-03T02:16:09.237250+0000","last_became_active":"2024-06-03T02:14:05.251152+0000","last_became_peered":"2024-06-03T02:14:05.251152+0000","last_unstale":"2024-06-03T02:16:09.237250+0000","last_undegraded":"2024-06-03T02:16:09.237250+0000","last_fullsized":"2024-06-03T02:16:09.237250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:24:32.802236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.190358+0000","last_change":"2024-06-03T02:14:09.764230+0000","last_active":"2024-06-03T02:17:22.190358+0000","last_peered":"2024-06-03T02:17:22.190358+0000","last_clean":"2024-06-03T02:17:22.190358+0000","last_became_active":"2024-06-03T02:14:09.761868+0000","last_became_peered":"2024-06-03T02:14:09.761868+0000","last_unstale":"2024-06-03T02:17:22.190358+0000","last_undegraded":"2024-06-03T02:17:22.190358+0000","last_fullsized":"2024-06-03T02:17:22.190358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:39:23.828546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:56.200917+0000","last_change":"2024-06-03T02:14:23.272169+0000","last_active":"2024-06-03T02:16:56.200917+0000","last_peered":"2024-06-03T02:16:56.200917+0000","last_clean":"2024-06-03T02:16:56.200917+0000","last_became_active":"2024-06-03T02:14:23.271636+0000","last_became_peered":"2024-06-03T02:14:23.271636+0000","last_unstale":"2024-06-03T02:16:56.200917+0000","last_undegraded":"2024-06-03T02:16:56.200917+0000","last_fullsized":"2024-06-03T02:16:56.200917+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:51:25.478028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"99'12","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:37.186203+0000","last_change":"2024-06-03T02:14:05.238590+0000","last_active":"2024-06-03T02:16:37.186203+0000","last_peered":"2024-06-03T02:16:37.186203+0000","last_clean":"2024-06-03T02:16:37.186203+0000","last_became_active":"2024-06-03T02:14:05.238128+0000","last_became_peered":"2024-06-03T02:14:05.238128+0000","last_unstale":"2024-06-03T02:16:37.186203+0000","last_undegraded":"2024-06-03T02:16:37.186203+0000","last_fullsized":"2024-06-03T02:16:37.186203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:21.298255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174715+0000","last_change":"2024-06-03T02:14:09.754489+0000","last_active":"2024-06-03T02:17:22.174715+0000","last_peered":"2024-06-03T02:17:22.174715+0000","last_clean":"2024-06-03T02:17:22.174715+0000","last_became_active":"2024-06-03T02:14:09.754155+0000","last_became_peered":"2024-06-03T02:14:09.754155+0000","last_unstale":"2024-06-03T02:17:22.174715+0000","last_undegraded":"2024-06-03T02:17:22.174715+0000","last_fullsized":"2024-06-03T02:17:22.174715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:20:43.841271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:56.200884+0000","last_change":"2024-06-03T02:14:23.261232+0000","last_active":"2024-06-03T02:16:56.200884+0000","last_peered":"2024-06-03T02:16:56.200884+0000","last_clean":"2024-06-03T02:16:56.200884+0000","last_became_active":"2024-06-03T02:14:23.260772+0000","last_became_peered":"2024-06-03T02:14:23.260772+0000","last_unstale":"2024-06-03T02:16:56.200884+0000","last_undegraded":"2024-06-03T02:16:56.200884+0000","last_fullsized":"2024-06-03T02:16:56.200884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:52.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"99'5","reported_seq":24,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:54.272244+0000","last_change":"2024-06-03T02:14:53.274084+0000","last_active":"2024-06-03T02:15:54.272244+0000","last_peered":"2024-06-03T02:15:54.272244+0000","last_clean":"2024-06-03T02:15:54.272244+0000","last_became_active":"2024-06-03T02:14:53.265253+0000","last_became_peered":"2024-06-03T02:14:53.265253+0000","last_unstale":"2024-06-03T02:15:54.272244+0000","last_undegraded":"2024-06-03T02:15:54.272244+0000","last_fullsized":"2024-06-03T02:15:54.272244+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:25:56.189308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"101'15","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.156115+0000","last_change":"2024-06-03T02:14:09.743450+0000","last_active":"2024-06-03T02:17:22.156115+0000","last_peered":"2024-06-03T02:17:22.156115+0000","last_clean":"2024-06-03T02:17:22.156115+0000","last_became_active":"2024-06-03T02:14:09.743071+0000","last_became_peered":"2024-06-03T02:14:09.743071+0000","last_unstale":"2024-06-03T02:17:22.156115+0000","last_undegraded":"2024-06-03T02:17:22.156115+0000","last_fullsized":"2024-06-03T02:17:22.156115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:43:44.278633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:57.213954+0000","last_change":"2024-06-03T02:15:54.342978+0000","last_active":"2024-06-03T02:16:57.213954+0000","last_peered":"2024-06-03T02:16:57.213954+0000","last_clean":"2024-06-03T02:16:57.213954+0000","last_became_active":"2024-06-03T02:15:54.342477+0000","last_became_peered":"2024-06-03T02:15:54.342477+0000","last_unstale":"2024-06-03T02:16:57.213954+0000","last_undegraded":"2024-06-03T02:16:57.213954+0000","last_fullsized":"2024-06-03T02:16:57.213954+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:47.980513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"101'8","reported_seq":290,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:25.810333+0000","last_change":"2024-06-03T02:14:05.237464+0000","last_active":"2024-06-03T02:19:25.810333+0000","last_peered":"2024-06-03T02:19:25.810333+0000","last_clean":"2024-06-03T02:19:25.810333+0000","last_became_active":"2024-06-03T02:14:05.237067+0000","last_became_peered":"2024-06-03T02:14:05.237067+0000","last_unstale":"2024-06-03T02:19:25.810333+0000","last_undegraded":"2024-06-03T02:19:25.810333+0000","last_fullsized":"2024-06-03T02:19:25.810333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:06:38.850484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.150011+0000","last_change":"2024-06-03T02:14:09.748741+0000","last_active":"2024-06-03T02:17:22.150011+0000","last_peered":"2024-06-03T02:17:22.150011+0000","last_clean":"2024-06-03T02:17:22.150011+0000","last_became_active":"2024-06-03T02:14:09.748402+0000","last_became_peered":"2024-06-03T02:14:09.748402+0000","last_unstale":"2024-06-03T02:17:22.150011+0000","last_undegraded":"2024-06-03T02:17:22.150011+0000","last_fullsized":"2024-06-03T02:17:22.150011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:03:25.664415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:57.213929+0000","last_change":"2024-06-03T02:14:23.260397+0000","last_active":"2024-06-03T02:16:57.213929+0000","last_peered":"2024-06-03T02:16:57.213929+0000","last_clean":"2024-06-03T02:16:57.213929+0000","last_became_active":"2024-06-03T02:14:23.259886+0000","last_became_peered":"2024-06-03T02:14:23.259886+0000","last_unstale":"2024-06-03T02:16:57.213929+0000","last_undegraded":"2024-06-03T02:16:57.213929+0000","last_fullsized":"2024-06-03T02:16:57.213929+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:46:18.562706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"99'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:38.829788+0000","last_change":"2024-06-03T02:14:05.241536+0000","last_active":"2024-06-03T02:16:38.829788+0000","last_peered":"2024-06-03T02:16:38.829788+0000","last_clean":"2024-06-03T02:16:38.829788+0000","last_became_active":"2024-06-03T02:14:05.241203+0000","last_became_peered":"2024-06-03T02:14:05.241203+0000","last_unstale":"2024-06-03T02:16:38.829788+0000","last_undegraded":"2024-06-03T02:16:38.829788+0000","last_fullsized":"2024-06-03T02:16:38.829788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:47.052563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"101'8","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185241+0000","last_change":"2024-06-03T02:14:53.475893+0000","last_active":"2024-06-03T02:17:22.185241+0000","last_peered":"2024-06-03T02:17:22.185241+0000","last_clean":"2024-06-03T02:17:22.185241+0000","last_became_active":"2024-06-03T02:14:53.475494+0000","last_became_peered":"2024-06-03T02:14:53.475494+0000","last_unstale":"2024-06-03T02:17:22.185241+0000","last_undegraded":"2024-06-03T02:17:22.185241+0000","last_fullsized":"2024-06-03T02:17:22.185241+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:27.113842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:28.911253+0000","last_change":"2024-06-03T02:15:54.344417+0000","last_active":"2024-06-03T02:16:28.911253+0000","last_peered":"2024-06-03T02:16:28.911253+0000","last_clean":"2024-06-03T02:16:28.911253+0000","last_became_active":"2024-06-03T02:15:54.343904+0000","last_became_peered":"2024-06-03T02:15:54.343904+0000","last_unstale":"2024-06-03T02:16:28.911253+0000","last_undegraded":"2024-06-03T02:16:28.911253+0000","last_fullsized":"2024-06-03T02:16:28.911253+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:42:18.121696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"99'3","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.217749+0000","last_change":"2024-06-03T02:14:05.237743+0000","last_active":"2024-06-03T02:16:31.217749+0000","last_peered":"2024-06-03T02:16:31.217749+0000","last_clean":"2024-06-03T02:16:31.217749+0000","last_became_active":"2024-06-03T02:14:05.236944+0000","last_became_peered":"2024-06-03T02:14:05.236944+0000","last_unstale":"2024-06-03T02:16:31.217749+0000","last_undegraded":"2024-06-03T02:16:31.217749+0000","last_fullsized":"2024-06-03T02:16:31.217749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:14:09.850862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185781+0000","last_change":"2024-06-03T02:14:09.761694+0000","last_active":"2024-06-03T02:17:22.185781+0000","last_peered":"2024-06-03T02:17:22.185781+0000","last_clean":"2024-06-03T02:17:22.185781+0000","last_became_active":"2024-06-03T02:14:09.761361+0000","last_became_peered":"2024-06-03T02:14:09.761361+0000","last_unstale":"2024-06-03T02:17:22.185781+0000","last_undegraded":"2024-06-03T02:17:22.185781+0000","last_fullsized":"2024-06-03T02:17:22.185781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:19:17.803088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:22.759262+0000","last_change":"2024-06-03T02:14:23.258634+0000","last_active":"2024-06-03T02:16:22.759262+0000","last_peered":"2024-06-03T02:16:22.759262+0000","last_clean":"2024-06-03T02:16:22.759262+0000","last_became_active":"2024-06-03T02:14:23.257799+0000","last_became_peered":"2024-06-03T02:14:23.257799+0000","last_unstale":"2024-06-03T02:16:22.759262+0000","last_undegraded":"2024-06-03T02:16:22.759262+0000","last_fullsized":"2024-06-03T02:16:22.759262+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:38:33.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"99'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.849678+0000","last_change":"2024-06-03T02:14:05.236790+0000","last_active":"2024-06-03T02:16:31.849678+0000","last_peered":"2024-06-03T02:16:31.849678+0000","last_clean":"2024-06-03T02:16:31.849678+0000","last_became_active":"2024-06-03T02:14:05.236419+0000","last_became_peered":"2024-06-03T02:14:05.236419+0000","last_unstale":"2024-06-03T02:16:31.849678+0000","last_undegraded":"2024-06-03T02:16:31.849678+0000","last_fullsized":"2024-06-03T02:16:31.849678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:44:00.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183352+0000","last_change":"2024-06-03T02:14:09.744680+0000","last_active":"2024-06-03T02:17:22.183352+0000","last_peered":"2024-06-03T02:17:22.183352+0000","last_clean":"2024-06-03T02:17:22.183352+0000","last_became_active":"2024-06-03T02:14:09.744250+0000","last_became_peered":"2024-06-03T02:14:09.744250+0000","last_unstale":"2024-06-03T02:17:22.183352+0000","last_undegraded":"2024-06-03T02:17:22.183352+0000","last_fullsized":"2024-06-03T02:17:22.183352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:58.027490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:44.242362+0000","last_change":"2024-06-03T02:14:23.259325+0000","last_active":"2024-06-03T02:16:44.242362+0000","last_peered":"2024-06-03T02:16:44.242362+0000","last_clean":"2024-06-03T02:16:44.242362+0000","last_became_active":"2024-06-03T02:14:23.258925+0000","last_became_peered":"2024-06-03T02:14:23.258925+0000","last_unstale":"2024-06-03T02:16:44.242362+0000","last_undegraded":"2024-06-03T02:16:44.242362+0000","last_fullsized":"2024-06-03T02:16:44.242362+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:32.038564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"101'7","reported_seq":519,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:28.661321+0000","last_change":"2024-06-03T02:14:05.242690+0000","last_active":"2024-06-03T02:19:28.661321+0000","last_peered":"2024-06-03T02:19:28.661321+0000","last_clean":"2024-06-03T02:19:28.661321+0000","last_became_active":"2024-06-03T02:14:05.242320+0000","last_became_peered":"2024-06-03T02:14:05.242320+0000","last_unstale":"2024-06-03T02:19:28.661321+0000","last_undegraded":"2024-06-03T02:19:28.661321+0000","last_fullsized":"2024-06-03T02:19:28.661321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:02.793186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":466,"num_read_kb":466,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.170769+0000","last_change":"2024-06-03T02:14:09.747146+0000","last_active":"2024-06-03T02:17:22.170769+0000","last_peered":"2024-06-03T02:17:22.170769+0000","last_clean":"2024-06-03T02:17:22.170769+0000","last_became_active":"2024-06-03T02:14:09.746803+0000","last_became_peered":"2024-06-03T02:14:09.746803+0000","last_unstale":"2024-06-03T02:17:22.170769+0000","last_undegraded":"2024-06-03T02:17:22.170769+0000","last_fullsized":"2024-06-03T02:17:22.170769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:04:36.262223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:45.418229+0000","last_change":"2024-06-03T02:14:23.270328+0000","last_active":"2024-06-03T02:16:45.418229+0000","last_peered":"2024-06-03T02:16:45.418229+0000","last_clean":"2024-06-03T02:16:45.418229+0000","last_became_active":"2024-06-03T02:14:23.264189+0000","last_became_peered":"2024-06-03T02:14:23.264189+0000","last_unstale":"2024-06-03T02:16:45.418229+0000","last_undegraded":"2024-06-03T02:16:45.418229+0000","last_fullsized":"2024-06-03T02:16:45.418229+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:36.248884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"99'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:32.401595+0000","last_change":"2024-06-03T02:14:05.238898+0000","last_active":"2024-06-03T02:16:32.401595+0000","last_peered":"2024-06-03T02:16:32.401595+0000","last_clean":"2024-06-03T02:16:32.401595+0000","last_became_active":"2024-06-03T02:14:05.238552+0000","last_became_peered":"2024-06-03T02:14:05.238552+0000","last_unstale":"2024-06-03T02:16:32.401595+0000","last_undegraded":"2024-06-03T02:16:32.401595+0000","last_fullsized":"2024-06-03T02:16:32.401595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:29:33.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.190294+0000","last_change":"2024-06-03T02:14:09.769973+0000","last_active":"2024-06-03T02:17:22.190294+0000","last_peered":"2024-06-03T02:17:22.190294+0000","last_clean":"2024-06-03T02:17:22.190294+0000","last_became_active":"2024-06-03T02:14:09.768984+0000","last_became_peered":"2024-06-03T02:14:09.768984+0000","last_unstale":"2024-06-03T02:17:22.190294+0000","last_undegraded":"2024-06-03T02:17:22.190294+0000","last_fullsized":"2024-06-03T02:17:22.190294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:48:33.720441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:46.199590+0000","last_change":"2024-06-03T02:14:23.265947+0000","last_active":"2024-06-03T02:16:46.199590+0000","last_peered":"2024-06-03T02:16:46.199590+0000","last_clean":"2024-06-03T02:16:46.199590+0000","last_became_active":"2024-06-03T02:14:23.265412+0000","last_became_peered":"2024-06-03T02:14:23.265412+0000","last_unstale":"2024-06-03T02:16:46.199590+0000","last_undegraded":"2024-06-03T02:16:46.199590+0000","last_fullsized":"2024-06-03T02:16:46.199590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:35:08.664884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"99'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.849724+0000","last_change":"2024-06-03T02:14:05.252751+0000","last_active":"2024-06-03T02:16:31.849724+0000","last_peered":"2024-06-03T02:16:31.849724+0000","last_clean":"2024-06-03T02:16:31.849724+0000","last_became_active":"2024-06-03T02:14:05.252267+0000","last_became_peered":"2024-06-03T02:14:05.252267+0000","last_unstale":"2024-06-03T02:16:31.849724+0000","last_undegraded":"2024-06-03T02:16:31.849724+0000","last_fullsized":"2024-06-03T02:16:31.849724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:58.091711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171446+0000","last_change":"2024-06-03T02:14:09.750757+0000","last_active":"2024-06-03T02:17:22.171446+0000","last_peered":"2024-06-03T02:17:22.171446+0000","last_clean":"2024-06-03T02:17:22.171446+0000","last_became_active":"2024-06-03T02:14:09.750414+0000","last_became_peered":"2024-06-03T02:14:09.750414+0000","last_unstale":"2024-06-03T02:17:22.171446+0000","last_undegraded":"2024-06-03T02:17:22.171446+0000","last_fullsized":"2024-06-03T02:17:22.171446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:29:55.435408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:45.418260+0000","last_change":"2024-06-03T02:14:23.263440+0000","last_active":"2024-06-03T02:16:45.418260+0000","last_peered":"2024-06-03T02:16:45.418260+0000","last_clean":"2024-06-03T02:16:45.418260+0000","last_became_active":"2024-06-03T02:14:23.254019+0000","last_became_peered":"2024-06-03T02:14:23.254019+0000","last_unstale":"2024-06-03T02:16:45.418260+0000","last_undegraded":"2024-06-03T02:16:45.418260+0000","last_fullsized":"2024-06-03T02:16:45.418260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:48:58.745990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'43","reported_seq":569,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:28.661292+0000","last_change":"2024-06-03T02:14:05.239353+0000","last_active":"2024-06-03T02:19:28.661292+0000","last_peered":"2024-06-03T02:19:28.661292+0000","last_clean":"2024-06-03T02:19:28.661292+0000","last_became_active":"2024-06-03T02:14:05.238976+0000","last_became_peered":"2024-06-03T02:14:05.238976+0000","last_unstale":"2024-06-03T02:19:28.661292+0000","last_undegraded":"2024-06-03T02:19:28.661292+0000","last_fullsized":"2024-06-03T02:19:28.661292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:58:51.598533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75317,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":480,"num_read_kb":313,"num_write":45,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.145081+0000","last_change":"2024-06-03T02:14:09.746546+0000","last_active":"2024-06-03T02:17:22.145081+0000","last_peered":"2024-06-03T02:17:22.145081+0000","last_clean":"2024-06-03T02:17:22.145081+0000","last_became_active":"2024-06-03T02:14:09.746049+0000","last_became_peered":"2024-06-03T02:14:09.746049+0000","last_unstale":"2024-06-03T02:17:22.145081+0000","last_undegraded":"2024-06-03T02:17:22.145081+0000","last_fullsized":"2024-06-03T02:17:22.145081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:43:34.667133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.851868+0000","last_change":"2024-06-03T02:14:23.257296+0000","last_active":"2024-06-03T02:16:51.851868+0000","last_peered":"2024-06-03T02:16:51.851868+0000","last_clean":"2024-06-03T02:16:51.851868+0000","last_became_active":"2024-06-03T02:14:23.256858+0000","last_became_peered":"2024-06-03T02:14:23.256858+0000","last_unstale":"2024-06-03T02:16:51.851868+0000","last_undegraded":"2024-06-03T02:16:51.851868+0000","last_fullsized":"2024-06-03T02:16:51.851868+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:00.464902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"99'9","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:27.210855+0000","last_change":"2024-06-03T02:14:05.245163+0000","last_active":"2024-06-03T02:16:27.210855+0000","last_peered":"2024-06-03T02:16:27.210855+0000","last_clean":"2024-06-03T02:16:27.210855+0000","last_became_active":"2024-06-03T02:14:05.244813+0000","last_became_peered":"2024-06-03T02:14:05.244813+0000","last_unstale":"2024-06-03T02:16:27.210855+0000","last_undegraded":"2024-06-03T02:16:27.210855+0000","last_fullsized":"2024-06-03T02:16:27.210855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:46:47.017859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181707+0000","last_change":"2024-06-03T02:14:09.757200+0000","last_active":"2024-06-03T02:17:22.181707+0000","last_peered":"2024-06-03T02:17:22.181707+0000","last_clean":"2024-06-03T02:17:22.181707+0000","last_became_active":"2024-06-03T02:14:09.756380+0000","last_became_peered":"2024-06-03T02:14:09.756380+0000","last_unstale":"2024-06-03T02:17:22.181707+0000","last_undegraded":"2024-06-03T02:17:22.181707+0000","last_fullsized":"2024-06-03T02:17:22.181707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:56.045338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:43.913099+0000","last_change":"2024-06-03T02:14:23.258441+0000","last_active":"2024-06-03T02:16:43.913099+0000","last_peered":"2024-06-03T02:16:43.913099+0000","last_clean":"2024-06-03T02:16:43.913099+0000","last_became_active":"2024-06-03T02:14:23.258015+0000","last_became_peered":"2024-06-03T02:14:23.258015+0000","last_unstale":"2024-06-03T02:16:43.913099+0000","last_undegraded":"2024-06-03T02:16:43.913099+0000","last_fullsized":"2024-06-03T02:16:43.913099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:03.123511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"99'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:26.849234+0000","last_change":"2024-06-03T02:14:05.240456+0000","last_active":"2024-06-03T02:16:26.849234+0000","last_peered":"2024-06-03T02:16:26.849234+0000","last_clean":"2024-06-03T02:16:26.849234+0000","last_became_active":"2024-06-03T02:14:05.240078+0000","last_became_peered":"2024-06-03T02:14:05.240078+0000","last_unstale":"2024-06-03T02:16:26.849234+0000","last_undegraded":"2024-06-03T02:16:26.849234+0000","last_fullsized":"2024-06-03T02:16:26.849234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:00:26.491773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"101'15","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.179670+0000","last_change":"2024-06-03T02:14:09.750876+0000","last_active":"2024-06-03T02:17:22.179670+0000","last_peered":"2024-06-03T02:17:22.179670+0000","last_clean":"2024-06-03T02:17:22.179670+0000","last_became_active":"2024-06-03T02:14:09.750547+0000","last_became_peered":"2024-06-03T02:14:09.750547+0000","last_unstale":"2024-06-03T02:17:22.179670+0000","last_undegraded":"2024-06-03T02:17:22.179670+0000","last_fullsized":"2024-06-03T02:17:22.179670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:08:41.136225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.241201+0000","last_change":"2024-06-03T02:14:23.276172+0000","last_active":"2024-06-03T02:16:51.241201+0000","last_peered":"2024-06-03T02:16:51.241201+0000","last_clean":"2024-06-03T02:16:51.241201+0000","last_became_active":"2024-06-03T02:14:23.275410+0000","last_became_peered":"2024-06-03T02:14:23.275410+0000","last_unstale":"2024-06-03T02:16:51.241201+0000","last_undegraded":"2024-06-03T02:16:51.241201+0000","last_fullsized":"2024-06-03T02:16:51.241201+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:16:30.859453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"101'8","reported_seq":289,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:24.802602+0000","last_change":"2024-06-03T02:14:05.237263+0000","last_active":"2024-06-03T02:19:24.802602+0000","last_peered":"2024-06-03T02:19:24.802602+0000","last_clean":"2024-06-03T02:19:24.802602+0000","last_became_active":"2024-06-03T02:14:05.236890+0000","last_became_peered":"2024-06-03T02:14:05.236890+0000","last_unstale":"2024-06-03T02:19:24.802602+0000","last_undegraded":"2024-06-03T02:19:24.802602+0000","last_fullsized":"2024-06-03T02:19:24.802602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:42.292545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171587+0000","last_change":"2024-06-03T02:14:09.746260+0000","last_active":"2024-06-03T02:17:22.171587+0000","last_peered":"2024-06-03T02:17:22.171587+0000","last_clean":"2024-06-03T02:17:22.171587+0000","last_became_active":"2024-06-03T02:14:09.745940+0000","last_became_peered":"2024-06-03T02:14:09.745940+0000","last_unstale":"2024-06-03T02:17:22.171587+0000","last_undegraded":"2024-06-03T02:17:22.171587+0000","last_fullsized":"2024-06-03T02:17:22.171587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:08:55.851464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:51.220996+0000","last_change":"2024-06-03T02:14:23.265440+0000","last_active":"2024-06-03T02:16:51.220996+0000","last_peered":"2024-06-03T02:16:51.220996+0000","last_clean":"2024-06-03T02:16:51.220996+0000","last_became_active":"2024-06-03T02:14:23.265028+0000","last_became_peered":"2024-06-03T02:14:23.265028+0000","last_unstale":"2024-06-03T02:16:51.220996+0000","last_undegraded":"2024-06-03T02:16:51.220996+0000","last_fullsized":"2024-06-03T02:16:51.220996+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:56.237500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"99'17","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:27.185001+0000","last_change":"2024-06-03T02:14:53.271821+0000","last_active":"2024-06-03T02:16:27.185001+0000","last_peered":"2024-06-03T02:16:27.185001+0000","last_clean":"2024-06-03T02:16:27.185001+0000","last_became_active":"2024-06-03T02:14:53.263047+0000","last_became_peered":"2024-06-03T02:14:53.263047+0000","last_unstale":"2024-06-03T02:16:27.185001+0000","last_undegraded":"2024-06-03T02:16:27.185001+0000","last_fullsized":"2024-06-03T02:16:27.185001+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:02:25.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.149642+0000","last_change":"2024-06-03T02:14:09.754819+0000","last_active":"2024-06-03T02:17:22.149642+0000","last_peered":"2024-06-03T02:17:22.149642+0000","last_clean":"2024-06-03T02:17:22.149642+0000","last_became_active":"2024-06-03T02:14:09.753995+0000","last_became_peered":"2024-06-03T02:14:09.753995+0000","last_unstale":"2024-06-03T02:17:22.149642+0000","last_undegraded":"2024-06-03T02:17:22.149642+0000","last_fullsized":"2024-06-03T02:17:22.149642+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:19:51.123774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.850243+0000","last_change":"2024-06-03T02:14:23.265003+0000","last_active":"2024-06-03T02:16:36.850243+0000","last_peered":"2024-06-03T02:16:36.850243+0000","last_clean":"2024-06-03T02:16:36.850243+0000","last_became_active":"2024-06-03T02:14:23.264529+0000","last_became_peered":"2024-06-03T02:14:23.264529+0000","last_unstale":"2024-06-03T02:16:36.850243+0000","last_undegraded":"2024-06-03T02:16:36.850243+0000","last_fullsized":"2024-06-03T02:16:36.850243+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:50:19.838274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"99'2","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:22.184389+0000","last_change":"2024-06-03T02:14:05.236901+0000","last_active":"2024-06-03T02:16:22.184389+0000","last_peered":"2024-06-03T02:16:22.184389+0000","last_clean":"2024-06-03T02:16:22.184389+0000","last_became_active":"2024-06-03T02:14:05.236460+0000","last_became_peered":"2024-06-03T02:14:05.236460+0000","last_unstale":"2024-06-03T02:16:22.184389+0000","last_undegraded":"2024-06-03T02:16:22.184389+0000","last_fullsized":"2024-06-03T02:16:22.184389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:59.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.160124+0000","last_change":"2024-06-03T02:14:09.743135+0000","last_active":"2024-06-03T02:17:22.160124+0000","last_peered":"2024-06-03T02:17:22.160124+0000","last_clean":"2024-06-03T02:17:22.160124+0000","last_became_active":"2024-06-03T02:14:09.742506+0000","last_became_peered":"2024-06-03T02:14:09.742506+0000","last_unstale":"2024-06-03T02:17:22.160124+0000","last_undegraded":"2024-06-03T02:17:22.160124+0000","last_fullsized":"2024-06-03T02:17:22.160124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:51.413537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:32.185624+0000","last_change":"2024-06-03T02:14:23.255512+0000","last_active":"2024-06-03T02:16:32.185624+0000","last_peered":"2024-06-03T02:16:32.185624+0000","last_clean":"2024-06-03T02:16:32.185624+0000","last_became_active":"2024-06-03T02:14:23.255103+0000","last_became_peered":"2024-06-03T02:14:23.255103+0000","last_unstale":"2024-06-03T02:16:32.185624+0000","last_undegraded":"2024-06-03T02:16:32.185624+0000","last_fullsized":"2024-06-03T02:16:32.185624+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:12:05.295848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"99'1","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:11.235176+0000","last_change":"2024-06-03T02:14:05.248728+0000","last_active":"2024-06-03T02:16:11.235176+0000","last_peered":"2024-06-03T02:16:11.235176+0000","last_clean":"2024-06-03T02:16:11.235176+0000","last_became_active":"2024-06-03T02:14:05.248144+0000","last_became_peered":"2024-06-03T02:14:05.248144+0000","last_unstale":"2024-06-03T02:16:11.235176+0000","last_undegraded":"2024-06-03T02:16:11.235176+0000","last_fullsized":"2024-06-03T02:16:11.235176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:53:27.814443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184912+0000","last_change":"2024-06-03T02:14:09.752029+0000","last_active":"2024-06-03T02:17:22.184912+0000","last_peered":"2024-06-03T02:17:22.184912+0000","last_clean":"2024-06-03T02:17:22.184912+0000","last_became_active":"2024-06-03T02:14:09.751638+0000","last_became_peered":"2024-06-03T02:14:09.751638+0000","last_unstale":"2024-06-03T02:17:22.184912+0000","last_undegraded":"2024-06-03T02:17:22.184912+0000","last_fullsized":"2024-06-03T02:17:22.184912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:51.201923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:31.217797+0000","last_change":"2024-06-03T02:14:23.250406+0000","last_active":"2024-06-03T02:16:31.217797+0000","last_peered":"2024-06-03T02:16:31.217797+0000","last_clean":"2024-06-03T02:16:31.217797+0000","last_became_active":"2024-06-03T02:14:23.249946+0000","last_became_peered":"2024-06-03T02:14:23.249946+0000","last_unstale":"2024-06-03T02:16:31.217797+0000","last_undegraded":"2024-06-03T02:16:31.217797+0000","last_fullsized":"2024-06-03T02:16:31.217797+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:04:47.714532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"101'7","reported_seq":287,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:25.839775+0000","last_change":"2024-06-03T02:14:05.241666+0000","last_active":"2024-06-03T02:19:25.839775+0000","last_peered":"2024-06-03T02:19:25.839775+0000","last_clean":"2024-06-03T02:19:25.839775+0000","last_became_active":"2024-06-03T02:14:05.241299+0000","last_became_peered":"2024-06-03T02:14:05.241299+0000","last_unstale":"2024-06-03T02:19:25.839775+0000","last_undegraded":"2024-06-03T02:19:25.839775+0000","last_fullsized":"2024-06-03T02:19:25.839775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:21:16.792022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":234,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.176295+0000","last_change":"2024-06-03T02:14:09.768602+0000","last_active":"2024-06-03T02:17:22.176295+0000","last_peered":"2024-06-03T02:17:22.176295+0000","last_clean":"2024-06-03T02:17:22.176295+0000","last_became_active":"2024-06-03T02:14:09.768195+0000","last_became_peered":"2024-06-03T02:14:09.768195+0000","last_unstale":"2024-06-03T02:17:22.176295+0000","last_undegraded":"2024-06-03T02:17:22.176295+0000","last_fullsized":"2024-06-03T02:17:22.176295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:45:42.361966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.218658+0000","last_change":"2024-06-03T02:14:23.252472+0000","last_active":"2024-06-03T02:16:36.218658+0000","last_peered":"2024-06-03T02:16:36.218658+0000","last_clean":"2024-06-03T02:16:36.218658+0000","last_became_active":"2024-06-03T02:14:23.252018+0000","last_became_peered":"2024-06-03T02:14:23.252018+0000","last_unstale":"2024-06-03T02:16:36.218658+0000","last_undegraded":"2024-06-03T02:16:36.218658+0000","last_fullsized":"2024-06-03T02:16:36.218658+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:28.468657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163186+0000","last_change":"2024-06-03T02:14:09.744443+0000","last_active":"2024-06-03T02:17:22.163186+0000","last_peered":"2024-06-03T02:17:22.163186+0000","last_clean":"2024-06-03T02:17:22.163186+0000","last_became_active":"2024-06-03T02:14:09.744100+0000","last_became_peered":"2024-06-03T02:14:09.744100+0000","last_unstale":"2024-06-03T02:17:22.163186+0000","last_undegraded":"2024-06-03T02:17:22.163186+0000","last_fullsized":"2024-06-03T02:17:22.163186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:44.566745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"101'7","reported_seq":284,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:24.801818+0000","last_change":"2024-06-03T02:14:05.250734+0000","last_active":"2024-06-03T02:19:24.801818+0000","last_peered":"2024-06-03T02:19:24.801818+0000","last_clean":"2024-06-03T02:19:24.801818+0000","last_became_active":"2024-06-03T02:14:05.250300+0000","last_became_peered":"2024-06-03T02:14:05.250300+0000","last_unstale":"2024-06-03T02:19:24.801818+0000","last_undegraded":"2024-06-03T02:19:24.801818+0000","last_fullsized":"2024-06-03T02:19:24.801818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:52.322895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":231,"num_read_kb":231,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"99'1","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:27.401054+0000","last_change":"2024-06-03T02:14:53.471397+0000","last_active":"2024-06-03T02:16:27.401054+0000","last_peered":"2024-06-03T02:16:27.401054+0000","last_clean":"2024-06-03T02:16:27.401054+0000","last_became_active":"2024-06-03T02:14:53.470868+0000","last_became_peered":"2024-06-03T02:14:53.470868+0000","last_unstale":"2024-06-03T02:16:27.401054+0000","last_undegraded":"2024-06-03T02:16:27.401054+0000","last_fullsized":"2024-06-03T02:16:27.401054+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:58.726070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.188260+0000","last_change":"2024-06-03T02:14:09.765193+0000","last_active":"2024-06-03T02:17:22.188260+0000","last_peered":"2024-06-03T02:17:22.188260+0000","last_clean":"2024-06-03T02:17:22.188260+0000","last_became_active":"2024-06-03T02:14:09.764834+0000","last_became_peered":"2024-06-03T02:14:09.764834+0000","last_unstale":"2024-06-03T02:17:22.188260+0000","last_undegraded":"2024-06-03T02:17:22.188260+0000","last_fullsized":"2024-06-03T02:17:22.188260+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:20:37.464286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"99'6","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:11.195852+0000","last_change":"2024-06-03T02:14:05.240118+0000","last_active":"2024-06-03T02:16:11.195852+0000","last_peered":"2024-06-03T02:16:11.195852+0000","last_clean":"2024-06-03T02:16:11.195852+0000","last_became_active":"2024-06-03T02:14:05.239771+0000","last_became_peered":"2024-06-03T02:14:05.239771+0000","last_unstale":"2024-06-03T02:16:11.195852+0000","last_undegraded":"2024-06-03T02:16:11.195852+0000","last_fullsized":"2024-06-03T02:16:11.195852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:08.256222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":372,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:07.405163+0000","last_change":"2024-06-03T02:12:31.842613+0000","last_active":"2024-06-03T02:17:07.405163+0000","last_peered":"2024-06-03T02:17:07.405163+0000","last_clean":"2024-06-03T02:17:07.405163+0000","last_became_active":"2024-06-03T02:12:31.842051+0000","last_became_peered":"2024-06-03T02:12:31.842051+0000","last_unstale":"2024-06-03T02:17:07.405163+0000","last_undegraded":"2024-06-03T02:17:07.405163+0000","last_fullsized":"2024-06-03T02:17:07.405163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:08:38.749702+0000","last_clean_scrub_stamp":"2024-06-03T02:08:38.749702+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:40:29.748789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.850310+0000","last_change":"2024-06-03T02:14:23.255849+0000","last_active":"2024-06-03T02:16:36.850310+0000","last_peered":"2024-06-03T02:16:36.850310+0000","last_clean":"2024-06-03T02:16:36.850310+0000","last_became_active":"2024-06-03T02:14:23.255380+0000","last_became_peered":"2024-06-03T02:14:23.255380+0000","last_unstale":"2024-06-03T02:16:36.850310+0000","last_undegraded":"2024-06-03T02:16:36.850310+0000","last_fullsized":"2024-06-03T02:16:36.850310+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:41:16.051326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"101'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.169554+0000","last_change":"2024-06-03T02:14:53.270423+0000","last_active":"2024-06-03T02:17:22.169554+0000","last_peered":"2024-06-03T02:17:22.169554+0000","last_clean":"2024-06-03T02:17:22.169554+0000","last_became_active":"2024-06-03T02:14:53.269260+0000","last_became_peered":"2024-06-03T02:14:53.269260+0000","last_unstale":"2024-06-03T02:17:22.169554+0000","last_undegraded":"2024-06-03T02:17:22.169554+0000","last_fullsized":"2024-06-03T02:17:22.169554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:57.392012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"99'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:07.399193+0000","last_change":"2024-06-03T02:14:05.235982+0000","last_active":"2024-06-03T02:16:07.399193+0000","last_peered":"2024-06-03T02:16:07.399193+0000","last_clean":"2024-06-03T02:16:07.399193+0000","last_became_active":"2024-06-03T02:14:05.235638+0000","last_became_peered":"2024-06-03T02:14:05.235638+0000","last_unstale":"2024-06-03T02:16:07.399193+0000","last_undegraded":"2024-06-03T02:16:07.399193+0000","last_fullsized":"2024-06-03T02:16:07.399193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:42:07.544005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:28.911203+0000","last_change":"2024-06-03T02:14:23.257790+0000","last_active":"2024-06-03T02:16:28.911203+0000","last_peered":"2024-06-03T02:16:28.911203+0000","last_clean":"2024-06-03T02:16:28.911203+0000","last_became_active":"2024-06-03T02:14:23.257297+0000","last_became_peered":"2024-06-03T02:14:23.257297+0000","last_unstale":"2024-06-03T02:16:28.911203+0000","last_undegraded":"2024-06-03T02:16:28.911203+0000","last_fullsized":"2024-06-03T02:16:28.911203+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:13.715455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"99'4","reported_seq":22,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:58.914613+0000","last_change":"2024-06-03T02:14:53.268495+0000","last_active":"2024-06-03T02:16:58.914613+0000","last_peered":"2024-06-03T02:16:58.914613+0000","last_clean":"2024-06-03T02:16:58.914613+0000","last_became_active":"2024-06-03T02:14:53.268147+0000","last_became_peered":"2024-06-03T02:14:53.268147+0000","last_unstale":"2024-06-03T02:16:58.914613+0000","last_undegraded":"2024-06-03T02:16:58.914613+0000","last_fullsized":"2024-06-03T02:16:58.914613+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:23.076319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172914+0000","last_change":"2024-06-03T02:14:09.758244+0000","last_active":"2024-06-03T02:17:22.172914+0000","last_peered":"2024-06-03T02:17:22.172914+0000","last_clean":"2024-06-03T02:17:22.172914+0000","last_became_active":"2024-06-03T02:14:09.757812+0000","last_became_peered":"2024-06-03T02:14:09.757812+0000","last_unstale":"2024-06-03T02:17:22.172914+0000","last_undegraded":"2024-06-03T02:17:22.172914+0000","last_fullsized":"2024-06-03T02:17:22.172914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:08.243703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:24.239124+0000","last_change":"2024-06-03T02:14:23.259980+0000","last_active":"2024-06-03T02:16:24.239124+0000","last_peered":"2024-06-03T02:16:24.239124+0000","last_clean":"2024-06-03T02:16:24.239124+0000","last_became_active":"2024-06-03T02:14:23.258308+0000","last_became_peered":"2024-06-03T02:14:23.258308+0000","last_unstale":"2024-06-03T02:16:24.239124+0000","last_undegraded":"2024-06-03T02:16:24.239124+0000","last_fullsized":"2024-06-03T02:16:24.239124+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:28.833994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'19","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:21.815379+0000","last_change":"2024-06-03T02:14:05.244855+0000","last_active":"2024-06-03T02:19:21.815379+0000","last_peered":"2024-06-03T02:19:21.815379+0000","last_clean":"2024-06-03T02:19:21.815379+0000","last_became_active":"2024-06-03T02:14:05.244335+0000","last_became_peered":"2024-06-03T02:14:05.244335+0000","last_unstale":"2024-06-03T02:19:21.815379+0000","last_undegraded":"2024-06-03T02:19:21.815379+0000","last_fullsized":"2024-06-03T02:19:21.815379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:53:19.065411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185900+0000","last_change":"2024-06-03T02:14:09.750761+0000","last_active":"2024-06-03T02:17:22.185900+0000","last_peered":"2024-06-03T02:17:22.185900+0000","last_clean":"2024-06-03T02:17:22.185900+0000","last_became_active":"2024-06-03T02:14:09.750336+0000","last_became_peered":"2024-06-03T02:14:09.750336+0000","last_unstale":"2024-06-03T02:17:22.185900+0000","last_undegraded":"2024-06-03T02:17:22.185900+0000","last_fullsized":"2024-06-03T02:17:22.185900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:59:46.651149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:32.211290+0000","last_change":"2024-06-03T02:14:23.243287+0000","last_active":"2024-06-03T02:16:32.211290+0000","last_peered":"2024-06-03T02:16:32.211290+0000","last_clean":"2024-06-03T02:16:32.211290+0000","last_became_active":"2024-06-03T02:14:23.242828+0000","last_became_peered":"2024-06-03T02:14:23.242828+0000","last_unstale":"2024-06-03T02:16:32.211290+0000","last_undegraded":"2024-06-03T02:16:32.211290+0000","last_fullsized":"2024-06-03T02:16:32.211290+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:54:12.312438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"99'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:16.196453+0000","last_change":"2024-06-03T02:14:05.596288+0000","last_active":"2024-06-03T02:16:16.196453+0000","last_peered":"2024-06-03T02:16:16.196453+0000","last_clean":"2024-06-03T02:16:16.196453+0000","last_became_active":"2024-06-03T02:14:05.595868+0000","last_became_peered":"2024-06-03T02:14:05.595868+0000","last_unstale":"2024-06-03T02:16:16.196453+0000","last_undegraded":"2024-06-03T02:16:16.196453+0000","last_fullsized":"2024-06-03T02:16:16.196453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:50.291541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.189478+0000","last_change":"2024-06-03T02:14:09.765317+0000","last_active":"2024-06-03T02:17:22.189478+0000","last_peered":"2024-06-03T02:17:22.189478+0000","last_clean":"2024-06-03T02:17:22.189478+0000","last_became_active":"2024-06-03T02:14:09.764983+0000","last_became_peered":"2024-06-03T02:14:09.764983+0000","last_unstale":"2024-06-03T02:17:22.189478+0000","last_undegraded":"2024-06-03T02:17:22.189478+0000","last_fullsized":"2024-06-03T02:17:22.189478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:20.075276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:41.198968+0000","last_change":"2024-06-03T02:14:23.265352+0000","last_active":"2024-06-03T02:16:41.198968+0000","last_peered":"2024-06-03T02:16:41.198968+0000","last_clean":"2024-06-03T02:16:41.198968+0000","last_became_active":"2024-06-03T02:14:23.264932+0000","last_became_peered":"2024-06-03T02:14:23.264932+0000","last_unstale":"2024-06-03T02:16:41.198968+0000","last_undegraded":"2024-06-03T02:16:41.198968+0000","last_fullsized":"2024-06-03T02:16:41.198968+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:43.803170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"99'2","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:18.326424+0000","last_change":"2024-06-03T02:14:05.248563+0000","last_active":"2024-06-03T02:16:18.326424+0000","last_peered":"2024-06-03T02:16:18.326424+0000","last_clean":"2024-06-03T02:16:18.326424+0000","last_became_active":"2024-06-03T02:14:05.248306+0000","last_became_peered":"2024-06-03T02:14:05.248306+0000","last_unstale":"2024-06-03T02:16:18.326424+0000","last_undegraded":"2024-06-03T02:16:18.326424+0000","last_fullsized":"2024-06-03T02:16:18.326424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:23:58.130157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"101'2","reported_seq":42,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.149643+0000","last_change":"2024-06-03T02:14:09.745948+0000","last_active":"2024-06-03T02:17:22.149643+0000","last_peered":"2024-06-03T02:17:22.149643+0000","last_clean":"2024-06-03T02:17:22.149643+0000","last_became_active":"2024-06-03T02:14:09.745626+0000","last_became_peered":"2024-06-03T02:14:09.745626+0000","last_unstale":"2024-06-03T02:17:22.149643+0000","last_undegraded":"2024-06-03T02:17:22.149643+0000","last_fullsized":"2024-06-03T02:17:22.149643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:34.742129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:15:57.181841+0000","last_change":"2024-06-03T02:15:54.341824+0000","last_active":"2024-06-03T02:15:57.181841+0000","last_peered":"2024-06-03T02:15:57.181841+0000","last_clean":"2024-06-03T02:15:57.181841+0000","last_became_active":"2024-06-03T02:15:54.341267+0000","last_became_peered":"2024-06-03T02:15:54.341267+0000","last_unstale":"2024-06-03T02:15:57.181841+0000","last_undegraded":"2024-06-03T02:15:57.181841+0000","last_fullsized":"2024-06-03T02:15:57.181841+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:21:47.714056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"99'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:19.238284+0000","last_change":"2024-06-03T02:14:05.244806+0000","last_active":"2024-06-03T02:16:19.238284+0000","last_peered":"2024-06-03T02:16:19.238284+0000","last_clean":"2024-06-03T02:16:19.238284+0000","last_became_active":"2024-06-03T02:14:05.242719+0000","last_became_peered":"2024-06-03T02:14:05.242719+0000","last_unstale":"2024-06-03T02:16:19.238284+0000","last_undegraded":"2024-06-03T02:16:19.238284+0000","last_fullsized":"2024-06-03T02:16:19.238284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:31.314266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.181636+0000","last_change":"2024-06-03T02:14:09.755102+0000","last_active":"2024-06-03T02:17:22.181636+0000","last_peered":"2024-06-03T02:17:22.181636+0000","last_clean":"2024-06-03T02:17:22.181636+0000","last_became_active":"2024-06-03T02:14:09.754748+0000","last_became_peered":"2024-06-03T02:14:09.754748+0000","last_unstale":"2024-06-03T02:17:22.181636+0000","last_undegraded":"2024-06-03T02:17:22.181636+0000","last_fullsized":"2024-06-03T02:17:22.181636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:48:59.920710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"101'1","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:41.198931+0000","last_change":"2024-06-03T02:14:23.271974+0000","last_active":"2024-06-03T02:16:41.198931+0000","last_peered":"2024-06-03T02:16:41.198931+0000","last_clean":"2024-06-03T02:16:41.198931+0000","last_became_active":"2024-06-03T02:14:23.271470+0000","last_became_peered":"2024-06-03T02:14:23.271470+0000","last_unstale":"2024-06-03T02:16:41.198931+0000","last_undegraded":"2024-06-03T02:16:41.198931+0000","last_fullsized":"2024-06-03T02:16:41.198931+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:12:03.159351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"101'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:18:27.223978+0000","last_change":"2024-06-03T02:14:05.242261+0000","last_active":"2024-06-03T02:18:27.223978+0000","last_peered":"2024-06-03T02:18:27.223978+0000","last_clean":"2024-06-03T02:18:27.223978+0000","last_became_active":"2024-06-03T02:14:05.241854+0000","last_became_peered":"2024-06-03T02:14:05.241854+0000","last_unstale":"2024-06-03T02:18:27.223978+0000","last_undegraded":"2024-06-03T02:18:27.223978+0000","last_fullsized":"2024-06-03T02:18:27.223978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:21.427703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186160+0000","last_change":"2024-06-03T02:14:09.763591+0000","last_active":"2024-06-03T02:17:22.186160+0000","last_peered":"2024-06-03T02:17:22.186160+0000","last_clean":"2024-06-03T02:17:22.186160+0000","last_became_active":"2024-06-03T02:14:09.763240+0000","last_became_peered":"2024-06-03T02:14:09.763240+0000","last_unstale":"2024-06-03T02:17:22.186160+0000","last_undegraded":"2024-06-03T02:17:22.186160+0000","last_fullsized":"2024-06-03T02:17:22.186160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:40.146898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:41.198957+0000","last_change":"2024-06-03T02:14:23.265011+0000","last_active":"2024-06-03T02:16:41.198957+0000","last_peered":"2024-06-03T02:16:41.198957+0000","last_clean":"2024-06-03T02:16:41.198957+0000","last_became_active":"2024-06-03T02:14:23.263767+0000","last_became_peered":"2024-06-03T02:14:23.263767+0000","last_unstale":"2024-06-03T02:16:41.198957+0000","last_undegraded":"2024-06-03T02:16:41.198957+0000","last_fullsized":"2024-06-03T02:16:41.198957+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:34:04.274545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"99'4","reported_seq":61,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:22.184364+0000","last_change":"2024-06-03T02:14:53.474879+0000","last_active":"2024-06-03T02:16:22.184364+0000","last_peered":"2024-06-03T02:16:22.184364+0000","last_clean":"2024-06-03T02:16:22.184364+0000","last_became_active":"2024-06-03T02:14:53.472209+0000","last_became_peered":"2024-06-03T02:14:53.472209+0000","last_unstale":"2024-06-03T02:16:22.184364+0000","last_undegraded":"2024-06-03T02:16:22.184364+0000","last_fullsized":"2024-06-03T02:16:22.184364+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:36:57.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"101'5","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.172711+0000","last_change":"2024-06-03T02:14:09.747756+0000","last_active":"2024-06-03T02:17:22.172711+0000","last_peered":"2024-06-03T02:17:22.172711+0000","last_clean":"2024-06-03T02:17:22.172711+0000","last_became_active":"2024-06-03T02:14:09.747453+0000","last_became_peered":"2024-06-03T02:14:09.747453+0000","last_unstale":"2024-06-03T02:17:22.172711+0000","last_undegraded":"2024-06-03T02:17:22.172711+0000","last_fullsized":"2024-06-03T02:17:22.172711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:19:02.341367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:33.328239+0000","last_change":"2024-06-03T02:14:23.248594+0000","last_active":"2024-06-03T02:16:33.328239+0000","last_peered":"2024-06-03T02:16:33.328239+0000","last_clean":"2024-06-03T02:16:33.328239+0000","last_became_active":"2024-06-03T02:14:23.248080+0000","last_became_peered":"2024-06-03T02:14:23.248080+0000","last_unstale":"2024-06-03T02:16:33.328239+0000","last_undegraded":"2024-06-03T02:16:33.328239+0000","last_fullsized":"2024-06-03T02:16:33.328239+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:24:42.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"101'6","reported_seq":286,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:19:25.657346+0000","last_change":"2024-06-03T02:14:05.596274+0000","last_active":"2024-06-03T02:19:25.657346+0000","last_peered":"2024-06-03T02:19:25.657346+0000","last_clean":"2024-06-03T02:19:25.657346+0000","last_became_active":"2024-06-03T02:14:05.595678+0000","last_became_peered":"2024-06-03T02:14:05.595678+0000","last_unstale":"2024-06-03T02:19:25.657346+0000","last_undegraded":"2024-06-03T02:19:25.657346+0000","last_fullsized":"2024-06-03T02:19:25.657346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:13:25.436688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"101'7","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.173298+0000","last_change":"2024-06-03T02:14:09.744934+0000","last_active":"2024-06-03T02:17:22.173298+0000","last_peered":"2024-06-03T02:17:22.173298+0000","last_clean":"2024-06-03T02:17:22.173298+0000","last_became_active":"2024-06-03T02:14:09.744621+0000","last_became_peered":"2024-06-03T02:14:09.744621+0000","last_unstale":"2024-06-03T02:17:22.173298+0000","last_undegraded":"2024-06-03T02:17:22.173298+0000","last_fullsized":"2024-06-03T02:17:22.173298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:01:51.970160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:37.761133+0000","last_change":"2024-06-03T02:14:23.273123+0000","last_active":"2024-06-03T02:16:37.761133+0000","last_peered":"2024-06-03T02:16:37.761133+0000","last_clean":"2024-06-03T02:16:37.761133+0000","last_became_active":"2024-06-03T02:14:23.272153+0000","last_became_peered":"2024-06-03T02:14:23.272153+0000","last_unstale":"2024-06-03T02:16:37.761133+0000","last_undegraded":"2024-06-03T02:16:37.761133+0000","last_fullsized":"2024-06-03T02:16:37.761133+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:01:43.374413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"99'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:21.848723+0000","last_change":"2024-06-03T02:14:05.245094+0000","last_active":"2024-06-03T02:16:21.848723+0000","last_peered":"2024-06-03T02:16:21.848723+0000","last_clean":"2024-06-03T02:16:21.848723+0000","last_became_active":"2024-06-03T02:14:05.244678+0000","last_became_peered":"2024-06-03T02:14:05.244678+0000","last_unstale":"2024-06-03T02:16:21.848723+0000","last_undegraded":"2024-06-03T02:16:21.848723+0000","last_fullsized":"2024-06-03T02:16:21.848723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:13:26.546135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.185237+0000","last_change":"2024-06-03T02:14:09.745919+0000","last_active":"2024-06-03T02:17:22.185237+0000","last_peered":"2024-06-03T02:17:22.185237+0000","last_clean":"2024-06-03T02:17:22.185237+0000","last_became_active":"2024-06-03T02:14:09.745576+0000","last_became_peered":"2024-06-03T02:14:09.745576+0000","last_unstale":"2024-06-03T02:17:22.185237+0000","last_undegraded":"2024-06-03T02:17:22.185237+0000","last_fullsized":"2024-06-03T02:17:22.185237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:29:31.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:36.850220+0000","last_change":"2024-06-03T02:14:23.244174+0000","last_active":"2024-06-03T02:16:36.850220+0000","last_peered":"2024-06-03T02:16:36.850220+0000","last_clean":"2024-06-03T02:16:36.850220+0000","last_became_active":"2024-06-03T02:14:23.243595+0000","last_became_peered":"2024-06-03T02:14:23.243595+0000","last_unstale":"2024-06-03T02:16:36.850220+0000","last_undegraded":"2024-06-03T02:16:36.850220+0000","last_fullsized":"2024-06-03T02:16:36.850220+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:22.213606+0000","last_clean_scrub_stamp":"2024-06-03T02:14:22.213606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:12:36.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"99'5","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:16:21.216403+0000","last_change":"2024-06-03T02:14:05.247669+0000","last_active":"2024-06-03T02:16:21.216403+0000","last_peered":"2024-06-03T02:16:21.216403+0000","last_clean":"2024-06-03T02:16:21.216403+0000","last_became_active":"2024-06-03T02:14:05.246685+0000","last_became_peered":"2024-06-03T02:14:05.246685+0000","last_unstale":"2024-06-03T02:16:21.216403+0000","last_undegraded":"2024-06-03T02:16:21.216403+0000","last_fullsized":"2024-06-03T02:16:21.216403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:04.204725+0000","last_clean_scrub_stamp":"2024-06-03T02:14:04.204725+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:53:39.454422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.171702+0000","last_change":"2024-06-03T02:14:09.749242+0000","last_active":"2024-06-03T02:17:22.171702+0000","last_peered":"2024-06-03T02:17:22.171702+0000","last_clean":"2024-06-03T02:17:22.171702+0000","last_became_active":"2024-06-03T02:14:09.748370+0000","last_became_peered":"2024-06-03T02:14:09.748370+0000","last_unstale":"2024-06-03T02:17:22.171702+0000","last_undegraded":"2024-06-03T02:17:22.171702+0000","last_fullsized":"2024-06-03T02:17:22.171702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:22:26.189960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"101'9","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.162116+0000","last_change":"2024-06-03T02:14:09.745966+0000","last_active":"2024-06-03T02:17:22.162116+0000","last_peered":"2024-06-03T02:17:22.162116+0000","last_clean":"2024-06-03T02:17:22.162116+0000","last_became_active":"2024-06-03T02:14:09.745590+0000","last_became_peered":"2024-06-03T02:14:09.745590+0000","last_unstale":"2024-06-03T02:17:22.162116+0000","last_undegraded":"2024-06-03T02:17:22.162116+0000","last_fullsized":"2024-06-03T02:17:22.162116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:46.744629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"101'12","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.163485+0000","last_change":"2024-06-03T02:14:09.744964+0000","last_active":"2024-06-03T02:17:22.163485+0000","last_peered":"2024-06-03T02:17:22.163485+0000","last_clean":"2024-06-03T02:17:22.163485+0000","last_became_active":"2024-06-03T02:14:09.743877+0000","last_became_peered":"2024-06-03T02:14:09.743877+0000","last_unstale":"2024-06-03T02:17:22.163485+0000","last_undegraded":"2024-06-03T02:17:22.163485+0000","last_fullsized":"2024-06-03T02:17:22.163485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:05:48.020670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186843+0000","last_change":"2024-06-03T02:14:09.761736+0000","last_active":"2024-06-03T02:17:22.186843+0000","last_peered":"2024-06-03T02:17:22.186843+0000","last_clean":"2024-06-03T02:17:22.186843+0000","last_became_active":"2024-06-03T02:14:09.761477+0000","last_became_peered":"2024-06-03T02:14:09.761477+0000","last_unstale":"2024-06-03T02:17:22.186843+0000","last_undegraded":"2024-06-03T02:17:22.186843+0000","last_fullsized":"2024-06-03T02:17:22.186843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:00:04.631330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"101'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183594+0000","last_change":"2024-06-03T02:14:53.266594+0000","last_active":"2024-06-03T02:17:22.183594+0000","last_peered":"2024-06-03T02:17:22.183594+0000","last_clean":"2024-06-03T02:17:22.183594+0000","last_became_active":"2024-06-03T02:14:53.266104+0000","last_became_peered":"2024-06-03T02:14:53.266104+0000","last_unstale":"2024-06-03T02:17:22.183594+0000","last_undegraded":"2024-06-03T02:17:22.183594+0000","last_fullsized":"2024-06-03T02:17:22.183594+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:12.018082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"101'4","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.174909+0000","last_change":"2024-06-03T02:14:09.772332+0000","last_active":"2024-06-03T02:17:22.174909+0000","last_peered":"2024-06-03T02:17:22.174909+0000","last_clean":"2024-06-03T02:17:22.174909+0000","last_became_active":"2024-06-03T02:14:09.771888+0000","last_became_peered":"2024-06-03T02:14:09.771888+0000","last_unstale":"2024-06-03T02:17:22.174909+0000","last_undegraded":"2024-06-03T02:17:22.174909+0000","last_fullsized":"2024-06-03T02:17:22.174909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:58:49.414986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"101'6","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.156930+0000","last_change":"2024-06-03T02:14:53.270828+0000","last_active":"2024-06-03T02:17:22.156930+0000","last_peered":"2024-06-03T02:17:22.156930+0000","last_clean":"2024-06-03T02:17:22.156930+0000","last_became_active":"2024-06-03T02:14:53.270365+0000","last_became_peered":"2024-06-03T02:14:53.270365+0000","last_unstale":"2024-06-03T02:17:22.156930+0000","last_undegraded":"2024-06-03T02:17:22.156930+0000","last_fullsized":"2024-06-03T02:17:22.156930+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.051756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"101'15","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.161647+0000","last_change":"2024-06-03T02:14:09.747709+0000","last_active":"2024-06-03T02:17:22.161647+0000","last_peered":"2024-06-03T02:17:22.161647+0000","last_clean":"2024-06-03T02:17:22.161647+0000","last_became_active":"2024-06-03T02:14:09.747359+0000","last_became_peered":"2024-06-03T02:14:09.747359+0000","last_unstale":"2024-06-03T02:17:22.161647+0000","last_undegraded":"2024-06-03T02:17:22.161647+0000","last_fullsized":"2024-06-03T02:17:22.161647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:37:24.873442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"101'2","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.145482+0000","last_change":"2024-06-03T02:14:53.473136+0000","last_active":"2024-06-03T02:17:22.145482+0000","last_peered":"2024-06-03T02:17:22.145482+0000","last_clean":"2024-06-03T02:17:22.145482+0000","last_became_active":"2024-06-03T02:14:53.471214+0000","last_became_peered":"2024-06-03T02:14:53.471214+0000","last_unstale":"2024-06-03T02:17:22.145482+0000","last_undegraded":"2024-06-03T02:17:22.145482+0000","last_fullsized":"2024-06-03T02:17:22.145482+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:35:25.822042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"101'6","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184932+0000","last_change":"2024-06-03T02:14:09.746463+0000","last_active":"2024-06-03T02:17:22.184932+0000","last_peered":"2024-06-03T02:17:22.184932+0000","last_clean":"2024-06-03T02:17:22.184932+0000","last_became_active":"2024-06-03T02:14:09.746115+0000","last_became_peered":"2024-06-03T02:14:09.746115+0000","last_unstale":"2024-06-03T02:17:22.184932+0000","last_undegraded":"2024-06-03T02:17:22.184932+0000","last_fullsized":"2024-06-03T02:17:22.184932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:51:23.050739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.184825+0000","last_change":"2024-06-03T02:14:09.755467+0000","last_active":"2024-06-03T02:17:22.184825+0000","last_peered":"2024-06-03T02:17:22.184825+0000","last_clean":"2024-06-03T02:17:22.184825+0000","last_became_active":"2024-06-03T02:14:09.755103+0000","last_became_peered":"2024-06-03T02:14:09.755103+0000","last_unstale":"2024-06-03T02:17:22.184825+0000","last_undegraded":"2024-06-03T02:17:22.184825+0000","last_fullsized":"2024-06-03T02:17:22.184825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:51:22.834519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"101'10","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.186153+0000","last_change":"2024-06-03T02:14:09.758201+0000","last_active":"2024-06-03T02:17:22.186153+0000","last_peered":"2024-06-03T02:17:22.186153+0000","last_clean":"2024-06-03T02:17:22.186153+0000","last_became_active":"2024-06-03T02:14:09.757819+0000","last_became_peered":"2024-06-03T02:14:09.757819+0000","last_unstale":"2024-06-03T02:17:22.186153+0000","last_undegraded":"2024-06-03T02:17:22.186153+0000","last_fullsized":"2024-06-03T02:17:22.186153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:44.802694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"101'11","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.183397+0000","last_change":"2024-06-03T02:14:09.768132+0000","last_active":"2024-06-03T02:17:22.183397+0000","last_peered":"2024-06-03T02:17:22.183397+0000","last_clean":"2024-06-03T02:17:22.183397+0000","last_became_active":"2024-06-03T02:14:09.767642+0000","last_became_peered":"2024-06-03T02:14:09.767642+0000","last_unstale":"2024-06-03T02:17:22.183397+0000","last_undegraded":"2024-06-03T02:17:22.183397+0000","last_fullsized":"2024-06-03T02:17:22.183397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:32:30.455418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"101'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-03T02:17:22.159132+0000","last_change":"2024-06-03T02:14:09.745603+0000","last_active":"2024-06-03T02:17:22.159132+0000","last_peered":"2024-06-03T02:17:22.159132+0000","last_clean":"2024-06-03T02:17:22.159132+0000","last_became_active":"2024-06-03T02:14:09.745245+0000","last_became_peered":"2024-06-03T02:14:09.745245+0000","last_unstale":"2024-06-03T02:17:22.159132+0000","last_undegraded":"2024-06-03T02:17:22.159132+0000","last_fullsized":"2024-06-03T02:17:22.159132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T02:14:08.698968+0000","last_clean_scrub_stamp":"2024-06-03T02:14:08.698968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:09:14.507101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":84502,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2368,"num_read_kb":1278,"num_write":279,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":614400,"data_stored":253506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318355,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":579107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579991,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37172,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707980,"statfs":{"total":95995035648,"available":95956971520,"internally_reserved":0,"allocated":1880064,"data_stored":1245173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743516,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":577953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005154,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":577637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266792,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1785856,"data_stored":1176810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430318,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":573988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28244,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716908,"statfs":{"total":95995035648,"available":95966113792,"internally_reserved":0,"allocated":1257472,"data_stored":650447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738512,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":575303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":651223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149822,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":573165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888184,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":578452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691956,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37088,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708064,"statfs":{"total":95995035648,"available":95957057536,"internally_reserved":0,"allocated":1794048,"data_stored":1170940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":78488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":77974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T02:19:30.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:19:30.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:30 smithi079 ceph-mon[31819]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:19:30.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:30 smithi079 ceph-mon[31819]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:19:30.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:30 smithi135 ceph-mon[36653]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:19:30.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:30 smithi135 ceph-mon[36653]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:19:30.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:30 smithi123 ceph-mon[36640]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:19:30.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:30 smithi123 ceph-mon[36640]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:19:31.066 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:31.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:31 smithi079 ceph-mon[31819]: pgmap v500: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-03T02:19:31.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:31 smithi079 ceph-mon[31819]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:19:31.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:31 smithi135 ceph-mon[36653]: pgmap v500: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-03T02:19:31.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:31 smithi135 ceph-mon[36653]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:19:31.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:31 smithi123 ceph-mon[36640]: pgmap v500: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-03T02:19:31.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:31 smithi123 ceph-mon[36640]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T02:19:33.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.382+0000 7fc338c00700 1 -- 172.21.15.79:0/459114065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc334108650 msgr2=0x7fc334108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:33.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.382+0000 7fc338c00700 1 --2- 172.21.15.79:0/459114065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc334108650 0x7fc334108a30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc3240098f0 tx=0x7fc324009c00 comp rx=0 tx=0).stop 2024-06-03T02:19:33.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.382+0000 7fc338c00700 1 -- 172.21.15.79:0/459114065 shutdown_connections 2024-06-03T02:19:33.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.382+0000 7fc338c00700 1 --2- 172.21.15.79:0/459114065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc334102e80 0x7fc33410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:33.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.382+0000 7fc338c00700 1 --2- 172.21.15.79:0/459114065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc334108650 0x7fc334108a30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:33.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.382+0000 7fc338c00700 1 --2- 172.21.15.79:0/459114065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3341024e0 0x7fc334102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:33.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.382+0000 7fc338c00700 1 -- 172.21.15.79:0/459114065 >> 172.21.15.79:0/459114065 conn(0x7fc3340795f0 msgr2=0x7fc334079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:33.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.382+0000 7fc338c00700 1 -- 172.21.15.79:0/459114065 shutdown_connections 2024-06-03T02:19:33.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.383+0000 7fc338c00700 1 -- 172.21.15.79:0/459114065 wait complete. 2024-06-03T02:19:33.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.384+0000 7fc338c00700 1 Processor -- start 2024-06-03T02:19:33.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.384+0000 7fc338c00700 1 -- start start 2024-06-03T02:19:33.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.384+0000 7fc338c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3341024e0 0x7fc33419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:33.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.384+0000 7fc338c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc334102e80 0x7fc33419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:33.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.384+0000 7fc338c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc334108650 0x7fc334198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:33.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.384+0000 7fc338c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341127c0 con 0x7fc3341024e0 2024-06-03T02:19:33.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.384+0000 7fc338c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc334112640 con 0x7fc334102e80 2024-06-03T02:19:33.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.384+0000 7fc338c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc334112940 con 0x7fc334108650 2024-06-03T02:19:33.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3341024e0 0x7fc33419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:33.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3341024e0 0x7fc33419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36604/0 (socket says 172.21.15.79:36604) 2024-06-03T02:19:33.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331e00700 1 -- 172.21.15.79:0/3215611489 learned_addr learned my addr 172.21.15.79:0/3215611489 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:33.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc332800700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc334108650 0x7fc334198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:33.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331400700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc334102e80 0x7fc33419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331e00700 1 -- 172.21.15.79:0/3215611489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc334108650 msgr2=0x7fc334198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331e00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc334108650 0x7fc334198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331e00700 1 -- 172.21.15.79:0/3215611489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc334102e80 msgr2=0x7fc33419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331e00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc334102e80 0x7fc33419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331e00700 1 -- 172.21.15.79:0/3215611489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3341991e0 con 0x7fc3341024e0 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc332800700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc334108650 0x7fc334198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.385+0000 7fc331400700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc334102e80 0x7fc33419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.386+0000 7fc331e00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3341024e0 0x7fc33419e870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc324009230 tx=0x7fc32400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.386+0000 7fc32aa00700 1 -- 172.21.15.79:0/3215611489 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324005180 con 0x7fc3341024e0 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.386+0000 7fc338c00700 1 -- 172.21.15.79:0/3215611489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc334199470 con 0x7fc3341024e0 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.386+0000 7fc32aa00700 1 -- 172.21.15.79:0/3215611489 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc32400b560 con 0x7fc3341024e0 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.386+0000 7fc32aa00700 1 -- 172.21.15.79:0/3215611489 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324003a70 con 0x7fc3341024e0 2024-06-03T02:19:33.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.386+0000 7fc338c00700 1 -- 172.21.15.79:0/3215611489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3340746c0 con 0x7fc3341024e0 2024-06-03T02:19:33.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.388+0000 7fc31fe00700 1 -- 172.21.15.79:0/3215611489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30c005330 con 0x7fc3341024e0 2024-06-03T02:19:33.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.390+0000 7fc32aa00700 1 -- 172.21.15.79:0/3215611489 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fc324029060 con 0x7fc3341024e0 2024-06-03T02:19:33.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.391+0000 7fc32aa00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc3180a70c0 0x7fc3180a9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:33.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.391+0000 7fc32aa00700 1 -- 172.21.15.79:0/3215611489 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc3240d03f0 con 0x7fc3341024e0 2024-06-03T02:19:33.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.391+0000 7fc331400700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc3180a70c0 0x7fc3180a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:33.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.392+0000 7fc331400700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc3180a70c0 0x7fc3180a9580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc320009750 tx=0x7fc320009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:33.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.394+0000 7fc32aa00700 1 -- 172.21.15.79:0/3215611489 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc32405d850 con 0x7fc3341024e0 2024-06-03T02:19:33.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.672+0000 7fc31fe00700 1 -- 172.21.15.79:0/3215611489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc30c005620 con 0x7fc3341024e0 2024-06-03T02:19:33.673 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:33 smithi079 ceph-mon[31819]: pgmap v501: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-03T02:19:33.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.674+0000 7fc32aa00700 1 -- 172.21.15.79:0/3215611489 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7fc324013030 con 0x7fc3341024e0 2024-06-03T02:19:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:33.675 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:19:33.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 -- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc3180a70c0 msgr2=0x7fc3180a9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:33.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc3180a70c0 0x7fc3180a9580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc320009750 tx=0x7fc320009310 comp rx=0 tx=0).stop 2024-06-03T02:19:33.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 -- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3341024e0 msgr2=0x7fc33419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:33.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3341024e0 0x7fc33419e870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc324009230 tx=0x7fc32400fa70 comp rx=0 tx=0).stop 2024-06-03T02:19:33.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 -- 172.21.15.79:0/3215611489 shutdown_connections 2024-06-03T02:19:33.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc334108650 0x7fc334198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:33.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc3180a70c0 0x7fc3180a9580 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:33.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3341024e0 0x7fc33419e870 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:33.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 --2- 172.21.15.79:0/3215611489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc334102e80 0x7fc33419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:33.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 -- 172.21.15.79:0/3215611489 >> 172.21.15.79:0/3215611489 conn(0x7fc3340795f0 msgr2=0x7fc3341011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:33.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 -- 172.21.15.79:0/3215611489 shutdown_connections 2024-06-03T02:19:33.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:33.677+0000 7fc31fe00700 1 -- 172.21.15.79:0/3215611489 wait complete. 2024-06-03T02:19:33.678 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:19:33.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:33 smithi135 ceph-mon[36653]: pgmap v501: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-03T02:19:33.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:33 smithi123 ceph-mon[36640]: pgmap v501: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-03T02:19:34.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:19:34.735 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:34 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3215611489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:19:34.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:34 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3215611489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:19:34.922 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:34.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:34 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3215611489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:19:35.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:35 smithi079 ceph-mon[31819]: pgmap v502: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-03T02:19:35.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:35 smithi135 ceph-mon[36653]: pgmap v502: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-03T02:19:35.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:35 smithi123 ceph-mon[36640]: pgmap v502: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-03T02:19:37.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.189+0000 7fdf15e00700 1 -- 172.21.15.79:0/52278139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1006a170 msgr2=0x7fdf1006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:37.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.189+0000 7fdf15e00700 1 --2- 172.21.15.79:0/52278139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1006a170 0x7fdf1006a5f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009990 tx=0x7fdefc009ca0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.189+0000 7fdf15e00700 1 -- 172.21.15.79:0/52278139 shutdown_connections 2024-06-03T02:19:37.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.189+0000 7fdf15e00700 1 --2- 172.21.15.79:0/52278139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf1006ab30 0x7fdf1010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.189+0000 7fdf15e00700 1 --2- 172.21.15.79:0/52278139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf10108500 0x7fdf101088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.189+0000 7fdf15e00700 1 --2- 172.21.15.79:0/52278139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1006a170 0x7fdf1006a5f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.189+0000 7fdf15e00700 1 -- 172.21.15.79:0/52278139 >> 172.21.15.79:0/52278139 conn(0x7fdf10078340 msgr2=0x7fdf10078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:37.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.189+0000 7fdf15e00700 1 -- 172.21.15.79:0/52278139 shutdown_connections 2024-06-03T02:19:37.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.189+0000 7fdf15e00700 1 -- 172.21.15.79:0/52278139 wait complete. 2024-06-03T02:19:37.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.190+0000 7fdf15e00700 1 Processor -- start 2024-06-03T02:19:37.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.190+0000 7fdf15e00700 1 -- start start 2024-06-03T02:19:37.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1006a170 0x7fdf10101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:37.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf1006ab30 0x7fdf101019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:37.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf10108500 0x7fdf10101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:37.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf15e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101127b0 con 0x7fdf1006ab30 2024-06-03T02:19:37.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf15e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf10112630 con 0x7fdf1006a170 2024-06-03T02:19:37.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf15e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf10112930 con 0x7fdf10108500 2024-06-03T02:19:37.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf0e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf1006ab30 0x7fdf101019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:37.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf0ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1006a170 0x7fdf10101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:37.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf0e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf1006ab30 0x7fdf101019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46670/0 (socket says 172.21.15.79:46670) 2024-06-03T02:19:37.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf0e000700 1 -- 172.21.15.79:0/3332123868 learned_addr learned my addr 172.21.15.79:0/3332123868 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:37.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.191+0000 7fdf0f400700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf10108500 0x7fdf10101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:37.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf0ea00700 1 -- 172.21.15.79:0/3332123868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf10108500 msgr2=0x7fdf10101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:37.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf0ea00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf10108500 0x7fdf10101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf0ea00700 1 -- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf1006ab30 msgr2=0x7fdf101019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:37.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf0ea00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf1006ab30 0x7fdf101019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf0ea00700 1 -- 172.21.15.79:0/3332123868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf101a7190 con 0x7fdf1006a170 2024-06-03T02:19:37.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf0e000700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf1006ab30 0x7fdf101019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:37.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf0ea00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1006a170 0x7fdf10101460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e9f0 tx=0x7fdf0000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:37.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf14a00700 1 -- 172.21.15.79:0/3332123868 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000c9c0 con 0x7fdf1006a170 2024-06-03T02:19:37.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf14a00700 1 -- 172.21.15.79:0/3332123868 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0000cb60 con 0x7fdf1006a170 2024-06-03T02:19:37.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf101a7480 con 0x7fdf1006a170 2024-06-03T02:19:37.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.192+0000 7fdf14a00700 1 -- 172.21.15.79:0/3332123868 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00018680 con 0x7fdf1006a170 2024-06-03T02:19:37.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.193+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf101a7a10 con 0x7fdf1006a170 2024-06-03T02:19:37.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.194+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf10069180 con 0x7fdf1006a170 2024-06-03T02:19:37.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.196+0000 7fdf14a00700 1 -- 172.21.15.79:0/3332123868 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fdf00010430 con 0x7fdf1006a170 2024-06-03T02:19:37.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.197+0000 7fdf14a00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdef40a2d30 0x7fdef40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:37.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.197+0000 7fdf14a00700 1 -- 172.21.15.79:0/3332123868 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fdf000cf7f0 con 0x7fdf1006a170 2024-06-03T02:19:37.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.198+0000 7fdf0e000700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdef40a2d30 0x7fdef40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:37.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.198+0000 7fdf0e000700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdef40a2d30 0x7fdef40a51f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009750 tx=0x7fdefc00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:37.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.199+0000 7fdf14a00700 1 -- 172.21.15.79:0/3332123868 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf0009c1e0 con 0x7fdf1006a170 2024-06-03T02:19:37.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.427+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf1002dca0 con 0x7fdf1006a170 2024-06-03T02:19:37.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.429+0000 7fdf14a00700 1 -- 172.21.15.79:0/3332123868 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+24805 (secure 0 0 0) 0x7fdf0009bae0 con 0x7fdf1006a170 2024-06-03T02:19:37.430 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:37.430 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":101,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:15:54.257434+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:19:37.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdef40a2d30 msgr2=0x7fdef40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:37.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdef40a2d30 0x7fdef40a51f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009750 tx=0x7fdefc00b280 comp rx=0 tx=0).stop 2024-06-03T02:19:37.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1006a170 msgr2=0x7fdf10101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:37.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1006a170 0x7fdf10101460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e9f0 tx=0x7fdf0000ed00 comp rx=0 tx=0).stop 2024-06-03T02:19:37.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 shutdown_connections 2024-06-03T02:19:37.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf10108500 0x7fdf10101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1006a170 0x7fdf10101460 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdef40a2d30 0x7fdef40a51f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 --2- 172.21.15.79:0/3332123868 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf1006ab30 0x7fdf101019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:37.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 >> 172.21.15.79:0/3332123868 conn(0x7fdf10078340 msgr2=0x7fdf100fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:37.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 shutdown_connections 2024-06-03T02:19:37.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:37.432+0000 7fdf15e00700 1 -- 172.21.15.79:0/3332123868 wait complete. 2024-06-03T02:19:37.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:37 smithi079 ceph-mon[31819]: pgmap v503: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 852 B/s wr, 10 op/s 2024-06-03T02:19:37.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:37 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3332123868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:19:37.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:37 smithi135 ceph-mon[36653]: pgmap v503: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 852 B/s wr, 10 op/s 2024-06-03T02:19:37.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:37 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3332123868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:19:37.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:37 smithi123 ceph-mon[36640]: pgmap v503: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 852 B/s wr, 10 op/s 2024-06-03T02:19:37.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:37 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3332123868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:19:38.384 DEBUG:tasks.fwd_scrub:Yielding 2024-06-03T02:19:38.385 INFO:teuthology.run_tasks:Running task exec... 2024-06-03T02:19:38.396 INFO:teuthology.task.exec:Executing custom commands... 2024-06-03T02:19:38.396 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi079.front.sepia.ceph.com 2024-06-03T02:19:38.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-06-03T02:19:38.397 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-03T02:19:38.397 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:19:38.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:19:38.500 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:38.545 INFO:teuthology.orchestra.run.smithi079.stdout:Filesystem Size Used Avail Use% Mounted on 2024-06-03T02:19:38.545 INFO:teuthology.orchestra.run.smithi079.stdout:0@7a1e9ad6-214d-11ef-bc9b-c7b262605968.cephfs=/volumes/qa/sv_1/384b230e-7fb1-414a-9276-a11ddd986ce8 1.1T 356M 1.1T 1% /tmp/cephfs 2024-06-03T02:19:38.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-06-03T02:19:38.687 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-06-03T02:19:38.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-06-03T02:19:38.846 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:39 smithi079 ceph-mon[31819]: pgmap v504: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-06-03T02:19:39.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:39 smithi135 ceph-mon[36653]: pgmap v504: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-06-03T02:19:39.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:39 smithi123 ceph-mon[36640]: pgmap v504: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-06-03T02:19:41.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-06-03T02:19:41.303 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-06-03T02:19:41.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-06-03T02:19:41.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-06-03T02:19:41.542 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/: 2024-06-03T02:19:41.542 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:41.542 INFO:teuthology.orchestra.run.smithi079.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:19:41.543 INFO:teuthology.orchestra.run.smithi079.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 .. 2024-06-03T02:19:41.543 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 data 2024-06-03T02:19:41.543 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:41.543 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data: 2024-06-03T02:19:41.543 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:41.544 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:41.544 INFO:teuthology.orchestra.run.smithi079.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 .. 2024-06-03T02:19:41.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-06-03T02:19:41.674 INFO:teuthology.orchestra.run.smithi079.stdout:[Service] 2024-06-03T02:19:41.675 INFO:teuthology.orchestra.run.smithi079.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-06-03T02:19:41.675 INFO:teuthology.orchestra.run.smithi079.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-06-03T02:19:41.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-06-03T02:19:41.745 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:41 smithi079 ceph-mon[31819]: pgmap v505: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.839+0000 7f1222600700 1 -- 172.21.15.79:0/1176998152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f121c1030f0 msgr2=0x7f121c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.839+0000 7f1222600700 1 --2- 172.21.15.79:0/1176998152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f121c1030f0 0x7f121c107110 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1204009920 tx=0x7f1204009c30 comp rx=0 tx=0).stop 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.839+0000 7f1222600700 1 -- 172.21.15.79:0/1176998152 shutdown_connections 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.839+0000 7f1222600700 1 --2- 172.21.15.79:0/1176998152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f121c107840 0x7f121c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.839+0000 7f1222600700 1 --2- 172.21.15.79:0/1176998152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f121c102740 0x7f121c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.839+0000 7f1222600700 1 --2- 172.21.15.79:0/1176998152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f121c1030f0 0x7f121c107110 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.839+0000 7f1222600700 1 -- 172.21.15.79:0/1176998152 >> 172.21.15.79:0/1176998152 conn(0x7f121c0795f0 msgr2=0x7f121c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.840+0000 7f1222600700 1 -- 172.21.15.79:0/1176998152 shutdown_connections 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.840+0000 7f1222600700 1 -- 172.21.15.79:0/1176998152 wait complete. 2024-06-03T02:19:41.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.841+0000 7f1222600700 1 Processor -- start 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.841+0000 7f1222600700 1 -- start start 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.841+0000 7f1222600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f121c102740 0x7f121c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.841+0000 7f1222600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f121c1030f0 0x7f121c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.841+0000 7f1222600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f121c107840 0x7f121c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.841+0000 7f1222600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1127a0 con 0x7f121c102740 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.841+0000 7f1222600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f121c112620 con 0x7f121c107840 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.841+0000 7f1222600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f121c112920 con 0x7f121c1030f0 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f1213e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f121c1030f0 0x7f121c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f121b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f121c102740 0x7f121c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f1213e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f121c1030f0 0x7f121c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33102/0 (socket says 172.21.15.79:33102) 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f1213e00700 1 -- 172.21.15.79:0/1188117134 learned_addr learned my addr 172.21.15.79:0/1188117134 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f121be00700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f121c107840 0x7f121c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f1213e00700 1 -- 172.21.15.79:0/1188117134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f121c107840 msgr2=0x7f121c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f1213e00700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f121c107840 0x7f121c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f1213e00700 1 -- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f121c102740 msgr2=0x7f121c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f1213e00700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f121c102740 0x7f121c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f1213e00700 1 -- 172.21.15.79:0/1188117134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c1991e0 con 0x7f121c1030f0 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.842+0000 7f121b400700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f121c102740 0x7f121c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.843+0000 7f1213e00700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f121c1030f0 0x7f121c19ee00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f120400fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.843+0000 7f1218c00700 1 -- 172.21.15.79:0/1188117134 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204027070 con 0x7f121c1030f0 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.843+0000 7f1218c00700 1 -- 172.21.15.79:0/1188117134 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1204005660 con 0x7f121c1030f0 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.843+0000 7f1222600700 1 -- 172.21.15.79:0/1188117134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c199470 con 0x7f121c1030f0 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.843+0000 7f1218c00700 1 -- 172.21.15.79:0/1188117134 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204004930 con 0x7f121c1030f0 2024-06-03T02:19:41.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.843+0000 7f1222600700 1 -- 172.21.15.79:0/1188117134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c1a5790 con 0x7f121c1030f0 2024-06-03T02:19:41.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:41 smithi135 ceph-mon[36653]: pgmap v505: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-03T02:19:41.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.848+0000 7f1218c00700 1 -- 172.21.15.79:0/1188117134 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f120402a070 con 0x7f121c1030f0 2024-06-03T02:19:41.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.849+0000 7f122cff9b80 1 Processor -- start 2024-06-03T02:19:41.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.849+0000 7f122cff9b80 1 -- start start 2024-06-03T02:19:41.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f1218c00700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12000a2cb0 0x7f12000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e7c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c940 0x5583c3e76190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e76b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f122cff9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5583c3d80a90 con 0x5583c3e7c940 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f122cff9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583c3dc3b80 con 0x5583c3e7c020 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f122cff9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5583c3d7e970 con 0x5583c3e766d0 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f1218c00700 1 -- 172.21.15.79:0/1188117134 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f12040cffb0 con 0x7f121c1030f0 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f121b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e7c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f1213e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c940 0x5583c3e76190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f121b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e7c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:39828/0 (socket says 172.21.15.79:39828) 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f121b400700 1 -- 172.21.15.79:0/1119367424 learned_addr learned my addr 172.21.15.79:0/1119367424 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f121be00700 1 --2- 172.21.15.79:0/1119367424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e76b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.850+0000 7f121be00700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12000a2cb0 0x7f12000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.851+0000 7f121b400700 1 -- 172.21.15.79:0/1119367424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 msgr2=0x5583c3e76b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:41.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.851+0000 7f121b400700 1 --2- 172.21.15.79:0/1119367424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e76b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.851+0000 7f121b400700 1 -- 172.21.15.79:0/1119367424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c940 msgr2=0x5583c3e76190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:41.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.851+0000 7f121b400700 1 --2- 172.21.15.79:0/1119367424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c940 0x5583c3e76190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.851+0000 7f121b400700 1 -- 172.21.15.79:0/1119367424 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c3d6c010 con 0x5583c3e7c020 2024-06-03T02:19:41.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.851+0000 7f1213e00700 1 --2- 172.21.15.79:0/1119367424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c940 0x5583c3e76190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:41.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.851+0000 7f121b400700 1 --2- 172.21.15.79:0/1119367424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e7c400 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f120c0115f0 tx=0x7f120c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9424505567d43d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:41.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.851+0000 7f1210c00700 1 -- 172.21.15.79:0/1119367424 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c004a30 con 0x5583c3e7c020 2024-06-03T02:19:41.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.851+0000 7f121be00700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12000a2cb0 0x7f12000a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1214009e60 tx=0x7f121400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:41.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.852+0000 7f1210c00700 1 -- 172.21.15.79:0/1119367424 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c005500 con 0x5583c3e7c020 2024-06-03T02:19:41.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.852+0000 7f1210c00700 1 -- 172.21.15.79:0/1119367424 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c019a10 con 0x5583c3e7c020 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.852+0000 7f122cff9b80 1 -- 172.21.15.79:0/1119367424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 msgr2=0x5583c3e7c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.852+0000 7f122cff9b80 1 --2- 172.21.15.79:0/1119367424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e7c400 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f120c0115f0 tx=0x7f120c011900 comp rx=0 tx=0).stop 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.853+0000 7f122cff9b80 1 -- 172.21.15.79:0/1119367424 shutdown_connections 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.853+0000 7f122cff9b80 1 --2- 172.21.15.79:0/1119367424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e76b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.853+0000 7f122cff9b80 1 --2- 172.21.15.79:0/1119367424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e7c400 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.853+0000 7f122cff9b80 1 --2- 172.21.15.79:0/1119367424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c940 0x5583c3e76190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.853+0000 7f122cff9b80 1 -- 172.21.15.79:0/1119367424 >> 172.21.15.79:0/1119367424 conn(0x5583c3d9e1f0 msgr2=0x5583c3e6d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.853+0000 7f122cff9b80 1 -- 172.21.15.79:0/1119367424 shutdown_connections 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.853+0000 7f122cff9b80 1 -- 172.21.15.79:0/1119367424 wait complete. 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.853+0000 7f122cff9b80 1 -- start start 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f122cff9b80 1 Processor -- start 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c020 0x5583c3e7f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c940 0x5583c3e7fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f122cff9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5583c3d5ffe0 con 0x5583c3e7c020 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f122cff9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583c3dbfb40 con 0x5583c3e7c940 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f122cff9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5583c3e83320 con 0x5583c3e766d0 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f122cff9b80 10 client.0 _ll_register_callbacks cb 0x5583c3d59400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f121b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f1213e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c020 0x5583c3e7f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f121b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e82680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33112/0 (socket says 172.21.15.79:33112) 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.854+0000 7f121b400700 1 -- 172.21.15.79:0/3194746703 learned_addr learned my addr 172.21.15.79:0/3194746703 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f121be00700 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c940 0x5583c3e7fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f121b400700 1 -- 172.21.15.79:0/3194746703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c940 msgr2=0x5583c3e7fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f121b400700 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c940 0x5583c3e7fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f121b400700 1 -- 172.21.15.79:0/3194746703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c020 msgr2=0x5583c3e7f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f121b400700 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c020 0x5583c3e7f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f121b400700 1 -- 172.21.15.79:0/3194746703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c3d12950 con 0x5583c3e766d0 2024-06-03T02:19:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f121be00700 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c940 0x5583c3e7fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:19:41.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f1213e00700 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c020 0x5583c3e7f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:41.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f121b400700 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e82680 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f120c0115c0 tx=0x7f120c0038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e2de18633c2796 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:41.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.855+0000 7f1209600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.856+0000 7f1209600700 1 -- 172.21.15.79:0/3194746703 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c027070 con 0x5583c3e766d0 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.856+0000 7f122cff9b80 1 -- 172.21.15.79:0/3194746703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c005500 con 0x5583c3e766d0 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.856+0000 7f1209600700 1 -- 172.21.15.79:0/3194746703 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f120c019570 con 0x5583c3e766d0 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.856+0000 7f1209600700 1 -- 172.21.15.79:0/3194746703 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f120c033050 con 0x5583c3e766d0 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.856+0000 7f122cff9b80 10 client.25282 fetch_fsmap learned FSMap version 30 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.856+0000 7f122cff9b80 1 -- 172.21.15.79:0/3194746703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5583c3e8da80 con 0x5583c3e766d0 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.856+0000 7f1209600700 1 -- 172.21.15.79:0/3194746703 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c005500 con 0x5583c3e766d0 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.857+0000 7f1209600700 1 -- 172.21.15.79:0/3194746703 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f120c0236c0 con 0x5583c3e766d0 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.857+0000 7f122cff9b80 10 client.25282 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.857+0000 7f122cff9b80 10 client.25282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.857+0000 7f122cff9b80 20 client.25282 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.857+0000 7f122cff9b80 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583c3e8ffb0 0x5583c3e923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.857+0000 7f122cff9b80 4 client.25282 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.857+0000 7f122cff9b80 1 -- 172.21.15.79:0/3194746703 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583c3dd1080 con 0x5583c3e8ffb0 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.857+0000 7f1213e00700 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583c3e8ffb0 0x5583c3e923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.858+0000 7f1213e00700 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583c3e8ffb0 0x5583c3e923a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2782e56ae73cbdc2 server_cookie=1bd6bcf56ec49dac in_seq=0 out_seq=0 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.858+0000 7f1209600700 10 client.25282 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f1209600700 1 -- 172.21.15.79:0/3194746703 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5583c3dd1080 con 0x5583c3e8ffb0 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f1209600700 10 client.25282 handle_command_reply: tid=0 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 client.25282 shutdown 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 -- 172.21.15.79:0/3194746703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 msgr2=0x5583c3e82680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e82680 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f120c0115c0 tx=0x7f120c0038b0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 -- 172.21.15.79:0/3194746703 shutdown_connections 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c940 0x5583c3e7fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e82680 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583c3e8ffb0 0x5583c3e923a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 --2- 172.21.15.79:0/3194746703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e7c020 0x5583c3e7f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f1209600700 0 client.25282 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 -- 172.21.15.79:0/3194746703 >> 172.21.15.79:0/3194746703 conn(0x5583c3d9e1f0 msgr2=0x5583c3e66810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 -- 172.21.15.79:0/3194746703 shutdown_connections 2024-06-03T02:19:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 1 -- 172.21.15.79:0/3194746703 wait complete. 2024-06-03T02:19:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.859+0000 7f122cff9b80 20 client.25282 trim_cache size 0 max 16384 2024-06-03T02:19:41.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f122cff9b80 1 Processor -- start 2024-06-03T02:19:41.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f122cff9b80 1 -- start start 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3d9e1f0 0x5583c3e76190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e766d0 0x5583c3e7d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e7c020 0x5583c3e671b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f122cff9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1ac060 con 0x5583c3e766d0 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f122cff9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1adba0 con 0x5583c3d9e1f0 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f122cff9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1ac730 con 0x5583c3e7c020 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f1213e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e766d0 0x5583c3e7d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.883+0000 7f121b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3d9e1f0 0x5583c3e76190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f1213e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e766d0 0x5583c3e7d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46716/0 (socket says 172.21.15.79:46716) 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f121be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e7c020 0x5583c3e671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f121b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3d9e1f0 0x5583c3e76190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:39854/0 (socket says 172.21.15.79:39854) 2024-06-03T02:19:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f1213e00700 1 -- 172.21.15.79:0/635186816 learned_addr learned my addr 172.21.15.79:0/635186816 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f121b400700 1 -- 172.21.15.79:0/635186816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e7c020 msgr2=0x5583c3e671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f121b400700 1 --2- 172.21.15.79:0/635186816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e7c020 0x5583c3e671b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f121b400700 1 -- 172.21.15.79:0/635186816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e766d0 msgr2=0x5583c3e7d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f121b400700 1 --2- 172.21.15.79:0/635186816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e766d0 0x5583c3e7d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f121b400700 1 -- 172.21.15.79:0/635186816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c3e8c470 con 0x5583c3d9e1f0 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f1213e00700 1 --2- 172.21.15.79:0/635186816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e766d0 0x5583c3e7d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.884+0000 7f121b400700 1 --2- 172.21.15.79:0/635186816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3d9e1f0 0x5583c3e76190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f120c023a90 tx=0x7f120c005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6df436e7c326d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.885+0000 7f120be00700 1 -- 172.21.15.79:0/635186816 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c027070 con 0x5583c3d9e1f0 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.885+0000 7f120be00700 1 -- 172.21.15.79:0/635186816 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c005040 con 0x5583c3d9e1f0 2024-06-03T02:19:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.885+0000 7f120be00700 1 -- 172.21.15.79:0/635186816 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c0197a0 con 0x5583c3d9e1f0 2024-06-03T02:19:41.887 INFO:teuthology.orchestra.run.smithi079.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-06-03T02:19:41.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 -- 172.21.15.79:0/635186816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3d9e1f0 msgr2=0x5583c3e76190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 --2- 172.21.15.79:0/635186816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3d9e1f0 0x5583c3e76190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f120c023a90 tx=0x7f120c005a00 comp rx=0 tx=0).stop 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 -- 172.21.15.79:0/635186816 shutdown_connections 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 --2- 172.21.15.79:0/635186816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e7c020 0x5583c3e671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 --2- 172.21.15.79:0/635186816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3d9e1f0 0x5583c3e76190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 --2- 172.21.15.79:0/635186816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3e766d0 0x5583c3e7d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 -- 172.21.15.79:0/635186816 >> 172.21.15.79:0/635186816 conn(0x5583c3e7a260 msgr2=0x5583c3e7e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 -- 172.21.15.79:0/635186816 shutdown_connections 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 -- 172.21.15.79:0/635186816 wait complete. 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.886+0000 7f122cff9b80 1 -- start start 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.887+0000 7f122cff9b80 1 Processor -- start 2024-06-03T02:19:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.887+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3d9e1f0 0x5583c3e885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.887+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e88b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.887+0000 7f122cff9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.887+0000 7f122cff9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1ad4d0 con 0x5583c3d9e1f0 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.887+0000 7f122cff9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1ace00 con 0x5583c3e7c020 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.887+0000 7f122cff9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1aa000 con 0x5583c3e766d0 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.887+0000 7f122cff9b80 10 client.0 _ll_register_callbacks cb 0x5583c3d59400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f121b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3d9e1f0 0x5583c3e885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f1213e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e88b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f121b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3d9e1f0 0x5583c3e885e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46728/0 (socket says 172.21.15.79:46728) 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f121b400700 1 -- 172.21.15.79:0/909754510 learned_addr learned my addr 172.21.15.79:0/909754510 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f121be00700 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f1213e00700 1 -- 172.21.15.79:0/909754510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 msgr2=0x5583c3e855b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f1213e00700 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e855b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f1213e00700 1 -- 172.21.15.79:0/909754510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3d9e1f0 msgr2=0x5583c3e885e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f1213e00700 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3d9e1f0 0x5583c3e885e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f1213e00700 1 -- 172.21.15.79:0/909754510 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c3e891d0 con 0x5583c3e766d0 2024-06-03T02:19:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f1213e00700 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e88b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1204009680 tx=0x7f12040bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7bbe27861367db3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.888+0000 7f1209600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:19:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.889+0000 7f1209600700 1 -- 172.21.15.79:0/909754510 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040a06e0 con 0x5583c3e766d0 2024-06-03T02:19:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.889+0000 7f1209600700 1 -- 172.21.15.79:0/909754510 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f120409a630 con 0x5583c3e766d0 2024-06-03T02:19:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.889+0000 7f122cff9b80 1 -- 172.21.15.79:0/909754510 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c005040 con 0x5583c3e766d0 2024-06-03T02:19:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.889+0000 7f1209600700 1 -- 172.21.15.79:0/909754510 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12040a4f10 con 0x5583c3e766d0 2024-06-03T02:19:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.889+0000 7f122cff9b80 10 client.25288 fetch_fsmap learned FSMap version 30 2024-06-03T02:19:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.889+0000 7f122cff9b80 1 -- 172.21.15.79:0/909754510 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5583c3efb130 con 0x5583c3e766d0 2024-06-03T02:19:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.889+0000 7f1209600700 1 -- 172.21.15.79:0/909754510 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c005040 con 0x5583c3e766d0 2024-06-03T02:19:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.889+0000 7f1209600700 1 -- 172.21.15.79:0/909754510 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f120409c590 con 0x5583c3e766d0 2024-06-03T02:19:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.890+0000 7f122cff9b80 10 client.25288 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:19:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.890+0000 7f122cff9b80 10 client.25288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:19:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.890+0000 7f122cff9b80 20 client.25288 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.890+0000 7f122cff9b80 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583c3e7c940 0x5583c3efd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.890+0000 7f122cff9b80 4 client.25288 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:19:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.890+0000 7f122cff9b80 1 -- 172.21.15.79:0/909754510 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5583c3dcb600 con 0x5583c3e7c940 2024-06-03T02:19:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.890+0000 7f121b400700 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583c3e7c940 0x5583c3efd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.890+0000 7f121b400700 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583c3e7c940 0x5583c3efd7e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890ff138c70dfbf7 server_cookie=360ab36b18e858a9 in_seq=0 out_seq=0 2024-06-03T02:19:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.890+0000 7f1209600700 10 client.25288 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f1209600700 1 -- 172.21.15.79:0/909754510 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5583c3dcb600 con 0x5583c3e7c940 2024-06-03T02:19:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f1209600700 10 client.25288 handle_command_reply: tid=0 2024-06-03T02:19:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 client.25288 shutdown 2024-06-03T02:19:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 -- 172.21.15.79:0/909754510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 msgr2=0x5583c3e88b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e88b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1204009680 tx=0x7f12040bc040 comp rx=0 tx=0).stop 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 -- 172.21.15.79:0/909754510 shutdown_connections 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583c3e7c020 0x5583c3e855b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583c3e7c940 0x5583c3efd7e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583c3d9e1f0 0x5583c3e885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 --2- 172.21.15.79:0/909754510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583c3e766d0 0x5583c3e88b20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f1209600700 0 client.25288 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 -- 172.21.15.79:0/909754510 >> 172.21.15.79:0/909754510 conn(0x5583c3e7a260 msgr2=0x5583c3e7eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.892+0000 7f122cff9b80 1 -- 172.21.15.79:0/909754510 shutdown_connections 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.893+0000 7f122cff9b80 1 -- 172.21.15.79:0/909754510 wait complete. 2024-06-03T02:19:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.893+0000 7f122cff9b80 20 client.25288 trim_cache size 0 max 16384 2024-06-03T02:19:41.896 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:19:41.896 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:19:41.896 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "9505095d-e7c5-4873-b408-174f166602ba", 2024-06-03T02:19:41.896 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:19:41.896 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:19:41.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.893+0000 7f1222600700 1 -- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12000a2cb0 msgr2=0x7f12000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.893+0000 7f1222600700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12000a2cb0 0x7f12000a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1214009e60 tx=0x7f121400b040 comp rx=0 tx=0).stop 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.893+0000 7f1222600700 1 -- 172.21.15.79:0/1188117134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f121c1030f0 msgr2=0x7f121c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.893+0000 7f1222600700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f121c1030f0 0x7f121c19ee00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f120400fa70 comp rx=0 tx=0).stop 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.894+0000 7f1222600700 1 -- 172.21.15.79:0/1188117134 shutdown_connections 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.894+0000 7f1222600700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f121c107840 0x7f121c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.894+0000 7f1222600700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12000a2cb0 0x7f12000a5170 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.894+0000 7f1222600700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f121c102740 0x7f121c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.894+0000 7f1222600700 1 --2- 172.21.15.79:0/1188117134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f121c1030f0 0x7f121c19ee00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.894+0000 7f1222600700 1 -- 172.21.15.79:0/1188117134 >> 172.21.15.79:0/1188117134 conn(0x7f121c0795f0 msgr2=0x7f121c1008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.894+0000 7f1222600700 1 -- 172.21.15.79:0/1188117134 shutdown_connections 2024-06-03T02:19:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:41.894+0000 7f1222600700 1 -- 172.21.15.79:0/1188117134 wait complete. 2024-06-03T02:19:41.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:41 smithi123 ceph-mon[36640]: pgmap v505: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-03T02:19:42.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:42 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:19:42.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:42 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:19:42.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:42 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:19:42.836 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:42 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:19:42.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:42 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:19:42.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:42 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:19:42.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:42 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:19:42.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:42 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:19:42.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:19:42.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:42 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:19:42.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:42 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:19:42.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:42 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:19:42.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:42 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:19:43.387 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:19:43.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:43 smithi079 ceph-mon[31819]: pgmap v506: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-03T02:19:43.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:43 smithi135 ceph-mon[36653]: pgmap v506: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-03T02:19:43.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:43 smithi123 ceph-mon[36640]: pgmap v506: 129 pgs: 129 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-03T02:19:45.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.833+0000 7f91a0200700 1 -- 172.21.15.79:0/3939483899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9198109480 msgr2=0x7f9198115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:45.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.833+0000 7f91a0200700 1 --2- 172.21.15.79:0/3939483899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9198109480 0x7f9198115990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9190009f90 tx=0x7f919000b330 comp rx=0 tx=0).stop 2024-06-03T02:19:45.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.833+0000 7f91a0200700 1 -- 172.21.15.79:0/3939483899 shutdown_connections 2024-06-03T02:19:45.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.833+0000 7f91a0200700 1 --2- 172.21.15.79:0/3939483899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9198109480 0x7f9198115990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.833+0000 7f91a0200700 1 --2- 172.21.15.79:0/3939483899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f919810eac0 0x7f919810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.833+0000 7f91a0200700 1 --2- 172.21.15.79:0/3939483899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9198108ac0 0x7f9198108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.833+0000 7f91a0200700 1 -- 172.21.15.79:0/3939483899 >> 172.21.15.79:0/3939483899 conn(0x7f9198075c60 msgr2=0x7f91981072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:45.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.833+0000 7f91a0200700 1 -- 172.21.15.79:0/3939483899 shutdown_connections 2024-06-03T02:19:45.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.833+0000 7f91a0200700 1 -- 172.21.15.79:0/3939483899 wait complete. 2024-06-03T02:19:45.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:45 smithi079 ceph-mon[31819]: pgmap v507: 129 pgs: 129 active+clean; 1.5 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 74 KiB/s wr, 14 op/s 2024-06-03T02:19:45.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.834+0000 7f91a0200700 1 Processor -- start 2024-06-03T02:19:45.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.834+0000 7f91a0200700 1 -- start start 2024-06-03T02:19:45.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f91a0200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9198108ac0 0x7f919819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f91a0200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9198109480 0x7f919819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f91a0200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919810eac0 0x7f91981a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f91a0200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9198118df0 con 0x7f9198108ac0 2024-06-03T02:19:45.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f91a0200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9198118c70 con 0x7f919810eac0 2024-06-03T02:19:45.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f91a0200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9198118f70 con 0x7f9198109480 2024-06-03T02:19:45.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f919d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9198108ac0 0x7f919819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f919d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9198108ac0 0x7f919819f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35144/0 (socket says 172.21.15.79:35144) 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f919ce00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9198109480 0x7f919819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f919d800700 1 -- 172.21.15.79:0/1639860287 learned_addr learned my addr 172.21.15.79:0/1639860287 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.835+0000 7f919e200700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919810eac0 0x7f91981a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.836+0000 7f919d800700 1 -- 172.21.15.79:0/1639860287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9198109480 msgr2=0x7f919819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.836+0000 7f919d800700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9198109480 0x7f919819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.836+0000 7f919d800700 1 -- 172.21.15.79:0/1639860287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919810eac0 msgr2=0x7f91981a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.836+0000 7f919d800700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919810eac0 0x7f91981a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.836+0000 7f919d800700 1 -- 172.21.15.79:0/1639860287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91981a41d0 con 0x7f9198108ac0 2024-06-03T02:19:45.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.836+0000 7f919ce00700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9198109480 0x7f919819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:19:45.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.836+0000 7f919d800700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9198108ac0 0x7f919819f1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f918c00ec10 tx=0x7f918c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:45.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.837+0000 7f9196000700 1 -- 172.21.15.79:0/1639860287 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c00cc90 con 0x7f9198108ac0 2024-06-03T02:19:45.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.837+0000 7f9196000700 1 -- 172.21.15.79:0/1639860287 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918c00ce30 con 0x7f9198108ac0 2024-06-03T02:19:45.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.837+0000 7f91a0200700 1 -- 172.21.15.79:0/1639860287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91981a44c0 con 0x7f9198108ac0 2024-06-03T02:19:45.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.837+0000 7f9196000700 1 -- 172.21.15.79:0/1639860287 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c010690 con 0x7f9198108ac0 2024-06-03T02:19:45.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.837+0000 7f91a0200700 1 -- 172.21.15.79:0/1639860287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91981abd40 con 0x7f9198108ac0 2024-06-03T02:19:45.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.842+0000 7f9196000700 1 -- 172.21.15.79:0/1639860287 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f918c003680 con 0x7f9198108ac0 2024-06-03T02:19:45.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f91aac4db80 1 Processor -- start 2024-06-03T02:19:45.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f91aac4db80 1 -- start start 2024-06-03T02:19:45.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f9196000700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f91800a2cb0 0x7f91800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c040e40 0x56026c0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 0x56026c0c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 0x56026c0d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f91aac4db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56026bfd3a10 con 0x56026c040e40 2024-06-03T02:19:45.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f91aac4db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56026c016b00 con 0x56026c0cf300 2024-06-03T02:19:45.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f91aac4db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56026bfd18f0 con 0x56026c0c45b0 2024-06-03T02:19:45.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.843+0000 7f9196000700 1 -- 172.21.15.79:0/1639860287 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f918c0cf800 con 0x7f9198108ac0 2024-06-03T02:19:45.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919e200700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f91800a2cb0 0x7f91800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c040e40 0x56026c0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919ce00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 0x56026c0c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c040e40 0x56026c0c4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35150/0 (socket says 172.21.15.79:35150) 2024-06-03T02:19:45.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919d800700 1 -- 172.21.15.79:0/3717412119 learned_addr learned my addr 172.21.15.79:0/3717412119 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:45.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919e200700 1 --2- 172.21.15.79:0/3717412119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 0x56026c0d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919d800700 1 -- 172.21.15.79:0/3717412119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 msgr2=0x56026c0c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919d800700 1 --2- 172.21.15.79:0/3717412119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 0x56026c0c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919d800700 1 -- 172.21.15.79:0/3717412119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 msgr2=0x56026c0d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919d800700 1 --2- 172.21.15.79:0/3717412119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 0x56026c0d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.844+0000 7f919d800700 1 -- 172.21.15.79:0/3717412119 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026bfbef90 con 0x56026c040e40 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.845+0000 7f919e200700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f91800a2cb0 0x7f91800a5170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f919000b6c0 tx=0x7f9190009bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.845+0000 7f919d800700 1 --2- 172.21.15.79:0/3717412119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c040e40 0x56026c0c4070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f918c00eff0 tx=0x7f918c09ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5067054175796281 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.845+0000 7f918aa00700 1 -- 172.21.15.79:0/3717412119 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c0b3020 con 0x56026c040e40 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.845+0000 7f918aa00700 1 -- 172.21.15.79:0/3717412119 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918c01fe20 con 0x56026c040e40 2024-06-03T02:19:45.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.845+0000 7f918aa00700 1 -- 172.21.15.79:0/3717412119 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c09bdb0 con 0x56026c040e40 2024-06-03T02:19:45.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:45 smithi135 ceph-mon[36653]: pgmap v507: 129 pgs: 129 active+clean; 1.5 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 74 KiB/s wr, 14 op/s 2024-06-03T02:19:45.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.846+0000 7f91aac4db80 1 -- 172.21.15.79:0/3717412119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c040e40 msgr2=0x56026c0c4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:45.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.846+0000 7f91aac4db80 1 --2- 172.21.15.79:0/3717412119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c040e40 0x56026c0c4070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f918c00eff0 tx=0x7f918c09ba50 comp rx=0 tx=0).stop 2024-06-03T02:19:45.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.846+0000 7f91aac4db80 1 -- 172.21.15.79:0/3717412119 shutdown_connections 2024-06-03T02:19:45.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.846+0000 7f91aac4db80 1 --2- 172.21.15.79:0/3717412119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 0x56026c0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.846+0000 7f91aac4db80 1 --2- 172.21.15.79:0/3717412119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c040e40 0x56026c0c4070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.846+0000 7f91aac4db80 1 --2- 172.21.15.79:0/3717412119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 0x56026c0c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.846+0000 7f91aac4db80 1 -- 172.21.15.79:0/3717412119 >> 172.21.15.79:0/3717412119 conn(0x56026bff1170 msgr2=0x56026c0c04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:45.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.846+0000 7f91aac4db80 1 -- 172.21.15.79:0/3717412119 shutdown_connections 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.846+0000 7f91aac4db80 1 -- 172.21.15.79:0/3717412119 wait complete. 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.847+0000 7f91aac4db80 1 -- start start 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.847+0000 7f91aac4db80 1 Processor -- start 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.847+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.847+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0c45b0 0x56026c0cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.847+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.847+0000 7f91aac4db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56026bfb2f60 con 0x56026c0cf300 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.847+0000 7f91aac4db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56026c012ac0 con 0x56026c0c45b0 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.847+0000 7f91aac4db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56026c0d1fc0 con 0x56026c040e40 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f91aac4db80 10 client.0 _ll_register_callbacks cb 0x56026bfb0720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919e200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0c45b0 0x56026c0cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0c45b0 0x56026c0cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:39008/0 (socket says 172.21.15.79:39008) 2024-06-03T02:19:45.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54188/0 (socket says 172.21.15.79:54188) 2024-06-03T02:19:45.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919e200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35152/0 (socket says 172.21.15.79:35152) 2024-06-03T02:19:45.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919ce00700 1 -- 172.21.15.79:0/1807005134 learned_addr learned my addr 172.21.15.79:0/1807005134 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919d800700 1 -- 172.21.15.79:0/1807005134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0c45b0 msgr2=0x56026c0cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919d800700 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0c45b0 0x56026c0cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919d800700 1 -- 172.21.15.79:0/1807005134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 msgr2=0x56026c0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919d800700 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.848+0000 7f919d800700 1 -- 172.21.15.79:0/1807005134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026bf658d0 con 0x56026c040e40 2024-06-03T02:19:45.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.849+0000 7f919d800700 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0ceaa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x56026c0c13b0 tx=0x7f918c0cd280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e276f216545ab5c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.849+0000 7f917f400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.849+0000 7f917f400700 1 -- 172.21.15.79:0/1807005134 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c0b3070 con 0x56026c040e40 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.849+0000 7f917f400700 1 -- 172.21.15.79:0/1807005134 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f918c0ba2e0 con 0x56026c040e40 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.849+0000 7f91aac4db80 1 -- 172.21.15.79:0/1807005134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f918c01fe20 con 0x56026c040e40 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.849+0000 7f917f400700 1 -- 172.21.15.79:0/1807005134 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f918c083050 con 0x56026c040e40 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.849+0000 7f91aac4db80 10 client.25300 fetch_fsmap learned FSMap version 30 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.849+0000 7f91aac4db80 1 -- 172.21.15.79:0/1807005134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56026c0dc710 con 0x56026c040e40 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.849+0000 7f917f400700 1 -- 172.21.15.79:0/1807005134 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918c01fe20 con 0x56026c040e40 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.850+0000 7f917f400700 1 -- 172.21.15.79:0/1807005134 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f918c0a4330 con 0x56026c040e40 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.850+0000 7f91aac4db80 10 client.25300 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.850+0000 7f91aac4db80 10 client.25300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.850+0000 7f91aac4db80 20 client.25300 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.850+0000 7f91aac4db80 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56026c0debe0 0x56026c0e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.850+0000 7f91aac4db80 4 client.25300 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:19:45.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.850+0000 7f91aac4db80 1 -- 172.21.15.79:0/1807005134 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56026c024000 con 0x56026c0debe0 2024-06-03T02:19:45.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.850+0000 7f919ce00700 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56026c0debe0 0x56026c0e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.851+0000 7f919ce00700 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56026c0debe0 0x56026c0e0fd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d981c4f4475f206 server_cookie=6aa8396069fead0b in_seq=0 out_seq=0 2024-06-03T02:19:45.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.851+0000 7f917f400700 10 client.25300 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.851+0000 7f917f400700 1 -- 172.21.15.79:0/1807005134 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56026c024000 con 0x56026c0debe0 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.851+0000 7f917f400700 10 client.25300 handle_command_reply: tid=0 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 client.25300 shutdown 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 -- 172.21.15.79:0/1807005134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 msgr2=0x56026c0ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0ceaa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x56026c0c13b0 tx=0x7f918c0cd280 comp rx=0 tx=0).stop 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 -- 172.21.15.79:0/1807005134 shutdown_connections 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56026c0debe0 0x56026c0e0fd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0ceaa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f917f400700 0 client.25300 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 --2- 172.21.15.79:0/1807005134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0c45b0 0x56026c0cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 -- 172.21.15.79:0/1807005134 >> 172.21.15.79:0/1807005134 conn(0x56026bff1170 msgr2=0x56026c0bf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:45.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 -- 172.21.15.79:0/1807005134 shutdown_connections 2024-06-03T02:19:45.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 1 -- 172.21.15.79:0/1807005134 wait complete. 2024-06-03T02:19:45.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.852+0000 7f91aac4db80 20 client.25300 trim_cache size 0 max 16384 2024-06-03T02:19:45.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.875+0000 7f91aac4db80 1 Processor -- start 2024-06-03T02:19:45.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.875+0000 7f91aac4db80 1 -- start start 2024-06-03T02:19:45.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c040e40 0x56026c0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 0x56026c0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f91aac4db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91981b4a20 con 0x56026c0cf300 2024-06-03T02:19:45.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f91aac4db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91981b6560 con 0x56026c040e40 2024-06-03T02:19:45.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f91aac4db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f91981b50f0 con 0x56026c0c45b0 2024-06-03T02:19:45.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f919e200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f919e200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35164/0 (socket says 172.21.15.79:35164) 2024-06-03T02:19:45.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f919e200700 1 -- 172.21.15.79:0/2566053856 learned_addr learned my addr 172.21.15.79:0/2566053856 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:45.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f919ce00700 1 --2- 172.21.15.79:0/2566053856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 0x56026c0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.876+0000 7f919d800700 1 --2- 172.21.15.79:0/2566053856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c040e40 0x56026c0cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.877+0000 7f919e200700 1 -- 172.21.15.79:0/2566053856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 msgr2=0x56026c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.877+0000 7f919e200700 1 --2- 172.21.15.79:0/2566053856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 0x56026c0ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.877+0000 7f919e200700 1 -- 172.21.15.79:0/2566053856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c040e40 msgr2=0x56026c0cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.877+0000 7f919e200700 1 --2- 172.21.15.79:0/2566053856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c040e40 0x56026c0cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.877+0000 7f919e200700 1 -- 172.21.15.79:0/2566053856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026c0b7e10 con 0x56026c0cf300 2024-06-03T02:19:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.877+0000 7f919e200700 1 --2- 172.21.15.79:0/2566053856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0d06f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f919000bd90 tx=0x7f9190029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf3c8eade4a1d58 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.877+0000 7f918a000700 1 -- 172.21.15.79:0/2566053856 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919000bdc0 con 0x56026c0cf300 2024-06-03T02:19:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.877+0000 7f918a000700 1 -- 172.21.15.79:0/2566053856 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9190021b50 con 0x56026c0cf300 2024-06-03T02:19:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.877+0000 7f918a000700 1 -- 172.21.15.79:0/2566053856 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919002e680 con 0x56026c0cf300 2024-06-03T02:19:45.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.878+0000 7f91aac4db80 1 -- 172.21.15.79:0/2566053856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 msgr2=0x56026c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:45.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.878+0000 7f91aac4db80 1 --2- 172.21.15.79:0/2566053856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0d06f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f919000bd90 tx=0x7f9190029000 comp rx=0 tx=0).stop 2024-06-03T02:19:45.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.878+0000 7f91aac4db80 1 -- 172.21.15.79:0/2566053856 shutdown_connections 2024-06-03T02:19:45.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.878+0000 7f91aac4db80 1 --2- 172.21.15.79:0/2566053856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0cf300 0x56026c0d06f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.878+0000 7f91aac4db80 1 --2- 172.21.15.79:0/2566053856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c040e40 0x56026c0cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.878+0000 7f91aac4db80 1 --2- 172.21.15.79:0/2566053856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c0c45b0 0x56026c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.878+0000 7f91aac4db80 1 -- 172.21.15.79:0/2566053856 >> 172.21.15.79:0/2566053856 conn(0x56026bff1170 msgr2=0x56026c0c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:45.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.878+0000 7f91aac4db80 1 -- 172.21.15.79:0/2566053856 shutdown_connections 2024-06-03T02:19:45.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.878+0000 7f91aac4db80 1 -- 172.21.15.79:0/2566053856 wait complete. 2024-06-03T02:19:45.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.879+0000 7f91aac4db80 1 -- start start 2024-06-03T02:19:45.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.879+0000 7f91aac4db80 1 Processor -- start 2024-06-03T02:19:45.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.879+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0c45b0 0x56026c0e4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f91aac4db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 0x56026c0c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f91aac4db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91981b5e90 con 0x56026c0c45b0 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f91aac4db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91981b57c0 con 0x56026c0cf300 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f91aac4db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f91981b29c0 con 0x56026c040e40 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f91aac4db80 10 client.0 _ll_register_callbacks cb 0x56026bfb0720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0c45b0 0x56026c0e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0c45b0 0x56026c0e4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35172/0 (socket says 172.21.15.79:35172) 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919ce00700 1 -- 172.21.15.79:0/641254870 learned_addr learned my addr 172.21.15.79:0/641254870 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919d800700 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919e200700 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 0x56026c0c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919ce00700 1 -- 172.21.15.79:0/641254870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 msgr2=0x56026c0e42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:45.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919ce00700 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0e42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919ce00700 1 -- 172.21.15.79:0/641254870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 msgr2=0x56026c0c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:19:45.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919ce00700 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 0x56026c0c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.880+0000 7f919ce00700 1 -- 172.21.15.79:0/641254870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026c0c9010 con 0x56026c0c45b0 2024-06-03T02:19:45.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.881+0000 7f919ce00700 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0c45b0 0x56026c0e4820 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56026c0c12a0 tx=0x7f91840043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d8d59393f8ec8a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:19:45.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.881+0000 7f917f400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.881+0000 7f917f400700 1 -- 172.21.15.79:0/641254870 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184021070 con 0x56026c0c45b0 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.881+0000 7f917f400700 1 -- 172.21.15.79:0/641254870 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f9184010cc0 con 0x56026c0c45b0 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.881+0000 7f91aac4db80 1 -- 172.21.15.79:0/641254870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9190021b50 con 0x56026c0c45b0 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.881+0000 7f917f400700 1 -- 172.21.15.79:0/641254870 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f918402e050 con 0x56026c0c45b0 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.881+0000 7f91aac4db80 10 client.15207 fetch_fsmap learned FSMap version 30 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.881+0000 7f91aac4db80 1 -- 172.21.15.79:0/641254870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56026c149190 con 0x56026c0c45b0 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.882+0000 7f917f400700 1 -- 172.21.15.79:0/641254870 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9190021b50 con 0x56026c0c45b0 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.882+0000 7f917f400700 1 -- 172.21.15.79:0/641254870 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f918401e830 con 0x56026c0c45b0 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.882+0000 7f91aac4db80 10 client.15207 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.882+0000 7f91aac4db80 10 client.15207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.882+0000 7f91aac4db80 20 client.15207 populate_metadata read hostname 'smithi079' 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.882+0000 7f91aac4db80 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56026c1498b0 0x56026c14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:19:45.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.882+0000 7f91aac4db80 4 client.15207 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.882+0000 7f91aac4db80 1 -- 172.21.15.79:0/641254870 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56026c01e580 con 0x56026c1498b0 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.882+0000 7f919d800700 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56026c1498b0 0x56026c14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f919d800700 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56026c1498b0 0x56026c14bcd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e483e4b80c2e01b9 server_cookie=6b38b50e830c77eb in_seq=0 out_seq=0 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f917f400700 10 client.15207 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f917f400700 1 -- 172.21.15.79:0/641254870 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56026c01e580 con 0x56026c1498b0 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f917f400700 10 client.15207 handle_command_reply: tid=0 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f91aac4db80 1 client.15207 shutdown 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f91aac4db80 1 -- 172.21.15.79:0/641254870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0c45b0 msgr2=0x56026c0e4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f91aac4db80 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0c45b0 0x56026c0e4820 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56026c0c12a0 tx=0x7f91840043b0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f91aac4db80 1 -- 172.21.15.79:0/641254870 shutdown_connections 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f91aac4db80 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026c0cf300 0x56026c0c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f91aac4db80 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56026c040e40 0x56026c0e42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f91aac4db80 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56026c1498b0 0x56026c14bcd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f91aac4db80 1 --2- 172.21.15.79:0/641254870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56026c0c45b0 0x56026c0e4820 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f91aac4db80 1 -- 172.21.15.79:0/641254870 >> 172.21.15.79:0/641254870 conn(0x56026bff1170 msgr2=0x56026c0cb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:45.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.883+0000 7f917f400700 0 client.15207 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:19:45.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.884+0000 7f91aac4db80 1 -- 172.21.15.79:0/641254870 shutdown_connections 2024-06-03T02:19:45.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.884+0000 7f91aac4db80 1 -- 172.21.15.79:0/641254870 wait complete. 2024-06-03T02:19:45.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.884+0000 7f91aac4db80 20 client.15207 trim_cache size 0 max 16384 2024-06-03T02:19:45.886 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:19:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-03T02:19:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": { 2024-06-03T02:19:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "9505095d-e7c5-4873-b408-174f166602ba": { 2024-06-03T02:19:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "path": "/", 2024-06-03T02:19:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "tag": "9505095d-e7c5-4873-b408-174f166602ba", 2024-06-03T02:19:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "options": "recursive,force" 2024-06-03T02:19:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:19:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:19:45.887 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:19:45.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.884+0000 7f91a0200700 1 -- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f91800a2cb0 msgr2=0x7f91800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.884+0000 7f91a0200700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f91800a2cb0 0x7f91800a5170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f919000b6c0 tx=0x7f9190009bf0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.884+0000 7f91a0200700 1 -- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9198108ac0 msgr2=0x7f919819f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.884+0000 7f91a0200700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9198108ac0 0x7f919819f1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f918c00ec10 tx=0x7f918c00ef20 comp rx=0 tx=0).stop 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.885+0000 7f91a0200700 1 -- 172.21.15.79:0/1639860287 shutdown_connections 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.885+0000 7f91a0200700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919810eac0 0x7f91981a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.885+0000 7f91a0200700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f91800a2cb0 0x7f91800a5170 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.885+0000 7f91a0200700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9198108ac0 0x7f919819f1c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.885+0000 7f91a0200700 1 --2- 172.21.15.79:0/1639860287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9198109480 0x7f919819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.885+0000 7f91a0200700 1 -- 172.21.15.79:0/1639860287 >> 172.21.15.79:0/1639860287 conn(0x7f9198075c60 msgr2=0x7f9198076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.885+0000 7f91a0200700 1 -- 172.21.15.79:0/1639860287 shutdown_connections 2024-06-03T02:19:45.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:19:45.885+0000 7f91a0200700 1 -- 172.21.15.79:0/1639860287 wait complete. 2024-06-03T02:19:45.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:45 smithi123 ceph-mon[36640]: pgmap v507: 129 pgs: 129 active+clean; 1.5 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 74 KiB/s wr, 14 op/s 2024-06-03T02:19:46.833 INFO:tasks.cephfs.filesystem:scrub status for tag:9505095d-e7c5-4873-b408-174f166602ba - {'path': '/', 'tag': '9505095d-e7c5-4873-b408-174f166602ba', 'options': 'recursive,force'} 2024-06-03T02:19:47.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:47 smithi079 ceph-mon[31819]: pgmap v508: 129 pgs: 129 active+clean; 10 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 845 KiB/s wr, 20 op/s 2024-06-03T02:19:47.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:47 smithi135 ceph-mon[36653]: pgmap v508: 129 pgs: 129 active+clean; 10 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 845 KiB/s wr, 20 op/s 2024-06-03T02:19:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:47 smithi123 ceph-mon[36640]: pgmap v508: 129 pgs: 129 active+clean; 10 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 845 KiB/s wr, 20 op/s 2024-06-03T02:19:49.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:49 smithi079 ceph-mon[31819]: pgmap v509: 129 pgs: 129 active+clean; 21 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.7 MiB/s wr, 25 op/s 2024-06-03T02:19:49.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:49 smithi135 ceph-mon[36653]: pgmap v509: 129 pgs: 129 active+clean; 21 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.7 MiB/s wr, 25 op/s 2024-06-03T02:19:49.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:49 smithi123 ceph-mon[36640]: pgmap v509: 129 pgs: 129 active+clean; 21 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.7 MiB/s wr, 25 op/s 2024-06-03T02:19:51.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:51 smithi079 ceph-mon[31819]: pgmap v510: 129 pgs: 129 active+clean; 22 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2024-06-03T02:19:51.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:51 smithi135 ceph-mon[36653]: pgmap v510: 129 pgs: 129 active+clean; 22 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2024-06-03T02:19:51.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:51 smithi123 ceph-mon[36640]: pgmap v510: 129 pgs: 129 active+clean; 22 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2024-06-03T02:19:53.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:53 smithi135 ceph-mon[36653]: pgmap v511: 129 pgs: 129 active+clean; 24 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.9 MiB/s wr, 35 op/s 2024-06-03T02:19:53.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:53 smithi123 ceph-mon[36640]: pgmap v511: 129 pgs: 129 active+clean; 24 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.9 MiB/s wr, 35 op/s 2024-06-03T02:19:54.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:53 smithi079 ceph-mon[31819]: pgmap v511: 129 pgs: 129 active+clean; 24 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 1.9 MiB/s wr, 35 op/s 2024-06-03T02:19:54.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:54 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:19:55.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:54 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:19:55.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:54 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:19:55.386 INFO:teuthology.orchestra.run.smithi079.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-06-03T02:19:55.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-06-03T02:19:55.528 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/: 2024-06-03T02:19:55.528 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.528 INFO:teuthology.orchestra.run.smithi079.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:19:55.528 INFO:teuthology.orchestra.run.smithi079.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 .. 2024-06-03T02:19:55.528 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 data 2024-06-03T02:19:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.528 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data: 2024-06-03T02:19:55.528 INFO:teuthology.orchestra.run.smithi079.stdout:total 35 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 . 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 .. 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 base 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Jun 3 02:19 global 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 log 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_commit_ts 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_dynshmem 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Jun 3 02:19 pg_hba.conf 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jun 3 02:19 pg_ident.conf 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 pg_logical 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 pg_multixact 2024-06-03T02:19:55.529 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_notify 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_replslot 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_serial 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_snapshots 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_stat 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_stat_tmp 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 pg_subtrans 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_tblspc 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_twophase 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 PG_VERSION 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 pg_wal 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 pg_xact 2024-06-03T02:19:55.530 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jun 3 02:19 postgresql.auto.conf 2024-06-03T02:19:55.531 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Jun 3 02:19 postgresql.conf 2024-06-03T02:19:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.531 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/base: 2024-06-03T02:19:55.531 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.531 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 . 2024-06-03T02:19:55.531 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.531 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 1 2024-06-03T02:19:55.531 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 13493 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 13494 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/base/1: 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:total 7777 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 . 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 .. 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 112 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 113 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 1247 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1247_fsm 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1247_vm 2024-06-03T02:19:55.532 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 3 02:19 1249 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1249_fsm 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1249_vm 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 3 02:19 1255 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1255_fsm 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1255_vm 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 3 02:19 1259 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1259_fsm 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1259_vm 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 3 02:19 13344 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13344_fsm 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13344_vm 2024-06-03T02:19:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13346 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13348 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13349_fsm 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349_vm 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13351 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13353 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13354_fsm 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354_vm 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13356 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13358 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359 2024-06-03T02:19:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13359_fsm 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359_vm 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13361 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13363 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1417 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1418 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 174 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 175 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2187 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2224 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2328 2024-06-03T02:19:55.535 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2336 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2337 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2579 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2600 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2600_fsm 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2600_vm 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2601_fsm 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601_vm 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2602 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2602_fsm 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2602_vm 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2603 2024-06-03T02:19:55.536 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2603_fsm 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2603_vm 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2604 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2605 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2605_fsm 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2605_vm 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2606_fsm 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606_vm 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2607 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2607_fsm 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2607_vm 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 3 02:19 2608 2024-06-03T02:19:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2608_fsm 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2608_vm 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 3 02:19 2609 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2609_fsm 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2609_vm 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2610 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2610_fsm 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2610_vm 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2611 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2612_fsm 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612_vm 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2613 2024-06-03T02:19:55.538 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2615_fsm 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615_vm 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2616 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2616_fsm 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2616_vm 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 3 02:19 2617 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2617_fsm 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2617_vm 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 3 02:19 2618 2024-06-03T02:19:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2618_fsm 2024-06-03T02:19:55.540 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2618_vm 2024-06-03T02:19:55.540 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 3 02:19 2619 2024-06-03T02:19:55.540 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2619_fsm 2024-06-03T02:19:55.540 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2619_vm 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2620 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2650 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2651 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2652 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2653 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2654 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2655 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2656 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2657 2024-06-03T02:19:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 3 02:19 2658 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 2659 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2660 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2661 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2662 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2663 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2664 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2665 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2666 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2667 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2668 2024-06-03T02:19:55.542 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2669 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2670 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 3 02:19 2673 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 3 02:19 2674 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 3 02:19 2675 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2678 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2679 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2680 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2681 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2682 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2683 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2684 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2685 2024-06-03T02:19:55.543 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2686 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2687 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2688 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2689 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 3 02:19 2690 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 3 02:19 2691 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2692 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2693 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2696 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2699 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2701 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2702 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2703 2024-06-03T02:19:55.544 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2704 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2753 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2753_fsm 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2753_vm 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2754 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2755 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2756 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2757 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2830 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2831 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2832 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2833 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2834 2024-06-03T02:19:55.545 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2835 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2836 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2837 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 3 02:19 2838 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2838_fsm 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2838_vm 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2839 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840_fsm 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2840_vm 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2841 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2995 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2996 2024-06-03T02:19:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3079_fsm 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079_vm 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3080 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3081 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3085 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3118 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3119 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 3 02:19 3164 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3256 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3257 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3258 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3350 2024-06-03T02:19:55.547 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3351 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3379 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3380 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3381 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3394 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3394_fsm 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3394_vm 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3395 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3429 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3430 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3431 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3433 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3439 2024-06-03T02:19:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3440 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3455 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 3 02:19 3456 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3456_fsm 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3456_vm 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3466 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3467 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3468 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3501 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3502 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3503 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3534 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3541_fsm 2024-06-03T02:19:55.549 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541_vm 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3542 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3574 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3575 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3576 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3596 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3597 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3598 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3599 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3600_fsm 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600_vm 2024-06-03T02:19:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3601_fsm 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601_vm 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3602_fsm 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602_vm 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603_fsm 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3603_vm 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3604 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3605 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3606 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3607 2024-06-03T02:19:55.551 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3608 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 3609 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3712 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3764_fsm 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764_vm 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3766 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3767 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3997 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4143 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4144 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4145 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4146 2024-06-03T02:19:55.552 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4147 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4148 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4149 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4150 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4151 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4152 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4153 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4154 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4155 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4156 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4157 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4158 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4159 2024-06-03T02:19:55.553 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4160 2024-06-03T02:19:55.554 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4161 2024-06-03T02:19:55.554 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4162 2024-06-03T02:19:55.554 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4163 2024-06-03T02:19:55.554 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4164 2024-06-03T02:19:55.554 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4165 2024-06-03T02:19:55.556 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4166 2024-06-03T02:19:55.556 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4167 2024-06-03T02:19:55.556 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4168 2024-06-03T02:19:55.556 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4169 2024-06-03T02:19:55.556 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4170 2024-06-03T02:19:55.556 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4171 2024-06-03T02:19:55.556 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4172 2024-06-03T02:19:55.556 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4173 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4174 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 5002 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 548 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 549 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6102 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6104 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6106 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6110 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6111 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6112 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6113 2024-06-03T02:19:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6117 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 826 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 827 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 828 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 3 02:19 pg_filenode.map 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 PG_VERSION 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:total 7777 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 . 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 .. 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 112 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 113 2024-06-03T02:19:55.558 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 1247 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1247_fsm 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1247_vm 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 3 02:19 1249 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1249_fsm 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1249_vm 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 3 02:19 1255 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1255_fsm 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1255_vm 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 3 02:19 1259 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1259_fsm 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1259_vm 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 3 02:19 13344 2024-06-03T02:19:55.559 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13344_fsm 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13344_vm 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13346 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13348 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13349_fsm 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349_vm 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13351 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13353 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13354_fsm 2024-06-03T02:19:55.560 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354_vm 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13356 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13358 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13359_fsm 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359_vm 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13361 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13363 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1417 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1418 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 174 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 175 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2187 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2224 2024-06-03T02:19:55.561 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2328 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2336 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2337 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2579 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2600 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2600_fsm 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2600_vm 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2601_fsm 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601_vm 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2602 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2602_fsm 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2602_vm 2024-06-03T02:19:55.562 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2603 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2603_fsm 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2603_vm 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2604 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2605 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2605_fsm 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2605_vm 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2606_fsm 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606_vm 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2607 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2607_fsm 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2607_vm 2024-06-03T02:19:55.563 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 3 02:19 2608 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2608_fsm 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2608_vm 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 3 02:19 2609 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2609_fsm 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2609_vm 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2610 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2610_fsm 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2610_vm 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2611 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2612_fsm 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612_vm 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2613 2024-06-03T02:19:55.564 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2615_fsm 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615_vm 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2616 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2616_fsm 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2616_vm 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 3 02:19 2617 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2617_fsm 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2617_vm 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 3 02:19 2618 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2618_fsm 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2618_vm 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 3 02:19 2619 2024-06-03T02:19:55.565 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2619_fsm 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2619_vm 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2620 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2650 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2651 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2652 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2653 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2654 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2655 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2656 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2657 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 3 02:19 2658 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 2659 2024-06-03T02:19:55.566 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2660 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2661 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2662 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2663 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2664 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2665 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2666 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2667 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2668 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2669 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2670 2024-06-03T02:19:55.567 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 3 02:19 2673 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 3 02:19 2674 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 3 02:19 2675 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2678 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2679 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2680 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2681 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2682 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2683 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2684 2024-06-03T02:19:55.568 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2685 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2686 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2687 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2688 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2689 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 3 02:19 2690 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 3 02:19 2691 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2692 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2693 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2696 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2699 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2701 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2702 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2703 2024-06-03T02:19:55.569 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2704 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2753 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2753_fsm 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2753_vm 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2754 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2755 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2756 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2757 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2830 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2831 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2832 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2833 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2834 2024-06-03T02:19:55.570 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2835 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2836 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2837 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 3 02:19 2838 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2838_fsm 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2838_vm 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2839 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840_fsm 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2840_vm 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2841 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2995 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2996 2024-06-03T02:19:55.571 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3079_fsm 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079_vm 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3080 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3081 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3085 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3118 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3119 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 3 02:19 3164 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3256 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3257 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3258 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3350 2024-06-03T02:19:55.572 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3351 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3379 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3380 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3381 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3394 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3394_fsm 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3394_vm 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3395 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3429 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3430 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3431 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3433 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3439 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3440 2024-06-03T02:19:55.573 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3455 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 3 02:19 3456 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3456_fsm 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3456_vm 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3466 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3467 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3468 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3501 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3502 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3503 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3534 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3541_fsm 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541_vm 2024-06-03T02:19:55.574 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3542 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3574 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3575 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3576 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3596 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3597 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3598 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3599 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3600_fsm 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600_vm 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3601_fsm 2024-06-03T02:19:55.575 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601_vm 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3602_fsm 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602_vm 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603_fsm 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3603_vm 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3604 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3605 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3606 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3607 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3608 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 3609 2024-06-03T02:19:55.576 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3712 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3764_fsm 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764_vm 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3766 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3767 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3997 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4143 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4144 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4145 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4146 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4147 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4148 2024-06-03T02:19:55.577 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4149 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4150 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4151 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4152 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4153 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4154 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4155 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4156 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4157 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4158 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4159 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4160 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4161 2024-06-03T02:19:55.578 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4162 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4163 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4164 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4165 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4166 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4167 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4168 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4169 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4170 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4171 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4172 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4173 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4174 2024-06-03T02:19:55.579 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 5002 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 548 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 549 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6102 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6104 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6106 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6110 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6111 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6112 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6113 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6117 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 826 2024-06-03T02:19:55.580 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 827 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 828 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 3 02:19 pg_filenode.map 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 PG_VERSION 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:total 7777 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 . 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 .. 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 112 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 113 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 1247 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1247_fsm 2024-06-03T02:19:55.581 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1247_vm 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 3 02:19 1249 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1249_fsm 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1249_vm 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 3 02:19 1255 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1255_fsm 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1255_vm 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 3 02:19 1259 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1259_fsm 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1259_vm 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 3 02:19 13344 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13344_fsm 2024-06-03T02:19:55.582 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13344_vm 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13346 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13348 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13349_fsm 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349_vm 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13351 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13353 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13354_fsm 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354_vm 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13356 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13358 2024-06-03T02:19:55.583 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13359_fsm 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359_vm 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13361 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13363 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1417 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1418 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 174 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 175 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2187 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2224 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2328 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2336 2024-06-03T02:19:55.584 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2337 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2579 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2600 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2600_fsm 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2600_vm 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2601_fsm 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601_vm 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2602 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2602_fsm 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2602_vm 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2603 2024-06-03T02:19:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2603_fsm 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2603_vm 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2604 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2605 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2605_fsm 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2605_vm 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2606_fsm 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606_vm 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2607 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2607_fsm 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2607_vm 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 3 02:19 2608 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2608_fsm 2024-06-03T02:19:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2608_vm 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 3 02:19 2609 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2609_fsm 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2609_vm 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2610 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2610_fsm 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2610_vm 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2611 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2612_fsm 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612_vm 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2613 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615 2024-06-03T02:19:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2615_fsm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615_vm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2616 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2616_fsm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2616_vm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 3 02:19 2617 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2617_fsm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2617_vm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 3 02:19 2618 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2618_fsm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2618_vm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 3 02:19 2619 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2619_fsm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2619_vm 2024-06-03T02:19:55.588 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2620 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2650 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2651 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2652 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2653 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2654 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2655 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2656 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2657 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 3 02:19 2658 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 2659 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2660 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2661 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2662 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2663 2024-06-03T02:19:55.589 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2664 2024-06-03T02:19:55.590 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2665 2024-06-03T02:19:55.590 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2666 2024-06-03T02:19:55.590 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2667 2024-06-03T02:19:55.590 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2668 2024-06-03T02:19:55.590 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2669 2024-06-03T02:19:55.590 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2670 2024-06-03T02:19:55.590 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 3 02:19 2673 2024-06-03T02:19:55.590 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 3 02:19 2674 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 3 02:19 2675 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2678 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2679 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2680 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2681 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2682 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2683 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2684 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2685 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2686 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2687 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2688 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2689 2024-06-03T02:19:55.591 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 3 02:19 2690 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 3 02:19 2691 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2692 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2693 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2696 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2699 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2701 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2702 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2703 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2704 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2753 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2753_fsm 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2753_vm 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2754 2024-06-03T02:19:55.592 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2755 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2756 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2757 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2830 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2831 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2832 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2833 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2834 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2835 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2836 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2837 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 3 02:19 2838 2024-06-03T02:19:55.593 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2838_fsm 2024-06-03T02:19:55.594 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2838_vm 2024-06-03T02:19:55.594 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2839 2024-06-03T02:19:55.594 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840 2024-06-03T02:19:55.594 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840_fsm 2024-06-03T02:19:55.594 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2840_vm 2024-06-03T02:19:55.594 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2841 2024-06-03T02:19:55.594 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2995 2024-06-03T02:19:55.594 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2996 2024-06-03T02:19:55.594 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3079_fsm 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079_vm 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3080 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3081 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3085 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3118 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3119 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 3 02:19 3164 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3256 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3257 2024-06-03T02:19:55.595 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3258 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3350 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3351 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3379 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3380 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3381 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3394 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3394_fsm 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3394_vm 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3395 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3429 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3430 2024-06-03T02:19:55.596 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3431 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3433 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3439 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3440 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3455 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 3 02:19 3456 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3456_fsm 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3456_vm 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3466 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3467 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3468 2024-06-03T02:19:55.597 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3501 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3502 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3503 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3534 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3541_fsm 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541_vm 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3542 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3574 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3575 2024-06-03T02:19:55.598 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3576 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3596 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3597 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3598 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3599 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3600_fsm 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600_vm 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3601_fsm 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601_vm 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3602_fsm 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602_vm 2024-06-03T02:19:55.599 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603_fsm 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3603_vm 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3604 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3605 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3606 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3607 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3608 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 3609 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3712 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3764_fsm 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764_vm 2024-06-03T02:19:55.600 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3766 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3767 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3997 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4143 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4144 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4145 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4146 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4147 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4148 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4149 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4150 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4151 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4152 2024-06-03T02:19:55.601 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4153 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4154 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4155 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4156 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4157 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4158 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4159 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4160 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4161 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4162 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4163 2024-06-03T02:19:55.602 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4164 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4165 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4166 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4167 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4168 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4169 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4170 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4171 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4172 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4173 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4174 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 5002 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 548 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 549 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6102 2024-06-03T02:19:55.603 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6104 2024-06-03T02:19:55.604 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6106 2024-06-03T02:19:55.604 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6110 2024-06-03T02:19:55.604 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6111 2024-06-03T02:19:55.604 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6112 2024-06-03T02:19:55.604 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6113 2024-06-03T02:19:55.604 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6117 2024-06-03T02:19:55.604 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 826 2024-06-03T02:19:55.605 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 827 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 828 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 3 02:19 pg_filenode.map 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 PG_VERSION 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/global: 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:total 537 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Jun 3 02:19 . 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1213 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1213_fsm 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1213_vm 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1214 2024-06-03T02:19:55.606 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1214_fsm 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1214_vm 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 1232 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 1233 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1260 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1260_fsm 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1260_vm 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1261 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1261_fsm 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1261_vm 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1262 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1262_fsm 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1262_vm 2024-06-03T02:19:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2396 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2396_fsm 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2396_vm 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2397 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2671 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2672 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2676 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2677 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2694 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2695 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2697 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2698 2024-06-03T02:19:55.608 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2846 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2847 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2964 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2965 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2966 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2967 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3592 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3593 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4060 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4061 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4175 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4176 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4177 2024-06-03T02:19:55.609 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4178 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4181 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4182 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4183 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4184 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4185 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4186 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6000 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6001 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6002 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6100 2024-06-03T02:19:55.610 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6114 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6115 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 pg_control 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 3 02:19 pg_filenode.map 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/log: 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-06-03T02:19:55.611 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:total 1 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 . 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.612 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 mappings 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 3 02:19 replorigin_checkpoint 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 snapshots 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 .. 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.613 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 .. 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 . 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 members 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 offsets 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.614 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout:total 8 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 .. 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 0000 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout:total 8 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 .. 2024-06-03T02:19:55.615 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 0000 2024-06-03T02:19:55.616 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.616 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-06-03T02:19:55.616 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.616 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.616 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.616 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.616 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-06-03T02:19:55.616 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.616 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-06-03T02:19:55.617 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:total 8 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 0000 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.619 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout:total 16384 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 . 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:19 000000010000000000000001 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 archive_status 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 .. 2024-06-03T02:19:55.620 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:19:55.621 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-06-03T02:19:55.621 INFO:teuthology.orchestra.run.smithi079.stdout:total 8 2024-06-03T02:19:55.621 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:19:55.621 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 3 02:19 .. 2024-06-03T02:19:55.621 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 0000 2024-06-03T02:19:55.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-06-03T02:19:55.695 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:55 smithi079 ceph-mon[31819]: pgmap v512: 129 pgs: 129 active+clean; 25 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.1 MiB/s wr, 36 op/s 2024-06-03T02:19:55.918 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-06-03T02:19:55.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:55 smithi123 ceph-mon[36640]: pgmap v512: 129 pgs: 129 active+clean; 25 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.1 MiB/s wr, 36 op/s 2024-06-03T02:19:56.030 INFO:teuthology.orchestra.run.smithi079.stderr:dropping old tables... 2024-06-03T02:19:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-06-03T02:19:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-06-03T02:19:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-06-03T02:19:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-06-03T02:19:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:creating tables... 2024-06-03T02:19:56.053 INFO:teuthology.orchestra.run.smithi079.stderr:generating data (client-side)... 2024-06-03T02:19:56.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:55 smithi135 ceph-mon[36653]: pgmap v512: 129 pgs: 129 active+clean; 25 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.1 MiB/s wr, 36 op/s 2024-06-03T02:19:56.213 INFO:teuthology.orchestra.run.smithi079.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.12 s, remaining 3.60 s) 2024-06-03T02:19:56.577 INFO:teuthology.orchestra.run.smithi079.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.48 s, remaining 7.21 s) 2024-06-03T02:19:56.898 INFO:teuthology.orchestra.run.smithi079.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.80 s, remaining 7.74 s) 2024-06-03T02:19:57.110 INFO:teuthology.orchestra.run.smithi079.stderr:400000 of 3200000 tuples (12%) done (elapsed 1.01 s, remaining 7.09 s) 2024-06-03T02:19:57.452 INFO:teuthology.orchestra.run.smithi079.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.35 s, remaining 7.32 s) 2024-06-03T02:19:57.613 INFO:teuthology.orchestra.run.smithi079.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.52 s, remaining 6.57 s) 2024-06-03T02:19:57.927 INFO:teuthology.orchestra.run.smithi079.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.83 s, remaining 6.54 s) 2024-06-03T02:19:57.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:57 smithi123 ceph-mon[36640]: pgmap v513: 129 pgs: 129 active+clean; 33 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.8 MiB/s wr, 56 op/s 2024-06-03T02:19:58.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:57 smithi079 ceph-mon[31819]: pgmap v513: 129 pgs: 129 active+clean; 33 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.8 MiB/s wr, 56 op/s 2024-06-03T02:19:58.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:57 smithi135 ceph-mon[36653]: pgmap v513: 129 pgs: 129 active+clean; 33 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.8 MiB/s wr, 56 op/s 2024-06-03T02:19:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:800000 of 3200000 tuples (25%) done (elapsed 2.21 s, remaining 6.64 s) 2024-06-03T02:19:58.464 INFO:teuthology.orchestra.run.smithi079.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.37 s, remaining 6.05 s) 2024-06-03T02:19:58.788 INFO:teuthology.orchestra.run.smithi079.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.69 s, remaining 5.92 s) 2024-06-03T02:19:59.168 INFO:teuthology.orchestra.run.smithi079.stderr:1100000 of 3200000 tuples (34%) done (elapsed 3.07 s, remaining 5.86 s) 2024-06-03T02:19:59.365 INFO:teuthology.orchestra.run.smithi079.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.27 s, remaining 5.45 s) 2024-06-03T02:19:59.672 INFO:teuthology.orchestra.run.smithi079.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.58 s, remaining 5.23 s) 2024-06-03T02:19:59.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:19:59 smithi123 ceph-mon[36640]: pgmap v514: 129 pgs: 129 active+clean; 48 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.8 MiB/s wr, 72 op/s 2024-06-03T02:19:59.983 INFO:teuthology.orchestra.run.smithi079.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.89 s, remaining 5.00 s) 2024-06-03T02:20:00.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:19:59 smithi079 ceph-mon[31819]: pgmap v514: 129 pgs: 129 active+clean; 48 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.8 MiB/s wr, 72 op/s 2024-06-03T02:20:00.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:19:59 smithi135 ceph-mon[36653]: pgmap v514: 129 pgs: 129 active+clean; 48 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.8 MiB/s wr, 72 op/s 2024-06-03T02:20:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:1500000 of 3200000 tuples (46%) done (elapsed 4.09 s, remaining 4.63 s) 2024-06-03T02:20:00.590 INFO:teuthology.orchestra.run.smithi079.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.49 s, remaining 4.49 s) 2024-06-03T02:20:00.888 INFO:teuthology.orchestra.run.smithi079.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.79 s, remaining 4.23 s) 2024-06-03T02:20:00.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:00 smithi123 ceph-mon[36640]: overall HEALTH_OK 2024-06-03T02:20:00.990 INFO:teuthology.orchestra.run.smithi079.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.89 s, remaining 3.81 s) 2024-06-03T02:20:01.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:00 smithi079 ceph-mon[31819]: overall HEALTH_OK 2024-06-03T02:20:01.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:00 smithi135 ceph-mon[36653]: overall HEALTH_OK 2024-06-03T02:20:01.421 INFO:teuthology.orchestra.run.smithi079.stderr:1900000 of 3200000 tuples (59%) done (elapsed 5.32 s, remaining 3.64 s) 2024-06-03T02:20:01.724 INFO:teuthology.orchestra.run.smithi079.stderr:2000000 of 3200000 tuples (62%) done (elapsed 5.63 s, remaining 3.38 s) 2024-06-03T02:20:01.834 INFO:teuthology.orchestra.run.smithi079.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.74 s, remaining 3.01 s) 2024-06-03T02:20:01.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:01 smithi123 ceph-mon[36640]: pgmap v515: 129 pgs: 129 active+clean; 84 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 9.6 MiB/s wr, 135 op/s 2024-06-03T02:20:02.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:01 smithi079 ceph-mon[31819]: pgmap v515: 129 pgs: 129 active+clean; 84 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 9.6 MiB/s wr, 135 op/s 2024-06-03T02:20:02.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:01 smithi135 ceph-mon[36653]: pgmap v515: 129 pgs: 129 active+clean; 84 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 9.6 MiB/s wr, 135 op/s 2024-06-03T02:20:02.209 INFO:teuthology.orchestra.run.smithi079.stderr:2200000 of 3200000 tuples (68%) done (elapsed 6.11 s, remaining 2.78 s) 2024-06-03T02:20:02.311 INFO:teuthology.orchestra.run.smithi079.stderr:2300000 of 3200000 tuples (71%) done (elapsed 6.21 s, remaining 2.43 s) 2024-06-03T02:20:02.626 INFO:teuthology.orchestra.run.smithi079.stderr:2400000 of 3200000 tuples (75%) done (elapsed 6.53 s, remaining 2.18 s) 2024-06-03T02:20:03.009 INFO:teuthology.orchestra.run.smithi079.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.91 s, remaining 1.94 s) 2024-06-03T02:20:03.116 INFO:teuthology.orchestra.run.smithi079.stderr:2600000 of 3200000 tuples (81%) done (elapsed 7.02 s, remaining 1.62 s) 2024-06-03T02:20:03.446 INFO:teuthology.orchestra.run.smithi079.stderr:2700000 of 3200000 tuples (84%) done (elapsed 7.35 s, remaining 1.36 s) 2024-06-03T02:20:03.611 INFO:teuthology.orchestra.run.smithi079.stderr:2800000 of 3200000 tuples (87%) done (elapsed 7.51 s, remaining 1.07 s) 2024-06-03T02:20:03.897 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:03 smithi135 ceph-mon[36653]: pgmap v516: 129 pgs: 129 active+clean; 208 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 30 MiB/s wr, 149 op/s 2024-06-03T02:20:03.960 INFO:teuthology.orchestra.run.smithi079.stderr:2900000 of 3200000 tuples (90%) done (elapsed 7.86 s, remaining 0.81 s) 2024-06-03T02:20:03.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:03 smithi123 ceph-mon[36640]: pgmap v516: 129 pgs: 129 active+clean; 208 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 30 MiB/s wr, 149 op/s 2024-06-03T02:20:04.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:03 smithi079 ceph-mon[31819]: pgmap v516: 129 pgs: 129 active+clean; 208 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 30 MiB/s wr, 149 op/s 2024-06-03T02:20:04.264 INFO:teuthology.orchestra.run.smithi079.stderr:3000000 of 3200000 tuples (93%) done (elapsed 8.17 s, remaining 0.54 s) 2024-06-03T02:20:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:3100000 of 3200000 tuples (96%) done (elapsed 8.33 s, remaining 0.27 s) 2024-06-03T02:20:04.758 INFO:teuthology.orchestra.run.smithi079.stderr:3200000 of 3200000 tuples (100%) done (elapsed 8.66 s, remaining 0.00 s) 2024-06-03T02:20:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:vacuuming... 2024-06-03T02:20:05.856 INFO:teuthology.orchestra.run.smithi079.stderr:creating primary keys... 2024-06-03T02:20:05.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:05 smithi123 ceph-mon[36640]: pgmap v517: 129 pgs: 129 active+clean; 284 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 42 MiB/s wr, 146 op/s 2024-06-03T02:20:06.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:05 smithi079 ceph-mon[31819]: pgmap v517: 129 pgs: 129 active+clean; 284 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 42 MiB/s wr, 146 op/s 2024-06-03T02:20:06.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:05 smithi135 ceph-mon[36653]: pgmap v517: 129 pgs: 129 active+clean; 284 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 42 MiB/s wr, 146 op/s 2024-06-03T02:20:07.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:07 smithi123 ceph-mon[36640]: pgmap v518: 129 pgs: 129 active+clean; 324 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 48 MiB/s wr, 146 op/s 2024-06-03T02:20:08.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:07 smithi079 ceph-mon[31819]: pgmap v518: 129 pgs: 129 active+clean; 324 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 48 MiB/s wr, 146 op/s 2024-06-03T02:20:08.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:07 smithi135 ceph-mon[36653]: pgmap v518: 129 pgs: 129 active+clean; 324 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 48 MiB/s wr, 146 op/s 2024-06-03T02:20:08.157 INFO:teuthology.orchestra.run.smithi079.stderr:done in 12.13 s (drop tables 0.00 s, create tables 0.02 s, client-side generate 8.79 s, vacuum 1.02 s, primary keys 2.30 s). 2024-06-03T02:20:08.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-06-03T02:20:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:starting vacuum...end. 2024-06-03T02:20:09.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:09 smithi123 ceph-mon[36640]: pgmap v519: 129 pgs: 129 active+clean; 389 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 59 MiB/s wr, 134 op/s 2024-06-03T02:20:10.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:09 smithi079 ceph-mon[31819]: pgmap v519: 129 pgs: 129 active+clean; 389 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 59 MiB/s wr, 134 op/s 2024-06-03T02:20:10.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:09 smithi135 ceph-mon[36653]: pgmap v519: 129 pgs: 129 active+clean; 389 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 59 MiB/s wr, 134 op/s 2024-06-03T02:20:11.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:11 smithi123 ceph-mon[36640]: pgmap v520: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 59 MiB/s wr, 112 op/s 2024-06-03T02:20:12.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:11 smithi079 ceph-mon[31819]: pgmap v520: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 59 MiB/s wr, 112 op/s 2024-06-03T02:20:12.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:11 smithi135 ceph-mon[36653]: pgmap v520: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 59 MiB/s wr, 112 op/s 2024-06-03T02:20:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 5.0 s, 2613.3 tps, lat 36.590 ms stddev 39.734 2024-06-03T02:20:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:13 smithi123 ceph-mon[36640]: pgmap v521: 129 pgs: 129 active+clean; 553 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 65 MiB/s wr, 137 op/s 2024-06-03T02:20:14.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:13 smithi079 ceph-mon[31819]: pgmap v521: 129 pgs: 129 active+clean; 553 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 65 MiB/s wr, 137 op/s 2024-06-03T02:20:14.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:13 smithi135 ceph-mon[36653]: pgmap v521: 129 pgs: 129 active+clean; 553 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 65 MiB/s wr, 137 op/s 2024-06-03T02:20:15.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:15 smithi079 ceph-mon[31819]: pgmap v522: 129 pgs: 129 active+clean; 641 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 46 MiB/s wr, 135 op/s 2024-06-03T02:20:15.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:15 smithi123 ceph-mon[36640]: pgmap v522: 129 pgs: 129 active+clean; 641 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 46 MiB/s wr, 135 op/s 2024-06-03T02:20:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:15 smithi135 ceph-mon[36653]: pgmap v522: 129 pgs: 129 active+clean; 641 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 46 MiB/s wr, 135 op/s 2024-06-03T02:20:16.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:20:17.515 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:20:17.948 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:17 smithi079 ceph-mon[31819]: pgmap v523: 129 pgs: 129 active+clean; 787 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 35 MiB/s wr, 231 op/s 2024-06-03T02:20:17.948 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:17 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:20:17.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:17 smithi123 ceph-mon[36640]: pgmap v523: 129 pgs: 129 active+clean; 787 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 35 MiB/s wr, 231 op/s 2024-06-03T02:20:17.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:17 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:20:18.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:17 smithi135 ceph-mon[36653]: pgmap v523: 129 pgs: 129 active+clean; 787 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 35 MiB/s wr, 231 op/s 2024-06-03T02:20:18.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:17 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:20:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 10.0 s, 2197.6 tps, lat 45.549 ms stddev 50.507 2024-06-03T02:20:19.961 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:19 smithi079 ceph-mon[31819]: pgmap v524: 129 pgs: 129 active+clean; 893 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 346 op/s 2024-06-03T02:20:19.961 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:20:19.961 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:20:19.961 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:20:19.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:19 smithi123 ceph-mon[36640]: pgmap v524: 129 pgs: 129 active+clean; 893 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 346 op/s 2024-06-03T02:20:19.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:20:19.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:20:19.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:20:20.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:19 smithi135 ceph-mon[36653]: pgmap v524: 129 pgs: 129 active+clean; 893 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 346 op/s 2024-06-03T02:20:20.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:20:20.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:20:20.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:20:20.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.311+0000 7fc525000700 1 -- 172.21.15.79:0/4261823929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520102ee0 msgr2=0x7fc52010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:20.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.311+0000 7fc525000700 1 --2- 172.21.15.79:0/4261823929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520102ee0 0x7fc52010f3c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc51800b0a0 tx=0x7fc51800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.311+0000 7fc525000700 1 -- 172.21.15.79:0/4261823929 shutdown_connections 2024-06-03T02:20:20.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.311+0000 7fc525000700 1 --2- 172.21.15.79:0/4261823929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520102ee0 0x7fc52010f3c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.312+0000 7fc525000700 1 --2- 172.21.15.79:0/4261823929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc520107450 0x7fc520107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.312+0000 7fc525000700 1 --2- 172.21.15.79:0/4261823929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520102540 0x7fc5201029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.312+0000 7fc525000700 1 -- 172.21.15.79:0/4261823929 >> 172.21.15.79:0/4261823929 conn(0x7fc5200795f0 msgr2=0x7fc520079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:20.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.312+0000 7fc525000700 1 -- 172.21.15.79:0/4261823929 shutdown_connections 2024-06-03T02:20:20.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.312+0000 7fc525000700 1 -- 172.21.15.79:0/4261823929 wait complete. 2024-06-03T02:20:20.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.314+0000 7fc525000700 1 Processor -- start 2024-06-03T02:20:20.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.314+0000 7fc525000700 1 -- start start 2024-06-03T02:20:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.314+0000 7fc525000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520102540 0x7fc520117ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.314+0000 7fc525000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc520102ee0 0x7fc520117ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.314+0000 7fc525000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520107450 0x7fc520111bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.314+0000 7fc525000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc520110660 con 0x7fc520102540 2024-06-03T02:20:20.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.314+0000 7fc525000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201104e0 con 0x7fc520107450 2024-06-03T02:20:20.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.314+0000 7fc525000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201107e0 con 0x7fc520102ee0 2024-06-03T02:20:20.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.315+0000 7fc51e800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520107450 0x7fc520111bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.315+0000 7fc51e800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520107450 0x7fc520111bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58190/0 (socket says 172.21.15.79:58190) 2024-06-03T02:20:20.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.315+0000 7fc51e800700 1 -- 172.21.15.79:0/2073835348 learned_addr learned my addr 172.21.15.79:0/2073835348 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:20.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.315+0000 7fc51de00700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520102540 0x7fc520117ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.315+0000 7fc51e800700 1 -- 172.21.15.79:0/2073835348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc520102ee0 msgr2=0x7fc520117ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:20:20.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.315+0000 7fc51e800700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc520102ee0 0x7fc520117ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.315+0000 7fc51e800700 1 -- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520102540 msgr2=0x7fc520117ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:20.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.315+0000 7fc51e800700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520102540 0x7fc520117ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.315+0000 7fc51e800700 1 -- 172.21.15.79:0/2073835348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc520112490 con 0x7fc520107450 2024-06-03T02:20:20.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.316+0000 7fc51e800700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520107450 0x7fc520111bd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc51800b070 tx=0x7fc518004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:20.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.317+0000 7fc516a00700 1 -- 172.21.15.79:0/2073835348 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518021070 con 0x7fc520107450 2024-06-03T02:20:20.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.317+0000 7fc516a00700 1 -- 172.21.15.79:0/2073835348 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc518007c00 con 0x7fc520107450 2024-06-03T02:20:20.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.317+0000 7fc516a00700 1 -- 172.21.15.79:0/2073835348 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5180048b0 con 0x7fc520107450 2024-06-03T02:20:20.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.318+0000 7fc525000700 1 -- 172.21.15.79:0/2073835348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc520112720 con 0x7fc520107450 2024-06-03T02:20:20.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.318+0000 7fc525000700 1 -- 172.21.15.79:0/2073835348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5201b46e0 con 0x7fc520107450 2024-06-03T02:20:20.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.324+0000 7fc516a00700 1 -- 172.21.15.79:0/2073835348 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fc518004000 con 0x7fc520107450 2024-06-03T02:20:20.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.326+0000 7fc516a00700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5040a2e60 0x7fc5040a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.327+0000 7fc516a00700 1 -- 172.21.15.79:0/2073835348 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc51800f020 con 0x7fc520107450 2024-06-03T02:20:20.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.327+0000 7fc51de00700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5040a2e60 0x7fc5040a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.327+0000 7fc52fa26b80 1 Processor -- start 2024-06-03T02:20:20.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.327+0000 7fc52fa26b80 1 -- start start 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b84d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b85680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555773a93a10 con 0x555773b85240 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555773ad6b00 con 0x555773b8b4b0 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555773a918f0 con 0x555773b8ab90 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc51de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc51e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b85680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc51de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40772/0 (socket says 172.21.15.79:40772) 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.328+0000 7fc51de00700 1 -- 172.21.15.79:0/1206090287 learned_addr learned my addr 172.21.15.79:0/1206090287 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.329+0000 7fc51d400700 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b84d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.329+0000 7fc51e800700 1 -- 172.21.15.79:0/1206090287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 msgr2=0x555773b8af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:20.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.329+0000 7fc51e800700 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.329+0000 7fc51e800700 1 -- 172.21.15.79:0/1206090287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 msgr2=0x555773b84d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:20.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.329+0000 7fc51e800700 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b84d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.329+0000 7fc51e800700 1 -- 172.21.15.79:0/1206090287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555773a7ef90 con 0x555773b85240 2024-06-03T02:20:20.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.329+0000 7fc51de00700 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:20:20.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.329+0000 7fc51d400700 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b84d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:20:20.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.330+0000 7fc51e800700 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b85680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc5180cd7c0 tx=0x7fc518004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7684f3f44cb206 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:20.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.330+0000 7fc51de00700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5040a2e60 0x7fc5040a5320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc51000e430 tx=0x7fc51000a960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:20.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.331+0000 7fc50b400700 1 -- 172.21.15.79:0/1206090287 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5180a0950 con 0x555773b85240 2024-06-03T02:20:20.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.331+0000 7fc50b400700 1 -- 172.21.15.79:0/1206090287 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5180a5180 con 0x555773b85240 2024-06-03T02:20:20.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.331+0000 7fc50b400700 1 -- 172.21.15.79:0/1206090287 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51809c800 con 0x555773b85240 2024-06-03T02:20:20.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.332+0000 7fc52fa26b80 1 -- 172.21.15.79:0/1206090287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 msgr2=0x555773b85680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.332+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b85680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc5180cd7c0 tx=0x7fc518004350 comp rx=0 tx=0).stop 2024-06-03T02:20:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.332+0000 7fc52fa26b80 1 -- 172.21.15.79:0/1206090287 shutdown_connections 2024-06-03T02:20:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.332+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b85680 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.332+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.332+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/1206090287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b84d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.332+0000 7fc52fa26b80 1 -- 172.21.15.79:0/1206090287 >> 172.21.15.79:0/1206090287 conn(0x555773ab1170 msgr2=0x555773b7fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.332+0000 7fc52fa26b80 1 -- 172.21.15.79:0/1206090287 shutdown_connections 2024-06-03T02:20:20.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.333+0000 7fc52fa26b80 1 -- 172.21.15.79:0/1206090287 wait complete. 2024-06-03T02:20:20.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.333+0000 7fc52fa26b80 1 -- start start 2024-06-03T02:20:20.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.334+0000 7fc52fa26b80 1 Processor -- start 2024-06-03T02:20:20.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.335+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.336+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.336+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b8ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.336+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555773a72f60 con 0x555773b85240 2024-06-03T02:20:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.336+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555773ad2ac0 con 0x555773b8b4b0 2024-06-03T02:20:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.336+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555773a93a10 con 0x555773b8ab90 2024-06-03T02:20:20.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.336+0000 7fc52fa26b80 10 client.0 _ll_register_callbacks cb 0x555773a6c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:20.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.337+0000 7fc51de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.337+0000 7fc51d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.337+0000 7fc51de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b911f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47802/0 (socket says 172.21.15.79:47802) 2024-06-03T02:20:20.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.337+0000 7fc51de00700 1 -- 172.21.15.79:0/2896701772 learned_addr learned my addr 172.21.15.79:0/2896701772 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:20.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.338+0000 7fc51e800700 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b8ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.339+0000 7fc51de00700 1 -- 172.21.15.79:0/2896701772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 msgr2=0x555773b8e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:20.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.339+0000 7fc51de00700 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.339+0000 7fc51de00700 1 -- 172.21.15.79:0/2896701772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 msgr2=0x555773b8ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:20.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.339+0000 7fc51de00700 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b8ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.339+0000 7fc51de00700 1 -- 172.21.15.79:0/2896701772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555773a258d0 con 0x555773b85240 2024-06-03T02:20:20.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.339+0000 7fc51d400700 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:20:20.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.339+0000 7fc51e800700 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b8ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:20:20.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.339+0000 7fc51de00700 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b911f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc51000f010 tx=0x7fc510004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba21c0b71396f9a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:20.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.339+0000 7fc4ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:20:20.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.340+0000 7fc4ffe00700 1 -- 172.21.15.79:0/2896701772 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510023070 con 0x555773b85240 2024-06-03T02:20:20.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.340+0000 7fc52fa26b80 1 -- 172.21.15.79:0/2896701772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5180a5180 con 0x555773b85240 2024-06-03T02:20:20.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.340+0000 7fc4ffe00700 1 -- 172.21.15.79:0/2896701772 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc5100106e0 con 0x555773b85240 2024-06-03T02:20:20.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.340+0000 7fc4ffe00700 1 -- 172.21.15.79:0/2896701772 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc51002f050 con 0x555773b85240 2024-06-03T02:20:20.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.340+0000 7fc52fa26b80 10 client.15219 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:20.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.340+0000 7fc52fa26b80 1 -- 172.21.15.79:0/2896701772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555773b9c1f0 con 0x555773b85240 2024-06-03T02:20:20.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.340+0000 7fc4ffe00700 1 -- 172.21.15.79:0/2896701772 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5180a5180 con 0x555773b85240 2024-06-03T02:20:20.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.341+0000 7fc4ffe00700 1 -- 172.21.15.79:0/2896701772 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc51001e040 con 0x555773b85240 2024-06-03T02:20:20.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.341+0000 7fc52fa26b80 10 client.15219 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:20.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.341+0000 7fc52fa26b80 10 client.15219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:20:20.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.341+0000 7fc52fa26b80 20 client.15219 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:20.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.341+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555773b9e7d0 0x555773ba0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.341+0000 7fc52fa26b80 4 client.15219 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:20:20.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.341+0000 7fc52fa26b80 1 -- 172.21.15.79:0/2896701772 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555773ae4000 con 0x555773b9e7d0 2024-06-03T02:20:20.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.341+0000 7fc51d400700 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555773b9e7d0 0x555773ba0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.342+0000 7fc51d400700 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555773b9e7d0 0x555773ba0bc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8944debbef292f server_cookie=1ce44b122c688a23 in_seq=0 out_seq=0 2024-06-03T02:20:20.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.342+0000 7fc4ffe00700 10 client.15219 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:20:20.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.343+0000 7fc4ffe00700 1 -- 172.21.15.79:0/2896701772 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555773ae4000 con 0x555773b9e7d0 2024-06-03T02:20:20.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.343+0000 7fc4ffe00700 10 client.15219 handle_command_reply: tid=0 2024-06-03T02:20:20.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.343+0000 7fc52fa26b80 1 client.15219 shutdown 2024-06-03T02:20:20.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.344+0000 7fc52fa26b80 1 -- 172.21.15.79:0/2896701772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 msgr2=0x555773b911f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:20.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.344+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b911f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc51000f010 tx=0x7fc510004750 comp rx=0 tx=0).stop 2024-06-03T02:20:20.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.344+0000 7fc52fa26b80 1 -- 172.21.15.79:0/2896701772 shutdown_connections 2024-06-03T02:20:20.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.344+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b8b4b0 0x555773b8ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.345+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555773b9e7d0 0x555773ba0bc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.345+0000 7fc4ffe00700 0 client.15219 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:20:20.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.345+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b85240 0x555773b911f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.345+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/2896701772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b8e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.345+0000 7fc52fa26b80 1 -- 172.21.15.79:0/2896701772 >> 172.21.15.79:0/2896701772 conn(0x555773ab1170 msgr2=0x555773b79790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:20.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.345+0000 7fc52fa26b80 1 -- 172.21.15.79:0/2896701772 shutdown_connections 2024-06-03T02:20:20.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.345+0000 7fc52fa26b80 1 -- 172.21.15.79:0/2896701772 wait complete. 2024-06-03T02:20:20.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.346+0000 7fc52fa26b80 20 client.15219 trim_cache size 0 max 16384 2024-06-03T02:20:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.385+0000 7fc52fa26b80 1 Processor -- start 2024-06-03T02:20:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.385+0000 7fc52fa26b80 1 -- start start 2024-06-03T02:20:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.386+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b84d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.386+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b8ab90 0x555773b88c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.386+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8b4b0 0x555773b7a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.386+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201bd0a0 con 0x555773b8ab90 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.386+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201bebe0 con 0x555773b85240 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.386+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201bd770 con 0x555773b8b4b0 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.386+0000 7fc51de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b84d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc51e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8b4b0 0x555773b7a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc51de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b84d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58226/0 (socket says 172.21.15.79:58226) 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc51de00700 1 -- 172.21.15.79:0/3034904163 learned_addr learned my addr 172.21.15.79:0/3034904163 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc51de00700 1 -- 172.21.15.79:0/3034904163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8b4b0 msgr2=0x555773b7a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc51de00700 1 --2- 172.21.15.79:0/3034904163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8b4b0 0x555773b7a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc51de00700 1 -- 172.21.15.79:0/3034904163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b8ab90 msgr2=0x555773b88c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc51de00700 1 --2- 172.21.15.79:0/3034904163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b8ab90 0x555773b88c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc51de00700 1 -- 172.21.15.79:0/3034904163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555773b8b2b0 con 0x555773b85240 2024-06-03T02:20:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc51de00700 1 --2- 172.21.15.79:0/3034904163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b84d00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc510005c20 tx=0x7fc510000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8906330447e82340 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.387+0000 7fc50aa00700 1 -- 172.21.15.79:0/3034904163 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510023070 con 0x555773b85240 2024-06-03T02:20:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.388+0000 7fc50aa00700 1 -- 172.21.15.79:0/3034904163 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51001f8e0 con 0x555773b85240 2024-06-03T02:20:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.388+0000 7fc50aa00700 1 -- 172.21.15.79:0/3034904163 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510018040 con 0x555773b85240 2024-06-03T02:20:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.389+0000 7fc52fa26b80 1 -- 172.21.15.79:0/3034904163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 msgr2=0x555773b84d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.389+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/3034904163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b84d00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc510005c20 tx=0x7fc510000bc0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.389+0000 7fc52fa26b80 1 -- 172.21.15.79:0/3034904163 shutdown_connections 2024-06-03T02:20:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.389+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/3034904163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8b4b0 0x555773b7a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.389+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/3034904163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b84d00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.389+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/3034904163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b8ab90 0x555773b88c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.389+0000 7fc52fa26b80 1 -- 172.21.15.79:0/3034904163 >> 172.21.15.79:0/3034904163 conn(0x555773ab1170 msgr2=0x555773b8b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.390+0000 7fc52fa26b80 1 -- 172.21.15.79:0/3034904163 shutdown_connections 2024-06-03T02:20:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.390+0000 7fc52fa26b80 1 -- 172.21.15.79:0/3034904163 wait complete. 2024-06-03T02:20:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.390+0000 7fc52fa26b80 1 -- start start 2024-06-03T02:20:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.391+0000 7fc52fa26b80 1 Processor -- start 2024-06-03T02:20:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.391+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b96ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.391+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b97430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.391+0000 7fc52fa26b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b8b4b0 0x555773ba5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.391+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201be510 con 0x555773b8b4b0 2024-06-03T02:20:20.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.391+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201bde40 con 0x555773b85240 2024-06-03T02:20:20.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.391+0000 7fc52fa26b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201bb040 con 0x555773b8ab90 2024-06-03T02:20:20.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.392+0000 7fc52fa26b80 10 client.0 _ll_register_callbacks cb 0x555773a6c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:20.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.393+0000 7fc51de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b96ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.393+0000 7fc51de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b96ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58230/0 (socket says 172.21.15.79:58230) 2024-06-03T02:20:20.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.393+0000 7fc51d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b97430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.393+0000 7fc51d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b97430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40796/0 (socket says 172.21.15.79:40796) 2024-06-03T02:20:20.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.393+0000 7fc51d400700 1 -- 172.21.15.79:0/4015408012 learned_addr learned my addr 172.21.15.79:0/4015408012 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:20.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.394+0000 7fc51e800700 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b8b4b0 0x555773ba5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.395+0000 7fc51d400700 1 -- 172.21.15.79:0/4015408012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 msgr2=0x555773b96ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:20.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.395+0000 7fc51d400700 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b96ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.395+0000 7fc51d400700 1 -- 172.21.15.79:0/4015408012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b8b4b0 msgr2=0x555773ba5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:20.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.395+0000 7fc51d400700 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b8b4b0 0x555773ba5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.395+0000 7fc51d400700 1 -- 172.21.15.79:0/4015408012 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555773ba60c0 con 0x555773b8ab90 2024-06-03T02:20:20.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.396+0000 7fc51d400700 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b97430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc520113380 tx=0x7fc50c012870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74adb0f41bc3c275 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:20.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.396+0000 7fc4ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:20:20.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.396+0000 7fc4ffe00700 1 -- 172.21.15.79:0/4015408012 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c01f030 con 0x555773b8ab90 2024-06-03T02:20:20.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.396+0000 7fc4ffe00700 1 -- 172.21.15.79:0/4015408012 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc50c004830 con 0x555773b8ab90 2024-06-03T02:20:20.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.396+0000 7fc52fa26b80 1 -- 172.21.15.79:0/4015408012 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc51001f8e0 con 0x555773b8ab90 2024-06-03T02:20:20.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc4ffe00700 1 -- 172.21.15.79:0/4015408012 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc50c004a30 con 0x555773b8ab90 2024-06-03T02:20:20.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc52fa26b80 10 client.25318 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:20.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc52fa26b80 1 -- 172.21.15.79:0/4015408012 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555773c08f60 con 0x555773b8ab90 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc4ffe00700 1 -- 172.21.15.79:0/4015408012 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51001f8e0 con 0x555773b8ab90 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc4ffe00700 1 -- 172.21.15.79:0/4015408012 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc50c00f040 con 0x555773b8ab90 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc52fa26b80 10 client.25318 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc52fa26b80 10 client.25318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc52fa26b80 20 client.25318 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555773b9e7d0 0x555773c0b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc52fa26b80 4 client.25318 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.397+0000 7fc52fa26b80 1 -- 172.21.15.79:0/4015408012 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555773ade580 con 0x555773b9e7d0 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.398+0000 7fc51de00700 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555773b9e7d0 0x555773c0b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.398+0000 7fc4ffe00700 10 client.25318 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:20:20.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.400+0000 7fc51de00700 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555773b9e7d0 0x555773c0b6e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7679f281265a1fb5 server_cookie=805e14cc3fd8b62f in_seq=0 out_seq=0 2024-06-03T02:20:20.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.400+0000 7fc4ffe00700 1 -- 172.21.15.79:0/4015408012 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555773ade580 con 0x555773b9e7d0 2024-06-03T02:20:20.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.400+0000 7fc4ffe00700 10 client.25318 handle_command_reply: tid=0 2024-06-03T02:20:20.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.400+0000 7fc52fa26b80 1 client.25318 shutdown 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 -- 172.21.15.79:0/4015408012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 msgr2=0x555773b97430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b97430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc520113380 tx=0x7fc50c012870 comp rx=0 tx=0).stop 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 -- 172.21.15.79:0/4015408012 shutdown_connections 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555773b9e7d0 0x555773c0b6e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555773b8b4b0 0x555773ba5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773b85240 0x555773b96ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc4ffe00700 0 client.25318 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 --2- 172.21.15.79:0/4015408012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555773b8ab90 0x555773b97430 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 -- 172.21.15.79:0/4015408012 >> 172.21.15.79:0/4015408012 conn(0x555773ab1170 msgr2=0x555773b89670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 -- 172.21.15.79:0/4015408012 shutdown_connections 2024-06-03T02:20:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.401+0000 7fc52fa26b80 1 -- 172.21.15.79:0/4015408012 wait complete. 2024-06-03T02:20:20.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.402+0000 7fc52fa26b80 20 client.25318 trim_cache size 0 max 16384 2024-06-03T02:20:20.403 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:20:20.403 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:20:20.403 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:20:20.403 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:20:20.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 -- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5040a2e60 msgr2=0x7fc5040a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:20.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5040a2e60 0x7fc5040a5320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc51000e430 tx=0x7fc51000a960 comp rx=0 tx=0).stop 2024-06-03T02:20:20.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 -- 172.21.15.79:0/2073835348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520107450 msgr2=0x7fc520111bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:20.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520107450 0x7fc520111bd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc51800b070 tx=0x7fc518004d10 comp rx=0 tx=0).stop 2024-06-03T02:20:20.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 -- 172.21.15.79:0/2073835348 shutdown_connections 2024-06-03T02:20:20.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520107450 0x7fc520111bd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5040a2e60 0x7fc5040a5320 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520102540 0x7fc520117ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 --2- 172.21.15.79:0/2073835348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc520102ee0 0x7fc520117ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:20.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 -- 172.21.15.79:0/2073835348 >> 172.21.15.79:0/2073835348 conn(0x7fc5200795f0 msgr2=0x7fc520100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:20.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 -- 172.21.15.79:0/2073835348 shutdown_connections 2024-06-03T02:20:20.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:20.403+0000 7fc525000700 1 -- 172.21.15.79:0/2073835348 wait complete. 2024-06-03T02:20:21.349 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:20:21.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:20:21.963 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:20:21.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:21 smithi123 ceph-mon[36640]: pgmap v525: 129 pgs: 129 active+clean; 901 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 362 op/s 2024-06-03T02:20:22.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:21 smithi079 ceph-mon[31819]: pgmap v525: 129 pgs: 129 active+clean; 901 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 362 op/s 2024-06-03T02:20:22.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:21 smithi135 ceph-mon[36653]: pgmap v525: 129 pgs: 129 active+clean; 901 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 362 op/s 2024-06-03T02:20:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 15.0 s, 1896.3 tps, lat 52.710 ms stddev 61.076 2024-06-03T02:20:23.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:23 smithi123 ceph-mon[36640]: pgmap v526: 129 pgs: 129 active+clean; 914 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 574 op/s 2024-06-03T02:20:24.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:23 smithi079 ceph-mon[31819]: pgmap v526: 129 pgs: 129 active+clean; 914 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 574 op/s 2024-06-03T02:20:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:23 smithi135 ceph-mon[36653]: pgmap v526: 129 pgs: 129 active+clean; 914 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 574 op/s 2024-06-03T02:20:25.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.288+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1997087704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bcc104ed0 msgr2=0x7f6bcc105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:25.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.288+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1997087704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bcc104ed0 0x7f6bcc105360 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400b0a0 tx=0x7f6bc400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.288+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1997087704 shutdown_connections 2024-06-03T02:20:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.288+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1997087704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bcc104ed0 0x7f6bcc105360 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.288+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1997087704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc1073c0 0x7f6bcc06a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.288+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1997087704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bcc06a6e0 0x7f6bcc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.288+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1997087704 >> 172.21.15.79:0/1997087704 conn(0x7f6bcc078340 msgr2=0x7f6bcc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.289+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1997087704 shutdown_connections 2024-06-03T02:20:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.289+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1997087704 wait complete. 2024-06-03T02:20:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.290+0000 7f6bd1a00700 1 Processor -- start 2024-06-03T02:20:25.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.290+0000 7f6bd1a00700 1 -- start start 2024-06-03T02:20:25.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.290+0000 7f6bd1a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bcc06a6e0 0x7f6bcc1154c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:25.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.290+0000 7f6bd1a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc104ed0 0x7f6bcc115a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:25.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.290+0000 7f6bd1a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bcc1073c0 0x7f6bcc10e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:25.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.291+0000 7f6bd1a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc118da0 con 0x7f6bcc1073c0 2024-06-03T02:20:25.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.291+0000 7f6bd1a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc118c20 con 0x7f6bcc06a6e0 2024-06-03T02:20:25.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.291+0000 7f6bd1a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc118f20 con 0x7f6bcc104ed0 2024-06-03T02:20:25.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.292+0000 7f6bca000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc104ed0 0x7f6bcc115a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:25.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.292+0000 7f6bca000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc104ed0 0x7f6bcc115a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40094/0 (socket says 172.21.15.79:40094) 2024-06-03T02:20:25.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.292+0000 7f6bca000700 1 -- 172.21.15.79:0/1500187670 learned_addr learned my addr 172.21.15.79:0/1500187670 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:25.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.292+0000 7f6bcaa00700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bcc06a6e0 0x7f6bcc1154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.293+0000 7f6bca000700 1 -- 172.21.15.79:0/1500187670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bcc06a6e0 msgr2=0x7f6bcc1154c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.293+0000 7f6bca000700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bcc06a6e0 0x7f6bcc1154c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.293+0000 7f6bca000700 1 -- 172.21.15.79:0/1500187670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bcc1073c0 msgr2=0x7f6bcc10e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.293+0000 7f6bca000700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bcc1073c0 0x7f6bcc10e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.293+0000 7f6bca000700 1 -- 172.21.15.79:0/1500187670 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc10ee00 con 0x7f6bcc104ed0 2024-06-03T02:20:25.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.293+0000 7f6bca000700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc104ed0 0x7f6bcc115a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00d980 tx=0x7f6bbc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:25.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.293+0000 7f6bc3400700 1 -- 172.21.15.79:0/1500187670 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc016070 con 0x7f6bcc104ed0 2024-06-03T02:20:25.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.294+0000 7f6bc3400700 1 -- 172.21.15.79:0/1500187670 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bbc004d10 con 0x7f6bcc104ed0 2024-06-03T02:20:25.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.294+0000 7f6bc3400700 1 -- 172.21.15.79:0/1500187670 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0055f0 con 0x7f6bcc104ed0 2024-06-03T02:20:25.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.294+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc10f0f0 con 0x7f6bcc104ed0 2024-06-03T02:20:25.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.295+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc1131c0 con 0x7f6bcc104ed0 2024-06-03T02:20:25.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.296+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc1019b0 con 0x7f6bcc104ed0 2024-06-03T02:20:25.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.301+0000 7f6bc3400700 1 -- 172.21.15.79:0/1500187670 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6bbc004750 con 0x7f6bcc104ed0 2024-06-03T02:20:25.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.302+0000 7f6bc3400700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bac0a2d30 0x7f6bac0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:25.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.303+0000 7f6bc3400700 1 -- 172.21.15.79:0/1500187670 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6bbc0cf500 con 0x7f6bcc104ed0 2024-06-03T02:20:25.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.303+0000 7f6bcaa00700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bac0a2d30 0x7f6bac0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:25.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.304+0000 7f6bc3400700 1 -- 172.21.15.79:0/1500187670 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6bbc09bf50 con 0x7f6bcc104ed0 2024-06-03T02:20:25.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.304+0000 7f6bcaa00700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bac0a2d30 0x7f6bac0a51f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009be0 tx=0x7f6bb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:25.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.800+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bcc069180 con 0x7f6bcc104ed0 2024-06-03T02:20:25.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.801+0000 7f6bc3400700 1 -- 172.21.15.79:0/1500187670 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f6bcc069180 con 0x7f6bcc104ed0 2024-06-03T02:20:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:20:25.802 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:20:25.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bac0a2d30 msgr2=0x7f6bac0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:25.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bac0a2d30 0x7f6bac0a51f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009be0 tx=0x7f6bb4009260 comp rx=0 tx=0).stop 2024-06-03T02:20:25.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc104ed0 msgr2=0x7f6bcc115a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:25.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc104ed0 0x7f6bcc115a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00d980 tx=0x7f6bbc00dc90 comp rx=0 tx=0).stop 2024-06-03T02:20:25.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 shutdown_connections 2024-06-03T02:20:25.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6bac0a2d30 0x7f6bac0a51f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bcc1073c0 0x7f6bcc10e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bcc06a6e0 0x7f6bcc1154c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 --2- 172.21.15.79:0/1500187670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc104ed0 0x7f6bcc115a00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:25.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.804+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 >> 172.21.15.79:0/1500187670 conn(0x7f6bcc078340 msgr2=0x7f6bcc103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:25.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.805+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 shutdown_connections 2024-06-03T02:20:25.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:25.805+0000 7f6bd1a00700 1 -- 172.21.15.79:0/1500187670 wait complete. 2024-06-03T02:20:25.808 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:20:25.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:25 smithi123 ceph-mon[36640]: pgmap v527: 129 pgs: 129 active+clean; 922 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.8 MiB/s wr, 563 op/s 2024-06-03T02:20:26.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:25 smithi079 ceph-mon[31819]: pgmap v527: 129 pgs: 129 active+clean; 922 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.8 MiB/s wr, 563 op/s 2024-06-03T02:20:26.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:25 smithi135 ceph-mon[36653]: pgmap v527: 129 pgs: 129 active+clean; 922 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.8 MiB/s wr, 563 op/s 2024-06-03T02:20:26.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:20:27.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:26 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1500187670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:20:27.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:26 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1500187670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:20:27.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:26 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1500187670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:20:27.422 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:20:27.849 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:27 smithi079 ceph-mon[31819]: pgmap v528: 129 pgs: 129 active+clean; 932 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 646 op/s 2024-06-03T02:20:28.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:27 smithi135 ceph-mon[36653]: pgmap v528: 129 pgs: 129 active+clean; 932 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 646 op/s 2024-06-03T02:20:28.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:27 smithi123 ceph-mon[36640]: pgmap v528: 129 pgs: 129 active+clean; 932 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 646 op/s 2024-06-03T02:20:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 20.0 s, 2046.4 tps, lat 48.928 ms stddev 56.094 2024-06-03T02:20:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.965+0000 7f6aaf000700 1 -- 172.21.15.79:0/4099392275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8105040 msgr2=0x7f6aa8111550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.965+0000 7f6aaf000700 1 --2- 172.21.15.79:0/4099392275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8105040 0x7f6aa8111550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009f90 tx=0x7f6aa000b330 comp rx=0 tx=0).stop 2024-06-03T02:20:29.966 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:29 smithi079 ceph-mon[31819]: pgmap v529: 129 pgs: 129 active+clean; 934 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 631 op/s 2024-06-03T02:20:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.966+0000 7f6aaf000700 1 -- 172.21.15.79:0/4099392275 shutdown_connections 2024-06-03T02:20:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.966+0000 7f6aaf000700 1 --2- 172.21.15.79:0/4099392275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8105040 0x7f6aa8111550 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.966+0000 7f6aaf000700 1 --2- 172.21.15.79:0/4099392275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa810a680 0x7f6aa810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.966+0000 7f6aaf000700 1 --2- 172.21.15.79:0/4099392275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aa8104680 0x7f6aa8104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.966+0000 7f6aaf000700 1 -- 172.21.15.79:0/4099392275 >> 172.21.15.79:0/4099392275 conn(0x7f6aa8100170 msgr2=0x7f6aa8102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.967+0000 7f6aaf000700 1 -- 172.21.15.79:0/4099392275 shutdown_connections 2024-06-03T02:20:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.967+0000 7f6aaf000700 1 -- 172.21.15.79:0/4099392275 wait complete. 2024-06-03T02:20:29.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.969+0000 7f6aaf000700 1 Processor -- start 2024-06-03T02:20:29.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.970+0000 7f6aaf000700 1 -- start start 2024-06-03T02:20:29.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.970+0000 7f6aaf000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8104680 0x7f6aa8073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.970+0000 7f6aaf000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aa8105040 0x7f6aa806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.970+0000 7f6aaf000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa810a680 0x7f6aa806e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.970+0000 7f6aaf000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa806c8c0 con 0x7f6aa8105040 2024-06-03T02:20:29.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.970+0000 7f6aaf000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa806c740 con 0x7f6aa810a680 2024-06-03T02:20:29.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.970+0000 7f6aaf000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa806ca40 con 0x7f6aa8104680 2024-06-03T02:20:29.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.970+0000 7f6aad000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa810a680 0x7f6aa806e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:29.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8104680 0x7f6aa8073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:29.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8104680 0x7f6aa8073cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40104/0 (socket says 172.21.15.79:40104) 2024-06-03T02:20:29.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7e00700 1 -- 172.21.15.79:0/952509318 learned_addr learned my addr 172.21.15.79:0/952509318 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:29.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.970+0000 7f6aad000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa810a680 0x7f6aa806e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34356/0 (socket says 172.21.15.79:34356) 2024-06-03T02:20:29.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7400700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aa8105040 0x7f6aa806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:29.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7e00700 1 -- 172.21.15.79:0/952509318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa810a680 msgr2=0x7f6aa806e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:29.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa810a680 0x7f6aa806e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7e00700 1 -- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aa8105040 msgr2=0x7f6aa806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:29.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aa8105040 0x7f6aa806dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7e00700 1 -- 172.21.15.79:0/952509318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa806e880 con 0x7f6aa8104680 2024-06-03T02:20:29.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.971+0000 7f6aa7400700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aa8105040 0x7f6aa806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:20:29.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.972+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8104680 0x7f6aa8073cb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c920 tx=0x7f6a9800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:29.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.972+0000 7f6aa4c00700 1 -- 172.21.15.79:0/952509318 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98007b00 con 0x7f6aa8104680 2024-06-03T02:20:29.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.972+0000 7f6aa4c00700 1 -- 172.21.15.79:0/952509318 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a98004d10 con 0x7f6aa8104680 2024-06-03T02:20:29.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.972+0000 7f6aa4c00700 1 -- 172.21.15.79:0/952509318 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98005680 con 0x7f6aa8104680 2024-06-03T02:20:29.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.972+0000 7f6aaf000700 1 -- 172.21.15.79:0/952509318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa806eb70 con 0x7f6aa8104680 2024-06-03T02:20:29.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.973+0000 7f6aaf000700 1 -- 172.21.15.79:0/952509318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa81b00c0 con 0x7f6aa8104680 2024-06-03T02:20:29.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.986+0000 7f6aa4c00700 1 -- 172.21.15.79:0/952509318 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6a98004750 con 0x7f6aa8104680 2024-06-03T02:20:29.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.987+0000 7f6ab99e7b80 1 Processor -- start 2024-06-03T02:20:29.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.988+0000 7f6ab99e7b80 1 -- start start 2024-06-03T02:20:29.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.990+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a70761f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.990+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7075bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.990+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7075c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.990+0000 7f6ab99e7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559a7066aa10 con 0x559a70761b90 2024-06-03T02:20:29.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.990+0000 7f6ab99e7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a706adb00 con 0x559a7075c240 2024-06-03T02:20:29.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.990+0000 7f6ab99e7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559a706688f0 con 0x559a707624b0 2024-06-03T02:20:29.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.990+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7075bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:29.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.991+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7075bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40110/0 (socket says 172.21.15.79:40110) 2024-06-03T02:20:29.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.991+0000 7f6aa7e00700 1 -- 172.21.15.79:0/1992985211 learned_addr learned my addr 172.21.15.79:0/1992985211 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:29.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.991+0000 7f6aa4c00700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a8c0a2e00 0x7f6a8c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.991+0000 7f6aa7400700 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7075c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:29.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.991+0000 7f6aa4c00700 1 -- 172.21.15.79:0/952509318 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6a980cf300 con 0x7f6aa8104680 2024-06-03T02:20:29.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.992+0000 7f6aa7e00700 1 -- 172.21.15.79:0/1992985211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 msgr2=0x559a7075c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:29.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.992+0000 7f6aad000700 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a70761f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:29.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.992+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7075c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.992+0000 7f6aa7e00700 1 -- 172.21.15.79:0/1992985211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 msgr2=0x559a70761f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:29.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.992+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a70761f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.992+0000 7f6aa7e00700 1 -- 172.21.15.79:0/1992985211 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a70655f90 con 0x559a707624b0 2024-06-03T02:20:29.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.993+0000 7f6aad000700 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a70761f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:20:29.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.993+0000 7f6aad000700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a8c0a2e00 0x7f6a8c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:29.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.993+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7075bd00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800a6b0 tx=0x7f6a98000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0ba2ef2dc078e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:29.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.993+0000 7f6a9d600700 1 -- 172.21.15.79:0/1992985211 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9801d040 con 0x559a707624b0 2024-06-03T02:20:29.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.993+0000 7f6a9d600700 1 -- 172.21.15.79:0/1992985211 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9809bd50 con 0x559a707624b0 2024-06-03T02:20:29.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.993+0000 7f6a9d600700 1 -- 172.21.15.79:0/1992985211 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98007d30 con 0x559a707624b0 2024-06-03T02:20:29.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.994+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/1992985211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 msgr2=0x559a7075bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:29.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.994+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7075bd00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800a6b0 tx=0x7f6a98000bc0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.994+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/1992985211 shutdown_connections 2024-06-03T02:20:29.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.994+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7075c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.994+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a70761f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.994+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/1992985211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7075bd00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:29.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.994+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/1992985211 >> 172.21.15.79:0/1992985211 conn(0x559a70688170 msgr2=0x559a70756ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:29.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.995+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/1992985211 shutdown_connections 2024-06-03T02:20:29.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.995+0000 7f6aad000700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a8c0a2e00 0x7f6a8c0a52c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009c10 tx=0x7f6aa0009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:29.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.995+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/1992985211 wait complete. 2024-06-03T02:20:29.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.996+0000 7f6ab99e7b80 1 -- start start 2024-06-03T02:20:29.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.996+0000 7f6ab99e7b80 1 Processor -- start 2024-06-03T02:20:29.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.997+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a7075c240 0x559a70768220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.997+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70765520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.997+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a707624b0 0x559a70765a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:29.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.997+0000 7f6ab99e7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559a70649f60 con 0x559a7075c240 2024-06-03T02:20:29.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.997+0000 7f6ab99e7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a706a9ac0 con 0x559a707624b0 2024-06-03T02:20:29.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.997+0000 7f6ab99e7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559a70768ed0 con 0x559a70761b90 2024-06-03T02:20:29.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.997+0000 7f6ab99e7b80 10 client.0 _ll_register_callbacks cb 0x559a70643380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:29.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.998+0000 7f6aa7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70765520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:29.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.998+0000 7f6aa7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70765520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40126/0 (socket says 172.21.15.79:40126) 2024-06-03T02:20:29.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.998+0000 7f6aa7400700 1 -- 172.21.15.79:0/2096761454 learned_addr learned my addr 172.21.15.79:0/2096761454 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:30.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.998+0000 7f6aa7400700 1 -- 172.21.15.79:0/2096761454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a707624b0 msgr2=0x559a70765a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:20:30.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.998+0000 7f6aa7400700 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a707624b0 0x559a70765a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.998+0000 7f6aa7400700 1 -- 172.21.15.79:0/2096761454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a7075c240 msgr2=0x559a70768220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:30.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.998+0000 7f6aa7400700 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a7075c240 0x559a70768220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.998+0000 7f6aa7400700 1 -- 172.21.15.79:0/2096761454 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a705fc8d0 con 0x559a70761b90 2024-06-03T02:20:30.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.999+0000 7f6aa7400700 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70765520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a94013790 tx=0x7f6a94013b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a212c6dcf53304ac server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:30.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.999+0000 7f6a92000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:20:30.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.999+0000 7f6a92000700 1 -- 172.21.15.79:0/2096761454 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94004a30 con 0x559a70761b90 2024-06-03T02:20:30.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:29.999+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/2096761454 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a9809bd50 con 0x559a70761b90 2024-06-03T02:20:30.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.000+0000 7f6a92000700 1 -- 172.21.15.79:0/2096761454 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6a94004bd0 con 0x559a70761b90 2024-06-03T02:20:30.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.001+0000 7f6a92000700 1 -- 172.21.15.79:0/2096761454 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a94011710 con 0x559a70761b90 2024-06-03T02:20:30.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.002+0000 7f6ab99e7b80 10 client.25336 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:30.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.002+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/2096761454 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a70773430 con 0x559a70761b90 2024-06-03T02:20:30.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.002+0000 7f6a92000700 1 -- 172.21.15.79:0/2096761454 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9809bd50 con 0x559a70761b90 2024-06-03T02:20:30.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.003+0000 7f6a92000700 1 -- 172.21.15.79:0/2096761454 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6a940054d0 con 0x559a70761b90 2024-06-03T02:20:30.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.003+0000 7f6ab99e7b80 10 client.25336 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:30.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.003+0000 7f6ab99e7b80 10 client.25336 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:20:30.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.003+0000 7f6ab99e7b80 20 client.25336 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:30.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.003+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559a707759c0 0x559a70777db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:30.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.003+0000 7f6ab99e7b80 4 client.25336 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:20:30.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.003+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/2096761454 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a706bb000 con 0x559a707759c0 2024-06-03T02:20:30.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.003+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559a707759c0 0x559a70777db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:30.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.005+0000 7f6a92000700 10 client.25336 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:20:30.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.005+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559a707759c0 0x559a70777db0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ec4193cbd99e3d8 server_cookie=7bb413f368452605 in_seq=0 out_seq=0 2024-06-03T02:20:30.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.006+0000 7f6a92000700 1 -- 172.21.15.79:0/2096761454 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559a706bb000 con 0x559a707759c0 2024-06-03T02:20:30.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.006+0000 7f6a92000700 10 client.25336 handle_command_reply: tid=0 2024-06-03T02:20:30.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.006+0000 7f6ab99e7b80 1 client.25336 shutdown 2024-06-03T02:20:30.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.006+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/2096761454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 msgr2=0x559a70765520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:30.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70765520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a94013790 tx=0x7f6a94013b50 comp rx=0 tx=0).stop 2024-06-03T02:20:30.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/2096761454 shutdown_connections 2024-06-03T02:20:30.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559a707759c0 0x559a70777db0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a707624b0 0x559a70765a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a7075c240 0x559a70768220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/2096761454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70765520 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6a92000700 0 client.25336 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:20:30.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/2096761454 >> 172.21.15.79:0/2096761454 conn(0x559a70688170 msgr2=0x559a70750790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:30.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/2096761454 shutdown_connections 2024-06-03T02:20:30.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/2096761454 wait complete. 2024-06-03T02:20:30.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.007+0000 7f6ab99e7b80 20 client.25336 trim_cache size 0 max 16384 2024-06-03T02:20:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.033+0000 7f6ab99e7b80 1 Processor -- start 2024-06-03T02:20:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.033+0000 7f6ab99e7b80 1 -- start start 2024-06-03T02:20:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.033+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7075bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.033+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70763160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.033+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a707624b0 0x559a70751130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6aa7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70763160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7075bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6aa7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70763160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40138/0 (socket says 172.21.15.79:40138) 2024-06-03T02:20:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7075bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34386/0 (socket says 172.21.15.79:34386) 2024-06-03T02:20:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6aa7400700 1 -- 172.21.15.79:0/3707757414 learned_addr learned my addr 172.21.15.79:0/3707757414 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/3707757414 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81b8ec0 con 0x559a707624b0 2024-06-03T02:20:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/3707757414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81baa00 con 0x559a7075c240 2024-06-03T02:20:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/3707757414 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81b9590 con 0x559a70761b90 2024-06-03T02:20:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6aad000700 1 --2- 172.21.15.79:0/3707757414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a707624b0 0x559a70751130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:30.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6aa7400700 1 -- 172.21.15.79:0/3707757414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 msgr2=0x559a7075bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:30.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6aa7400700 1 --2- 172.21.15.79:0/3707757414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7075bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.034+0000 7f6aa7400700 1 -- 172.21.15.79:0/3707757414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a707624b0 msgr2=0x559a70751130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:30.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.035+0000 7f6aa7400700 1 --2- 172.21.15.79:0/3707757414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a707624b0 0x559a70751130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.035+0000 7f6aa7400700 1 -- 172.21.15.79:0/3707757414 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a707783e0 con 0x559a70761b90 2024-06-03T02:20:30.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.035+0000 7f6aa7400700 1 --2- 172.21.15.79:0/3707757414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70763160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d7e0 tx=0x7f6a94009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36c655ca533c2785 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:30.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.035+0000 7f6a9cc00700 1 -- 172.21.15.79:0/3707757414 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94011710 con 0x559a70761b90 2024-06-03T02:20:30.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.035+0000 7f6a9cc00700 1 -- 172.21.15.79:0/3707757414 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9402c020 con 0x559a70761b90 2024-06-03T02:20:30.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.035+0000 7f6a9cc00700 1 -- 172.21.15.79:0/3707757414 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9401c3a0 con 0x559a70761b90 2024-06-03T02:20:30.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.036+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/3707757414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 msgr2=0x559a70763160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.036+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/3707757414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70763160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d7e0 tx=0x7f6a94009ea0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.037+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/3707757414 shutdown_connections 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.037+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/3707757414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a707624b0 0x559a70751130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.037+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/3707757414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7075bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.037+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/3707757414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a70761b90 0x559a70763160 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.037+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/3707757414 >> 172.21.15.79:0/3707757414 conn(0x559a70688170 msgr2=0x559a70764dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.037+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/3707757414 shutdown_connections 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.037+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/3707757414 wait complete. 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.037+0000 7f6ab99e7b80 1 -- start start 2024-06-03T02:20:30.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.038+0000 7f6ab99e7b80 1 Processor -- start 2024-06-03T02:20:30.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.038+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7076e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:30.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.038+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a7076e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:30.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.039+0000 7f6ab99e7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7077c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:30.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.039+0000 7f6ab99e7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81ba330 con 0x559a70761b90 2024-06-03T02:20:30.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.039+0000 7f6ab99e7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81b9c60 con 0x559a7075c240 2024-06-03T02:20:30.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.039+0000 7f6ab99e7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81b6e00 con 0x559a707624b0 2024-06-03T02:20:30.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.039+0000 7f6ab99e7b80 10 client.0 _ll_register_callbacks cb 0x559a70643380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:30.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.039+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7076e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:30.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.039+0000 7f6aa7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7076e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34402/0 (socket says 172.21.15.79:34402) 2024-06-03T02:20:30.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.040+0000 7f6aad000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7077c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:30.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.040+0000 7f6aa7e00700 1 -- 172.21.15.79:0/771439127 learned_addr learned my addr 172.21.15.79:0/771439127 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:30.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.040+0000 7f6aa7400700 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a7076e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:30.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.041+0000 7f6aad000700 1 -- 172.21.15.79:0/771439127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 msgr2=0x559a7076e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:30.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.041+0000 7f6aad000700 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7076e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.041+0000 7f6aad000700 1 -- 172.21.15.79:0/771439127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 msgr2=0x559a7076e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:30.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.041+0000 7f6aad000700 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a7076e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.041+0000 7f6aad000700 1 -- 172.21.15.79:0/771439127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7077cba0 con 0x559a707624b0 2024-06-03T02:20:30.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.041+0000 7f6aa7400700 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a7076e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:20:30.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.041+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7076e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:20:30.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.042+0000 7f6aad000700 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7077c660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x559a707517d0 tx=0x7f6aa0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95096a86e64116a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:30.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.042+0000 7f6a92000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:20:30.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.042+0000 7f6a92000700 1 -- 172.21.15.79:0/771439127 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000be20 con 0x559a707624b0 2024-06-03T02:20:30.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.043+0000 7f6a92000700 1 -- 172.21.15.79:0/771439127 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6aa001e430 con 0x559a707624b0 2024-06-03T02:20:30.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.043+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/771439127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a9402c020 con 0x559a707624b0 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.043+0000 7f6a92000700 1 -- 172.21.15.79:0/771439127 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6aa001e630 con 0x559a707624b0 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.043+0000 7f6ab99e7b80 10 client.25342 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.043+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/771439127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a707d8d10 con 0x559a707624b0 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.044+0000 7f6a92000700 1 -- 172.21.15.79:0/771439127 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9402c020 con 0x559a707624b0 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.044+0000 7f6a92000700 1 -- 172.21.15.79:0/771439127 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6aa0011070 con 0x559a707624b0 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.044+0000 7f6ab99e7b80 10 client.25342 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.044+0000 7f6ab99e7b80 10 client.25342 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.044+0000 7f6ab99e7b80 20 client.25342 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.044+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559a707d9420 0x559a707db840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.044+0000 7f6ab99e7b80 4 client.25342 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.044+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/771439127 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a706b5580 con 0x559a707d9420 2024-06-03T02:20:30.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.044+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559a707d9420 0x559a707db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:30.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.045+0000 7f6aa7e00700 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559a707d9420 0x559a707db840 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f0010087e39387a2 server_cookie=bb60c4f9aec7aba9 in_seq=0 out_seq=0 2024-06-03T02:20:30.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.045+0000 7f6a92000700 10 client.25342 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:20:30.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.045+0000 7f6a92000700 1 -- 172.21.15.79:0/771439127 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a706b5580 con 0x559a707d9420 2024-06-03T02:20:30.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.045+0000 7f6a92000700 10 client.25342 handle_command_reply: tid=0 2024-06-03T02:20:30.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.045+0000 7f6ab99e7b80 1 client.25342 shutdown 2024-06-03T02:20:30.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.046+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/771439127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 msgr2=0x559a7077c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:30.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.046+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7077c660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x559a707517d0 tx=0x7f6aa0009920 comp rx=0 tx=0).stop 2024-06-03T02:20:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.046+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/771439127 shutdown_connections 2024-06-03T02:20:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.046+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559a707d9420 0x559a707db840 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.046+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a707624b0 0x559a7077c660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.046+0000 7f6a92000700 0 client.25342 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:20:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.047+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a7075c240 0x559a7076e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.047+0000 7f6ab99e7b80 1 --2- 172.21.15.79:0/771439127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a70761b90 0x559a7076e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.047+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/771439127 >> 172.21.15.79:0/771439127 conn(0x559a70688170 msgr2=0x559a70765660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.047+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/771439127 shutdown_connections 2024-06-03T02:20:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.047+0000 7f6ab99e7b80 1 -- 172.21.15.79:0/771439127 wait complete. 2024-06-03T02:20:30.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.047+0000 7f6ab99e7b80 20 client.25342 trim_cache size 0 max 16384 2024-06-03T02:20:30.049 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:20:30.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.048+0000 7f6aaf000700 1 -- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a8c0a2e00 msgr2=0x7f6a8c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:30.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.048+0000 7f6aaf000700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a8c0a2e00 0x7f6a8c0a52c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009c10 tx=0x7f6aa0009700 comp rx=0 tx=0).stop 2024-06-03T02:20:30.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.048+0000 7f6aaf000700 1 -- 172.21.15.79:0/952509318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8104680 msgr2=0x7f6aa8073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:30.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.048+0000 7f6aaf000700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8104680 0x7f6aa8073cb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c920 tx=0x7f6a9800cc30 comp rx=0 tx=0).stop 2024-06-03T02:20:30.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.049+0000 7f6aaf000700 1 -- 172.21.15.79:0/952509318 shutdown_connections 2024-06-03T02:20:30.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.049+0000 7f6aaf000700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa810a680 0x7f6aa806e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.049+0000 7f6aaf000700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aa8104680 0x7f6aa8073cb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.049+0000 7f6aaf000700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a8c0a2e00 0x7f6a8c0a52c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.049+0000 7f6aaf000700 1 --2- 172.21.15.79:0/952509318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aa8105040 0x7f6aa806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:30.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.049+0000 7f6aaf000700 1 -- 172.21.15.79:0/952509318 >> 172.21.15.79:0/952509318 conn(0x7f6aa8100170 msgr2=0x7f6aa8100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:30.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.049+0000 7f6aaf000700 1 -- 172.21.15.79:0/952509318 shutdown_connections 2024-06-03T02:20:30.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:30.049+0000 7f6aaf000700 1 -- 172.21.15.79:0/952509318 wait complete. 2024-06-03T02:20:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:29 smithi135 ceph-mon[36653]: pgmap v529: 129 pgs: 129 active+clean; 934 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 631 op/s 2024-06-03T02:20:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:29 smithi123 ceph-mon[36640]: pgmap v529: 129 pgs: 129 active+clean; 934 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 631 op/s 2024-06-03T02:20:31.038 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:20:31.709 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:20:32.087 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:31 smithi079 ceph-mon[31819]: pgmap v530: 129 pgs: 129 active+clean; 935 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 552 op/s 2024-06-03T02:20:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:31 smithi135 ceph-mon[36653]: pgmap v530: 129 pgs: 129 active+clean; 935 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 552 op/s 2024-06-03T02:20:32.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:31 smithi123 ceph-mon[36640]: pgmap v530: 129 pgs: 129 active+clean; 935 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 552 op/s 2024-06-03T02:20:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 25.0 s, 1731.4 tps, lat 48.579 ms stddev 59.267 2024-06-03T02:20:34.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:33 smithi079 ceph-mon[31819]: pgmap v531: 129 pgs: 129 active+clean; 939 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 MiB/s wr, 719 op/s 2024-06-03T02:20:34.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:33 smithi135 ceph-mon[36653]: pgmap v531: 129 pgs: 129 active+clean; 939 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 MiB/s wr, 719 op/s 2024-06-03T02:20:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:33 smithi123 ceph-mon[36640]: pgmap v531: 129 pgs: 129 active+clean; 939 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 MiB/s wr, 719 op/s 2024-06-03T02:20:36.021 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:35 smithi079 ceph-mon[31819]: pgmap v532: 129 pgs: 129 active+clean; 940 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 613 op/s 2024-06-03T02:20:36.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:35 smithi135 ceph-mon[36653]: pgmap v532: 129 pgs: 129 active+clean; 940 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 613 op/s 2024-06-03T02:20:36.227 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:35 smithi123 ceph-mon[36640]: pgmap v532: 129 pgs: 129 active+clean; 940 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 613 op/s 2024-06-03T02:20:36.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.363+0000 7fe3cb000700 1 -- 172.21.15.79:0/3263069764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3c406a6e0 msgr2=0x7fe3c406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:36.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.363+0000 7fe3cb000700 1 --2- 172.21.15.79:0/3263069764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3c406a6e0 0x7fe3c406ab40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009990 tx=0x7fe3b0009ca0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.364+0000 7fe3cb000700 1 -- 172.21.15.79:0/3263069764 shutdown_connections 2024-06-03T02:20:36.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.365+0000 7fe3cb000700 1 --2- 172.21.15.79:0/3263069764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c4104ed0 0x7fe3c4105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.365+0000 7fe3cb000700 1 --2- 172.21.15.79:0/3263069764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3c41073c0 0x7fe3c406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.365+0000 7fe3cb000700 1 --2- 172.21.15.79:0/3263069764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3c406a6e0 0x7fe3c406ab40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.365+0000 7fe3cb000700 1 -- 172.21.15.79:0/3263069764 >> 172.21.15.79:0/3263069764 conn(0x7fe3c4078340 msgr2=0x7fe3c4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:36.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.365+0000 7fe3cb000700 1 -- 172.21.15.79:0/3263069764 shutdown_connections 2024-06-03T02:20:36.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.365+0000 7fe3cb000700 1 -- 172.21.15.79:0/3263069764 wait complete. 2024-06-03T02:20:36.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.367+0000 7fe3cb000700 1 Processor -- start 2024-06-03T02:20:36.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.367+0000 7fe3cb000700 1 -- start start 2024-06-03T02:20:36.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.367+0000 7fe3cb000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3c406a6e0 0x7fe3c419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.367+0000 7fe3cb000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3c4104ed0 0x7fe3c419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.367+0000 7fe3cb000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c41073c0 0x7fe3c41a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.367+0000 7fe3cb000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4118e40 con 0x7fe3c406a6e0 2024-06-03T02:20:36.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.367+0000 7fe3cb000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4118cc0 con 0x7fe3c41073c0 2024-06-03T02:20:36.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.367+0000 7fe3cb000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4118fc0 con 0x7fe3c4104ed0 2024-06-03T02:20:36.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.368+0000 7fe3c3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3c406a6e0 0x7fe3c419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.368+0000 7fe3c9000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c41073c0 0x7fe3c41a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.368+0000 7fe3c9000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c41073c0 0x7fe3c41a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60880/0 (socket says 172.21.15.79:60880) 2024-06-03T02:20:36.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.368+0000 7fe3c9000700 1 -- 172.21.15.79:0/1848726567 learned_addr learned my addr 172.21.15.79:0/1848726567 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:36.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.368+0000 7fe3c3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3c4104ed0 0x7fe3c419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.369+0000 7fe3c9000700 1 -- 172.21.15.79:0/1848726567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3c4104ed0 msgr2=0x7fe3c419f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:36.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.369+0000 7fe3c9000700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3c4104ed0 0x7fe3c419f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.369+0000 7fe3c9000700 1 -- 172.21.15.79:0/1848726567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3c406a6e0 msgr2=0x7fe3c419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:36.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.369+0000 7fe3c9000700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3c406a6e0 0x7fe3c419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.369+0000 7fe3c9000700 1 -- 172.21.15.79:0/1848726567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c41a4370 con 0x7fe3c41073c0 2024-06-03T02:20:36.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.369+0000 7fe3c3400700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3c4104ed0 0x7fe3c419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:20:36.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.369+0000 7fe3c9000700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c41073c0 0x7fe3c41a3c30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00ada0 tx=0x7fe3bc00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:36.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.370+0000 7fe3c0c00700 1 -- 172.21.15.79:0/1848726567 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc017020 con 0x7fe3c41073c0 2024-06-03T02:20:36.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.370+0000 7fe3c0c00700 1 -- 172.21.15.79:0/1848726567 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3bc004bb0 con 0x7fe3c41073c0 2024-06-03T02:20:36.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.370+0000 7fe3c0c00700 1 -- 172.21.15.79:0/1848726567 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc005620 con 0x7fe3c41073c0 2024-06-03T02:20:36.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.370+0000 7fe3cb000700 1 -- 172.21.15.79:0/1848726567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c41a4660 con 0x7fe3c41073c0 2024-06-03T02:20:36.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.371+0000 7fe3cb000700 1 -- 172.21.15.79:0/1848726567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c406beb0 con 0x7fe3c41073c0 2024-06-03T02:20:36.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.376+0000 7fe3c0c00700 1 -- 172.21.15.79:0/1848726567 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fe3bc01a070 con 0x7fe3c41073c0 2024-06-03T02:20:36.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.377+0000 7fe3c0c00700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3a80a2d30 0x7fe3a80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.377+0000 7fe3c0c00700 1 -- 172.21.15.79:0/1848726567 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe3bc0ce4e0 con 0x7fe3c41073c0 2024-06-03T02:20:36.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.379+0000 7fe3c3e00700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3a80a2d30 0x7fe3a80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.380+0000 7fe3d59ecb80 1 Processor -- start 2024-06-03T02:20:36.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.380+0000 7fe3c3e00700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3a80a2d30 0x7fe3a80a51f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009910 tx=0x7fe3b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:36.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.380+0000 7fe3d59ecb80 1 -- start start 2024-06-03T02:20:36.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.381+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65ad9e40 0x55ff65b5d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.381+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b5da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.381+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b6a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.381+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ff65a6ca10 con 0x55ff65ad9e40 2024-06-03T02:20:36.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.381+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ff65aafb00 con 0x55ff65b5d5b0 2024-06-03T02:20:36.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.381+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ff65a6a8f0 con 0x55ff65b68300 2024-06-03T02:20:36.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.381+0000 7fe3c9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b6a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b5da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b6a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53584/0 (socket says 172.21.15.79:53584) 2024-06-03T02:20:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c9000700 1 -- 172.21.15.79:0/3339758832 learned_addr learned my addr 172.21.15.79:0/3339758832 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c3e00700 1 --2- 172.21.15.79:0/3339758832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65ad9e40 0x55ff65b5d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c9000700 1 -- 172.21.15.79:0/3339758832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 msgr2=0x55ff65b5da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c9000700 1 --2- 172.21.15.79:0/3339758832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b5da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c9000700 1 -- 172.21.15.79:0/3339758832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65ad9e40 msgr2=0x55ff65b5d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c9000700 1 --2- 172.21.15.79:0/3339758832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65ad9e40 0x55ff65b5d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c9000700 1 -- 172.21.15.79:0/3339758832 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff65a57f90 con 0x55ff65b68300 2024-06-03T02:20:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.382+0000 7fe3c9000700 1 --2- 172.21.15.79:0/3339758832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b6a6f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0cbb10 tx=0x7fe3bc00a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75661fc531c90042 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:36.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.383+0000 7fe3b9600700 1 -- 172.21.15.79:0/3339758832 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc09ebe0 con 0x55ff65b68300 2024-06-03T02:20:36.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.383+0000 7fe3b9600700 1 -- 172.21.15.79:0/3339758832 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3bc0a34a0 con 0x55ff65b68300 2024-06-03T02:20:36.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.383+0000 7fe3b9600700 1 -- 172.21.15.79:0/3339758832 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc09aa90 con 0x55ff65b68300 2024-06-03T02:20:36.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.384+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3339758832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 msgr2=0x55ff65b6a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:36.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.384+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3339758832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b6a6f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0cbb10 tx=0x7fe3bc00a2e0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.385+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3339758832 shutdown_connections 2024-06-03T02:20:36.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.385+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3339758832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b6a6f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.385+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3339758832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65ad9e40 0x55ff65b5d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.385+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3339758832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b5da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.385+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3339758832 >> 172.21.15.79:0/3339758832 conn(0x55ff65a8a170 msgr2=0x55ff65b594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:36.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.386+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3339758832 shutdown_connections 2024-06-03T02:20:36.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.386+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3339758832 wait complete. 2024-06-03T02:20:36.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.387+0000 7fe3d59ecb80 1 -- start start 2024-06-03T02:20:36.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.387+0000 7fe3d59ecb80 1 Processor -- start 2024-06-03T02:20:36.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.389+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 0x55ff65b679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.389+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b64ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.389+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b65220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.389+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ff65a4bf60 con 0x55ff65b5d5b0 2024-06-03T02:20:36.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.389+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ff65aabac0 con 0x55ff65ad9e40 2024-06-03T02:20:36.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.390+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ff65b6afc0 con 0x55ff65b68300 2024-06-03T02:20:36.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.390+0000 7fe3c9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b65220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.390+0000 7fe3c9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b65220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53590/0 (socket says 172.21.15.79:53590) 2024-06-03T02:20:36.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.390+0000 7fe3c9000700 1 -- 172.21.15.79:0/38290914 learned_addr learned my addr 172.21.15.79:0/38290914 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:36.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.390+0000 7fe3c3400700 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b64ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.390+0000 7fe3d59ecb80 10 client.0 _ll_register_callbacks cb 0x55ff65a49720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.390+0000 7fe3c3e00700 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 0x55ff65b679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.391+0000 7fe3c9000700 1 -- 172.21.15.79:0/38290914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 msgr2=0x55ff65b679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.391+0000 7fe3c9000700 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 0x55ff65b679e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.391+0000 7fe3c9000700 1 -- 172.21.15.79:0/38290914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 msgr2=0x55ff65b64ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.391+0000 7fe3c9000700 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b64ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.391+0000 7fe3c9000700 1 -- 172.21.15.79:0/38290914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff659fe8d0 con 0x55ff65b68300 2024-06-03T02:20:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.391+0000 7fe3c3400700 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b64ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:20:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.392+0000 7fe3ae000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.392+0000 7fe3c9000700 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b65220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0a2f70 tx=0x7fe3bc0cbf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673dc4a3737c33e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.392+0000 7fe3ae000700 1 -- 172.21.15.79:0/38290914 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc09aa90 con 0x55ff65b68300 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.392+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/38290914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3bc0a34a0 con 0x55ff65b68300 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.392+0000 7fe3ae000700 1 -- 172.21.15.79:0/38290914 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe3bc0b8020 con 0x55ff65b68300 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3ae000700 1 -- 172.21.15.79:0/38290914 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3bc09aa90 con 0x55ff65b68300 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3d59ecb80 10 client.25351 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/38290914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff65b75480 con 0x55ff65b68300 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3ae000700 1 -- 172.21.15.79:0/38290914 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3bc0a34a0 con 0x55ff65b68300 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3ae000700 1 -- 172.21.15.79:0/38290914 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe3bc012070 con 0x55ff65b68300 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3d59ecb80 10 client.25351 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3d59ecb80 10 client.25351 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:20:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3d59ecb80 20 client.25351 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:36.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff65b77950 0x55ff65b79d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3d59ecb80 4 client.25351 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:20:36.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.393+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/38290914 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff65abd000 con 0x55ff65b77950 2024-06-03T02:20:36.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.394+0000 7fe3c3e00700 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff65b77950 0x55ff65b79d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.394+0000 7fe3ae000700 10 client.25351 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:20:36.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.395+0000 7fe3c3e00700 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff65b77950 0x55ff65b79d40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17b61e6b8c36278c server_cookie=166f652b7cf8d9bd in_seq=0 out_seq=0 2024-06-03T02:20:36.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.396+0000 7fe3ae000700 1 -- 172.21.15.79:0/38290914 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ff65abd000 con 0x55ff65b77950 2024-06-03T02:20:36.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.396+0000 7fe3ae000700 10 client.25351 handle_command_reply: tid=0 2024-06-03T02:20:36.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.396+0000 7fe3d59ecb80 1 client.25351 shutdown 2024-06-03T02:20:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.397+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/38290914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 msgr2=0x55ff65b65220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.397+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b65220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0a2f70 tx=0x7fe3bc0cbf60 comp rx=0 tx=0).stop 2024-06-03T02:20:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.397+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/38290914 shutdown_connections 2024-06-03T02:20:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.397+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65b65220 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.397+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 0x55ff65b679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.397+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff65b77950 0x55ff65b79d40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.397+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/38290914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b64ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.397+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/38290914 >> 172.21.15.79:0/38290914 conn(0x55ff65a8a170 msgr2=0x55ff65b58740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.397+0000 7fe3ae000700 0 client.25351 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:20:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.398+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/38290914 shutdown_connections 2024-06-03T02:20:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.399+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/38290914 wait complete. 2024-06-03T02:20:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.399+0000 7fe3d59ecb80 20 client.25351 trim_cache size 0 max 16384 2024-06-03T02:20:36.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.421+0000 7fe3d59ecb80 1 Processor -- start 2024-06-03T02:20:36.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.421+0000 7fe3d59ecb80 1 -- start start 2024-06-03T02:20:36.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.422+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65ad9e40 0x55ff65b696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.422+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b58fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.422+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b68300 0x55ff65b594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.422+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41bd410 con 0x55ff65b68300 2024-06-03T02:20:36.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.422+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41bef50 con 0x55ff65b5d5b0 2024-06-03T02:20:36.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.422+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41bdae0 con 0x55ff65ad9e40 2024-06-03T02:20:36.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.422+0000 7fe3c3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b58fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.422+0000 7fe3c3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b58fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60912/0 (socket says 172.21.15.79:60912) 2024-06-03T02:20:36.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.423+0000 7fe3c3400700 1 -- 172.21.15.79:0/3981561576 learned_addr learned my addr 172.21.15.79:0/3981561576 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:36.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.422+0000 7fe3c9000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b68300 0x55ff65b594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.423+0000 7fe3c3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65ad9e40 0x55ff65b696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.424+0000 7fe3c3400700 1 -- 172.21.15.79:0/3981561576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65ad9e40 msgr2=0x55ff65b696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:36.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.424+0000 7fe3c3400700 1 --2- 172.21.15.79:0/3981561576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65ad9e40 0x55ff65b696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.424+0000 7fe3c3400700 1 -- 172.21.15.79:0/3981561576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b68300 msgr2=0x55ff65b594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:36.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.424+0000 7fe3c3400700 1 --2- 172.21.15.79:0/3981561576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b68300 0x55ff65b594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.424+0000 7fe3c3400700 1 -- 172.21.15.79:0/3981561576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff65b5eb00 con 0x55ff65b5d5b0 2024-06-03T02:20:36.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.424+0000 7fe3c9000700 1 --2- 172.21.15.79:0/3981561576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b68300 0x55ff65b594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:20:36.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.425+0000 7fe3c3400700 1 --2- 172.21.15.79:0/3981561576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b58fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009990 tx=0x7fe3b00096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a795b3040e43cb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:36.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.425+0000 7fe3b8c00700 1 -- 172.21.15.79:0/3981561576 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00050c0 con 0x55ff65b5d5b0 2024-06-03T02:20:36.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.425+0000 7fe3b8c00700 1 -- 172.21.15.79:0/3981561576 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b000b7d0 con 0x55ff65b5d5b0 2024-06-03T02:20:36.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.425+0000 7fe3b8c00700 1 -- 172.21.15.79:0/3981561576 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00246d0 con 0x55ff65b5d5b0 2024-06-03T02:20:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.426+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3981561576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 msgr2=0x55ff65b58fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.426+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3981561576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b58fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009990 tx=0x7fe3b00096b0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.426+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3981561576 shutdown_connections 2024-06-03T02:20:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.426+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3981561576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b68300 0x55ff65b594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.426+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3981561576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65ad9e40 0x55ff65b696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.426+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3981561576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65b5d5b0 0x55ff65b58fb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.426+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3981561576 >> 172.21.15.79:0/3981561576 conn(0x55ff65a8a170 msgr2=0x55ff65b6a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.427+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3981561576 shutdown_connections 2024-06-03T02:20:36.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.427+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3981561576 wait complete. 2024-06-03T02:20:36.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.428+0000 7fe3d59ecb80 1 -- start start 2024-06-03T02:20:36.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.428+0000 7fe3d59ecb80 1 Processor -- start 2024-06-03T02:20:36.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.429+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 0x55ff65b77f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.429+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b78450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.429+0000 7fe3d59ecb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65be0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41be880 con 0x55ff65b5d5b0 2024-06-03T02:20:36.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41be1b0 con 0x55ff65ad9e40 2024-06-03T02:20:36.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3d59ecb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4072b20 con 0x55ff65b68300 2024-06-03T02:20:36.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3d59ecb80 10 client.0 _ll_register_callbacks cb 0x55ff65a49720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:36.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 0x55ff65b77f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 0x55ff65b77f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60918/0 (socket says 172.21.15.79:60918) 2024-06-03T02:20:36.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3c3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b78450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3c3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b78450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54422/0 (socket says 172.21.15.79:54422) 2024-06-03T02:20:36.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3c3400700 1 -- 172.21.15.79:0/3656288442 learned_addr learned my addr 172.21.15.79:0/3656288442 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:36.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.430+0000 7fe3c9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65be0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.431+0000 7fe3c3400700 1 -- 172.21.15.79:0/3656288442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 msgr2=0x55ff65be0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:36.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.431+0000 7fe3c3400700 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65be0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.431+0000 7fe3c3400700 1 -- 172.21.15.79:0/3656288442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 msgr2=0x55ff65b77f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:36.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.431+0000 7fe3c3400700 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 0x55ff65b77f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.431+0000 7fe3c3400700 1 -- 172.21.15.79:0/3656288442 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff65be07b0 con 0x55ff65b5d5b0 2024-06-03T02:20:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.432+0000 7fe3c3400700 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b78450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00098e0 tx=0x7fe3b0005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0a72d313c7da77 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.432+0000 7fe3ae000700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:20:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.432+0000 7fe3ae000700 1 -- 172.21.15.79:0/3656288442 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b002c060 con 0x55ff65b5d5b0 2024-06-03T02:20:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.433+0000 7fe3ae000700 1 -- 172.21.15.79:0/3656288442 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fe3b0003e00 con 0x55ff65b5d5b0 2024-06-03T02:20:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.433+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3656288442 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b000b7d0 con 0x55ff65b5d5b0 2024-06-03T02:20:36.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.433+0000 7fe3ae000700 1 -- 172.21.15.79:0/3656288442 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3b000f460 con 0x55ff65b5d5b0 2024-06-03T02:20:36.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.433+0000 7fe3d59ecb80 10 client.15252 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:36.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.433+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3656288442 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff65be18d0 con 0x55ff65b5d5b0 2024-06-03T02:20:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.434+0000 7fe3ae000700 1 -- 172.21.15.79:0/3656288442 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b000b7d0 con 0x55ff65b5d5b0 2024-06-03T02:20:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.434+0000 7fe3ae000700 1 -- 172.21.15.79:0/3656288442 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe3b00249d0 con 0x55ff65b5d5b0 2024-06-03T02:20:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.434+0000 7fe3d59ecb80 10 client.15252 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.434+0000 7fe3d59ecb80 10 client.15252 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:20:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.434+0000 7fe3d59ecb80 20 client.15252 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.434+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff65be1db0 0x55ff65be41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.435+0000 7fe3d59ecb80 4 client.15252 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:20:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.435+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3656288442 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff65ab7580 con 0x55ff65be1db0 2024-06-03T02:20:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.435+0000 7fe3c3e00700 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff65be1db0 0x55ff65be41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.436+0000 7fe3c3e00700 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff65be1db0 0x55ff65be41d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27e8d8172fe2e7e2 server_cookie=ffce0fb3e39d7e8f in_seq=0 out_seq=0 2024-06-03T02:20:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.436+0000 7fe3ae000700 10 client.15252 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:20:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.436+0000 7fe3ae000700 1 -- 172.21.15.79:0/3656288442 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff65ab7580 con 0x55ff65be1db0 2024-06-03T02:20:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.436+0000 7fe3ae000700 10 client.15252 handle_command_reply: tid=0 2024-06-03T02:20:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.436+0000 7fe3d59ecb80 1 client.15252 shutdown 2024-06-03T02:20:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.436+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3656288442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 msgr2=0x55ff65b78450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.436+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b78450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00098e0 tx=0x7fe3b0005280 comp rx=0 tx=0).stop 2024-06-03T02:20:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3656288442 shutdown_connections 2024-06-03T02:20:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff65b68300 0x55ff65be0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff65ad9e40 0x55ff65b77f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff65be1db0 0x55ff65be41d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3d59ecb80 1 --2- 172.21.15.79:0/3656288442 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff65b5d5b0 0x55ff65b78450 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3656288442 >> 172.21.15.79:0/3656288442 conn(0x55ff65a8a170 msgr2=0x55ff65b635d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:36.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3ae000700 0 client.15252 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:20:36.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3656288442 shutdown_connections 2024-06-03T02:20:36.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3d59ecb80 1 -- 172.21.15.79:0/3656288442 wait complete. 2024-06-03T02:20:36.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.437+0000 7fe3d59ecb80 20 client.15252 trim_cache size 0 max 16384 2024-06-03T02:20:36.439 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:20:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.439+0000 7fe3cb000700 1 -- 172.21.15.79:0/1848726567 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3a80a2d30 msgr2=0x7fe3a80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.439+0000 7fe3cb000700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3a80a2d30 0x7fe3a80a51f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009910 tx=0x7fe3b4008040 comp rx=0 tx=0).stop 2024-06-03T02:20:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.439+0000 7fe3cb000700 1 -- 172.21.15.79:0/1848726567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c41073c0 msgr2=0x7fe3c41a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.439+0000 7fe3cb000700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c41073c0 0x7fe3c41a3c30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00ada0 tx=0x7fe3bc00c330 comp rx=0 tx=0).stop 2024-06-03T02:20:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.440+0000 7fe3cb000700 1 -- 172.21.15.79:0/1848726567 shutdown_connections 2024-06-03T02:20:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.440+0000 7fe3cb000700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c41073c0 0x7fe3c41a3c30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.440+0000 7fe3cb000700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fe3a80a2d30 0x7fe3a80a51f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.440+0000 7fe3cb000700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3c406a6e0 0x7fe3c419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.440+0000 7fe3cb000700 1 --2- 172.21.15.79:0/1848726567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3c4104ed0 0x7fe3c419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:36.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.440+0000 7fe3cb000700 1 -- 172.21.15.79:0/1848726567 >> 172.21.15.79:0/1848726567 conn(0x7fe3c4078340 msgr2=0x7fe3c4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:36.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.440+0000 7fe3cb000700 1 -- 172.21.15.79:0/1848726567 shutdown_connections 2024-06-03T02:20:36.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:36.440+0000 7fe3cb000700 1 -- 172.21.15.79:0/1848726567 wait complete. 2024-06-03T02:20:37.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:20:38.058 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:20:38.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:37 smithi079 ceph-mon[31819]: pgmap v533: 129 pgs: 129 active+clean; 961 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 691 op/s 2024-06-03T02:20:38.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:37 smithi135 ceph-mon[36653]: pgmap v533: 129 pgs: 129 active+clean; 961 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 691 op/s 2024-06-03T02:20:38.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:37 smithi123 ceph-mon[36640]: pgmap v533: 129 pgs: 129 active+clean; 961 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 691 op/s 2024-06-03T02:20:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 30.0 s, 1956.6 tps, lat 59.066 ms stddev 111.844 2024-06-03T02:20:40.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:39 smithi079 ceph-mon[31819]: pgmap v534: 129 pgs: 129 active+clean; 971 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 728 op/s 2024-06-03T02:20:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:39 smithi135 ceph-mon[36653]: pgmap v534: 129 pgs: 129 active+clean; 971 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 728 op/s 2024-06-03T02:20:40.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:39 smithi123 ceph-mon[36640]: pgmap v534: 129 pgs: 129 active+clean; 971 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 728 op/s 2024-06-03T02:20:41.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:41 smithi079 ceph-mon[31819]: pgmap v535: 129 pgs: 129 active+clean; 976 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 673 op/s 2024-06-03T02:20:42.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:41 smithi135 ceph-mon[36653]: pgmap v535: 129 pgs: 129 active+clean; 976 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 673 op/s 2024-06-03T02:20:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:41 smithi123 ceph-mon[36640]: pgmap v535: 129 pgs: 129 active+clean; 976 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 673 op/s 2024-06-03T02:20:42.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.541+0000 7fab04600700 1 -- 172.21.15.79:0/4000082872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc10ef40 msgr2=0x7faafc111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.541+0000 7fab04600700 1 --2- 172.21.15.79:0/4000082872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc10ef40 0x7faafc111330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b0a0 tx=0x7faaf400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.541+0000 7fab04600700 1 -- 172.21.15.79:0/4000082872 shutdown_connections 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.541+0000 7fab04600700 1 --2- 172.21.15.79:0/4000082872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc10ef40 0x7faafc111330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.541+0000 7fab04600700 1 --2- 172.21.15.79:0/4000082872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faafc06a1a0 0x7faafc06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.541+0000 7fab04600700 1 --2- 172.21.15.79:0/4000082872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc06aac0 0x7faafc10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.541+0000 7fab04600700 1 -- 172.21.15.79:0/4000082872 >> 172.21.15.79:0/4000082872 conn(0x7faafc078340 msgr2=0x7faafc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.541+0000 7fab04600700 1 -- 172.21.15.79:0/4000082872 shutdown_connections 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.541+0000 7fab04600700 1 -- 172.21.15.79:0/4000082872 wait complete. 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.543+0000 7fab04600700 1 Processor -- start 2024-06-03T02:20:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.543+0000 7fab04600700 1 -- start start 2024-06-03T02:20:42.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.543+0000 7fab04600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc06a1a0 0x7faafc109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.543+0000 7fab04600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc06aac0 0x7faafc10a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.543+0000 7fab04600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faafc10ef40 0x7faafc105520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.543+0000 7fab04600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faafc104d80 con 0x7faafc10ef40 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.543+0000 7fab04600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faafc104c00 con 0x7faafc06aac0 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.543+0000 7fab04600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faafc104f00 con 0x7faafc06a1a0 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.543+0000 7fab01c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc06a1a0 0x7faafc109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.544+0000 7fab01c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc06a1a0 0x7faafc109f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53632/0 (socket says 172.21.15.79:53632) 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.544+0000 7fab01c00700 1 -- 172.21.15.79:0/2704865787 learned_addr learned my addr 172.21.15.79:0/2704865787 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.544+0000 7fab02600700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faafc10ef40 0x7faafc105520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.544+0000 7fab01200700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc06aac0 0x7faafc10a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.544+0000 7fab01c00700 1 -- 172.21.15.79:0/2704865787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc06aac0 msgr2=0x7faafc10a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.544+0000 7fab01c00700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc06aac0 0x7faafc10a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.544+0000 7fab01c00700 1 -- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faafc10ef40 msgr2=0x7faafc105520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.544+0000 7fab01c00700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faafc10ef40 0x7faafc105520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.544+0000 7fab01c00700 1 -- 172.21.15.79:0/2704865787 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc105e20 con 0x7faafc06a1a0 2024-06-03T02:20:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.545+0000 7fab01c00700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc06a1a0 0x7faafc109f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faaf000d980 tx=0x7faaf000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.545+0000 7faafa000700 1 -- 172.21.15.79:0/2704865787 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0017070 con 0x7faafc06a1a0 2024-06-03T02:20:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.545+0000 7faafa000700 1 -- 172.21.15.79:0/2704865787 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf0004d10 con 0x7faafc06a1a0 2024-06-03T02:20:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.545+0000 7faafa000700 1 -- 172.21.15.79:0/2704865787 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf00055c0 con 0x7faafc06a1a0 2024-06-03T02:20:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.546+0000 7fab04600700 1 -- 172.21.15.79:0/2704865787 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc106110 con 0x7faafc06a1a0 2024-06-03T02:20:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.546+0000 7fab04600700 1 -- 172.21.15.79:0/2704865787 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc10de00 con 0x7faafc06a1a0 2024-06-03T02:20:42.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.553+0000 7faafa000700 1 -- 172.21.15.79:0/2704865787 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7faaf0004750 con 0x7faafc06a1a0 2024-06-03T02:20:42.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.555+0000 7faafa000700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7faae40a2df0 0x7faae40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.556+0000 7faafa000700 1 -- 172.21.15.79:0/2704865787 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7faaf00cf260 con 0x7faafc06a1a0 2024-06-03T02:20:42.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.556+0000 7fab01200700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7faae40a2df0 0x7faae40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.557+0000 7fab01200700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7faae40a2df0 0x7faae40a52b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faae8005fd0 tx=0x7faae800af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:42.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.557+0000 7fab0f053b80 1 Processor -- start 2024-06-03T02:20:42.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.558+0000 7fab0f053b80 1 -- start start 2024-06-03T02:20:42.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.558+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c57b90 0x565050c57f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.558+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.558+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c52240 0x565050c52680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.558+0000 7fab0f053b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565050b60a10 con 0x565050c584b0 2024-06-03T02:20:42.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.558+0000 7fab0f053b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565050ba3b00 con 0x565050c52240 2024-06-03T02:20:42.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.558+0000 7fab0f053b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565050b5e8f0 con 0x565050c57b90 2024-06-03T02:20:42.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.558+0000 7fab01c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c57b90 0x565050c57f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c57b90 0x565050c57f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53646/0 (socket says 172.21.15.79:53646) 2024-06-03T02:20:42.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01c00700 1 -- 172.21.15.79:0/4259249139 learned_addr learned my addr 172.21.15.79:0/4259249139 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:42.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01200700 1 --2- 172.21.15.79:0/4259249139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01c00700 1 -- 172.21.15.79:0/4259249139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c52240 msgr2=0x565050c52680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:42.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01c00700 1 --2- 172.21.15.79:0/4259249139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c52240 0x565050c52680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01c00700 1 -- 172.21.15.79:0/4259249139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 msgr2=0x565050c51d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:42.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01c00700 1 --2- 172.21.15.79:0/4259249139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c51d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01c00700 1 -- 172.21.15.79:0/4259249139 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565050b4bf90 con 0x565050c57b90 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01200700 1 --2- 172.21.15.79:0/4259249139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7fab01c00700 1 --2- 172.21.15.79:0/4259249139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c57b90 0x565050c57f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faaf00a3c90 tx=0x7faaf000de60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=456ec188bf0e18c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7faaeea00700 1 -- 172.21.15.79:0/4259249139 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf00b4070 con 0x565050c57b90 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7faaeea00700 1 -- 172.21.15.79:0/4259249139 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf009b550 con 0x565050c57b90 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.559+0000 7faaeea00700 1 -- 172.21.15.79:0/4259249139 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf00a3d30 con 0x565050c57b90 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.560+0000 7fab0f053b80 1 -- 172.21.15.79:0/4259249139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c57b90 msgr2=0x565050c57f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.560+0000 7fab0f053b80 1 --2- 172.21.15.79:0/4259249139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c57b90 0x565050c57f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faaf00a3c90 tx=0x7faaf000de60 comp rx=0 tx=0).stop 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.561+0000 7fab0f053b80 1 -- 172.21.15.79:0/4259249139 shutdown_connections 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.561+0000 7fab0f053b80 1 --2- 172.21.15.79:0/4259249139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c52240 0x565050c52680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.561+0000 7fab0f053b80 1 --2- 172.21.15.79:0/4259249139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c57b90 0x565050c57f70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.561+0000 7fab0f053b80 1 --2- 172.21.15.79:0/4259249139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.561+0000 7fab0f053b80 1 -- 172.21.15.79:0/4259249139 >> 172.21.15.79:0/4259249139 conn(0x565050b7e170 msgr2=0x565050c4cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.561+0000 7fab0f053b80 1 -- 172.21.15.79:0/4259249139 shutdown_connections 2024-06-03T02:20:42.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.561+0000 7fab0f053b80 1 -- 172.21.15.79:0/4259249139 wait complete. 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.562+0000 7fab0f053b80 1 -- start start 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.563+0000 7fab0f053b80 1 Processor -- start 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.563+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c52240 0x565050c5e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.563+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c5b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.563+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c5bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.563+0000 7fab0f053b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565050b3ff60 con 0x565050c52240 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.563+0000 7fab0f053b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565050b9fac0 con 0x565050c57b90 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.563+0000 7fab0f053b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565050c5ee30 con 0x565050c584b0 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.563+0000 7fab0f053b80 10 client.0 _ll_register_callbacks cb 0x565050b39380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab01c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c52240 0x565050c5e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab02600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c5bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab02600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c5bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53652/0 (socket says 172.21.15.79:53652) 2024-06-03T02:20:42.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab02600700 1 -- 172.21.15.79:0/3685504781 learned_addr learned my addr 172.21.15.79:0/3685504781 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:42.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab01200700 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c5b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab02600700 1 -- 172.21.15.79:0/3685504781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 msgr2=0x565050c5b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:42.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab02600700 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c5b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab02600700 1 -- 172.21.15.79:0/3685504781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c52240 msgr2=0x565050c5e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:42.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab02600700 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c52240 0x565050c5e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.564+0000 7fab02600700 1 -- 172.21.15.79:0/3685504781 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565050af28d0 con 0x565050c584b0 2024-06-03T02:20:42.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.565+0000 7fab02600700 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c5bb30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b070 tx=0x7faaf40095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49fc208267857fb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:42.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.565+0000 7faae3400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:20:42.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.565+0000 7faae3400700 1 -- 172.21.15.79:0/3685504781 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf401f030 con 0x565050c584b0 2024-06-03T02:20:42.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.565+0000 7faae3400700 1 -- 172.21.15.79:0/3685504781 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7faaf4009bc0 con 0x565050c584b0 2024-06-03T02:20:42.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.566+0000 7fab0f053b80 1 -- 172.21.15.79:0/3685504781 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf009b550 con 0x565050c584b0 2024-06-03T02:20:42.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.566+0000 7faae3400700 1 -- 172.21.15.79:0/3685504781 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faaf40074e0 con 0x565050c584b0 2024-06-03T02:20:42.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.566+0000 7fab0f053b80 10 client.25369 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:42.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.566+0000 7fab0f053b80 1 -- 172.21.15.79:0/3685504781 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565050c695d0 con 0x565050c584b0 2024-06-03T02:20:42.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.567+0000 7faae3400700 1 -- 172.21.15.79:0/3685504781 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf009b550 con 0x565050c584b0 2024-06-03T02:20:42.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.567+0000 7faae3400700 1 -- 172.21.15.79:0/3685504781 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7faaf4011070 con 0x565050c584b0 2024-06-03T02:20:42.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.568+0000 7fab0f053b80 10 client.25369 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:42.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.568+0000 7fab0f053b80 10 client.25369 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:20:42.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.568+0000 7fab0f053b80 20 client.25369 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:42.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.568+0000 7fab0f053b80 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x565050c6bc00 0x565050c6e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.568+0000 7fab0f053b80 4 client.25369 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:20:42.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.568+0000 7fab0f053b80 1 -- 172.21.15.79:0/3685504781 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565050bb1000 con 0x565050c6bc00 2024-06-03T02:20:42.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.569+0000 7fab01c00700 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x565050c6bc00 0x565050c6e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.569+0000 7fab01c00700 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x565050c6bc00 0x565050c6e010 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdca1761e6654be8 server_cookie=a88fa65a3ba9e4c in_seq=0 out_seq=0 2024-06-03T02:20:42.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.569+0000 7faae3400700 10 client.25369 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:20:42.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.570+0000 7faae3400700 1 -- 172.21.15.79:0/3685504781 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565050bb1000 con 0x565050c6bc00 2024-06-03T02:20:42.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.570+0000 7faae3400700 10 client.25369 handle_command_reply: tid=0 2024-06-03T02:20:42.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7fab0f053b80 1 client.25369 shutdown 2024-06-03T02:20:42.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7fab0f053b80 1 -- 172.21.15.79:0/3685504781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 msgr2=0x565050c5bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:42.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7fab0f053b80 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c5bb30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b070 tx=0x7faaf40095d0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7fab0f053b80 1 -- 172.21.15.79:0/3685504781 shutdown_connections 2024-06-03T02:20:42.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7fab0f053b80 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c5bb30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7fab0f053b80 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c52240 0x565050c5e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7fab0f053b80 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x565050c6bc00 0x565050c6e010 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7fab0f053b80 1 --2- 172.21.15.79:0/3685504781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c5b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7faae3400700 0 client.25369 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:20:42.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.571+0000 7fab0f053b80 1 -- 172.21.15.79:0/3685504781 >> 172.21.15.79:0/3685504781 conn(0x565050b7e170 msgr2=0x565050c46790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:42.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.572+0000 7fab0f053b80 1 -- 172.21.15.79:0/3685504781 shutdown_connections 2024-06-03T02:20:42.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.572+0000 7fab0f053b80 1 -- 172.21.15.79:0/3685504781 wait complete. 2024-06-03T02:20:42.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.573+0000 7fab0f053b80 20 client.25369 trim_cache size 0 max 16384 2024-06-03T02:20:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.604+0000 7fab0f053b80 1 Processor -- start 2024-06-03T02:20:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.604+0000 7fab0f053b80 1 -- start start 2024-06-03T02:20:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.605+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c52240 0x565050c67f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.605+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c57b90 0x565050c51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.605+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c47130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.605+0000 7fab0f053b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1bd130 con 0x565050c57b90 2024-06-03T02:20:42.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.605+0000 7fab0f053b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1bec70 con 0x565050c52240 2024-06-03T02:20:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.605+0000 7fab0f053b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1bd800 con 0x565050c584b0 2024-06-03T02:20:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.606+0000 7fab02600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c47130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.606+0000 7fab02600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c47130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53660/0 (socket says 172.21.15.79:53660) 2024-06-03T02:20:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.606+0000 7fab02600700 1 -- 172.21.15.79:0/2549453570 learned_addr learned my addr 172.21.15.79:0/2549453570 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.607+0000 7fab02600700 1 -- 172.21.15.79:0/2549453570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c52240 msgr2=0x565050c67f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.607+0000 7fab02600700 1 --2- 172.21.15.79:0/2549453570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c52240 0x565050c67f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.607+0000 7fab02600700 1 -- 172.21.15.79:0/2549453570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c57b90 msgr2=0x565050c51d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.607+0000 7fab02600700 1 --2- 172.21.15.79:0/2549453570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c57b90 0x565050c51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.607+0000 7fab02600700 1 -- 172.21.15.79:0/2549453570 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565050c6e690 con 0x565050c584b0 2024-06-03T02:20:42.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.607+0000 7fab02600700 1 --2- 172.21.15.79:0/2549453570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c47130 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faaf40062a0 tx=0x7faaf4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f859e2c7cb85e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:42.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.608+0000 7faaee000700 1 -- 172.21.15.79:0/2549453570 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf401f070 con 0x565050c584b0 2024-06-03T02:20:42.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.608+0000 7faaee000700 1 -- 172.21.15.79:0/2549453570 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf40058d0 con 0x565050c584b0 2024-06-03T02:20:42.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.608+0000 7faaee000700 1 -- 172.21.15.79:0/2549453570 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4007be0 con 0x565050c584b0 2024-06-03T02:20:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.609+0000 7fab0f053b80 1 -- 172.21.15.79:0/2549453570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 msgr2=0x565050c47130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.609+0000 7fab0f053b80 1 --2- 172.21.15.79:0/2549453570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c47130 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faaf40062a0 tx=0x7faaf4009c00 comp rx=0 tx=0).stop 2024-06-03T02:20:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.610+0000 7fab0f053b80 1 -- 172.21.15.79:0/2549453570 shutdown_connections 2024-06-03T02:20:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.610+0000 7fab0f053b80 1 --2- 172.21.15.79:0/2549453570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c584b0 0x565050c47130 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.610+0000 7fab0f053b80 1 --2- 172.21.15.79:0/2549453570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c52240 0x565050c67f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.610+0000 7fab0f053b80 1 --2- 172.21.15.79:0/2549453570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c57b90 0x565050c51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.610+0000 7fab0f053b80 1 -- 172.21.15.79:0/2549453570 >> 172.21.15.79:0/2549453570 conn(0x565050b7e170 msgr2=0x565050c5a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.610+0000 7fab0f053b80 1 -- 172.21.15.79:0/2549453570 shutdown_connections 2024-06-03T02:20:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.610+0000 7fab0f053b80 1 -- 172.21.15.79:0/2549453570 wait complete. 2024-06-03T02:20:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.611+0000 7fab0f053b80 1 -- start start 2024-06-03T02:20:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.611+0000 7fab0f053b80 1 Processor -- start 2024-06-03T02:20:42.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.612+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c52240 0x565050c64510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.612+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c72eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.613+0000 7fab0f053b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.613+0000 7fab0f053b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1be5a0 con 0x565050c584b0 2024-06-03T02:20:42.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.613+0000 7fab0f053b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1bded0 con 0x565050c57b90 2024-06-03T02:20:42.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.613+0000 7fab0f053b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1bb0f0 con 0x565050c52240 2024-06-03T02:20:42.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.613+0000 7fab0f053b80 10 client.0 _ll_register_callbacks cb 0x565050b39380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.613+0000 7fab01200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c72eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.613+0000 7fab02600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.614+0000 7fab01200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c72eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60962/0 (socket says 172.21.15.79:60962) 2024-06-03T02:20:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.614+0000 7fab02600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54480/0 (socket says 172.21.15.79:54480) 2024-06-03T02:20:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.614+0000 7fab01200700 1 -- 172.21.15.79:0/1390486576 learned_addr learned my addr 172.21.15.79:0/1390486576 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.615+0000 7fab01c00700 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c52240 0x565050c64510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.615+0000 7fab01200700 1 -- 172.21.15.79:0/1390486576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c52240 msgr2=0x565050c64510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:42.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.615+0000 7fab01200700 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c52240 0x565050c64510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.615+0000 7fab01200700 1 -- 172.21.15.79:0/1390486576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 msgr2=0x565050c733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:42.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.615+0000 7fab01200700 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.615+0000 7fab01200700 1 -- 172.21.15.79:0/1390486576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565050c73930 con 0x565050c57b90 2024-06-03T02:20:42.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.616+0000 7fab01200700 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c72eb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faafc106800 tx=0x7faae8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=350ab24bce56b1a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:42.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.616+0000 7faae3400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:20:42.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.616+0000 7fab01c00700 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c52240 0x565050c64510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:20:42.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.618+0000 7faae3400700 1 -- 172.21.15.79:0/1390486576 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8010900 con 0x565050c57b90 2024-06-03T02:20:42.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.619+0000 7faae3400700 1 -- 172.21.15.79:0/1390486576 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7faae8010aa0 con 0x565050c57b90 2024-06-03T02:20:42.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.619+0000 7fab0f053b80 1 -- 172.21.15.79:0/1390486576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf40058d0 con 0x565050c57b90 2024-06-03T02:20:42.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.619+0000 7faae3400700 1 -- 172.21.15.79:0/1390486576 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faae8010ca0 con 0x565050c57b90 2024-06-03T02:20:42.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.619+0000 7fab0f053b80 10 client.25382 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:42.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.619+0000 7fab0f053b80 1 -- 172.21.15.79:0/1390486576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565050cd6670 con 0x565050c57b90 2024-06-03T02:20:42.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.620+0000 7faae3400700 1 -- 172.21.15.79:0/1390486576 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf40058d0 con 0x565050c57b90 2024-06-03T02:20:42.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.621+0000 7faae3400700 1 -- 172.21.15.79:0/1390486576 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7faae8025070 con 0x565050c57b90 2024-06-03T02:20:42.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.621+0000 7fab0f053b80 10 client.25382 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:42.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.621+0000 7fab0f053b80 10 client.25382 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:20:42.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.621+0000 7fab0f053b80 20 client.25382 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:42.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.621+0000 7fab0f053b80 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x565050cd6db0 0x565050cd91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:42.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.621+0000 7fab0f053b80 4 client.25382 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:20:42.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.621+0000 7fab0f053b80 1 -- 172.21.15.79:0/1390486576 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565050bab580 con 0x565050cd6db0 2024-06-03T02:20:42.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.622+0000 7fab01c00700 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x565050cd6db0 0x565050cd91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:42.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.622+0000 7fab01c00700 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x565050cd6db0 0x565050cd91d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae26e8aa4050fef7 server_cookie=bd38a24cad6f208b in_seq=0 out_seq=0 2024-06-03T02:20:42.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.622+0000 7faae3400700 10 client.25382 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:20:42.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.623+0000 7faae3400700 1 -- 172.21.15.79:0/1390486576 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565050bab580 con 0x565050cd6db0 2024-06-03T02:20:42.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.623+0000 7faae3400700 10 client.25382 handle_command_reply: tid=0 2024-06-03T02:20:42.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.623+0000 7fab0f053b80 1 client.25382 shutdown 2024-06-03T02:20:42.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.625+0000 7fab0f053b80 1 -- 172.21.15.79:0/1390486576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 msgr2=0x565050c72eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:42.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.625+0000 7fab0f053b80 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c72eb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faafc106800 tx=0x7faae8004750 comp rx=0 tx=0).stop 2024-06-03T02:20:42.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.625+0000 7fab0f053b80 1 -- 172.21.15.79:0/1390486576 shutdown_connections 2024-06-03T02:20:42.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.625+0000 7fab0f053b80 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565050c584b0 0x565050c733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.625+0000 7fab0f053b80 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565050c52240 0x565050c64510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.625+0000 7fab0f053b80 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x565050cd6db0 0x565050cd91d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.625+0000 7fab0f053b80 1 --2- 172.21.15.79:0/1390486576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565050c57b90 0x565050c72eb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.625+0000 7fab0f053b80 1 -- 172.21.15.79:0/1390486576 >> 172.21.15.79:0/1390486576 conn(0x565050b7e170 msgr2=0x565050c56770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:42.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.625+0000 7faae3400700 0 client.25382 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:20:42.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.626+0000 7fab0f053b80 1 -- 172.21.15.79:0/1390486576 shutdown_connections 2024-06-03T02:20:42.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.626+0000 7fab0f053b80 1 -- 172.21.15.79:0/1390486576 wait complete. 2024-06-03T02:20:42.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.626+0000 7fab0f053b80 20 client.25382 trim_cache size 0 max 16384 2024-06-03T02:20:42.627 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:20:42.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 -- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7faae40a2df0 msgr2=0x7faae40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7faae40a2df0 0x7faae40a52b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faae8005fd0 tx=0x7faae800af20 comp rx=0 tx=0).stop 2024-06-03T02:20:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 -- 172.21.15.79:0/2704865787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc06a1a0 msgr2=0x7faafc109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc06a1a0 0x7faafc109f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faaf000d980 tx=0x7faaf000dd40 comp rx=0 tx=0).stop 2024-06-03T02:20:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 -- 172.21.15.79:0/2704865787 shutdown_connections 2024-06-03T02:20:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7faae40a2df0 0x7faae40a52b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faafc10ef40 0x7faafc105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faafc06a1a0 0x7faafc109f20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 --2- 172.21.15.79:0/2704865787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc06aac0 0x7faafc10a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.628+0000 7fab04600700 1 -- 172.21.15.79:0/2704865787 >> 172.21.15.79:0/2704865787 conn(0x7faafc078340 msgr2=0x7faafc110390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:42.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.629+0000 7fab04600700 1 -- 172.21.15.79:0/2704865787 shutdown_connections 2024-06-03T02:20:42.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:42.629+0000 7fab04600700 1 -- 172.21.15.79:0/2704865787 wait complete. 2024-06-03T02:20:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 35.0 s, 1964.6 tps, lat 50.945 ms stddev 64.292 2024-06-03T02:20:44.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:43 smithi079 ceph-mon[31819]: pgmap v536: 129 pgs: 129 active+clean; 1002 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 844 op/s 2024-06-03T02:20:44.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:43 smithi135 ceph-mon[36653]: pgmap v536: 129 pgs: 129 active+clean; 1002 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 844 op/s 2024-06-03T02:20:44.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:43 smithi123 ceph-mon[36640]: pgmap v536: 129 pgs: 129 active+clean; 1002 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 844 op/s 2024-06-03T02:20:46.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:20:46.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:46 smithi079 ceph-mon[31819]: pgmap v537: 129 pgs: 129 active+clean; 1022 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 691 op/s 2024-06-03T02:20:46.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:46 smithi123 ceph-mon[36640]: pgmap v537: 129 pgs: 129 active+clean; 1022 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 691 op/s 2024-06-03T02:20:46.512 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:46 smithi135 ceph-mon[36653]: pgmap v537: 129 pgs: 129 active+clean; 1022 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 691 op/s 2024-06-03T02:20:46.710 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:20:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 40.0 s, 1648.2 tps, lat 60.318 ms stddev 82.378 2024-06-03T02:20:48.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:48 smithi123 ceph-mon[36640]: pgmap v538: 129 pgs: 129 active+clean; 996 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 748 op/s 2024-06-03T02:20:48.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:48 smithi079 ceph-mon[31819]: pgmap v538: 129 pgs: 129 active+clean; 996 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 748 op/s 2024-06-03T02:20:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:48 smithi135 ceph-mon[36653]: pgmap v538: 129 pgs: 129 active+clean; 996 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 748 op/s 2024-06-03T02:20:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.194+0000 7f4d6e400700 1 -- 172.21.15.79:0/375601658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6810eef0 msgr2=0x7f4d681112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.194+0000 7f4d6e400700 1 --2- 172.21.15.79:0/375601658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6810eef0 0x7f4d681112e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000b0a0 tx=0x7f4d6000b3b0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.194+0000 7f4d6e400700 1 -- 172.21.15.79:0/375601658 shutdown_connections 2024-06-03T02:20:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.194+0000 7f4d6e400700 1 --2- 172.21.15.79:0/375601658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d6810eef0 0x7f4d681112e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.194+0000 7f4d6e400700 1 --2- 172.21.15.79:0/375601658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d680746c0 0x7f4d68074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.194+0000 7f4d6e400700 1 --2- 172.21.15.79:0/375601658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d68074fe0 0x7f4d6810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.194+0000 7f4d6e400700 1 -- 172.21.15.79:0/375601658 >> 172.21.15.79:0/375601658 conn(0x7f4d680795f0 msgr2=0x7f4d68079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.194+0000 7f4d6e400700 1 -- 172.21.15.79:0/375601658 shutdown_connections 2024-06-03T02:20:49.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.194+0000 7f4d6e400700 1 -- 172.21.15.79:0/375601658 wait complete. 2024-06-03T02:20:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.196+0000 7f4d6e400700 1 Processor -- start 2024-06-03T02:20:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.196+0000 7f4d6e400700 1 -- start start 2024-06-03T02:20:49.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d6e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d680746c0 0x7f4d681a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d6e400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d68074fe0 0x7f4d681a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d6e400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d6810eef0 0x7f4d6810dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d6e400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68104190 con 0x7f4d6810eef0 2024-06-03T02:20:49.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d6e400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68104010 con 0x7f4d680746c0 2024-06-03T02:20:49.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d6e400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68104310 con 0x7f4d68074fe0 2024-06-03T02:20:49.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d67400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d680746c0 0x7f4d681a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d67400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d680746c0 0x7f4d681a3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54664/0 (socket says 172.21.15.79:54664) 2024-06-03T02:20:49.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d67400700 1 -- 172.21.15.79:0/77855318 learned_addr learned my addr 172.21.15.79:0/77855318 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:49.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.197+0000 7f4d66a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d68074fe0 0x7f4d681a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.198+0000 7f4d67e00700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d6810eef0 0x7f4d6810dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.198+0000 7f4d66a00700 1 -- 172.21.15.79:0/77855318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d680746c0 msgr2=0x7f4d681a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.198+0000 7f4d66a00700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d680746c0 0x7f4d681a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.198+0000 7f4d66a00700 1 -- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d6810eef0 msgr2=0x7f4d6810dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:49.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.198+0000 7f4d66a00700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d6810eef0 0x7f4d6810dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.198+0000 7f4d66a00700 1 -- 172.21.15.79:0/77855318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6810e3f0 con 0x7f4d68074fe0 2024-06-03T02:20:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.198+0000 7f4d67400700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d680746c0 0x7f4d681a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:20:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.198+0000 7f4d66a00700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d68074fe0 0x7f4d681a4330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400d750 tx=0x7f4d5400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.199+0000 7f4d5fe00700 1 -- 172.21.15.79:0/77855318 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54016070 con 0x7f4d68074fe0 2024-06-03T02:20:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.199+0000 7f4d6e400700 1 -- 172.21.15.79:0/77855318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6810e6e0 con 0x7f4d68074fe0 2024-06-03T02:20:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.199+0000 7f4d5fe00700 1 -- 172.21.15.79:0/77855318 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d54004d10 con 0x7f4d68074fe0 2024-06-03T02:20:49.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.199+0000 7f4d6e400700 1 -- 172.21.15.79:0/77855318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d681abd90 con 0x7f4d68074fe0 2024-06-03T02:20:49.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.199+0000 7f4d5fe00700 1 -- 172.21.15.79:0/77855318 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d540056a0 con 0x7f4d68074fe0 2024-06-03T02:20:49.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.209+0000 7f4d78c80b80 1 Processor -- start 2024-06-03T02:20:49.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.209+0000 7f4d78c80b80 1 -- start start 2024-06-03T02:20:49.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d5fe00700 1 -- 172.21.15.79:0/77855318 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4d54004830 con 0x7f4d68074fe0 2024-06-03T02:20:49.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1ef4b0 0x55e53d1e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 0x55e53d1e9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d78c80b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e53d0f7a10 con 0x55e53d1e9240 2024-06-03T02:20:49.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d78c80b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e53d13ab00 con 0x55e53d1eeb90 2024-06-03T02:20:49.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d78c80b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e53d0f58f0 con 0x55e53d1ef4b0 2024-06-03T02:20:49.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d67400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1ef4b0 0x55e53d1e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d67400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1ef4b0 0x55e53d1e8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44220/0 (socket says 172.21.15.79:44220) 2024-06-03T02:20:49.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d67400700 1 -- 172.21.15.79:0/2316239343 learned_addr learned my addr 172.21.15.79:0/2316239343 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:49.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.210+0000 7f4d67e00700 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.211+0000 7f4d67e00700 1 -- 172.21.15.79:0/2316239343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1ef4b0 msgr2=0x55e53d1e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:49.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.211+0000 7f4d67e00700 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1ef4b0 0x55e53d1e8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.211+0000 7f4d67e00700 1 -- 172.21.15.79:0/2316239343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 msgr2=0x55e53d1e9680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:20:49.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.212+0000 7f4d66a00700 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 0x55e53d1e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.212+0000 7f4d67e00700 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 0x55e53d1e9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.212+0000 7f4d67e00700 1 -- 172.21.15.79:0/2316239343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e53d0e2f90 con 0x55e53d1eeb90 2024-06-03T02:20:49.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.213+0000 7f4d66a00700 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 0x55e53d1e9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:20:49.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.213+0000 7f4d67e00700 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1eef70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000b780 tx=0x7f4d600092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73abd967dfeecef5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:49.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.213+0000 7f4d5cc00700 1 -- 172.21.15.79:0/2316239343 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000bcb0 con 0x55e53d1eeb90 2024-06-03T02:20:49.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.213+0000 7f4d5cc00700 1 -- 172.21.15.79:0/2316239343 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d6000be50 con 0x55e53d1eeb90 2024-06-03T02:20:49.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.213+0000 7f4d5cc00700 1 -- 172.21.15.79:0/2316239343 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000bcb0 con 0x55e53d1eeb90 2024-06-03T02:20:49.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.214+0000 7f4d5fe00700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4d4c0a2db0 0x7f4d4c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.214+0000 7f4d78c80b80 1 -- 172.21.15.79:0/2316239343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 msgr2=0x55e53d1eef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:49.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.214+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1eef70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000b780 tx=0x7f4d600092e0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.215+0000 7f4d5fe00700 1 -- 172.21.15.79:0/77855318 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4d540cf080 con 0x7f4d68074fe0 2024-06-03T02:20:49.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.215+0000 7f4d78c80b80 1 -- 172.21.15.79:0/2316239343 shutdown_connections 2024-06-03T02:20:49.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.215+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 0x55e53d1e9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.215+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1eef70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.215+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/2316239343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1ef4b0 0x55e53d1e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.215+0000 7f4d78c80b80 1 -- 172.21.15.79:0/2316239343 >> 172.21.15.79:0/2316239343 conn(0x55e53d115170 msgr2=0x55e53d1e3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:49.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.215+0000 7f4d67400700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4d4c0a2db0 0x7f4d4c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.216+0000 7f4d78c80b80 1 -- 172.21.15.79:0/2316239343 shutdown_connections 2024-06-03T02:20:49.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.216+0000 7f4d78c80b80 1 -- 172.21.15.79:0/2316239343 wait complete. 2024-06-03T02:20:49.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.217+0000 7f4d78c80b80 1 -- start start 2024-06-03T02:20:49.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.217+0000 7f4d67400700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4d4c0a2db0 0x7f4d4c0a5270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009fd0 tx=0x7f4d58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:49.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.218+0000 7f4d78c80b80 1 Processor -- start 2024-06-03T02:20:49.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.219+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1e9240 0x55e53d1f5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.219+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1f2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.219+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 0x55e53d1f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.219+0000 7f4d67400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1e9240 0x55e53d1f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.219+0000 7f4d78c80b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e53d0d6f60 con 0x55e53d1ef4b0 2024-06-03T02:20:49.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.219+0000 7f4d78c80b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e53d136ac0 con 0x55e53d1eeb90 2024-06-03T02:20:49.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.219+0000 7f4d78c80b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e53d206e40 con 0x55e53d1e9240 2024-06-03T02:20:49.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.219+0000 7f4d78c80b80 10 client.0 _ll_register_callbacks cb 0x55e53d0d0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:49.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.220+0000 7f4d67e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 0x55e53d1f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.220+0000 7f4d67400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1e9240 0x55e53d1f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44228/0 (socket says 172.21.15.79:44228) 2024-06-03T02:20:49.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.220+0000 7f4d67400700 1 -- 172.21.15.79:0/3681317942 learned_addr learned my addr 172.21.15.79:0/3681317942 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:49.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.220+0000 7f4d66a00700 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.220+0000 7f4d66a00700 1 -- 172.21.15.79:0/3681317942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1e9240 msgr2=0x55e53d1f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:49.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.220+0000 7f4d66a00700 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1e9240 0x55e53d1f5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.220+0000 7f4d66a00700 1 -- 172.21.15.79:0/3681317942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 msgr2=0x55e53d1f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:49.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.220+0000 7f4d66a00700 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 0x55e53d1f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.220+0000 7f4d66a00700 1 -- 172.21.15.79:0/3681317942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e53d0898d0 con 0x55e53d1eeb90 2024-06-03T02:20:49.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.221+0000 7f4d66a00700 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1f2510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d540cc620 tx=0x7f4d5409b230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d01468a49780569 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:49.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.221+0000 7f4d51600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:20:49.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.221+0000 7f4d51600700 1 -- 172.21.15.79:0/3681317942 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5409ba70 con 0x55e53d1eeb90 2024-06-03T02:20:49.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.221+0000 7f4d51600700 1 -- 172.21.15.79:0/3681317942 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4d5409b5d0 con 0x55e53d1eeb90 2024-06-03T02:20:49.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.221+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3681317942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d6000be50 con 0x55e53d1eeb90 2024-06-03T02:20:49.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.222+0000 7f4d51600700 1 -- 172.21.15.79:0/3681317942 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d5409b370 con 0x55e53d1eeb90 2024-06-03T02:20:49.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.222+0000 7f4d78c80b80 10 client.25394 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:49.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.222+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3681317942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e53d211350 con 0x55e53d1eeb90 2024-06-03T02:20:49.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.222+0000 7f4d51600700 1 -- 172.21.15.79:0/3681317942 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d6000be50 con 0x55e53d1eeb90 2024-06-03T02:20:49.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.223+0000 7f4d51600700 1 -- 172.21.15.79:0/3681317942 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4d540a3b50 con 0x55e53d1eeb90 2024-06-03T02:20:49.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.224+0000 7f4d78c80b80 10 client.25394 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:49.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.224+0000 7f4d78c80b80 10 client.25394 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:20:49.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.224+0000 7f4d78c80b80 20 client.25394 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:49.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.224+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e53d213990 0x55e53d215d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.224+0000 7f4d78c80b80 4 client.25394 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:20:49.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.224+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3681317942 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e53d148000 con 0x55e53d213990 2024-06-03T02:20:49.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.225+0000 7f4d67400700 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e53d213990 0x55e53d215d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.226+0000 7f4d51600700 10 client.25394 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:20:49.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.226+0000 7f4d67400700 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e53d213990 0x55e53d215d80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30a60e6c89626610 server_cookie=118d57be66c22c39 in_seq=0 out_seq=0 2024-06-03T02:20:49.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.227+0000 7f4d51600700 1 -- 172.21.15.79:0/3681317942 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e53d148000 con 0x55e53d213990 2024-06-03T02:20:49.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.227+0000 7f4d51600700 10 client.25394 handle_command_reply: tid=0 2024-06-03T02:20:49.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.227+0000 7f4d78c80b80 1 client.25394 shutdown 2024-06-03T02:20:49.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.227+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3681317942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 msgr2=0x55e53d1f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:49.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.227+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1f2510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d540cc620 tx=0x7f4d5409b230 comp rx=0 tx=0).stop 2024-06-03T02:20:49.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3681317942 shutdown_connections 2024-06-03T02:20:49.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 0x55e53d1f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e53d213990 0x55e53d215d80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1e9240 0x55e53d1f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d51600700 0 client.25394 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:20:49.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3681317942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1eeb90 0x55e53d1f2510 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3681317942 >> 172.21.15.79:0/3681317942 conn(0x55e53d115170 msgr2=0x55e53d1dd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:49.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3681317942 shutdown_connections 2024-06-03T02:20:49.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3681317942 wait complete. 2024-06-03T02:20:49.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.228+0000 7f4d78c80b80 20 client.25394 trim_cache size 0 max 16384 2024-06-03T02:20:49.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.262+0000 7f4d78c80b80 1 Processor -- start 2024-06-03T02:20:49.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.262+0000 7f4d78c80b80 1 -- start start 2024-06-03T02:20:49.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.262+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 0x55e53d1e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.262+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 0x55e53d1f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.262+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1ef4b0 0x55e53d1de130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.262+0000 7f4d78c80b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4800eb80 con 0x55e53d1e9240 2024-06-03T02:20:49.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.262+0000 7f4d78c80b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d480106c0 con 0x55e53d1ef4b0 2024-06-03T02:20:49.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.262+0000 7f4d78c80b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4800f250 con 0x55e53d1eeb90 2024-06-03T02:20:49.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.263+0000 7f4d66a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 0x55e53d1f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.263+0000 7f4d66a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 0x55e53d1f0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44230/0 (socket says 172.21.15.79:44230) 2024-06-03T02:20:49.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.263+0000 7f4d66a00700 1 -- 172.21.15.79:0/3435979805 learned_addr learned my addr 172.21.15.79:0/3435979805 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:49.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.263+0000 7f4d67e00700 1 --2- 172.21.15.79:0/3435979805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1ef4b0 0x55e53d1de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.263+0000 7f4d66a00700 1 -- 172.21.15.79:0/3435979805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1ef4b0 msgr2=0x55e53d1de130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:49.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.263+0000 7f4d66a00700 1 --2- 172.21.15.79:0/3435979805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1ef4b0 0x55e53d1de130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.263+0000 7f4d66a00700 1 -- 172.21.15.79:0/3435979805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 msgr2=0x55e53d1e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:49.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.263+0000 7f4d66a00700 1 --2- 172.21.15.79:0/3435979805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 0x55e53d1e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.263+0000 7f4d66a00700 1 -- 172.21.15.79:0/3435979805 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e53d1f34b0 con 0x55e53d1eeb90 2024-06-03T02:20:49.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.264+0000 7f4d66a00700 1 --2- 172.21.15.79:0/3435979805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 0x55e53d1f0060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4d54005210 tx=0x7f4d54099880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b273bbd6d06951 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:49.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.264+0000 7f4d53e00700 1 -- 172.21.15.79:0/3435979805 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5409ba70 con 0x55e53d1eeb90 2024-06-03T02:20:49.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.264+0000 7f4d53e00700 1 -- 172.21.15.79:0/3435979805 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d54014030 con 0x55e53d1eeb90 2024-06-03T02:20:49.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.264+0000 7f4d53e00700 1 -- 172.21.15.79:0/3435979805 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5409b5d0 con 0x55e53d1eeb90 2024-06-03T02:20:49.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.265+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3435979805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 msgr2=0x55e53d1f0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:49.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.265+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3435979805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 0x55e53d1f0060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4d54005210 tx=0x7f4d54099880 comp rx=0 tx=0).stop 2024-06-03T02:20:49.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.266+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3435979805 shutdown_connections 2024-06-03T02:20:49.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.266+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3435979805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1ef4b0 0x55e53d1de130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.266+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3435979805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1e9240 0x55e53d1e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.266+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/3435979805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 0x55e53d1f0060 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.266+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3435979805 >> 172.21.15.79:0/3435979805 conn(0x55e53d115170 msgr2=0x55e53d1ef950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:49.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.266+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3435979805 shutdown_connections 2024-06-03T02:20:49.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.266+0000 7f4d78c80b80 1 -- 172.21.15.79:0/3435979805 wait complete. 2024-06-03T02:20:49.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.266+0000 7f4d78c80b80 1 -- start start 2024-06-03T02:20:49.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.267+0000 7f4d78c80b80 1 Processor -- start 2024-06-03T02:20:49.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.268+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1e9240 0x55e53d20c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.268+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 0x55e53d20c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.268+0000 7f4d78c80b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 0x55e53d21a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d67400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1e9240 0x55e53d20c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d67400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1e9240 0x55e53d20c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54716/0 (socket says 172.21.15.79:54716) 2024-06-03T02:20:49.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d67400700 1 -- 172.21.15.79:0/1219362404 learned_addr learned my addr 172.21.15.79:0/1219362404 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:49.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4800fff0 con 0x55e53d1ef4b0 2024-06-03T02:20:49.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4800f920 con 0x55e53d1e9240 2024-06-03T02:20:49.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4800cb20 con 0x55e53d1eeb90 2024-06-03T02:20:49.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d78c80b80 10 client.0 _ll_register_callbacks cb 0x55e53d0d0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:49.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d67e00700 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 0x55e53d21a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d67400700 1 -- 172.21.15.79:0/1219362404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 msgr2=0x55e53d20c670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:20:49.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d67400700 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 0x55e53d20c670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d67400700 1 -- 172.21.15.79:0/1219362404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 msgr2=0x55e53d21a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:49.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d67400700 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 0x55e53d21a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.269+0000 7f4d67400700 1 -- 172.21.15.79:0/1219362404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e53d21aa80 con 0x55e53d1e9240 2024-06-03T02:20:49.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.270+0000 7f4d67e00700 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 0x55e53d21a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:20:49.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.270+0000 7f4d67400700 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1e9240 0x55e53d20c130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d580094f0 tx=0x7f4d58004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=853c729ef222d32c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:49.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.270+0000 7f4d51600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:20:49.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.270+0000 7f4d51600700 1 -- 172.21.15.79:0/1219362404 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58004c10 con 0x55e53d1e9240 2024-06-03T02:20:49.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.270+0000 7f4d51600700 1 -- 172.21.15.79:0/1219362404 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4d58018030 con 0x55e53d1e9240 2024-06-03T02:20:49.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.270+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d54014030 con 0x55e53d1e9240 2024-06-03T02:20:49.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.270+0000 7f4d51600700 1 -- 172.21.15.79:0/1219362404 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d5803a050 con 0x55e53d1e9240 2024-06-03T02:20:49.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.270+0000 7f4d78c80b80 10 client.25400 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:49.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.270+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e53d27dcc0 con 0x55e53d1e9240 2024-06-03T02:20:49.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.271+0000 7f4d51600700 1 -- 172.21.15.79:0/1219362404 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d54014030 con 0x55e53d1e9240 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.271+0000 7f4d51600700 1 -- 172.21.15.79:0/1219362404 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4d5802a5c0 con 0x55e53d1e9240 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.271+0000 7f4d78c80b80 10 client.25400 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.271+0000 7f4d78c80b80 10 client.25400 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.271+0000 7f4d78c80b80 20 client.25400 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.271+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e53d27e380 0x55e53d2807a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.271+0000 7f4d78c80b80 4 client.25400 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.271+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e53d142580 con 0x55e53d27e380 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.272+0000 7f4d66a00700 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e53d27e380 0x55e53d2807a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.273+0000 7f4d66a00700 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e53d27e380 0x55e53d2807a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9586f8c2ca52164a server_cookie=7e8fe679b88ad755 in_seq=0 out_seq=0 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.273+0000 7f4d51600700 10 client.25400 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.274+0000 7f4d51600700 1 -- 172.21.15.79:0/1219362404 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e53d142580 con 0x55e53d27e380 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.274+0000 7f4d51600700 10 client.25400 handle_command_reply: tid=0 2024-06-03T02:20:49.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.274+0000 7f4d78c80b80 1 client.25400 shutdown 2024-06-03T02:20:49.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.275+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1e9240 msgr2=0x55e53d20c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:49.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.275+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1e9240 0x55e53d20c130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d580094f0 tx=0x7f4d58004750 comp rx=0 tx=0).stop 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.275+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 shutdown_connections 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.275+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e53d1ef4b0 0x55e53d21a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.275+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53d1e9240 0x55e53d20c130 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.275+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e53d27e380 0x55e53d2807a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.275+0000 7f4d78c80b80 1 --2- 172.21.15.79:0/1219362404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e53d1eeb90 0x55e53d20c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.275+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 >> 172.21.15.79:0/1219362404 conn(0x55e53d115170 msgr2=0x55e53d1ed650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.276+0000 7f4d51600700 0 client.25400 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.276+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 shutdown_connections 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.277+0000 7f4d78c80b80 1 -- 172.21.15.79:0/1219362404 wait complete. 2024-06-03T02:20:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.277+0000 7f4d78c80b80 20 client.25400 trim_cache size 0 max 16384 2024-06-03T02:20:49.286 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:20:49.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.278+0000 7f4d6e400700 1 -- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4d4c0a2db0 msgr2=0x7f4d4c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.278+0000 7f4d6e400700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4d4c0a2db0 0x7f4d4c0a5270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009fd0 tx=0x7f4d58009260 comp rx=0 tx=0).stop 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.279+0000 7f4d6e400700 1 -- 172.21.15.79:0/77855318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d68074fe0 msgr2=0x7f4d681a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.279+0000 7f4d6e400700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d68074fe0 0x7f4d681a4330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400d750 tx=0x7f4d5400da60 comp rx=0 tx=0).stop 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.279+0000 7f4d6e400700 1 -- 172.21.15.79:0/77855318 shutdown_connections 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.279+0000 7f4d6e400700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4d4c0a2db0 0x7f4d4c0a5270 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.279+0000 7f4d6e400700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d6810eef0 0x7f4d6810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.280+0000 7f4d6e400700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d680746c0 0x7f4d681a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.280+0000 7f4d6e400700 1 --2- 172.21.15.79:0/77855318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d68074fe0 0x7f4d681a4330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.280+0000 7f4d6e400700 1 -- 172.21.15.79:0/77855318 >> 172.21.15.79:0/77855318 conn(0x7f4d680795f0 msgr2=0x7f4d68100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.280+0000 7f4d6e400700 1 -- 172.21.15.79:0/77855318 shutdown_connections 2024-06-03T02:20:49.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:49.280+0000 7f4d6e400700 1 -- 172.21.15.79:0/77855318 wait complete. 2024-06-03T02:20:50.398 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:50 smithi079 ceph-mon[31819]: pgmap v539: 129 pgs: 129 active+clean; 978 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 753 op/s 2024-06-03T02:20:50.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:20:50.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:50 smithi123 ceph-mon[36640]: pgmap v539: 129 pgs: 129 active+clean; 978 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 753 op/s 2024-06-03T02:20:50.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:50 smithi135 ceph-mon[36653]: pgmap v539: 129 pgs: 129 active+clean; 978 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 753 op/s 2024-06-03T02:20:51.112 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:20:52.450 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:52 smithi079 ceph-mon[31819]: pgmap v540: 129 pgs: 129 active+clean; 963 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 671 op/s 2024-06-03T02:20:52.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:52 smithi123 ceph-mon[36640]: pgmap v540: 129 pgs: 129 active+clean; 963 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 671 op/s 2024-06-03T02:20:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:52 smithi135 ceph-mon[36653]: pgmap v540: 129 pgs: 129 active+clean; 963 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 671 op/s 2024-06-03T02:20:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 45.0 s, 1809.8 tps, lat 55.565 ms stddev 62.984 2024-06-03T02:20:54.306 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:54 smithi079 ceph-mon[31819]: pgmap v541: 129 pgs: 129 active+clean; 963 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 837 op/s 2024-06-03T02:20:54.306 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:54 smithi079 ceph-mon[31819]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-06-03T02:20:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:54 smithi123 ceph-mon[36640]: pgmap v541: 129 pgs: 129 active+clean; 963 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 837 op/s 2024-06-03T02:20:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:54 smithi123 ceph-mon[36640]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-06-03T02:20:54.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.551+0000 7f25dfe00700 1 -- 172.21.15.79:0/1981712256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d80746c0 msgr2=0x7f25d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:54.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.551+0000 7f25dfe00700 1 --2- 172.21.15.79:0/1981712256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d80746c0 0x7f25d8074aa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009990 tx=0x7f25cc009ca0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.552+0000 7f25dfe00700 1 -- 172.21.15.79:0/1981712256 shutdown_connections 2024-06-03T02:20:54.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.552+0000 7f25dfe00700 1 --2- 172.21.15.79:0/1981712256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d8113660 0x7f25d8115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.552+0000 7f25dfe00700 1 --2- 172.21.15.79:0/1981712256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d80746c0 0x7f25d8074aa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.552+0000 7f25dfe00700 1 --2- 172.21.15.79:0/1981712256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8074fe0 0x7f25d8113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.552+0000 7f25dfe00700 1 -- 172.21.15.79:0/1981712256 >> 172.21.15.79:0/1981712256 conn(0x7f25d8079640 msgr2=0x7f25d8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:54.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.552+0000 7f25dfe00700 1 -- 172.21.15.79:0/1981712256 shutdown_connections 2024-06-03T02:20:54.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.553+0000 7f25dfe00700 1 -- 172.21.15.79:0/1981712256 wait complete. 2024-06-03T02:20:54.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.554+0000 7f25dfe00700 1 Processor -- start 2024-06-03T02:20:54.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.555+0000 7f25dfe00700 1 -- start start 2024-06-03T02:20:54.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dfe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d80746c0 0x7f25d819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dfe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8074fe0 0x7f25d819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d8113660 0x7f25d81a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dfe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8118e80 con 0x7f25d8074fe0 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dfe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8118d00 con 0x7f25d80746c0 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dfe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8119000 con 0x7f25d8113660 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dde00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d8113660 0x7f25d81a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dde00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d8113660 0x7f25d81a3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44242/0 (socket says 172.21.15.79:44242) 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dca00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8074fe0 0x7f25d819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dde00700 1 -- 172.21.15.79:0/341652278 learned_addr learned my addr 172.21.15.79:0/341652278 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.556+0000 7f25dd400700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d80746c0 0x7f25d819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.557+0000 7f25dde00700 1 -- 172.21.15.79:0/341652278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d80746c0 msgr2=0x7f25d819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.557+0000 7f25dde00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d80746c0 0x7f25d819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.557+0000 7f25dde00700 1 -- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8074fe0 msgr2=0x7f25d819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.557+0000 7f25dde00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8074fe0 0x7f25d819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.557+0000 7f25dde00700 1 -- 172.21.15.79:0/341652278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d81a4260 con 0x7f25d8113660 2024-06-03T02:20:54.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.557+0000 7f25dca00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8074fe0 0x7f25d819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:20:54.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.557+0000 7f25dde00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d8113660 0x7f25d81a3b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25d000b6d0 tx=0x7f25d000b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:54.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.560+0000 7f25d6000700 1 -- 172.21.15.79:0/341652278 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00115a0 con 0x7f25d8113660 2024-06-03T02:20:54.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.560+0000 7f25d6000700 1 -- 172.21.15.79:0/341652278 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25d0017070 con 0x7f25d8113660 2024-06-03T02:20:54.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.560+0000 7f25d6000700 1 -- 172.21.15.79:0/341652278 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0012be0 con 0x7f25d8113660 2024-06-03T02:20:54.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.560+0000 7f25dfe00700 1 -- 172.21.15.79:0/341652278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d81a4550 con 0x7f25d8113660 2024-06-03T02:20:54.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.561+0000 7f25dfe00700 1 -- 172.21.15.79:0/341652278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d81abdd0 con 0x7f25d8113660 2024-06-03T02:20:54.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.571+0000 7f25ea69fb80 1 Processor -- start 2024-06-03T02:20:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.572+0000 7f25d6000700 1 -- 172.21.15.79:0/341652278 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f25d0011740 con 0x7f25d8113660 2024-06-03T02:20:54.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.572+0000 7f25ea69fb80 1 -- start start 2024-06-03T02:20:54.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609027f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c609021d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609022680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c608f30a10 con 0x55c6090284b0 2024-06-03T02:20:54.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c608f73b00 con 0x55c609022240 2024-06-03T02:20:54.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c608f2e8f0 con 0x55c609027b90 2024-06-03T02:20:54.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25dca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25dca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609027f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44244/0 (socket says 172.21.15.79:44244) 2024-06-03T02:20:54.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25dca00700 1 -- 172.21.15.79:0/3777347770 learned_addr learned my addr 172.21.15.79:0/3777347770 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:54.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.573+0000 7f25dca00700 1 --2- 172.21.15.79:0/3777347770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609022680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.574+0000 7f25d6000700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25c00a2ca0 0x7f25c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.574+0000 7f25dca00700 1 -- 172.21.15.79:0/3777347770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 msgr2=0x55c609027f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.574+0000 7f25dca00700 1 --2- 172.21.15.79:0/3777347770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609027f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.574+0000 7f25dca00700 1 -- 172.21.15.79:0/3777347770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 msgr2=0x55c609021d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:20:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.574+0000 7f25dca00700 1 --2- 172.21.15.79:0/3777347770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c609021d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.574+0000 7f25dca00700 1 -- 172.21.15.79:0/3777347770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c608f1bf90 con 0x55c609022240 2024-06-03T02:20:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.575+0000 7f25d6000700 1 -- 172.21.15.79:0/341652278 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f25d00ceca0 con 0x7f25d8113660 2024-06-03T02:20:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.575+0000 7f25dca00700 1 --2- 172.21.15.79:0/3777347770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609022680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f25c4011c20 tx=0x7f25c400a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ffeb318f57af79 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.575+0000 7f25dde00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25c00a2ca0 0x7f25c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.575+0000 7f25caa00700 1 -- 172.21.15.79:0/3777347770 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4004020 con 0x55c609022240 2024-06-03T02:20:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.576+0000 7f25caa00700 1 -- 172.21.15.79:0/3777347770 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c40118e0 con 0x55c609022240 2024-06-03T02:20:54.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.576+0000 7f25caa00700 1 -- 172.21.15.79:0/3777347770 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4004020 con 0x55c609022240 2024-06-03T02:20:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.577+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3777347770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 msgr2=0x55c609022680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.577+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3777347770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609022680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f25c4011c20 tx=0x7f25c400a380 comp rx=0 tx=0).stop 2024-06-03T02:20:54.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.577+0000 7f25dde00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25c00a2ca0 0x7f25c00a5160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25d009b0e0 tx=0x7f25d0012430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.577+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3777347770 shutdown_connections 2024-06-03T02:20:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.577+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3777347770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609022680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.577+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3777347770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609027f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.577+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3777347770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c609021d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.577+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3777347770 >> 172.21.15.79:0/3777347770 conn(0x55c608f4e170 msgr2=0x55c60901cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.578+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3777347770 shutdown_connections 2024-06-03T02:20:54.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.578+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3777347770 wait complete. 2024-06-03T02:20:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.579+0000 7f25ea69fb80 1 -- start start 2024-06-03T02:20:54.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.579+0000 7f25ea69fb80 1 Processor -- start 2024-06-03T02:20:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.579+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 0x55c60902e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60902b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c60902baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c608f0ff60 con 0x55c6090284b0 2024-06-03T02:20:54.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c608f6fac0 con 0x55c609027b90 2024-06-03T02:20:54.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c60902ee90 con 0x55c609022240 2024-06-03T02:20:54.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25dd400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60902b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25dca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 0x55c60902e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25dca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 0x55c60902e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44248/0 (socket says 172.21.15.79:44248) 2024-06-03T02:20:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25dca00700 1 -- 172.21.15.79:0/3670428918 learned_addr learned my addr 172.21.15.79:0/3670428918 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.580+0000 7f25ea69fb80 10 client.0 _ll_register_callbacks cb 0x55c608f09380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.581+0000 7f25dd400700 1 -- 172.21.15.79:0/3670428918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 msgr2=0x55c60902e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.581+0000 7f25dd400700 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 0x55c60902e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.581+0000 7f25dd400700 1 -- 172.21.15.79:0/3670428918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 msgr2=0x55c60902baf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:20:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.581+0000 7f25dd400700 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c60902baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.581+0000 7f25dd400700 1 -- 172.21.15.79:0/3670428918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c608ec28d0 con 0x55c609027b90 2024-06-03T02:20:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.582+0000 7f25dca00700 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 0x55c60902e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:20:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.583+0000 7f25dd400700 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60902b5b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f25cc006010 tx=0x7f25cc005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c6a8c8f3cba6ec8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.583+0000 7f25bf400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:20:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.583+0000 7f25bf400700 1 -- 172.21.15.79:0/3670428918 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc027060 con 0x55c609027b90 2024-06-03T02:20:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.583+0000 7f25bf400700 1 -- 172.21.15.79:0/3670428918 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f25cc003b40 con 0x55c609027b90 2024-06-03T02:20:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.583+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3670428918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c40118e0 con 0x55c609027b90 2024-06-03T02:20:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.584+0000 7f25bf400700 1 -- 172.21.15.79:0/3670428918 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25cc02e050 con 0x55c609027b90 2024-06-03T02:20:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.584+0000 7f25ea69fb80 10 client.25409 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:54.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.584+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3670428918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c609039490 con 0x55c609027b90 2024-06-03T02:20:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.585+0000 7f25bf400700 1 -- 172.21.15.79:0/3670428918 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c40118e0 con 0x55c609027b90 2024-06-03T02:20:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.585+0000 7f25bf400700 1 -- 172.21.15.79:0/3670428918 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f25cc00f730 con 0x55c609027b90 2024-06-03T02:20:54.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.585+0000 7f25ea69fb80 10 client.25409 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.585+0000 7f25ea69fb80 10 client.25409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:20:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.585+0000 7f25ea69fb80 20 client.25409 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.585+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c60903bad0 0x55c60903df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.585+0000 7f25ea69fb80 4 client.25409 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:20:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.585+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3670428918 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c608f81000 con 0x55c60903bad0 2024-06-03T02:20:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.586+0000 7f25dca00700 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c60903bad0 0x55c60903df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.587+0000 7f25dca00700 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c60903bad0 0x55c60903df00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eccdce7fc445d5f server_cookie=a61746a46be3509c in_seq=0 out_seq=0 2024-06-03T02:20:54.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.587+0000 7f25bf400700 10 client.25409 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:20:54.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.588+0000 7f25bf400700 1 -- 172.21.15.79:0/3670428918 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c608f81000 con 0x55c60903bad0 2024-06-03T02:20:54.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.588+0000 7f25bf400700 10 client.25409 handle_command_reply: tid=0 2024-06-03T02:20:54.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.588+0000 7f25ea69fb80 1 client.25409 shutdown 2024-06-03T02:20:54.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.589+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3670428918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 msgr2=0x55c60902b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:54.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.589+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60902b5b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f25cc006010 tx=0x7f25cc005ed0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.589+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3670428918 shutdown_connections 2024-06-03T02:20:54.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.589+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c60903bad0 0x55c60903df00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.589+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c60902baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.589+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 0x55c60902e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.589+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/3670428918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60902b5b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.589+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3670428918 >> 172.21.15.79:0/3670428918 conn(0x55c608f4e170 msgr2=0x55c609016790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:54.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.589+0000 7f25bf400700 0 client.25409 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:20:54.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.590+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3670428918 shutdown_connections 2024-06-03T02:20:54.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.590+0000 7f25ea69fb80 1 -- 172.21.15.79:0/3670428918 wait complete. 2024-06-03T02:20:54.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.590+0000 7f25ea69fb80 20 client.25409 trim_cache size 0 max 16384 2024-06-03T02:20:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:54 smithi135 ceph-mon[36653]: pgmap v541: 129 pgs: 129 active+clean; 963 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 837 op/s 2024-06-03T02:20:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:54 smithi135 ceph-mon[36653]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-06-03T02:20:54.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.622+0000 7f25ea69fb80 1 Processor -- start 2024-06-03T02:20:54.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.622+0000 7f25ea69fb80 1 -- start start 2024-06-03T02:20:54.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.623+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 0x55c609021d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.623+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60901d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.623+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c609017130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.623+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81b4a40 con 0x55c6090284b0 2024-06-03T02:20:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.623+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81b6580 con 0x55c609027b90 2024-06-03T02:20:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.623+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81b5110 con 0x55c609022240 2024-06-03T02:20:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.623+0000 7f25dd400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60901d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.624+0000 7f25dd400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60901d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54764/0 (socket says 172.21.15.79:54764) 2024-06-03T02:20:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.624+0000 7f25dd400700 1 -- 172.21.15.79:0/4059001669 learned_addr learned my addr 172.21.15.79:0/4059001669 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.624+0000 7f25dd400700 1 -- 172.21.15.79:0/4059001669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 msgr2=0x55c609021d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:20:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.624+0000 7f25dd400700 1 --2- 172.21.15.79:0/4059001669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 0x55c609021d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.624+0000 7f25dd400700 1 -- 172.21.15.79:0/4059001669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 msgr2=0x55c609017130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:20:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.624+0000 7f25dd400700 1 --2- 172.21.15.79:0/4059001669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c609017130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.624+0000 7f25dd400700 1 -- 172.21.15.79:0/4059001669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c609029b40 con 0x55c609027b90 2024-06-03T02:20:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.624+0000 7f25dd400700 1 --2- 172.21.15.79:0/4059001669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60901d1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f25cc001eb0 tx=0x7f25cc00b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408944cde5e454b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.626+0000 7f25ca000700 1 -- 172.21.15.79:0/4059001669 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc059070 con 0x55c609027b90 2024-06-03T02:20:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.627+0000 7f25ca000700 1 -- 172.21.15.79:0/4059001669 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25cc003940 con 0x55c609027b90 2024-06-03T02:20:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.627+0000 7f25ca000700 1 -- 172.21.15.79:0/4059001669 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc027070 con 0x55c609027b90 2024-06-03T02:20:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.628+0000 7f25ea69fb80 1 -- 172.21.15.79:0/4059001669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 msgr2=0x55c60901d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.628+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/4059001669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60901d1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f25cc001eb0 tx=0x7f25cc00b6d0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.628+0000 7f25ea69fb80 1 -- 172.21.15.79:0/4059001669 shutdown_connections 2024-06-03T02:20:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.628+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/4059001669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c609017130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.628+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/4059001669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609022240 0x55c609021d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.629+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/4059001669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609027b90 0x55c60901d1a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.629+0000 7f25ea69fb80 1 -- 172.21.15.79:0/4059001669 >> 172.21.15.79:0/4059001669 conn(0x55c608f4e170 msgr2=0x55c60901ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.630+0000 7f25ea69fb80 1 -- 172.21.15.79:0/4059001669 shutdown_connections 2024-06-03T02:20:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.630+0000 7f25ea69fb80 1 -- 172.21.15.79:0/4059001669 wait complete. 2024-06-03T02:20:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.631+0000 7f25ea69fb80 1 -- start start 2024-06-03T02:20:54.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.631+0000 7f25ea69fb80 1 Processor -- start 2024-06-03T02:20:54.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.633+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609043e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.633+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609034150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.633+0000 7f25ea69fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c609034690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.633+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81b5eb0 con 0x55c6090284b0 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.633+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81b57e0 con 0x55c609022240 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.633+0000 7f25ea69fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81b29e0 con 0x55c609027b90 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.633+0000 7f25ea69fb80 10 client.0 _ll_register_callbacks cb 0x55c608f09380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.635+0000 7f25dca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609043e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.635+0000 7f25dca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609043e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54776/0 (socket says 172.21.15.79:54776) 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.635+0000 7f25dca00700 1 -- 172.21.15.79:0/1777383637 learned_addr learned my addr 172.21.15.79:0/1777383637 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.635+0000 7f25dd400700 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609034150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.636+0000 7f25dd400700 1 -- 172.21.15.79:0/1777383637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 msgr2=0x55c609043e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:54.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.636+0000 7f25dd400700 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609043e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.636+0000 7f25dd400700 1 -- 172.21.15.79:0/1777383637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 msgr2=0x55c609034690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:20:54.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.636+0000 7f25dd400700 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c609034690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.636+0000 7f25dd400700 1 -- 172.21.15.79:0/1777383637 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c609034de0 con 0x55c609027b90 2024-06-03T02:20:54.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.636+0000 7f25dca00700 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609043e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:20:54.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.639+0000 7f25dd400700 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609034150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25cc000c00 tx=0x7f25cc0098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd4f0e3ecd405c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:20:54.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.639+0000 7f25bf400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:20:54.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.639+0000 7f25bf400700 1 -- 172.21.15.79:0/1777383637 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc059070 con 0x55c609027b90 2024-06-03T02:20:54.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.639+0000 7f25bf400700 1 -- 172.21.15.79:0/1777383637 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f25cc00f660 con 0x55c609027b90 2024-06-03T02:20:54.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.640+0000 7f25ea69fb80 1 -- 172.21.15.79:0/1777383637 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25cc003940 con 0x55c609027b90 2024-06-03T02:20:54.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.640+0000 7f25bf400700 1 -- 172.21.15.79:0/1777383637 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25cc027070 con 0x55c609027b90 2024-06-03T02:20:54.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.640+0000 7f25ea69fb80 10 client.25399 fetch_fsmap learned FSMap version 30 2024-06-03T02:20:54.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.641+0000 7f25ea69fb80 1 -- 172.21.15.79:0/1777383637 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c6090a58b0 con 0x55c609027b90 2024-06-03T02:20:54.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.641+0000 7f25bf400700 1 -- 172.21.15.79:0/1777383637 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25cc003940 con 0x55c609027b90 2024-06-03T02:20:54.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.642+0000 7f25bf400700 1 -- 172.21.15.79:0/1777383637 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f25cc017860 con 0x55c609027b90 2024-06-03T02:20:54.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.642+0000 7f25ea69fb80 10 client.25399 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:20:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.642+0000 7f25ea69fb80 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:20:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.642+0000 7f25ea69fb80 20 client.25399 populate_metadata read hostname 'smithi079' 2024-06-03T02:20:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.642+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c6090a5dd0 0x55c6090a81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:20:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.642+0000 7f25ea69fb80 4 client.25399 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:20:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.642+0000 7f25ea69fb80 1 -- 172.21.15.79:0/1777383637 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c608f7b580 con 0x55c6090a5dd0 2024-06-03T02:20:54.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.644+0000 7f25dca00700 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c6090a5dd0 0x55c6090a81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:20:54.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.646+0000 7f25bf400700 10 client.25399 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:20:54.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.646+0000 7f25dca00700 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c6090a5dd0 0x55c6090a81f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db85ea8e7bc02a5b server_cookie=5d27656096455731 in_seq=0 out_seq=0 2024-06-03T02:20:54.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.647+0000 7f25bf400700 1 -- 172.21.15.79:0/1777383637 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c608f7b580 con 0x55c6090a5dd0 2024-06-03T02:20:54.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.647+0000 7f25bf400700 10 client.25399 handle_command_reply: tid=0 2024-06-03T02:20:54.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.647+0000 7f25ea69fb80 1 client.25399 shutdown 2024-06-03T02:20:54.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.649+0000 7f25ea69fb80 1 -- 172.21.15.79:0/1777383637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 msgr2=0x55c609034150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:54.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.649+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609034150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25cc000c00 tx=0x7f25cc0098c0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.651+0000 7f25ea69fb80 1 -- 172.21.15.79:0/1777383637 shutdown_connections 2024-06-03T02:20:54.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.651+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c6090a5dd0 0x55c6090a81f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.651+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c6090284b0 0x55c609034690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.651+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c609022240 0x55c609043e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.651+0000 7f25ea69fb80 1 --2- 172.21.15.79:0/1777383637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c609027b90 0x55c609034150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.651+0000 7f25bf400700 0 client.25399 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:20:54.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.651+0000 7f25ea69fb80 1 -- 172.21.15.79:0/1777383637 >> 172.21.15.79:0/1777383637 conn(0x55c608f4e170 msgr2=0x55c609028890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:54.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.652+0000 7f25ea69fb80 1 -- 172.21.15.79:0/1777383637 shutdown_connections 2024-06-03T02:20:54.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.652+0000 7f25ea69fb80 1 -- 172.21.15.79:0/1777383637 wait complete. 2024-06-03T02:20:54.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.652+0000 7f25ea69fb80 20 client.25399 trim_cache size 0 max 16384 2024-06-03T02:20:54.653 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:20:54.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.653+0000 7f25dfe00700 1 -- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25c00a2ca0 msgr2=0x7f25c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:54.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.653+0000 7f25dfe00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25c00a2ca0 0x7f25c00a5160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25d009b0e0 tx=0x7f25d0012430 comp rx=0 tx=0).stop 2024-06-03T02:20:54.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.653+0000 7f25dfe00700 1 -- 172.21.15.79:0/341652278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d8113660 msgr2=0x7f25d81a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:20:54.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.653+0000 7f25dfe00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d8113660 0x7f25d81a3b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25d000b6d0 tx=0x7f25d000b9e0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.654+0000 7f25dfe00700 1 -- 172.21.15.79:0/341652278 shutdown_connections 2024-06-03T02:20:54.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.654+0000 7f25dfe00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25d8113660 0x7f25d81a3b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.654+0000 7f25dfe00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d80746c0 0x7f25d819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.654+0000 7f25dfe00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f25c00a2ca0 0x7f25c00a5160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.654+0000 7f25dfe00700 1 --2- 172.21.15.79:0/341652278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8074fe0 0x7f25d819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:20:54.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.654+0000 7f25dfe00700 1 -- 172.21.15.79:0/341652278 >> 172.21.15.79:0/341652278 conn(0x7f25d8079640 msgr2=0x7f25d8112350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:20:54.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.654+0000 7f25dfe00700 1 -- 172.21.15.79:0/341652278 shutdown_connections 2024-06-03T02:20:54.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:20:54.654+0000 7f25dfe00700 1 -- 172.21.15.79:0/341652278 wait complete. 2024-06-03T02:20:56.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:56 smithi123 ceph-mon[36640]: pgmap v542: 129 pgs: 129 active+clean; 964 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 680 op/s 2024-06-03T02:20:56.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:56 smithi079 ceph-mon[31819]: pgmap v542: 129 pgs: 129 active+clean; 964 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 680 op/s 2024-06-03T02:20:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:56 smithi135 ceph-mon[36653]: pgmap v542: 129 pgs: 129 active+clean; 964 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 680 op/s 2024-06-03T02:20:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 50.0 s, 2410.2 tps, lat 41.484 ms stddev 59.599 2024-06-03T02:20:58.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:20:58 smithi123 ceph-mon[36640]: pgmap v543: 129 pgs: 129 active+clean; 964 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.9 MiB/s wr, 756 op/s 2024-06-03T02:20:58.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:20:58 smithi079 ceph-mon[31819]: pgmap v543: 129 pgs: 129 active+clean; 964 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.9 MiB/s wr, 756 op/s 2024-06-03T02:20:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:20:58 smithi135 ceph-mon[36653]: pgmap v543: 129 pgs: 129 active+clean; 964 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.9 MiB/s wr, 756 op/s 2024-06-03T02:21:00.179 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:21:00.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:21:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:00 smithi123 ceph-mon[36640]: pgmap v544: 129 pgs: 129 active+clean; 973 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 724 op/s 2024-06-03T02:21:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:00 smithi079 ceph-mon[31819]: pgmap v544: 129 pgs: 129 active+clean; 973 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 724 op/s 2024-06-03T02:21:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:00 smithi135 ceph-mon[36653]: pgmap v544: 129 pgs: 129 active+clean; 973 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 724 op/s 2024-06-03T02:21:00.833 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:02.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:02 smithi079 ceph-mon[31819]: pgmap v545: 129 pgs: 129 active+clean; 973 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 592 op/s 2024-06-03T02:21:02.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:02 smithi123 ceph-mon[36640]: pgmap v545: 129 pgs: 129 active+clean; 973 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 592 op/s 2024-06-03T02:21:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:02 smithi135 ceph-mon[36653]: pgmap v545: 129 pgs: 129 active+clean; 973 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 592 op/s 2024-06-03T02:21:03.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 55.0 s, 2365.4 tps, lat 42.204 ms stddev 59.400 2024-06-03T02:21:04.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:04 smithi079 ceph-mon[31819]: pgmap v546: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 821 op/s 2024-06-03T02:21:04.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:04 smithi123 ceph-mon[36640]: pgmap v546: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 821 op/s 2024-06-03T02:21:04.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:04 smithi135 ceph-mon[36653]: pgmap v546: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 821 op/s 2024-06-03T02:21:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.956+0000 7f98a9400700 1 -- 172.21.15.79:0/2627800887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98a4105080 msgr2=0x7f98a4111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.956+0000 7f98a9400700 1 --2- 172.21.15.79:0/2627800887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98a4105080 0x7f98a4111590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f989c00b0a0 tx=0x7f989c00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.956+0000 7f98a9400700 1 -- 172.21.15.79:0/2627800887 shutdown_connections 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.956+0000 7f98a9400700 1 --2- 172.21.15.79:0/2627800887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98a4105080 0x7f98a4111590 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.956+0000 7f98a9400700 1 --2- 172.21.15.79:0/2627800887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a410a6c0 0x7f98a410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.956+0000 7f98a9400700 1 --2- 172.21.15.79:0/2627800887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a41046c0 0x7f98a4104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.956+0000 7f98a9400700 1 -- 172.21.15.79:0/2627800887 >> 172.21.15.79:0/2627800887 conn(0x7f98a41001b0 msgr2=0x7f98a41025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.957+0000 7f98a9400700 1 -- 172.21.15.79:0/2627800887 shutdown_connections 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.957+0000 7f98a9400700 1 -- 172.21.15.79:0/2627800887 wait complete. 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a9400700 1 Processor -- start 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a9400700 1 -- start start 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a9400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98a41046c0 0x7f98a419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a9400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4105080 0x7f98a419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a9400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a410a6c0 0x7f98a419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a9400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4114a50 con 0x7f98a41046c0 2024-06-03T02:21:04.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a9400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41148d0 con 0x7f98a4105080 2024-06-03T02:21:04.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a9400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4114bd0 con 0x7f98a410a6c0 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a1e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4105080 0x7f98a419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a1e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4105080 0x7f98a419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36358/0 (socket says 172.21.15.79:36358) 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a2800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98a41046c0 0x7f98a419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.958+0000 7f98a1e00700 1 -- 172.21.15.79:0/756767171 learned_addr learned my addr 172.21.15.79:0/756767171 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.959+0000 7f98a3200700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a410a6c0 0x7f98a419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.959+0000 7f98a1e00700 1 -- 172.21.15.79:0/756767171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a410a6c0 msgr2=0x7f98a419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.959+0000 7f98a1e00700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a410a6c0 0x7f98a419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.959+0000 7f98a1e00700 1 -- 172.21.15.79:0/756767171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98a41046c0 msgr2=0x7f98a419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.959+0000 7f98a1e00700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98a41046c0 0x7f98a419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.959+0000 7f98a1e00700 1 -- 172.21.15.79:0/756767171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a419feb0 con 0x7f98a4105080 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.959+0000 7f98a1e00700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4105080 0x7f98a419b3e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f989000d980 tx=0x7f989000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.959+0000 7f989b400700 1 -- 172.21.15.79:0/756767171 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890016070 con 0x7f98a4105080 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.960+0000 7f989b400700 1 -- 172.21.15.79:0/756767171 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9890004d10 con 0x7f98a4105080 2024-06-03T02:21:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.960+0000 7f989b400700 1 -- 172.21.15.79:0/756767171 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890005670 con 0x7f98a4105080 2024-06-03T02:21:04.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.960+0000 7f98a9400700 1 -- 172.21.15.79:0/756767171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a41a01a0 con 0x7f98a4105080 2024-06-03T02:21:04.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.961+0000 7f98a9400700 1 -- 172.21.15.79:0/756767171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a406beb0 con 0x7f98a4105080 2024-06-03T02:21:04.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.970+0000 7f989b400700 1 -- 172.21.15.79:0/756767171 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9890004830 con 0x7f98a4105080 2024-06-03T02:21:04.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.974+0000 7f989b400700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f98880a2ca0 0x7f98880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.974+0000 7f98b3e78b80 1 Processor -- start 2024-06-03T02:21:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.974+0000 7f989b400700 1 -- 172.21.15.79:0/756767171 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f98900cf120 con 0x7f98a4105080 2024-06-03T02:21:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.975+0000 7f98a2800700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f98880a2ca0 0x7f98880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.975+0000 7f98b3e78b80 1 -- start start 2024-06-03T02:21:04.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.976+0000 7f98a2800700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f98880a2ca0 0x7f98880a5160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9894009680 tx=0x7f9894009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.976+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.976+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.976+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 0x5579561aa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.976+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5579560b8a90 con 0x5579561afc10 2024-06-03T02:21:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.976+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5579560fbb80 con 0x5579561b0530 2024-06-03T02:21:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.976+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5579560b6970 con 0x5579561aa2c0 2024-06-03T02:21:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a2800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a2800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54136/0 (socket says 172.21.15.79:54136) 2024-06-03T02:21:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a2800700 1 -- 172.21.15.79:0/1233649028 learned_addr learned my addr 172.21.15.79:0/1233649028 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a1e00700 1 --2- 172.21.15.79:0/1233649028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a3200700 1 --2- 172.21.15.79:0/1233649028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 0x5579561aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a2800700 1 -- 172.21.15.79:0/1233649028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 msgr2=0x5579561aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a2800700 1 --2- 172.21.15.79:0/1233649028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 0x5579561aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a2800700 1 -- 172.21.15.79:0/1233649028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 msgr2=0x5579561a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a2800700 1 --2- 172.21.15.79:0/1233649028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.977+0000 7f98a2800700 1 -- 172.21.15.79:0/1233649028 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579560a4010 con 0x5579561afc10 2024-06-03T02:21:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.978+0000 7f98a2800700 1 --2- 172.21.15.79:0/1233649028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561afff0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9894011d60 tx=0x7f9894014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce5f4c1e6bb68be server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.979+0000 7f988fe00700 1 -- 172.21.15.79:0/1233649028 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894011d90 con 0x5579561afc10 2024-06-03T02:21:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.979+0000 7f988fe00700 1 -- 172.21.15.79:0/1233649028 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9894004020 con 0x5579561afc10 2024-06-03T02:21:04.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.980+0000 7f988fe00700 1 -- 172.21.15.79:0/1233649028 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894003cf0 con 0x5579561afc10 2024-06-03T02:21:04.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.980+0000 7f98b3e78b80 1 -- 172.21.15.79:0/1233649028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 msgr2=0x5579561afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:04.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.980+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/1233649028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561afff0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9894011d60 tx=0x7f9894014000 comp rx=0 tx=0).stop 2024-06-03T02:21:04.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.981+0000 7f98b3e78b80 1 -- 172.21.15.79:0/1233649028 shutdown_connections 2024-06-03T02:21:04.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.981+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/1233649028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 0x5579561aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.981+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/1233649028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561afff0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.981+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/1233649028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.981+0000 7f98b3e78b80 1 -- 172.21.15.79:0/1233649028 >> 172.21.15.79:0/1233649028 conn(0x5579560d61f0 msgr2=0x5579561a5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:04.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.982+0000 7f98b3e78b80 1 -- 172.21.15.79:0/1233649028 shutdown_connections 2024-06-03T02:21:04.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.982+0000 7f98b3e78b80 1 -- 172.21.15.79:0/1233649028 wait complete. 2024-06-03T02:21:04.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.983+0000 7f98b3e78b80 1 -- start start 2024-06-03T02:21:04.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.983+0000 7f98b3e78b80 1 Processor -- start 2024-06-03T02:21:04.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.985+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 0x5579561aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.985+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.985+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.986+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557956097fe0 con 0x5579561afc10 2024-06-03T02:21:04.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.986+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5579560f7b40 con 0x5579561b0530 2024-06-03T02:21:04.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.986+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5579561bf7c0 con 0x5579561aa2c0 2024-06-03T02:21:04.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.986+0000 7f98a3200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.986+0000 7f98a3200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561ac810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36372/0 (socket says 172.21.15.79:36372) 2024-06-03T02:21:04.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.986+0000 7f98a3200700 1 -- 172.21.15.79:0/3696842307 learned_addr learned my addr 172.21.15.79:0/3696842307 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:04.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.986+0000 7f98a2800700 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 0x5579561aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.986+0000 7f98a1e00700 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.987+0000 7f98a3200700 1 -- 172.21.15.79:0/3696842307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 msgr2=0x5579561aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.987+0000 7f98a3200700 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 0x5579561aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.987+0000 7f98a3200700 1 -- 172.21.15.79:0/3696842307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 msgr2=0x5579561ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.987+0000 7f98a3200700 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.987+0000 7f98a3200700 1 -- 172.21.15.79:0/3696842307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55795604a950 con 0x5579561b0530 2024-06-03T02:21:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.987+0000 7f98a3200700 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561ac810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f989c015000 tx=0x7f989c00be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f113858aad0eac8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.987+0000 7f98b3e78b80 10 client.0 _ll_register_callbacks cb 0x557956091400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.987+0000 7f988cc00700 10 client.25427 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:21:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.987+0000 7f98b3e78b80 1 -- 172.21.15.79:0/3696842307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9894004020 con 0x5579561b0530 2024-06-03T02:21:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.988+0000 7f988cc00700 1 -- 172.21.15.79:0/3696842307 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c0074e0 con 0x5579561b0530 2024-06-03T02:21:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.988+0000 7f988cc00700 1 -- 172.21.15.79:0/3696842307 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f989c007680 con 0x5579561b0530 2024-06-03T02:21:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.988+0000 7f988cc00700 1 -- 172.21.15.79:0/3696842307 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f989c02f050 con 0x5579561b0530 2024-06-03T02:21:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.989+0000 7f98b3e78b80 10 client.25427 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.989+0000 7f98b3e78b80 1 -- 172.21.15.79:0/3696842307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5579561c9d70 con 0x5579561b0530 2024-06-03T02:21:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.989+0000 7f988cc00700 1 -- 172.21.15.79:0/3696842307 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f989c011070 con 0x5579561b0530 2024-06-03T02:21:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.989+0000 7f98b3e78b80 10 client.25427 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.989+0000 7f98b3e78b80 10 client.25427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.989+0000 7f98b3e78b80 20 client.25427 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.989+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579561cc240 0x5579561ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.989+0000 7f98b3e78b80 4 client.25427 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:21:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.989+0000 7f98b3e78b80 1 -- 172.21.15.79:0/3696842307 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557956109080 con 0x5579561cc240 2024-06-03T02:21:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.990+0000 7f98a2800700 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579561cc240 0x5579561ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:04.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.992+0000 7f98a2800700 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579561cc240 0x5579561ce650 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e475adb2c1ea4993 server_cookie=8eabad0f17311139 in_seq=0 out_seq=0 2024-06-03T02:21:04.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.992+0000 7f988cc00700 10 client.25427 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:04.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.997+0000 7f988cc00700 1 -- 172.21.15.79:0/3696842307 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557956109080 con 0x5579561cc240 2024-06-03T02:21:04.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.997+0000 7f988cc00700 10 client.25427 handle_command_reply: tid=0 2024-06-03T02:21:04.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.997+0000 7f98b3e78b80 1 client.25427 shutdown 2024-06-03T02:21:04.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.998+0000 7f98b3e78b80 1 -- 172.21.15.79:0/3696842307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 msgr2=0x5579561ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:04.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.998+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561ac810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f989c015000 tx=0x7f989c00be60 comp rx=0 tx=0).stop 2024-06-03T02:21:04.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.999+0000 7f98b3e78b80 1 -- 172.21.15.79:0/3696842307 shutdown_connections 2024-06-03T02:21:04.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.999+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561b0530 0x5579561ac810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.999+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561aa2c0 0x5579561aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.999+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579561cc240 0x5579561ce650 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.999+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/3696842307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561afc10 0x5579561ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:04.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.999+0000 7f988cc00700 0 client.25427 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:04.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.999+0000 7f98b3e78b80 1 -- 172.21.15.79:0/3696842307 >> 172.21.15.79:0/3696842307 conn(0x5579560d61f0 msgr2=0x55795619e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:05.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.999+0000 7f98b3e78b80 1 -- 172.21.15.79:0/3696842307 shutdown_connections 2024-06-03T02:21:05.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:04.999+0000 7f98b3e78b80 1 -- 172.21.15.79:0/3696842307 wait complete. 2024-06-03T02:21:05.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.001+0000 7f98b3e78b80 20 client.25427 trim_cache size 0 max 16384 2024-06-03T02:21:05.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.053+0000 7f98b3e78b80 1 Processor -- start 2024-06-03T02:21:05.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.054+0000 7f98b3e78b80 1 -- start start 2024-06-03T02:21:05.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.055+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561aa2c0 0x55795619f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:05.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.055+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561afc10 0x5579561bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:05.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.055+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561b0530 0x5579561d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:05.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.055+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f98a40705e0 con 0x5579561aa2c0 2024-06-03T02:21:05.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.055+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4072120 con 0x5579561afc10 2024-06-03T02:21:05.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.055+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4070cb0 con 0x5579561b0530 2024-06-03T02:21:05.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.056+0000 7f98a3200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561b0530 0x5579561d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:05.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.056+0000 7f98a3200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561b0530 0x5579561d3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:47608/0 (socket says 172.21.15.79:47608) 2024-06-03T02:21:05.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.056+0000 7f98a1e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561afc10 0x5579561bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.056+0000 7f98a3200700 1 -- 172.21.15.79:0/420322015 learned_addr learned my addr 172.21.15.79:0/420322015 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.057+0000 7f98a2800700 1 --2- 172.21.15.79:0/420322015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561aa2c0 0x55795619f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.057+0000 7f98a1e00700 1 -- 172.21.15.79:0/420322015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561b0530 msgr2=0x5579561d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.058+0000 7f98a1e00700 1 --2- 172.21.15.79:0/420322015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561b0530 0x5579561d3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.058+0000 7f98a1e00700 1 -- 172.21.15.79:0/420322015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561aa2c0 msgr2=0x55795619f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:05.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.058+0000 7f98a1e00700 1 --2- 172.21.15.79:0/420322015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561aa2c0 0x55795619f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.058+0000 7f98a1e00700 1 -- 172.21.15.79:0/420322015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579561ac0a0 con 0x5579561afc10 2024-06-03T02:21:05.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.060+0000 7f98a1e00700 1 --2- 172.21.15.79:0/420322015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561afc10 0x5579561bedb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98900cc8c0 tx=0x7f9890005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=577ac65a1cfec458 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:05.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.061+0000 7f988f400700 1 -- 172.21.15.79:0/420322015 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890014070 con 0x5579561afc10 2024-06-03T02:21:05.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.061+0000 7f988f400700 1 -- 172.21.15.79:0/420322015 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f989009bcd0 con 0x5579561afc10 2024-06-03T02:21:05.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.061+0000 7f988f400700 1 -- 172.21.15.79:0/420322015 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98900998d0 con 0x5579561afc10 2024-06-03T02:21:05.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.063+0000 7f98b3e78b80 1 -- 172.21.15.79:0/420322015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561afc10 msgr2=0x5579561bedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:05.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.063+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/420322015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561afc10 0x5579561bedb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98900cc8c0 tx=0x7f9890005890 comp rx=0 tx=0).stop 2024-06-03T02:21:05.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.064+0000 7f98b3e78b80 1 -- 172.21.15.79:0/420322015 shutdown_connections 2024-06-03T02:21:05.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.064+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/420322015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561b0530 0x5579561d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.064+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/420322015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561aa2c0 0x55795619f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.064+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/420322015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561afc10 0x5579561bedb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.064+0000 7f98b3e78b80 1 -- 172.21.15.79:0/420322015 >> 172.21.15.79:0/420322015 conn(0x5579560d61f0 msgr2=0x5579561be6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:05.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.064+0000 7f98b3e78b80 1 -- 172.21.15.79:0/420322015 shutdown_connections 2024-06-03T02:21:05.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.065+0000 7f98b3e78b80 1 -- 172.21.15.79:0/420322015 wait complete. 2024-06-03T02:21:05.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.065+0000 7f98b3e78b80 1 -- start start 2024-06-03T02:21:05.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.065+0000 7f98b3e78b80 1 Processor -- start 2024-06-03T02:21:05.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.068+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561aa2c0 0x5579561c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:05.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.068+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561afc10 0x5579561c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:05.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.069+0000 7f98a2800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561aa2c0 0x5579561c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:05.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.070+0000 7f98b3e78b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561b0530 0x5579561c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:05.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.070+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4071a50 con 0x5579561b0530 2024-06-03T02:21:05.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.070+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4071380 con 0x5579561aa2c0 2024-06-03T02:21:05.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.070+0000 7f98b3e78b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98a406e580 con 0x5579561afc10 2024-06-03T02:21:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.070+0000 7f98b3e78b80 10 client.0 _ll_register_callbacks cb 0x557956091400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.070+0000 7f98a3200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561b0530 0x5579561c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.070+0000 7f98a3200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561b0530 0x5579561c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54168/0 (socket says 172.21.15.79:54168) 2024-06-03T02:21:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.070+0000 7f98a3200700 1 -- 172.21.15.79:0/2783836803 learned_addr learned my addr 172.21.15.79:0/2783836803 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.071+0000 7f98a3200700 1 -- 172.21.15.79:0/2783836803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561afc10 msgr2=0x5579561c1c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:21:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.071+0000 7f98a3200700 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561afc10 0x5579561c1c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.071+0000 7f98a3200700 1 -- 172.21.15.79:0/2783836803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561aa2c0 msgr2=0x5579561c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.071+0000 7f98a3200700 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561aa2c0 0x5579561c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.071+0000 7f98a3200700 1 -- 172.21.15.79:0/2783836803 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557956236f00 con 0x5579561b0530 2024-06-03T02:21:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.071+0000 7f98a2800700 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561aa2c0 0x5579561c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.072+0000 7f98a3200700 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561b0530 0x5579561c21d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f989c00b900 tx=0x7f989c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3448be8556861c05 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.072+0000 7f988cc00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:21:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.073+0000 7f988cc00700 1 -- 172.21.15.79:0/2783836803 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c004d20 con 0x5579561b0530 2024-06-03T02:21:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.073+0000 7f988cc00700 1 -- 172.21.15.79:0/2783836803 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f989c00f040 con 0x5579561b0530 2024-06-03T02:21:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.073+0000 7f98b3e78b80 1 -- 172.21.15.79:0/2783836803 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f989009bcd0 con 0x5579561b0530 2024-06-03T02:21:05.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.075+0000 7f988cc00700 1 -- 172.21.15.79:0/2783836803 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f989c03d050 con 0x5579561b0530 2024-06-03T02:21:05.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.075+0000 7f98b3e78b80 10 client.15288 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:05.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.075+0000 7f98b3e78b80 1 -- 172.21.15.79:0/2783836803 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557956238160 con 0x5579561b0530 2024-06-03T02:21:05.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.076+0000 7f988cc00700 1 -- 172.21.15.79:0/2783836803 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f989009bcd0 con 0x5579561b0530 2024-06-03T02:21:05.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.076+0000 7f988cc00700 1 -- 172.21.15.79:0/2783836803 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f989c011040 con 0x5579561b0530 2024-06-03T02:21:05.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.076+0000 7f98b3e78b80 10 client.15288 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:05.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.076+0000 7f98b3e78b80 10 client.15288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:05.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.076+0000 7f98b3e78b80 20 client.15288 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:05.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.076+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x557956238750 0x55795623ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:05.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.076+0000 7f98b3e78b80 4 client.15288 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:21:05.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.076+0000 7f98b3e78b80 1 -- 172.21.15.79:0/2783836803 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557956103600 con 0x557956238750 2024-06-03T02:21:05.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.077+0000 7f98a2800700 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x557956238750 0x55795623ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:05.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.078+0000 7f98a2800700 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x557956238750 0x55795623ab40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=539ff80088274e05 server_cookie=9bd51e921a42393d in_seq=0 out_seq=0 2024-06-03T02:21:05.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.078+0000 7f988cc00700 10 client.15288 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:05.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.079+0000 7f988cc00700 1 -- 172.21.15.79:0/2783836803 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557956103600 con 0x557956238750 2024-06-03T02:21:05.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.079+0000 7f988cc00700 10 client.15288 handle_command_reply: tid=0 2024-06-03T02:21:05.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.079+0000 7f98b3e78b80 1 client.15288 shutdown 2024-06-03T02:21:05.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.080+0000 7f98b3e78b80 1 -- 172.21.15.79:0/2783836803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561b0530 msgr2=0x5579561c21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:05.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.080+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561b0530 0x5579561c21d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f989c00b900 tx=0x7f989c0099e0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.080+0000 7f98b3e78b80 1 -- 172.21.15.79:0/2783836803 shutdown_connections 2024-06-03T02:21:05.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.080+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x557956238750 0x55795623ab40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.080+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579561b0530 0x5579561c21d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.080+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579561aa2c0 0x5579561c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.080+0000 7f98b3e78b80 1 --2- 172.21.15.79:0/2783836803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579561afc10 0x5579561c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.080+0000 7f98b3e78b80 1 -- 172.21.15.79:0/2783836803 >> 172.21.15.79:0/2783836803 conn(0x5579560d61f0 msgr2=0x5579561adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:05.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.081+0000 7f988cc00700 0 client.15288 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:05.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.082+0000 7f98b3e78b80 1 -- 172.21.15.79:0/2783836803 shutdown_connections 2024-06-03T02:21:05.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.082+0000 7f98b3e78b80 1 -- 172.21.15.79:0/2783836803 wait complete. 2024-06-03T02:21:05.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.082+0000 7f98b3e78b80 20 client.15288 trim_cache size 0 max 16384 2024-06-03T02:21:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:21:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:21:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "e0163085-2799-4ed8-a3a3-42722ddb22a2", 2024-06-03T02:21:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:21:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:21:05.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 -- 172.21.15.79:0/756767171 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f98880a2ca0 msgr2=0x7f98880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:05.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f98880a2ca0 0x7f98880a5160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9894009680 tx=0x7f9894009260 comp rx=0 tx=0).stop 2024-06-03T02:21:05.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 -- 172.21.15.79:0/756767171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4105080 msgr2=0x7f98a419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:05.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4105080 0x7f98a419b3e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f989000d980 tx=0x7f989000dc90 comp rx=0 tx=0).stop 2024-06-03T02:21:05.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 -- 172.21.15.79:0/756767171 shutdown_connections 2024-06-03T02:21:05.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a410a6c0 0x7f98a419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f98880a2ca0 0x7f98880a5160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98a41046c0 0x7f98a419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 --2- 172.21.15.79:0/756767171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98a4105080 0x7f98a419b3e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:05.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 -- 172.21.15.79:0/756767171 >> 172.21.15.79:0/756767171 conn(0x7f98a41001b0 msgr2=0x7f98a4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:05.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.084+0000 7f98a9400700 1 -- 172.21.15.79:0/756767171 shutdown_connections 2024-06-03T02:21:05.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:05.085+0000 7f98a9400700 1 -- 172.21.15.79:0/756767171 wait complete. 2024-06-03T02:21:05.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:05 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:21:05.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:05 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:05.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:05 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:21:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:05 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:21:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:05 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:05 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:21:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:05 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:21:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:05 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:05 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:21:06.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:21:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:06 smithi123 ceph-mon[36640]: pgmap v547: 129 pgs: 129 active+clean; 981 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 695 op/s 2024-06-03T02:21:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:06 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:06.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:06 smithi079 ceph-mon[31819]: pgmap v547: 129 pgs: 129 active+clean; 981 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 695 op/s 2024-06-03T02:21:06.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:06 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:06 smithi135 ceph-mon[36653]: pgmap v547: 129 pgs: 129 active+clean; 981 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 695 op/s 2024-06-03T02:21:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:06 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:06.924 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 60.0 s, 1728.4 tps, lat 57.980 ms stddev 66.163 2024-06-03T02:21:08.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:08 smithi079 ceph-mon[31819]: pgmap v548: 129 pgs: 129 active+clean; 983 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 795 op/s 2024-06-03T02:21:08.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:08 smithi123 ceph-mon[36640]: pgmap v548: 129 pgs: 129 active+clean; 983 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 795 op/s 2024-06-03T02:21:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:08 smithi135 ceph-mon[36653]: pgmap v548: 129 pgs: 129 active+clean; 983 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 795 op/s 2024-06-03T02:21:09.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.486+0000 7ff50de00700 1 -- 172.21.15.79:0/1736649315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff508109bf0 msgr2=0x7ff50810e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:09.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.486+0000 7ff50de00700 1 --2- 172.21.15.79:0/1736649315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff508109bf0 0x7ff50810e6f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009990 tx=0x7ff4fc009ca0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.486+0000 7ff50de00700 1 -- 172.21.15.79:0/1736649315 shutdown_connections 2024-06-03T02:21:09.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.486+0000 7ff50de00700 1 --2- 172.21.15.79:0/1736649315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff508113880 0x7ff508115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.486+0000 7ff50de00700 1 --2- 172.21.15.79:0/1736649315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff508109bf0 0x7ff50810e6f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.486+0000 7ff50de00700 1 --2- 172.21.15.79:0/1736649315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff50810a060 0x7ff50810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.486+0000 7ff50de00700 1 -- 172.21.15.79:0/1736649315 >> 172.21.15.79:0/1736649315 conn(0x7ff50806d470 msgr2=0x7ff50806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:09.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.486+0000 7ff50de00700 1 -- 172.21.15.79:0/1736649315 shutdown_connections 2024-06-03T02:21:09.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.486+0000 7ff50de00700 1 -- 172.21.15.79:0/1736649315 wait complete. 2024-06-03T02:21:09.489 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:09 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:21:09.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.488+0000 7ff50de00700 1 Processor -- start 2024-06-03T02:21:09.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.488+0000 7ff50de00700 1 -- start start 2024-06-03T02:21:09.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff50de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff508109bf0 0x7ff5081a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff50de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50810a060 0x7ff5081a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff50de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff508113880 0x7ff5081a7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff50de00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff508119eb0 con 0x7ff508109bf0 2024-06-03T02:21:09.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff50de00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff508119d30 con 0x7ff508113880 2024-06-03T02:21:09.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff50de00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff50811a030 con 0x7ff50810a060 2024-06-03T02:21:09.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff507e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50810a060 0x7ff5081a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff50d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff508113880 0x7ff5081a7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff50d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff508113880 0x7ff5081a7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36414/0 (socket says 172.21.15.79:36414) 2024-06-03T02:21:09.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.489+0000 7ff50d400700 1 -- 172.21.15.79:0/4007803471 learned_addr learned my addr 172.21.15.79:0/4007803471 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:09.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.491+0000 7ff50d400700 1 -- 172.21.15.79:0/4007803471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50810a060 msgr2=0x7ff5081a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:09.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.491+0000 7ff50d400700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50810a060 0x7ff5081a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.491+0000 7ff50d400700 1 -- 172.21.15.79:0/4007803471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff508109bf0 msgr2=0x7ff5081a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:09.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.491+0000 7ff50d400700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff508109bf0 0x7ff5081a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.491+0000 7ff50d400700 1 -- 172.21.15.79:0/4007803471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5081a8650 con 0x7ff508113880 2024-06-03T02:21:09.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.492+0000 7ff50d400700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff508113880 0x7ff5081a7f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff50000ef90 tx=0x7ff50000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:09.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.500+0000 7ff505600700 1 -- 172.21.15.79:0/4007803471 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000092e0 con 0x7ff508113880 2024-06-03T02:21:09.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.500+0000 7ff505600700 1 -- 172.21.15.79:0/4007803471 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff500004bb0 con 0x7ff508113880 2024-06-03T02:21:09.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.500+0000 7ff505600700 1 -- 172.21.15.79:0/4007803471 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500004280 con 0x7ff508113880 2024-06-03T02:21:09.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.500+0000 7ff50de00700 1 -- 172.21.15.79:0/4007803471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5081a88e0 con 0x7ff508113880 2024-06-03T02:21:09.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.500+0000 7ff50de00700 1 -- 172.21.15.79:0/4007803471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5081b01c0 con 0x7ff508113880 2024-06-03T02:21:09.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.510+0000 7ff519e4ab80 1 Processor -- start 2024-06-03T02:21:09.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.511+0000 7ff505600700 1 -- 172.21.15.79:0/4007803471 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff50001b030 con 0x7ff508113880 2024-06-03T02:21:09.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.513+0000 7ff505600700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff4f00a2d70 0x7ff4f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.514+0000 7ff507e00700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff4f00a2d70 0x7ff4f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.514+0000 7ff507e00700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff4f00a2d70 0x7ff4f00a5230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff508073380 tx=0x7ff4f4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:09.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.515+0000 7ff505600700 1 -- 172.21.15.79:0/4007803471 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff500099560 con 0x7ff508113880 2024-06-03T02:21:09.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.516+0000 7ff519e4ab80 1 -- start start 2024-06-03T02:21:09.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 0x564449cae420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449caede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 0x564449cc2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564449bbba10 con 0x564449cae960 2024-06-03T02:21:09.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564449bfeb00 con 0x564449cbfd80 2024-06-03T02:21:09.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564449bb98f0 con 0x564449c28e40 2024-06-03T02:21:09.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff50ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 0x564449cae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff507e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449caede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff507e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449caede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54204/0 (socket says 172.21.15.79:54204) 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.518+0000 7ff507e00700 1 -- 172.21.15.79:0/507517243 learned_addr learned my addr 172.21.15.79:0/507517243 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.519+0000 7ff50d400700 1 --2- 172.21.15.79:0/507517243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 0x564449cc2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.519+0000 7ff507e00700 1 -- 172.21.15.79:0/507517243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 msgr2=0x564449cae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.519+0000 7ff507e00700 1 --2- 172.21.15.79:0/507517243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 0x564449cae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.519+0000 7ff507e00700 1 -- 172.21.15.79:0/507517243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 msgr2=0x564449cc2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.519+0000 7ff507e00700 1 --2- 172.21.15.79:0/507517243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 0x564449cc2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.519+0000 7ff507e00700 1 -- 172.21.15.79:0/507517243 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564449ba6f90 con 0x564449cae960 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.520+0000 7ff507e00700 1 --2- 172.21.15.79:0/507517243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449caede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009fd0 tx=0x7ff4f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aeb488bae240054 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.521+0000 7ff4fa000700 1 -- 172.21.15.79:0/507517243 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f401b040 con 0x564449cae960 2024-06-03T02:21:09.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.521+0000 7ff4fa000700 1 -- 172.21.15.79:0/507517243 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f4024070 con 0x564449cae960 2024-06-03T02:21:09.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.521+0000 7ff4fa000700 1 -- 172.21.15.79:0/507517243 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f40137d0 con 0x564449cae960 2024-06-03T02:21:09.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.522+0000 7ff519e4ab80 1 -- 172.21.15.79:0/507517243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 msgr2=0x564449caede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:09.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.522+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/507517243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449caede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009fd0 tx=0x7ff4f4011000 comp rx=0 tx=0).stop 2024-06-03T02:21:09.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.524+0000 7ff519e4ab80 1 -- 172.21.15.79:0/507517243 shutdown_connections 2024-06-03T02:21:09.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.524+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/507517243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 0x564449cc2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.524+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/507517243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 0x564449cae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.524+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/507517243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449caede0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.524+0000 7ff519e4ab80 1 -- 172.21.15.79:0/507517243 >> 172.21.15.79:0/507517243 conn(0x564449bd9170 msgr2=0x564449ca8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:09.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.524+0000 7ff519e4ab80 1 -- 172.21.15.79:0/507517243 shutdown_connections 2024-06-03T02:21:09.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.524+0000 7ff519e4ab80 1 -- 172.21.15.79:0/507517243 wait complete. 2024-06-03T02:21:09.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.525+0000 7ff519e4ab80 1 -- start start 2024-06-03T02:21:09.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.525+0000 7ff519e4ab80 1 Processor -- start 2024-06-03T02:21:09.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.526+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 0x564449cb9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.526+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449cb96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.526+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 0x564449cbeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.526+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564449b9af60 con 0x564449cae960 2024-06-03T02:21:09.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.526+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564449bfaac0 con 0x564449cbfd80 2024-06-03T02:21:09.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.526+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564449caf700 con 0x564449c28e40 2024-06-03T02:21:09.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.526+0000 7ff519e4ab80 10 client.0 _ll_register_callbacks cb 0x564449b98720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:09.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.528+0000 7ff507e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449cb96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.528+0000 7ff50d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 0x564449cbeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.528+0000 7ff50d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 0x564449cbeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36442/0 (socket says 172.21.15.79:36442) 2024-06-03T02:21:09.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.528+0000 7ff50d400700 1 -- 172.21.15.79:0/2396810963 learned_addr learned my addr 172.21.15.79:0/2396810963 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:09.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.528+0000 7ff50ca00700 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 0x564449cb9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.529+0000 7ff50ca00700 1 -- 172.21.15.79:0/2396810963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 msgr2=0x564449cbeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:09.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.529+0000 7ff50ca00700 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 0x564449cbeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.529+0000 7ff50ca00700 1 -- 172.21.15.79:0/2396810963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 msgr2=0x564449cb96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:09.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.529+0000 7ff50ca00700 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449cb96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.529+0000 7ff50ca00700 1 -- 172.21.15.79:0/2396810963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564449b4d8d0 con 0x564449c28e40 2024-06-03T02:21:09.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.529+0000 7ff50ca00700 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 0x564449cb9160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc006010 tx=0x7ff4fc00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f1a7e7cec903c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:09.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.529+0000 7ff4eea00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:21:09.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.530+0000 7ff4eea00700 1 -- 172.21.15.79:0/2396810963 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc027060 con 0x564449c28e40 2024-06-03T02:21:09.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.530+0000 7ff4eea00700 1 -- 172.21.15.79:0/2396810963 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff4fc0053b0 con 0x564449c28e40 2024-06-03T02:21:09.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.530+0000 7ff519e4ab80 1 -- 172.21.15.79:0/2396810963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f4024070 con 0x564449c28e40 2024-06-03T02:21:09.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.531+0000 7ff4eea00700 1 -- 172.21.15.79:0/2396810963 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4fc0055d0 con 0x564449c28e40 2024-06-03T02:21:09.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.533+0000 7ff519e4ab80 10 client.25426 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:09.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.535+0000 7ff519e4ab80 1 -- 172.21.15.79:0/2396810963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564449cb5190 con 0x564449c28e40 2024-06-03T02:21:09.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.536+0000 7ff4eea00700 1 -- 172.21.15.79:0/2396810963 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f4024070 con 0x564449c28e40 2024-06-03T02:21:09.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.536+0000 7ff4eea00700 1 -- 172.21.15.79:0/2396810963 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff4fc021070 con 0x564449c28e40 2024-06-03T02:21:09.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.537+0000 7ff519e4ab80 10 client.25426 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:09.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.537+0000 7ff519e4ab80 10 client.25426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:09.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.537+0000 7ff519e4ab80 20 client.25426 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:09.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.537+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564449cc6fa0 0x564449cc9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.537+0000 7ff519e4ab80 4 client.25426 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:21:09.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.537+0000 7ff519e4ab80 1 -- 172.21.15.79:0/2396810963 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564449c0c000 con 0x564449cc6fa0 2024-06-03T02:21:09.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.537+0000 7ff507e00700 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564449cc6fa0 0x564449cc9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.538+0000 7ff4eea00700 10 client.25426 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:09.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.538+0000 7ff507e00700 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564449cc6fa0 0x564449cc9390 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57da48ab1849e324 server_cookie=1b2d2936f7256c6d in_seq=0 out_seq=0 2024-06-03T02:21:09.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.539+0000 7ff4eea00700 1 -- 172.21.15.79:0/2396810963 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564449c0c000 con 0x564449cc6fa0 2024-06-03T02:21:09.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.539+0000 7ff4eea00700 10 client.25426 handle_command_reply: tid=0 2024-06-03T02:21:09.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.539+0000 7ff519e4ab80 1 client.25426 shutdown 2024-06-03T02:21:09.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.540+0000 7ff519e4ab80 1 -- 172.21.15.79:0/2396810963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 msgr2=0x564449cb9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:09.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.540+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 0x564449cb9160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc006010 tx=0x7ff4fc00bf90 comp rx=0 tx=0).stop 2024-06-03T02:21:09.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.540+0000 7ff519e4ab80 1 -- 172.21.15.79:0/2396810963 shutdown_connections 2024-06-03T02:21:09.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.540+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cbfd80 0x564449cbeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.540+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449c28e40 0x564449cb9160 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.540+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564449cc6fa0 0x564449cc9390 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.540+0000 7ff4eea00700 0 client.25426 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:09.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.540+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/2396810963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449cb96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.540+0000 7ff519e4ab80 1 -- 172.21.15.79:0/2396810963 >> 172.21.15.79:0/2396810963 conn(0x564449bd9170 msgr2=0x564449ca8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:09.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.541+0000 7ff519e4ab80 1 -- 172.21.15.79:0/2396810963 shutdown_connections 2024-06-03T02:21:09.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.541+0000 7ff519e4ab80 1 -- 172.21.15.79:0/2396810963 wait complete. 2024-06-03T02:21:09.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.541+0000 7ff519e4ab80 20 client.25426 trim_cache size 0 max 16384 2024-06-03T02:21:09.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.583+0000 7ff519e4ab80 1 Processor -- start 2024-06-03T02:21:09.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.583+0000 7ff519e4ab80 1 -- start start 2024-06-03T02:21:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.583+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae960 0x564449ca8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.583+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cbfd80 0x564449cbe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.583+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cae260 0x564449cceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.583+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081c5830 con 0x564449cbfd80 2024-06-03T02:21:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.583+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081c7370 con 0x564449cae960 2024-06-03T02:21:09.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.583+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081c5f00 con 0x564449cae260 2024-06-03T02:21:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.584+0000 7ff50d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cae260 0x564449cceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.584+0000 7ff50d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cae260 0x564449cceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:47646/0 (socket says 172.21.15.79:47646) 2024-06-03T02:21:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.584+0000 7ff50d400700 1 -- 172.21.15.79:0/3444827989 learned_addr learned my addr 172.21.15.79:0/3444827989 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.584+0000 7ff50d400700 1 -- 172.21.15.79:0/3444827989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae960 msgr2=0x564449ca8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:09.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.584+0000 7ff50ca00700 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae960 0x564449ca8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.585+0000 7ff507e00700 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cbfd80 0x564449cbe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.586+0000 7ff50d400700 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae960 0x564449ca8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.586+0000 7ff50d400700 1 -- 172.21.15.79:0/3444827989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cbfd80 msgr2=0x564449cbe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:09.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.586+0000 7ff50d400700 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cbfd80 0x564449cbe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.586+0000 7ff50d400700 1 -- 172.21.15.79:0/3444827989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564449ca4190 con 0x564449cae260 2024-06-03T02:21:09.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.587+0000 7ff50ca00700 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae960 0x564449ca8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:09.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.587+0000 7ff507e00700 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cbfd80 0x564449cbe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:09.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.588+0000 7ff50d400700 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cae260 0x564449cceb70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5000a3b70 tx=0x7ff5000a2af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19b44337ac817235 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:09.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.589+0000 7ff4f9600700 1 -- 172.21.15.79:0/3444827989 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500097ef0 con 0x564449cae260 2024-06-03T02:21:09.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.590+0000 7ff4f9600700 1 -- 172.21.15.79:0/3444827989 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5000a40a0 con 0x564449cae260 2024-06-03T02:21:09.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.590+0000 7ff4f9600700 1 -- 172.21.15.79:0/3444827989 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500098090 con 0x564449cae260 2024-06-03T02:21:09.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.590+0000 7ff519e4ab80 1 -- 172.21.15.79:0/3444827989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cae260 msgr2=0x564449cceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:09.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.590+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cae260 0x564449cceb70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5000a3b70 tx=0x7ff5000a2af0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.591+0000 7ff519e4ab80 1 -- 172.21.15.79:0/3444827989 shutdown_connections 2024-06-03T02:21:09.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.591+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cae260 0x564449cceb70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.591+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae960 0x564449ca8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.591+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/3444827989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cbfd80 0x564449cbe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.591+0000 7ff519e4ab80 1 -- 172.21.15.79:0/3444827989 >> 172.21.15.79:0/3444827989 conn(0x564449bd9170 msgr2=0x564449cbde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:09.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.591+0000 7ff519e4ab80 1 -- 172.21.15.79:0/3444827989 shutdown_connections 2024-06-03T02:21:09.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.591+0000 7ff519e4ab80 1 -- 172.21.15.79:0/3444827989 wait complete. 2024-06-03T02:21:09.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.592+0000 7ff519e4ab80 1 -- start start 2024-06-03T02:21:09.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.592+0000 7ff519e4ab80 1 Processor -- start 2024-06-03T02:21:09.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.594+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae260 0x564449cdaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.594+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449cd7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:09 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:21:09.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.595+0000 7ff519e4ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cbfd80 0x564449cd7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.595+0000 7ff50ca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae260 0x564449cdaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.595+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081c6ca0 con 0x564449cae960 2024-06-03T02:21:09.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.595+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081c65d0 con 0x564449cae260 2024-06-03T02:21:09.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.595+0000 7ff519e4ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081c37d0 con 0x564449cbfd80 2024-06-03T02:21:09.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.595+0000 7ff519e4ab80 10 client.0 _ll_register_callbacks cb 0x564449b98720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:09.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.595+0000 7ff50ca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae260 0x564449cdaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36468/0 (socket says 172.21.15.79:36468) 2024-06-03T02:21:09.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.596+0000 7ff50ca00700 1 -- 172.21.15.79:0/1290438259 learned_addr learned my addr 172.21.15.79:0/1290438259 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:09.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.596+0000 7ff50d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cbfd80 0x564449cd7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.596+0000 7ff507e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449cd7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.597+0000 7ff50ca00700 1 -- 172.21.15.79:0/1290438259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cbfd80 msgr2=0x564449cd7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:09.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.597+0000 7ff50ca00700 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cbfd80 0x564449cd7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.597+0000 7ff50ca00700 1 -- 172.21.15.79:0/1290438259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 msgr2=0x564449cd7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:09.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.597+0000 7ff50ca00700 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449cd7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.597+0000 7ff50ca00700 1 -- 172.21.15.79:0/1290438259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564449cd85f0 con 0x564449cae260 2024-06-03T02:21:09.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.598+0000 7ff50ca00700 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae260 0x564449cdaff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0098e0 tx=0x7ff4fc00b2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c107edeadb13850 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:09.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.598+0000 7ff4eea00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:21:09.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.598+0000 7ff507e00700 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449cd7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:09.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.598+0000 7ff4eea00700 1 -- 172.21.15.79:0/1290438259 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc031080 con 0x564449cae260 2024-06-03T02:21:09.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.598+0000 7ff4eea00700 1 -- 172.21.15.79:0/1290438259 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff4fc003680 con 0x564449cae260 2024-06-03T02:21:09.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.598+0000 7ff519e4ab80 1 -- 172.21.15.79:0/1290438259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5000a40a0 con 0x564449cae260 2024-06-03T02:21:09.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff4eea00700 1 -- 172.21.15.79:0/1290438259 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4fc027020 con 0x564449cae260 2024-06-03T02:21:09.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff519e4ab80 10 client.25445 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:09.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff519e4ab80 1 -- 172.21.15.79:0/1290438259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564449d329a0 con 0x564449cae260 2024-06-03T02:21:09.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff4eea00700 1 -- 172.21.15.79:0/1290438259 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5000a40a0 con 0x564449cae260 2024-06-03T02:21:09.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff4eea00700 1 -- 172.21.15.79:0/1290438259 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff4fc021070 con 0x564449cae260 2024-06-03T02:21:09.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff519e4ab80 10 client.25445 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:09.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff519e4ab80 10 client.25445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:09.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff519e4ab80 20 client.25445 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:09.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564449d33080 0x564449d354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:09.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff519e4ab80 4 client.25445 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:21:09.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.599+0000 7ff519e4ab80 1 -- 172.21.15.79:0/1290438259 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564449c06580 con 0x564449d33080 2024-06-03T02:21:09.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.600+0000 7ff507e00700 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564449d33080 0x564449d354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:09.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.601+0000 7ff507e00700 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564449d33080 0x564449d354a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e89ddbe805f137 server_cookie=3c2e98373d5d2793 in_seq=0 out_seq=0 2024-06-03T02:21:09.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.601+0000 7ff4eea00700 10 client.25445 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:09.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.601+0000 7ff4eea00700 1 -- 172.21.15.79:0/1290438259 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564449c06580 con 0x564449d33080 2024-06-03T02:21:09.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.601+0000 7ff4eea00700 10 client.25445 handle_command_reply: tid=0 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.601+0000 7ff519e4ab80 1 client.25445 shutdown 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff519e4ab80 1 -- 172.21.15.79:0/1290438259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae260 msgr2=0x564449cdaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae260 0x564449cdaff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0098e0 tx=0x7ff4fc00b2a0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff519e4ab80 1 -- 172.21.15.79:0/1290438259 shutdown_connections 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564449cbfd80 0x564449cd7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564449cae260 0x564449cdaff0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564449d33080 0x564449d354a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff519e4ab80 1 --2- 172.21.15.79:0/1290438259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564449cae960 0x564449cd7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff519e4ab80 1 -- 172.21.15.79:0/1290438259 >> 172.21.15.79:0/1290438259 conn(0x564449bd9170 msgr2=0x564449cbe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff4eea00700 0 client.25445 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:09.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.602+0000 7ff519e4ab80 1 -- 172.21.15.79:0/1290438259 shutdown_connections 2024-06-03T02:21:09.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.603+0000 7ff519e4ab80 1 -- 172.21.15.79:0/1290438259 wait complete. 2024-06-03T02:21:09.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.603+0000 7ff519e4ab80 20 client.25445 trim_cache size 0 max 16384 2024-06-03T02:21:09.604 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:21:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:21:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:21:09.605 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:21:09.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.604+0000 7ff50de00700 1 -- 172.21.15.79:0/4007803471 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff4f00a2d70 msgr2=0x7ff4f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:09.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.604+0000 7ff50de00700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff4f00a2d70 0x7ff4f00a5230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff508073380 tx=0x7ff4f4009190 comp rx=0 tx=0).stop 2024-06-03T02:21:09.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.604+0000 7ff50de00700 1 -- 172.21.15.79:0/4007803471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff508113880 msgr2=0x7ff5081a7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:09.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.604+0000 7ff50de00700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff508113880 0x7ff5081a7f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff50000ef90 tx=0x7ff50000c3a0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.605+0000 7ff50de00700 1 -- 172.21.15.79:0/4007803471 shutdown_connections 2024-06-03T02:21:09.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.605+0000 7ff50de00700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff508113880 0x7ff5081a7f50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.605+0000 7ff50de00700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff4f00a2d70 0x7ff4f00a5230 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.605+0000 7ff50de00700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff508109bf0 0x7ff5081a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.605+0000 7ff50de00700 1 --2- 172.21.15.79:0/4007803471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50810a060 0x7ff5081a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.605+0000 7ff50de00700 1 -- 172.21.15.79:0/4007803471 >> 172.21.15.79:0/4007803471 conn(0x7ff50806d470 msgr2=0x7ff508116d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.605+0000 7ff50de00700 1 -- 172.21.15.79:0/4007803471 shutdown_connections 2024-06-03T02:21:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:09.605+0000 7ff50de00700 1 -- 172.21.15.79:0/4007803471 wait complete. 2024-06-03T02:21:09.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:09 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:21:10.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:10 smithi079 ceph-mon[31819]: pgmap v549: 129 pgs: 129 active+clean; 983 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 851 op/s 2024-06-03T02:21:10.575 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:21:10.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:21:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:10 smithi135 ceph-mon[36653]: pgmap v549: 129 pgs: 129 active+clean; 983 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 851 op/s 2024-06-03T02:21:10.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:10 smithi123 ceph-mon[36640]: pgmap v549: 129 pgs: 129 active+clean; 983 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 851 op/s 2024-06-03T02:21:11.224 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:12 smithi079 ceph-mon[31819]: pgmap v550: 129 pgs: 129 active+clean; 983 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-06-03T02:21:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:12 smithi135 ceph-mon[36653]: pgmap v550: 129 pgs: 129 active+clean; 983 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-06-03T02:21:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:12 smithi123 ceph-mon[36640]: pgmap v550: 129 pgs: 129 active+clean; 983 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 742 op/s 2024-06-03T02:21:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 65.0 s, 1936.0 tps, lat 51.862 ms stddev 64.697 2024-06-03T02:21:14.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.128+0000 7f5ba2000700 1 -- 172.21.15.79:0/4148648605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c1073c0 msgr2=0x7f5b9c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:14.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.128+0000 7f5ba2000700 1 --2- 172.21.15.79:0/4148648605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c1073c0 0x7f5b9c06a1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009f90 tx=0x7f5b9400b330 comp rx=0 tx=0).stop 2024-06-03T02:21:14.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.130+0000 7f5ba2000700 1 -- 172.21.15.79:0/4148648605 shutdown_connections 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.130+0000 7f5ba2000700 1 --2- 172.21.15.79:0/4148648605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b9c104ed0 0x7f5b9c105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.130+0000 7f5ba2000700 1 --2- 172.21.15.79:0/4148648605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c1073c0 0x7f5b9c06a1a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.130+0000 7f5ba2000700 1 --2- 172.21.15.79:0/4148648605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06a6e0 0x7f5b9c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.130+0000 7f5ba2000700 1 -- 172.21.15.79:0/4148648605 >> 172.21.15.79:0/4148648605 conn(0x7f5b9c078340 msgr2=0x7f5b9c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.130+0000 7f5ba2000700 1 -- 172.21.15.79:0/4148648605 shutdown_connections 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.130+0000 7f5ba2000700 1 -- 172.21.15.79:0/4148648605 wait complete. 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.132+0000 7f5ba2000700 1 Processor -- start 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.132+0000 7f5ba2000700 1 -- start start 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.132+0000 7f5ba2000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06a6e0 0x7f5b9c115750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.132+0000 7f5ba2000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c104ed0 0x7f5b9c115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:14.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.132+0000 7f5ba2000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b9c1073c0 0x7f5b9c111a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.132+0000 7f5ba2000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10a2d0 con 0x7f5b9c06a6e0 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.132+0000 7f5ba2000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10a150 con 0x7f5b9c104ed0 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.132+0000 7f5ba2000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10a450 con 0x7f5b9c1073c0 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c104ed0 0x7f5b9c115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b9c1073c0 0x7f5b9c111a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c104ed0 0x7f5b9c115c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36484/0 (socket says 172.21.15.79:36484) 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9aa00700 1 -- 172.21.15.79:0/1739800115 learned_addr learned my addr 172.21.15.79:0/1739800115 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9b400700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06a6e0 0x7f5b9c115750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9be00700 1 -- 172.21.15.79:0/1739800115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c104ed0 msgr2=0x7f5b9c115c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9be00700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c104ed0 0x7f5b9c115c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9be00700 1 -- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06a6e0 msgr2=0x7f5b9c115750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:14.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9be00700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06a6e0 0x7f5b9c115750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9be00700 1 -- 172.21.15.79:0/1739800115 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c112350 con 0x7f5b9c1073c0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.133+0000 7f5b9b400700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06a6e0 0x7f5b9c115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.135+0000 7f5b9be00700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b9c1073c0 0x7f5b9c111a50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b8801e8f0 tx=0x7f5b8801ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.135+0000 7f5b93e00700 1 -- 172.21.15.79:0/1739800115 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88028070 con 0x7f5b9c1073c0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.135+0000 7f5b93e00700 1 -- 172.21.15.79:0/1739800115 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b88004d10 con 0x7f5b9c1073c0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.135+0000 7f5b93e00700 1 -- 172.21.15.79:0/1739800115 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8801a9d0 con 0x7f5b9c1073c0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.136+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c112640 con 0x7f5b9c1073c0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.136+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1a34d0 con 0x7f5b9c1073c0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.140+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c1019b0 con 0x7f5b9c1073c0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.141+0000 7f5b93e00700 1 -- 172.21.15.79:0/1739800115 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5b8801ab90 con 0x7f5b9c1073c0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.143+0000 7f5b93e00700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b800a2ca0 0x7f5b800a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.143+0000 7f5b93e00700 1 -- 172.21.15.79:0/1739800115 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5b880dfba0 con 0x7f5b9c1073c0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.144+0000 7f5b9b400700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b800a2ca0 0x7f5b800a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:14.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.146+0000 7f5b9b400700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b800a2ca0 0x7f5b800a5160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400b770 tx=0x7f5b94009c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:14.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.149+0000 7f5b93e00700 1 -- 172.21.15.79:0/1739800115 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b880ac750 con 0x7f5b9c1073c0 2024-06-03T02:21:14.383 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:14 smithi079 ceph-mon[31819]: pgmap v551: 129 pgs: 129 active+clean; 984 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 922 op/s 2024-06-03T02:21:14.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:14 smithi135 ceph-mon[36653]: pgmap v551: 129 pgs: 129 active+clean; 984 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 922 op/s 2024-06-03T02:21:14.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.617+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b9c1a3730 con 0x7f5b9c1073c0 2024-06-03T02:21:14.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.618+0000 7f5b93e00700 1 -- 172.21.15.79:0/1739800115 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f5b880ac050 con 0x7f5b9c1073c0 2024-06-03T02:21:14.619 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:21:14.619 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:21:14.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.622+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b800a2ca0 msgr2=0x7f5b800a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:14.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.622+0000 7f5ba2000700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b800a2ca0 0x7f5b800a5160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400b770 tx=0x7f5b94009c00 comp rx=0 tx=0).stop 2024-06-03T02:21:14.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.622+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b9c1073c0 msgr2=0x7f5b9c111a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:14.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.622+0000 7f5ba2000700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b9c1073c0 0x7f5b9c111a50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b8801e8f0 tx=0x7f5b8801ec00 comp rx=0 tx=0).stop 2024-06-03T02:21:14.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.623+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 shutdown_connections 2024-06-03T02:21:14.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.623+0000 7f5ba2000700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b9c1073c0 0x7f5b9c111a50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:14.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.623+0000 7f5ba2000700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b800a2ca0 0x7f5b800a5160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:14.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.623+0000 7f5ba2000700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06a6e0 0x7f5b9c115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:14.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.623+0000 7f5ba2000700 1 --2- 172.21.15.79:0/1739800115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c104ed0 0x7f5b9c115c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:14.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.623+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 >> 172.21.15.79:0/1739800115 conn(0x7f5b9c078340 msgr2=0x7f5b9c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:14.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.624+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 shutdown_connections 2024-06-03T02:21:14.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:14.624+0000 7f5ba2000700 1 -- 172.21.15.79:0/1739800115 wait complete. 2024-06-03T02:21:14.626 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:21:14.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:14 smithi123 ceph-mon[36640]: pgmap v551: 129 pgs: 129 active+clean; 984 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 922 op/s 2024-06-03T02:21:15.257 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:15 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1739800115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:21:15.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:15 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1739800115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:21:15.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:21:15.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:15 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1739800115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:21:16.258 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:16.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:16 smithi079 ceph-mon[31819]: pgmap v552: 129 pgs: 129 active+clean; 988 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 677 op/s 2024-06-03T02:21:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:16 smithi135 ceph-mon[36653]: pgmap v552: 129 pgs: 129 active+clean; 988 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 677 op/s 2024-06-03T02:21:16.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:16 smithi123 ceph-mon[36640]: pgmap v552: 129 pgs: 129 active+clean; 988 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 677 op/s 2024-06-03T02:21:18.265 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:18 smithi079 ceph-mon[31819]: pgmap v553: 129 pgs: 129 active+clean; 1000 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 722 op/s 2024-06-03T02:21:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 70.0 s, 1739.0 tps, lat 57.128 ms stddev 63.994 2024-06-03T02:21:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:18 smithi135 ceph-mon[36653]: pgmap v553: 129 pgs: 129 active+clean; 1000 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 722 op/s 2024-06-03T02:21:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:18 smithi123 ceph-mon[36640]: pgmap v553: 129 pgs: 129 active+clean; 1000 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 722 op/s 2024-06-03T02:21:19.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:19 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:21:19.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:19 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:21:19.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:19 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:21:20.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:20 smithi079 ceph-mon[31819]: pgmap v554: 129 pgs: 129 active+clean; 1000 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 687 op/s 2024-06-03T02:21:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:20 smithi135 ceph-mon[36653]: pgmap v554: 129 pgs: 129 active+clean; 1000 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 687 op/s 2024-06-03T02:21:20.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:20 smithi123 ceph-mon[36640]: pgmap v554: 129 pgs: 129 active+clean; 1000 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 687 op/s 2024-06-03T02:21:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:21:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:21:21.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:21:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:21:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:21:21.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:21:21.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:21:21.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:21:21.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:21:22.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:22 smithi079 ceph-mon[31819]: pgmap v555: 129 pgs: 129 active+clean; 1000 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 MiB/s wr, 538 op/s 2024-06-03T02:21:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:22 smithi135 ceph-mon[36653]: pgmap v555: 129 pgs: 129 active+clean; 1000 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 MiB/s wr, 538 op/s 2024-06-03T02:21:22.625 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:22 smithi123 ceph-mon[36640]: pgmap v555: 129 pgs: 129 active+clean; 1000 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 MiB/s wr, 538 op/s 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.632+0000 7f3667400700 1 -- 172.21.15.79:0/3523247940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36601073c0 msgr2=0x7f366006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.632+0000 7f3667400700 1 --2- 172.21.15.79:0/3523247940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36601073c0 0x7f366006a1a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3654009990 tx=0x7f3654009ca0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.635+0000 7f3667400700 1 -- 172.21.15.79:0/3523247940 shutdown_connections 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.635+0000 7f3667400700 1 --2- 172.21.15.79:0/3523247940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3660104ed0 0x7f3660105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.635+0000 7f3667400700 1 --2- 172.21.15.79:0/3523247940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36601073c0 0x7f366006a1a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.635+0000 7f3667400700 1 --2- 172.21.15.79:0/3523247940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f366006a6e0 0x7f366006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.635+0000 7f3667400700 1 -- 172.21.15.79:0/3523247940 >> 172.21.15.79:0/3523247940 conn(0x7f3660078340 msgr2=0x7f3660078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.635+0000 7f3667400700 1 -- 172.21.15.79:0/3523247940 shutdown_connections 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.635+0000 7f3667400700 1 -- 172.21.15.79:0/3523247940 wait complete. 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.637+0000 7f3667400700 1 Processor -- start 2024-06-03T02:21:22.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.638+0000 7f3667400700 1 -- start start 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.643+0000 7f3667400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366006a6e0 0x7f3660073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.645+0000 7f3667400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3660104ed0 0x7f366006cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.645+0000 7f3664a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366006a6e0 0x7f3660073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.645+0000 7f3664a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366006a6e0 0x7f3660073c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48926/0 (socket says 172.21.15.79:48926) 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.645+0000 7f3667400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36601073c0 0x7f366006d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.645+0000 7f3667400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3660118da0 con 0x7f3660104ed0 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.646+0000 7f3664a00700 1 -- 172.21.15.79:0/3982005509 learned_addr learned my addr 172.21.15.79:0/3982005509 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.646+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3660118c20 con 0x7f366006a6e0 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.647+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3660118f20 con 0x7f36601073c0 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.647+0000 7f3665400700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36601073c0 0x7f366006d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.649+0000 7f365fe00700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3660104ed0 0x7f366006cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.649+0000 7f3664a00700 1 -- 172.21.15.79:0/3982005509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36601073c0 msgr2=0x7f366006d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.649+0000 7f3664a00700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36601073c0 0x7f366006d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.649+0000 7f3664a00700 1 -- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3660104ed0 msgr2=0x7f366006cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.649+0000 7f3664a00700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3660104ed0 0x7f366006cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.649+0000 7f3664a00700 1 -- 172.21.15.79:0/3982005509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366006d8d0 con 0x7f366006a6e0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.649+0000 7f3665400700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36601073c0 0x7f366006d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.651+0000 7f3664a00700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366006a6e0 0x7f3660073c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3654005910 tx=0x7f3654003870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.655+0000 7f365d600700 1 -- 172.21.15.79:0/3982005509 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654003d00 con 0x7f366006a6e0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.656+0000 7f365d600700 1 -- 172.21.15.79:0/3982005509 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3654004830 con 0x7f366006a6e0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.656+0000 7f365d600700 1 -- 172.21.15.79:0/3982005509 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654005580 con 0x7f366006a6e0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.656+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366006db60 con 0x7f366006a6e0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.656+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3660071970 con 0x7f366006a6e0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.663+0000 7f365d600700 1 -- 172.21.15.79:0/3982005509 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f36540049e0 con 0x7f366006a6e0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.666+0000 7f365d600700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f36480a2df0 0x7f36480a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.666+0000 7f365d600700 1 -- 172.21.15.79:0/3982005509 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f36540d03b0 con 0x7f366006a6e0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.666+0000 7f365fe00700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f36480a2df0 0x7f36480a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.668+0000 7f365fe00700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f36480a2df0 0x7f36480a52b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f366006e4c0 tx=0x7f364c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.679+0000 7f3671d4eb80 1 Processor -- start 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.680+0000 7f3671d4eb80 1 -- start start 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.681+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.681+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.681+0000 7f3664a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.681+0000 7f3664a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5e2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37814/0 (socket says 172.21.15.79:37814) 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.682+0000 7f3664a00700 1 -- 172.21.15.79:0/1800330179 learned_addr learned my addr 172.21.15.79:0/1800330179 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.682+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5dd240 0x55748d5dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.682+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1800330179 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55748d4eba10 con 0x55748d5e34b0 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.682+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1800330179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55748d52eb00 con 0x55748d5dd240 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.682+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1800330179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55748d4e98f0 con 0x55748d5e2b90 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.682+0000 7f365fe00700 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.684+0000 7f365fe00700 1 -- 172.21.15.79:0/1800330179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 msgr2=0x55748d5e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:22.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.684+0000 7f3665400700 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5dd240 0x55748d5dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.684+0000 7f365fe00700 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5e2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.684+0000 7f365fe00700 1 -- 172.21.15.79:0/1800330179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5dd240 msgr2=0x55748d5dd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.684+0000 7f365fe00700 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5dd240 0x55748d5dd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.684+0000 7f365fe00700 1 -- 172.21.15.79:0/1800330179 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748d4d6f90 con 0x55748d5e34b0 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.684+0000 7f3665400700 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5dd240 0x55748d5dd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.684+0000 7f365fe00700 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5dcd00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f364c00b840 tx=0x7f364c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2fe29e86d0f8a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.685+0000 7f3652000700 1 -- 172.21.15.79:0/1800330179 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c020070 con 0x55748d5e34b0 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.685+0000 7f3652000700 1 -- 172.21.15.79:0/1800330179 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c01cd40 con 0x55748d5e34b0 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.685+0000 7f3652000700 1 -- 172.21.15.79:0/1800330179 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c020070 con 0x55748d5e34b0 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.686+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1800330179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 msgr2=0x55748d5dcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.686+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5dcd00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f364c00b840 tx=0x7f364c010000 comp rx=0 tx=0).stop 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.691+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1800330179 shutdown_connections 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.691+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5dd240 0x55748d5dd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.691+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.691+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1800330179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5dcd00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.691+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1800330179 >> 172.21.15.79:0/1800330179 conn(0x55748d509170 msgr2=0x55748d5d7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.691+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1800330179 shutdown_connections 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.691+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1800330179 wait complete. 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.692+0000 7f3671d4eb80 1 -- start start 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.692+0000 7f3671d4eb80 1 Processor -- start 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.693+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5dd240 0x55748d5e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.693+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 0x55748d5e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.693+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.693+0000 7f3671d4eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55748d4caf60 con 0x55748d5e34b0 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.693+0000 7f3671d4eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55748d52aac0 con 0x55748d5e2b90 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.693+0000 7f3671d4eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55748d5e9e90 con 0x55748d5dd240 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.693+0000 7f3671d4eb80 10 client.0 _ll_register_callbacks cb 0x55748d4c4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:22.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.695+0000 7f365fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 0x55748d5e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.695+0000 7f3664a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5dd240 0x55748d5e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.695+0000 7f3664a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5dd240 0x55748d5e91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37816/0 (socket says 172.21.15.79:37816) 2024-06-03T02:21:22.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.695+0000 7f3664a00700 1 -- 172.21.15.79:0/938117044 learned_addr learned my addr 172.21.15.79:0/938117044 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:22.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.696+0000 7f3665400700 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.697+0000 7f3665400700 1 -- 172.21.15.79:0/938117044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5dd240 msgr2=0x55748d5e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:22.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.697+0000 7f3665400700 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5dd240 0x55748d5e91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.697+0000 7f3665400700 1 -- 172.21.15.79:0/938117044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 msgr2=0x55748d5e64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.697+0000 7f3665400700 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 0x55748d5e64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.697+0000 7f3665400700 1 -- 172.21.15.79:0/938117044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748d47d8d0 con 0x55748d5e34b0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.697+0000 7f3665400700 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5e6a30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3658008bb0 tx=0x7f3658008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805168463fe64566 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.697+0000 7f3646a00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.698+0000 7f3646a00700 1 -- 172.21.15.79:0/938117044 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658017020 con 0x55748d5e34b0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.698+0000 7f3646a00700 1 -- 172.21.15.79:0/938117044 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f365800ea50 con 0x55748d5e34b0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.699+0000 7f3671d4eb80 1 -- 172.21.15.79:0/938117044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364c01cd40 con 0x55748d5e34b0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.699+0000 7f3646a00700 1 -- 172.21.15.79:0/938117044 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f365800ec50 con 0x55748d5e34b0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.699+0000 7f3671d4eb80 10 client.15303 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.699+0000 7f3671d4eb80 1 -- 172.21.15.79:0/938117044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55748d5f45f0 con 0x55748d5e34b0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.700+0000 7f3646a00700 1 -- 172.21.15.79:0/938117044 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c01cd40 con 0x55748d5e34b0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.700+0000 7f3646a00700 1 -- 172.21.15.79:0/938117044 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3658024020 con 0x55748d5e34b0 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.700+0000 7f3671d4eb80 10 client.15303 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:22.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.700+0000 7f3671d4eb80 10 client.15303 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.700+0000 7f3671d4eb80 20 client.15303 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.700+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55748d5f6c80 0x55748d5f9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.700+0000 7f3671d4eb80 4 client.15303 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.700+0000 7f3671d4eb80 1 -- 172.21.15.79:0/938117044 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55748d53c000 con 0x55748d5f6c80 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.700+0000 7f3664a00700 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55748d5f6c80 0x55748d5f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.701+0000 7f3664a00700 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55748d5f6c80 0x55748d5f9070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=912554180ed4f75d server_cookie=e467cb4aa30b9f75 in_seq=0 out_seq=0 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.703+0000 7f3646a00700 10 client.15303 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.703+0000 7f3646a00700 1 -- 172.21.15.79:0/938117044 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55748d53c000 con 0x55748d5f6c80 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.703+0000 7f3646a00700 10 client.15303 handle_command_reply: tid=0 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.703+0000 7f3671d4eb80 1 client.15303 shutdown 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.704+0000 7f3671d4eb80 1 -- 172.21.15.79:0/938117044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 msgr2=0x55748d5e6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.704+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5e6a30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3658008bb0 tx=0x7f3658008ec0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.704+0000 7f3671d4eb80 1 -- 172.21.15.79:0/938117044 shutdown_connections 2024-06-03T02:21:22.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.704+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55748d5f6c80 0x55748d5f9070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.704+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5e34b0 0x55748d5e6a30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.704+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5dd240 0x55748d5e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.704+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/938117044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 0x55748d5e64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.704+0000 7f3671d4eb80 1 -- 172.21.15.79:0/938117044 >> 172.21.15.79:0/938117044 conn(0x55748d509170 msgr2=0x55748d5d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:22.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.704+0000 7f3646a00700 0 client.15303 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:21:22.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.705+0000 7f3671d4eb80 1 -- 172.21.15.79:0/938117044 shutdown_connections 2024-06-03T02:21:22.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.705+0000 7f3671d4eb80 1 -- 172.21.15.79:0/938117044 wait complete. 2024-06-03T02:21:22.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.707+0000 7f3671d4eb80 20 client.15303 trim_cache size 0 max 16384 2024-06-03T02:21:22.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.743+0000 7f3671d4eb80 1 Processor -- start 2024-06-03T02:21:22.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.743+0000 7f3671d4eb80 1 -- start start 2024-06-03T02:21:22.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.744+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.745+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5d81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.745+0000 7f3664a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.745+0000 7f3664a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5dcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47434/0 (socket says 172.21.15.79:47434) 2024-06-03T02:21:22.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.745+0000 7f3664a00700 1 -- 172.21.15.79:0/3379018343 learned_addr learned my addr 172.21.15.79:0/3379018343 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:22.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.745+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/3379018343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e34b0 0x55748d5d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.745+0000 7f3671d4eb80 1 -- 172.21.15.79:0/3379018343 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f36601bd270 con 0x55748d5dd240 2024-06-03T02:21:22.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.746+0000 7f3671d4eb80 1 -- 172.21.15.79:0/3379018343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36601bedb0 con 0x55748d5e34b0 2024-06-03T02:21:22.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.746+0000 7f3671d4eb80 1 -- 172.21.15.79:0/3379018343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36601bd940 con 0x55748d5e2b90 2024-06-03T02:21:22.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.746+0000 7f365fe00700 1 --2- 172.21.15.79:0/3379018343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5d81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.746+0000 7f365fe00700 1 -- 172.21.15.79:0/3379018343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e34b0 msgr2=0x55748d5d2130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:21:22.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.746+0000 7f365fe00700 1 --2- 172.21.15.79:0/3379018343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e34b0 0x55748d5d2130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.746+0000 7f365fe00700 1 -- 172.21.15.79:0/3379018343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 msgr2=0x55748d5dcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:22.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.746+0000 7f365fe00700 1 --2- 172.21.15.79:0/3379018343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5dcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.747+0000 7f365fe00700 1 -- 172.21.15.79:0/3379018343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748d5e4a80 con 0x55748d5e2b90 2024-06-03T02:21:22.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.748+0000 7f365fe00700 1 --2- 172.21.15.79:0/3379018343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5d81a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f364c00fa60 tx=0x7f364c00fd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcadd445bde2a116 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:22.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.750+0000 7f3651600700 1 -- 172.21.15.79:0/3379018343 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c034070 con 0x55748d5e2b90 2024-06-03T02:21:22.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.750+0000 7f3651600700 1 -- 172.21.15.79:0/3379018343 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c005530 con 0x55748d5e2b90 2024-06-03T02:21:22.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.750+0000 7f3651600700 1 -- 172.21.15.79:0/3379018343 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c020040 con 0x55748d5e2b90 2024-06-03T02:21:22.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.751+0000 7f3671d4eb80 1 -- 172.21.15.79:0/3379018343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 msgr2=0x55748d5d81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:22.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.751+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/3379018343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5d81a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f364c00fa60 tx=0x7f364c00fd40 comp rx=0 tx=0).stop 2024-06-03T02:21:22.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.752+0000 7f3671d4eb80 1 -- 172.21.15.79:0/3379018343 shutdown_connections 2024-06-03T02:21:22.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.752+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/3379018343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e34b0 0x55748d5d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.752+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/3379018343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.752+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/3379018343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e2b90 0x55748d5d81a0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f364c00fa60 tx=0x7f364c00fd40 comp rx=0 tx=0).stop 2024-06-03T02:21:22.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.752+0000 7f3671d4eb80 1 -- 172.21.15.79:0/3379018343 >> 172.21.15.79:0/3379018343 conn(0x55748d509170 msgr2=0x55748d5d7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:22.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.753+0000 7f3671d4eb80 1 -- 172.21.15.79:0/3379018343 shutdown_connections 2024-06-03T02:21:22.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.754+0000 7f3671d4eb80 1 -- 172.21.15.79:0/3379018343 wait complete. 2024-06-03T02:21:22.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.754+0000 7f3671d4eb80 1 -- start start 2024-06-03T02:21:22.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.755+0000 7f3671d4eb80 1 Processor -- start 2024-06-03T02:21:22.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.758+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5ef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.758+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 0x55748d5ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.758+0000 7f3671d4eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e34b0 0x55748d5fdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.759+0000 7f3665400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e34b0 0x55748d5fdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.759+0000 7f3665400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e34b0 0x55748d5fdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37838/0 (socket says 172.21.15.79:37838) 2024-06-03T02:21:22.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.759+0000 7f3671d4eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f36601be6e0 con 0x55748d5dd240 2024-06-03T02:21:22.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.759+0000 7f3665400700 1 -- 172.21.15.79:0/1451177427 learned_addr learned my addr 172.21.15.79:0/1451177427 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:22.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.759+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36601be010 con 0x55748d5e2b90 2024-06-03T02:21:22.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.759+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36601bb210 con 0x55748d5e34b0 2024-06-03T02:21:22.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.759+0000 7f3671d4eb80 10 client.0 _ll_register_callbacks cb 0x55748d4c4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:22.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.759+0000 7f3665400700 1 -- 172.21.15.79:0/1451177427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 msgr2=0x55748d5ef8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:21:22.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.759+0000 7f3664a00700 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.760+0000 7f365fe00700 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 0x55748d5ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.761+0000 7f3665400700 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 0x55748d5ef8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.761+0000 7f3665400700 1 -- 172.21.15.79:0/1451177427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 msgr2=0x55748d5ef360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:22.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.761+0000 7f3665400700 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5ef360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.761+0000 7f3665400700 1 -- 172.21.15.79:0/1451177427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748d5fe430 con 0x55748d5e34b0 2024-06-03T02:21:22.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.761+0000 7f3664a00700 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:22.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.762+0000 7f3665400700 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e34b0 0x55748d5fdd10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3658002230 tx=0x7f36580134e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa39be5b411c531a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:22.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.762+0000 7f3646a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:21:22.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.763+0000 7f3646a00700 1 -- 172.21.15.79:0/1451177427 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658017040 con 0x55748d5e34b0 2024-06-03T02:21:22.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.764+0000 7f3646a00700 1 -- 172.21.15.79:0/1451177427 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3658013de0 con 0x55748d5e34b0 2024-06-03T02:21:22.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.764+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364c005530 con 0x55748d5e34b0 2024-06-03T02:21:22.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.765+0000 7f3646a00700 1 -- 172.21.15.79:0/1451177427 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3658035050 con 0x55748d5e34b0 2024-06-03T02:21:22.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.765+0000 7f3671d4eb80 10 client.25450 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:22.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.765+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55748d661310 con 0x55748d5e34b0 2024-06-03T02:21:22.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.765+0000 7f3646a00700 1 -- 172.21.15.79:0/1451177427 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c005530 con 0x55748d5e34b0 2024-06-03T02:21:22.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.766+0000 7f3646a00700 1 -- 172.21.15.79:0/1451177427 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3658030c20 con 0x55748d5e34b0 2024-06-03T02:21:22.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.766+0000 7f3671d4eb80 10 client.25450 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:22.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.766+0000 7f3671d4eb80 10 client.25450 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:21:22.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.766+0000 7f3671d4eb80 20 client.25450 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:22.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.766+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55748d661a70 0x55748d663e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:22.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.767+0000 7f3671d4eb80 4 client.25450 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:21:22.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.767+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55748d536580 con 0x55748d661a70 2024-06-03T02:21:22.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.768+0000 7f3664a00700 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55748d661a70 0x55748d663e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:22.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.769+0000 7f3664a00700 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55748d661a70 0x55748d663e90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f536b28ddf4af4a server_cookie=9b5ab01bd057a6fe in_seq=0 out_seq=0 2024-06-03T02:21:22.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.769+0000 7f3646a00700 10 client.25450 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:21:22.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.769+0000 7f3646a00700 1 -- 172.21.15.79:0/1451177427 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55748d536580 con 0x55748d661a70 2024-06-03T02:21:22.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.769+0000 7f3646a00700 10 client.25450 handle_command_reply: tid=0 2024-06-03T02:21:22.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.769+0000 7f3671d4eb80 1 client.25450 shutdown 2024-06-03T02:21:22.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.770+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e34b0 msgr2=0x55748d5fdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:22.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.770+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e34b0 0x55748d5fdd10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3658002230 tx=0x7f36580134e0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.771+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 shutdown_connections 2024-06-03T02:21:22.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.771+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55748d661a70 0x55748d663e90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.771+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55748d5e34b0 0x55748d5fdd10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.771+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55748d5dd240 0x55748d5ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.771+0000 7f3671d4eb80 1 --2- 172.21.15.79:0/1451177427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55748d5e2b90 0x55748d5ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.771+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 >> 172.21.15.79:0/1451177427 conn(0x55748d509170 msgr2=0x55748d5e3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:22.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.771+0000 7f3646a00700 0 client.25450 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:21:22.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.772+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 shutdown_connections 2024-06-03T02:21:22.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.773+0000 7f3671d4eb80 1 -- 172.21.15.79:0/1451177427 wait complete. 2024-06-03T02:21:22.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.773+0000 7f3671d4eb80 20 client.25450 trim_cache size 0 max 16384 2024-06-03T02:21:22.775 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:21:22.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f36480a2df0 msgr2=0x7f36480a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:22.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f36480a2df0 0x7f36480a52b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f366006e4c0 tx=0x7f364c005d50 comp rx=0 tx=0).stop 2024-06-03T02:21:22.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366006a6e0 msgr2=0x7f3660073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:22.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366006a6e0 0x7f3660073c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3654005910 tx=0x7f3654003870 comp rx=0 tx=0).stop 2024-06-03T02:21:22.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 shutdown_connections 2024-06-03T02:21:22.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36601073c0 0x7f366006d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366006a6e0 0x7f3660073c70 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3654005910 tx=0x7f3654003870 comp rx=0 tx=0).stop 2024-06-03T02:21:22.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f36480a2df0 0x7f36480a52b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 --2- 172.21.15.79:0/3982005509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3660104ed0 0x7f366006cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:22.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.779+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 >> 172.21.15.79:0/3982005509 conn(0x7f3660078340 msgr2=0x7f3660100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:22.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.780+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 shutdown_connections 2024-06-03T02:21:22.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:22.780+0000 7f3667400700 1 -- 172.21.15.79:0/3982005509 wait complete. 2024-06-03T02:21:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 75.0 s, 1127.6 tps, lat 77.116 ms stddev 89.933 2024-06-03T02:21:24.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:21:24.583 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:24 smithi079 ceph-mon[31819]: pgmap v556: 129 pgs: 129 active+clean; 1000 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.1 MiB/s wr, 731 op/s 2024-06-03T02:21:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:24 smithi135 ceph-mon[36653]: pgmap v556: 129 pgs: 129 active+clean; 1000 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.1 MiB/s wr, 731 op/s 2024-06-03T02:21:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:24 smithi123 ceph-mon[36640]: pgmap v556: 129 pgs: 129 active+clean; 1000 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.1 MiB/s wr, 731 op/s 2024-06-03T02:21:25.160 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:26 smithi079 ceph-mon[31819]: pgmap v557: 129 pgs: 129 active+clean; 1001 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.5 MiB/s wr, 598 op/s 2024-06-03T02:21:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:26 smithi135 ceph-mon[36653]: pgmap v557: 129 pgs: 129 active+clean; 1001 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.5 MiB/s wr, 598 op/s 2024-06-03T02:21:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:26 smithi123 ceph-mon[36640]: pgmap v557: 129 pgs: 129 active+clean; 1001 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.5 MiB/s wr, 598 op/s 2024-06-03T02:21:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 80.0 s, 1327.4 tps, lat 85.098 ms stddev 136.658 2024-06-03T02:21:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:28 smithi079 ceph-mon[31819]: pgmap v558: 129 pgs: 129 active+clean; 1002 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 12 MiB/s wr, 810 op/s 2024-06-03T02:21:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:28 smithi135 ceph-mon[36653]: pgmap v558: 129 pgs: 129 active+clean; 1002 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 12 MiB/s wr, 810 op/s 2024-06-03T02:21:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:28 smithi123 ceph-mon[36640]: pgmap v558: 129 pgs: 129 active+clean; 1002 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 12 MiB/s wr, 810 op/s 2024-06-03T02:21:29.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.416+0000 7fdd5c400700 1 -- 172.21.15.79:0/898341713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 msgr2=0x7fdd54109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:29.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.416+0000 7fdd5c400700 1 --2- 172.21.15.79:0/898341713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 0x7fdd54109490 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009990 tx=0x7fdd40009ca0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.416+0000 7fdd5c400700 1 -- 172.21.15.79:0/898341713 shutdown_connections 2024-06-03T02:21:29.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.416+0000 7fdd5c400700 1 --2- 172.21.15.79:0/898341713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd54117520 0x7fdd54119910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.416+0000 7fdd5c400700 1 --2- 172.21.15.79:0/898341713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 0x7fdd54109490 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.416+0000 7fdd5c400700 1 --2- 172.21.15.79:0/898341713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd541099d0 0x7fdd54109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.416+0000 7fdd5c400700 1 -- 172.21.15.79:0/898341713 >> 172.21.15.79:0/898341713 conn(0x7fdd5406d600 msgr2=0x7fdd5406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:29.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.419+0000 7fdd5c400700 1 -- 172.21.15.79:0/898341713 shutdown_connections 2024-06-03T02:21:29.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.419+0000 7fdd5c400700 1 -- 172.21.15.79:0/898341713 wait complete. 2024-06-03T02:21:29.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.422+0000 7fdd5c400700 1 Processor -- start 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.422+0000 7fdd5c400700 1 -- start start 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.423+0000 7fdd5c400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 0x7fdd541a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.423+0000 7fdd5c400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd54117520 0x7fdd541a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.423+0000 7fdd5c400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd541a3690 0x7fdd541a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.424+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 0x7fdd541a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.424+0000 7fdd5c400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5411d270 con 0x7fdd54117520 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.424+0000 7fdd5c400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5411d0f0 con 0x7fdd5410cbb0 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.424+0000 7fdd5c400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5411d3f0 con 0x7fdd541a3690 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.424+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 0x7fdd541a95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47264/0 (socket says 172.21.15.79:47264) 2024-06-03T02:21:29.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.424+0000 7fdd59a00700 1 -- 172.21.15.79:0/2478966117 learned_addr learned my addr 172.21.15.79:0/2478966117 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:29.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.424+0000 7fdd59a00700 1 -- 172.21.15.79:0/2478966117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd541a3690 msgr2=0x7fdd541a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:29.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.424+0000 7fdd59a00700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd541a3690 0x7fdd541a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.425+0000 7fdd59a00700 1 -- 172.21.15.79:0/2478966117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd54117520 msgr2=0x7fdd541a9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:29.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.425+0000 7fdd59a00700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd54117520 0x7fdd541a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.425+0000 7fdd59a00700 1 -- 172.21.15.79:0/2478966117 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd541a4270 con 0x7fdd5410cbb0 2024-06-03T02:21:29.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.425+0000 7fdd59a00700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 0x7fdd541a95c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000b5c0 tx=0x7fdd40005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:29.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.425+0000 7fdd52000700 1 -- 172.21.15.79:0/2478966117 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40026070 con 0x7fdd5410cbb0 2024-06-03T02:21:29.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.426+0000 7fdd52000700 1 -- 172.21.15.79:0/2478966117 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd40004170 con 0x7fdd5410cbb0 2024-06-03T02:21:29.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.426+0000 7fdd52000700 1 -- 172.21.15.79:0/2478966117 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40021410 con 0x7fdd5410cbb0 2024-06-03T02:21:29.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.426+0000 7fdd5c400700 1 -- 172.21.15.79:0/2478966117 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541a4500 con 0x7fdd5410cbb0 2024-06-03T02:21:29.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.426+0000 7fdd5c400700 1 -- 172.21.15.79:0/2478966117 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd54076900 con 0x7fdd5410cbb0 2024-06-03T02:21:29.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.433+0000 7fdd52000700 1 -- 172.21.15.79:0/2478966117 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdd4002a3f0 con 0x7fdd5410cbb0 2024-06-03T02:21:29.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.436+0000 7fdd52000700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdd3c0a2d20 0x7fdd3c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.436+0000 7fdd52000700 1 -- 172.21.15.79:0/2478966117 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fdd400d11d0 con 0x7fdd5410cbb0 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.437+0000 7fdd59000700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdd3c0a2d20 0x7fdd3c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.437+0000 7fdd66d98b80 1 Processor -- start 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.437+0000 7fdd66d98b80 1 -- start start 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e66070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e665b0 0x557702e66a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e71300 0x557702e736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd66d98b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557702d75a10 con 0x557702e71300 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd66d98b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557702db8b00 con 0x557702e665b0 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd66d98b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557702d738f0 con 0x557702de2e40 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e66070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e66070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46744/0 (socket says 172.21.15.79:46744) 2024-06-03T02:21:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd59a00700 1 -- 172.21.15.79:0/1242314635 learned_addr learned my addr 172.21.15.79:0/1242314635 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:29.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd59a00700 1 -- 172.21.15.79:0/1242314635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e665b0 msgr2=0x557702e66a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:21:29.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd59a00700 1 --2- 172.21.15.79:0/1242314635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e665b0 0x557702e66a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd59a00700 1 -- 172.21.15.79:0/1242314635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e71300 msgr2=0x557702e736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:29.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.438+0000 7fdd59a00700 1 --2- 172.21.15.79:0/1242314635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e71300 0x557702e736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.439+0000 7fdd59a00700 1 -- 172.21.15.79:0/1242314635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557702d60f90 con 0x557702de2e40 2024-06-03T02:21:29.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.439+0000 7fdd59a00700 1 --2- 172.21.15.79:0/1242314635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e66070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdd4009d640 tx=0x7fdd400054f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=226e868b02ccabcc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:29.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.439+0000 7fdd46a00700 1 -- 172.21.15.79:0/1242314635 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4009ddb0 con 0x557702de2e40 2024-06-03T02:21:29.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.439+0000 7fdd46a00700 1 -- 172.21.15.79:0/1242314635 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4009d6b0 con 0x557702de2e40 2024-06-03T02:21:29.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.439+0000 7fdd46a00700 1 -- 172.21.15.79:0/1242314635 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd400a6060 con 0x557702de2e40 2024-06-03T02:21:29.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.440+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1242314635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 msgr2=0x557702e66070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:29.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.440+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1242314635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e66070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdd4009d640 tx=0x7fdd400054f0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.441+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1242314635 shutdown_connections 2024-06-03T02:21:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.441+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1242314635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e71300 0x557702e736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.441+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1242314635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e66070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.441+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1242314635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e665b0 0x557702e66a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.441+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1242314635 >> 172.21.15.79:0/1242314635 conn(0x557702d93170 msgr2=0x557702e624b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.442+0000 7fdd59000700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdd3c0a2d20 0x7fdd3c0a51e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x557702e63920 tx=0x7fdd48009230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.442+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1242314635 shutdown_connections 2024-06-03T02:21:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.443+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1242314635 wait complete. 2024-06-03T02:21:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.443+0000 7fdd66d98b80 1 -- start start 2024-06-03T02:21:29.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.444+0000 7fdd66d98b80 1 Processor -- start 2024-06-03T02:21:29.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.445+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e70a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.445+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e6dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.445+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e6e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.445+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e70a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.445+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e70a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46760/0 (socket says 172.21.15.79:46760) 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.446+0000 7fdd59a00700 1 -- 172.21.15.79:0/1150126659 learned_addr learned my addr 172.21.15.79:0/1150126659 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.445+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557702d54f60 con 0x557702e665b0 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.446+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557702db4ac0 con 0x557702e71300 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.446+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557702e73d70 con 0x557702de2e40 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.446+0000 7fdd66d98b80 10 client.0 _ll_register_callbacks cb 0x557702d52720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.446+0000 7fdd5a400700 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e6e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.446+0000 7fdd59000700 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e6dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.447+0000 7fdd59a00700 1 -- 172.21.15.79:0/1150126659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 msgr2=0x557702e6e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.447+0000 7fdd59a00700 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e6e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.447+0000 7fdd59a00700 1 -- 172.21.15.79:0/1150126659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 msgr2=0x557702e6dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.447+0000 7fdd59a00700 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e6dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.447+0000 7fdd59a00700 1 -- 172.21.15.79:0/1150126659 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557702d078d0 con 0x557702de2e40 2024-06-03T02:21:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.447+0000 7fdd59000700 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e6dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:21:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.447+0000 7fdd5a400700 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e6e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:21:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.448+0000 7fdd59a00700 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e70a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd40006010 tx=0x7fdd400a40d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be5ad67d824aff server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.448+0000 7fdd3b400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:21:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.448+0000 7fdd3b400700 1 -- 172.21.15.79:0/1150126659 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd400b6070 con 0x557702de2e40 2024-06-03T02:21:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.448+0000 7fdd3b400700 1 -- 172.21.15.79:0/1150126659 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fdd400b1b40 con 0x557702de2e40 2024-06-03T02:21:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.448+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd4009d6b0 con 0x557702de2e40 2024-06-03T02:21:29.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.450+0000 7fdd3b400700 1 -- 172.21.15.79:0/1150126659 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd400a1a60 con 0x557702de2e40 2024-06-03T02:21:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.452+0000 7fdd66d98b80 10 client.25456 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.452+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557702e86c40 con 0x557702de2e40 2024-06-03T02:21:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.453+0000 7fdd3b400700 1 -- 172.21.15.79:0/1150126659 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4009d6b0 con 0x557702de2e40 2024-06-03T02:21:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.453+0000 7fdd3b400700 1 -- 172.21.15.79:0/1150126659 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdd40099400 con 0x557702de2e40 2024-06-03T02:21:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.453+0000 7fdd66d98b80 10 client.25456 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.453+0000 7fdd66d98b80 10 client.25456 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:21:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.453+0000 7fdd66d98b80 20 client.25456 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.453+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x557702e88f40 0x557702e8b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.453+0000 7fdd66d98b80 4 client.25456 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:21:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.453+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557702dc6000 con 0x557702e88f40 2024-06-03T02:21:29.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.455+0000 7fdd59000700 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x557702e88f40 0x557702e8b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.456+0000 7fdd59000700 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x557702e88f40 0x557702e8b330 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97706ad7cd2dd152 server_cookie=9720140e998b9e40 in_seq=0 out_seq=0 2024-06-03T02:21:29.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.456+0000 7fdd3b400700 10 client.25456 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:21:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.457+0000 7fdd3b400700 1 -- 172.21.15.79:0/1150126659 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557702dc6000 con 0x557702e88f40 2024-06-03T02:21:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.457+0000 7fdd3b400700 10 client.25456 handle_command_reply: tid=0 2024-06-03T02:21:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.458+0000 7fdd66d98b80 1 client.25456 shutdown 2024-06-03T02:21:29.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.459+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 msgr2=0x557702e70a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:29.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.459+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e70a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd40006010 tx=0x7fdd400a40d0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.459+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 shutdown_connections 2024-06-03T02:21:29.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.459+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e6e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.460+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e70a10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.460+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x557702e88f40 0x557702e8b330 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.460+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/1150126659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e6dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.460+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 >> 172.21.15.79:0/1150126659 conn(0x557702d93170 msgr2=0x557702e61740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:29.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.460+0000 7fdd3b400700 0 client.25456 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:21:29.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.461+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 shutdown_connections 2024-06-03T02:21:29.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.461+0000 7fdd66d98b80 1 -- 172.21.15.79:0/1150126659 wait complete. 2024-06-03T02:21:29.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.461+0000 7fdd66d98b80 20 client.25456 trim_cache size 0 max 16384 2024-06-03T02:21:29.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.489+0000 7fdd66d98b80 1 Processor -- start 2024-06-03T02:21:29.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.489+0000 7fdd66d98b80 1 -- start start 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e70550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd66d98b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3400eb90 con 0x557702e665b0 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd66d98b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdd340106d0 con 0x557702e71300 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd66d98b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3400f260 con 0x557702de2e40 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd5a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e70550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd5a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e70550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47290/0 (socket says 172.21.15.79:47290) 2024-06-03T02:21:29.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.490+0000 7fdd5a400700 1 -- 172.21.15.79:0/440578130 learned_addr learned my addr 172.21.15.79:0/440578130 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:29.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.491+0000 7fdd59000700 1 --2- 172.21.15.79:0/440578130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.491+0000 7fdd5a400700 1 -- 172.21.15.79:0/440578130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 msgr2=0x557702e737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:29.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.491+0000 7fdd5a400700 1 --2- 172.21.15.79:0/440578130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e737c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.491+0000 7fdd5a400700 1 -- 172.21.15.79:0/440578130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 msgr2=0x557702e726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:29.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.491+0000 7fdd5a400700 1 --2- 172.21.15.79:0/440578130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.491+0000 7fdd5a400700 1 -- 172.21.15.79:0/440578130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557702e66d90 con 0x557702e71300 2024-06-03T02:21:29.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.491+0000 7fdd59a00700 1 --2- 172.21.15.79:0/440578130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:21:29.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.491+0000 7fdd5a400700 1 --2- 172.21.15.79:0/440578130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e70550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0036b0 tx=0x7fdd4c0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c6a6941cac38a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:29.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.491+0000 7fdd46000700 1 -- 172.21.15.79:0/440578130 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004a30 con 0x557702e71300 2024-06-03T02:21:29.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.492+0000 7fdd46000700 1 -- 172.21.15.79:0/440578130 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4c015070 con 0x557702e71300 2024-06-03T02:21:29.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.492+0000 7fdd46000700 1 -- 172.21.15.79:0/440578130 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00ba00 con 0x557702e71300 2024-06-03T02:21:29.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.492+0000 7fdd66d98b80 1 -- 172.21.15.79:0/440578130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 msgr2=0x557702e70550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:29.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.492+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/440578130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e70550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0036b0 tx=0x7fdd4c0039c0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.493+0000 7fdd66d98b80 1 -- 172.21.15.79:0/440578130 shutdown_connections 2024-06-03T02:21:29.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.493+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/440578130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e70550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.493+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/440578130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.493+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/440578130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.493+0000 7fdd66d98b80 1 -- 172.21.15.79:0/440578130 >> 172.21.15.79:0/440578130 conn(0x557702d93170 msgr2=0x557702e6c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:29.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.493+0000 7fdd66d98b80 1 -- 172.21.15.79:0/440578130 shutdown_connections 2024-06-03T02:21:29.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.493+0000 7fdd66d98b80 1 -- 172.21.15.79:0/440578130 wait complete. 2024-06-03T02:21:29.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.494+0000 7fdd66d98b80 1 -- start start 2024-06-03T02:21:29.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.494+0000 7fdd66d98b80 1 Processor -- start 2024-06-03T02:21:29.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.496+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.496+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e81dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.496+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.496+0000 7fdd66d98b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e6afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.497+0000 7fdd59000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e81dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.497+0000 7fdd59000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e81dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46380/0 (socket says 172.21.15.79:46380) 2024-06-03T02:21:29.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.497+0000 7fdd59000700 1 -- 172.21.15.79:0/247239618 learned_addr learned my addr 172.21.15.79:0/247239618 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:29.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.497+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34010000 con 0x557702e665b0 2024-06-03T02:21:29.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.497+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3400f930 con 0x557702e71300 2024-06-03T02:21:29.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.497+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3400cb30 con 0x557702de2e40 2024-06-03T02:21:29.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.497+0000 7fdd5a400700 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e6afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.498+0000 7fdd66d98b80 10 client.0 _ll_register_callbacks cb 0x557702e670f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:29.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.498+0000 7fdd59000700 1 -- 172.21.15.79:0/247239618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 msgr2=0x557702e81890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:29.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.498+0000 7fdd59000700 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e81890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.498+0000 7fdd59000700 1 -- 172.21.15.79:0/247239618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 msgr2=0x557702e6afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:29.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.498+0000 7fdd59000700 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e6afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.498+0000 7fdd59000700 1 -- 172.21.15.79:0/247239618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557702e91610 con 0x557702e665b0 2024-06-03T02:21:29.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.499+0000 7fdd3b400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:21:29.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.499+0000 7fdd59000700 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e81dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd5406e550 tx=0x7fdd48004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2019f60c921d57 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:29.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.499+0000 7fdd3b400700 1 -- 172.21.15.79:0/247239618 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4801a220 con 0x557702e665b0 2024-06-03T02:21:29.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.499+0000 7fdd3b400700 1 -- 172.21.15.79:0/247239618 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fdd4801a020 con 0x557702e665b0 2024-06-03T02:21:29.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.500+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd4c015070 con 0x557702e665b0 2024-06-03T02:21:29.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.500+0000 7fdd3b400700 1 -- 172.21.15.79:0/247239618 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd4801a220 con 0x557702e665b0 2024-06-03T02:21:29.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.500+0000 7fdd66d98b80 10 client.15318 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:29.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.500+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557702e90040 con 0x557702e665b0 2024-06-03T02:21:29.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.500+0000 7fdd3b400700 1 -- 172.21.15.79:0/247239618 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4c015070 con 0x557702e665b0 2024-06-03T02:21:29.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.500+0000 7fdd3b400700 1 -- 172.21.15.79:0/247239618 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdd48035620 con 0x557702e665b0 2024-06-03T02:21:29.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.501+0000 7fdd66d98b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:29.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.501+0000 7fdd66d98b80 10 client.15318 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:21:29.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.501+0000 7fdd66d98b80 20 client.15318 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:29.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.501+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x557702ef3b10 0x557702ef5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:29.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.501+0000 7fdd66d98b80 4 client.15318 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:21:29.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.501+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557702dc0580 con 0x557702ef3b10 2024-06-03T02:21:29.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.501+0000 7fdd59a00700 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x557702ef3b10 0x557702ef5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:29.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.502+0000 7fdd59a00700 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x557702ef3b10 0x557702ef5f00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23e8979366ccb19f server_cookie=32bc063ee18cc28d in_seq=0 out_seq=0 2024-06-03T02:21:29.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.502+0000 7fdd3b400700 10 client.15318 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:21:29.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd3b400700 1 -- 172.21.15.79:0/247239618 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557702dc0580 con 0x557702ef3b10 2024-06-03T02:21:29.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd3b400700 10 client.15318 handle_command_reply: tid=0 2024-06-03T02:21:29.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd66d98b80 1 client.15318 shutdown 2024-06-03T02:21:29.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 msgr2=0x557702e81dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:29.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e81dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd5406e550 tx=0x7fdd48004bf0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 shutdown_connections 2024-06-03T02:21:29.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557702e71300 0x557702e6afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557702de2e40 0x557702e81890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x557702ef3b10 0x557702ef5f00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd66d98b80 1 --2- 172.21.15.79:0/247239618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557702e665b0 0x557702e81dd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.503+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 >> 172.21.15.79:0/247239618 conn(0x557702d93170 msgr2=0x557702e60eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:29.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.504+0000 7fdd3b400700 0 client.15318 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:21:29.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.504+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 shutdown_connections 2024-06-03T02:21:29.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.504+0000 7fdd66d98b80 1 -- 172.21.15.79:0/247239618 wait complete. 2024-06-03T02:21:29.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.504+0000 7fdd66d98b80 20 client.15318 trim_cache size 0 max 16384 2024-06-03T02:21:29.505 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:21:29.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.505+0000 7fdd5c400700 1 -- 172.21.15.79:0/2478966117 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdd3c0a2d20 msgr2=0x7fdd3c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:29.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.505+0000 7fdd5c400700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdd3c0a2d20 0x7fdd3c0a51e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x557702e63920 tx=0x7fdd48009230 comp rx=0 tx=0).stop 2024-06-03T02:21:29.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.505+0000 7fdd5c400700 1 -- 172.21.15.79:0/2478966117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 msgr2=0x7fdd541a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:29.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.505+0000 7fdd5c400700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 0x7fdd541a95c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000b5c0 tx=0x7fdd40005640 comp rx=0 tx=0).stop 2024-06-03T02:21:29.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.506+0000 7fdd5c400700 1 -- 172.21.15.79:0/2478966117 shutdown_connections 2024-06-03T02:21:29.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.506+0000 7fdd5c400700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd541a3690 0x7fdd541a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.506+0000 7fdd5c400700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd5410cbb0 0x7fdd541a95c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.506+0000 7fdd5c400700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdd3c0a2d20 0x7fdd3c0a51e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.506+0000 7fdd5c400700 1 --2- 172.21.15.79:0/2478966117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd54117520 0x7fdd541a9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.506+0000 7fdd5c400700 1 -- 172.21.15.79:0/2478966117 >> 172.21.15.79:0/2478966117 conn(0x7fdd5406d600 msgr2=0x7fdd54117900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.506+0000 7fdd5c400700 1 -- 172.21.15.79:0/2478966117 shutdown_connections 2024-06-03T02:21:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:29.506+0000 7fdd5c400700 1 -- 172.21.15.79:0/2478966117 wait complete. 2024-06-03T02:21:29.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:29 smithi079 ceph-mon[31819]: pgmap v559: 129 pgs: 129 active+clean; 1003 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 986 op/s 2024-06-03T02:21:29.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:29 smithi123 ceph-mon[36640]: pgmap v559: 129 pgs: 129 active+clean; 1003 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 986 op/s 2024-06-03T02:21:29.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:29 smithi135 ceph-mon[36653]: pgmap v559: 129 pgs: 129 active+clean; 1003 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 986 op/s 2024-06-03T02:21:30.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:21:31.303 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:31.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:31 smithi123 ceph-mon[36640]: pgmap v560: 129 pgs: 129 active+clean; 1003 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 889 op/s 2024-06-03T02:21:31.755 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:31 smithi079 ceph-mon[31819]: pgmap v560: 129 pgs: 129 active+clean; 1003 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 889 op/s 2024-06-03T02:21:31.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:31 smithi135 ceph-mon[36653]: pgmap v560: 129 pgs: 129 active+clean; 1003 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 889 op/s 2024-06-03T02:21:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 85.0 s, 1818.0 tps, lat 54.426 ms stddev 66.810 2024-06-03T02:21:33.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:33 smithi123 ceph-mon[36640]: pgmap v561: 129 pgs: 129 active+clean; 1003 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 1.04k op/s 2024-06-03T02:21:33.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:33 smithi079 ceph-mon[31819]: pgmap v561: 129 pgs: 129 active+clean; 1003 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 1.04k op/s 2024-06-03T02:21:33.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:33 smithi135 ceph-mon[36653]: pgmap v561: 129 pgs: 129 active+clean; 1003 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 1.04k op/s 2024-06-03T02:21:35.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:35 smithi079 ceph-mon[31819]: pgmap v562: 129 pgs: 129 active+clean; 1003 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 MiB/s wr, 922 op/s 2024-06-03T02:21:35.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:35 smithi135 ceph-mon[36653]: pgmap v562: 129 pgs: 129 active+clean; 1003 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 MiB/s wr, 922 op/s 2024-06-03T02:21:35.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:35 smithi123 ceph-mon[36640]: pgmap v562: 129 pgs: 129 active+clean; 1003 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 MiB/s wr, 922 op/s 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.956+0000 7f3bf6c00700 1 -- 172.21.15.79:0/4175918444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 msgr2=0x7f3bf0113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.956+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/4175918444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 0x7f3bf0113120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80098f0 tx=0x7f3bd8009c00 comp rx=0 tx=0).stop 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.957+0000 7f3bf6c00700 1 -- 172.21.15.79:0/4175918444 shutdown_connections 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.957+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/4175918444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bf0113660 0x7f3bf0115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.957+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/4175918444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bf00746c0 0x7f3bf0074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.957+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/4175918444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 0x7f3bf0113120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.957+0000 7f3bf6c00700 1 -- 172.21.15.79:0/4175918444 >> 172.21.15.79:0/4175918444 conn(0x7f3bf0079640 msgr2=0x7f3bf0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.957+0000 7f3bf6c00700 1 -- 172.21.15.79:0/4175918444 shutdown_connections 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.957+0000 7f3bf6c00700 1 -- 172.21.15.79:0/4175918444 wait complete. 2024-06-03T02:21:35.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.958+0000 7f3bf6c00700 1 Processor -- start 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.959+0000 7f3bf6c00700 1 -- start start 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.959+0000 7f3bf6c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bf00746c0 0x7f3bf019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.959+0000 7f3bf6c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 0x7f3bf019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.959+0000 7f3bf6c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bf0113660 0x7f3bf01a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.959+0000 7f3bf6c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0118e40 con 0x7f3bf00746c0 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.959+0000 7f3bf6c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0118cc0 con 0x7f3bf0074fe0 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.959+0000 7f3bf6c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0118fc0 con 0x7f3bf0113660 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3bef400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 0x7f3bf019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3bef400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 0x7f3bf019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40854/0 (socket says 172.21.15.79:40854) 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3bef400700 1 -- 172.21.15.79:0/2778756321 learned_addr learned my addr 172.21.15.79:0/2778756321 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3befe00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bf00746c0 0x7f3bf019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3bf4c00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bf0113660 0x7f3bf01a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3bef400700 1 -- 172.21.15.79:0/2778756321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bf0113660 msgr2=0x7f3bf01a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:35.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3bef400700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bf0113660 0x7f3bf01a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3bef400700 1 -- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bf00746c0 msgr2=0x7f3bf019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:35.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3bef400700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bf00746c0 0x7f3bf019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.960+0000 7f3bef400700 1 -- 172.21.15.79:0/2778756321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf01a42c0 con 0x7f3bf0074fe0 2024-06-03T02:21:35.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.962+0000 7f3bef400700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 0x7f3bf019f7f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c010 tx=0x7f3bd8010fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:35.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.964+0000 7f3becc00700 1 -- 172.21.15.79:0/2778756321 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8027070 con 0x7f3bf0074fe0 2024-06-03T02:21:35.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.964+0000 7f3becc00700 1 -- 172.21.15.79:0/2778756321 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd800b560 con 0x7f3bf0074fe0 2024-06-03T02:21:35.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.964+0000 7f3becc00700 1 -- 172.21.15.79:0/2778756321 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8004a90 con 0x7f3bf0074fe0 2024-06-03T02:21:35.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.964+0000 7f3bf6c00700 1 -- 172.21.15.79:0/2778756321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf01a4550 con 0x7f3bf0074fe0 2024-06-03T02:21:35.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.964+0000 7f3bf6c00700 1 -- 172.21.15.79:0/2778756321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf01abe30 con 0x7f3bf0074fe0 2024-06-03T02:21:35.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.969+0000 7f3becc00700 1 -- 172.21.15.79:0/2778756321 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3bd802a070 con 0x7f3bf0074fe0 2024-06-03T02:21:35.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.972+0000 7f3c014a1b80 1 Processor -- start 2024-06-03T02:21:35.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.972+0000 7f3c014a1b80 1 -- start start 2024-06-03T02:21:35.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3becc00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3bd40a2d70 0x7f3bd40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bd48f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd494b0 0x56110bd42d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd43240 0x56110bd43680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3c014a1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56110bc51a10 con 0x56110bd43240 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3c014a1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56110bc94b00 con 0x56110bd48b90 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3c014a1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56110bc4f8f0 con 0x56110bd494b0 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3befe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bd48f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3befe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bd48f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40858/0 (socket says 172.21.15.79:40858) 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3befe00700 1 -- 172.21.15.79:0/1554121698 learned_addr learned my addr 172.21.15.79:0/1554121698 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3bef400700 1 --2- 172.21.15.79:0/1554121698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd494b0 0x56110bd42d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3bf4c00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3bd40a2d70 0x7f3bd40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3becc00700 1 -- 172.21.15.79:0/2778756321 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3bd80cff60 con 0x7f3bf0074fe0 2024-06-03T02:21:35.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.973+0000 7f3bf4c00700 1 --2- 172.21.15.79:0/1554121698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd43240 0x56110bd43680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.974+0000 7f3befe00700 1 -- 172.21.15.79:0/1554121698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd494b0 msgr2=0x56110bd42d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:35.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.974+0000 7f3befe00700 1 --2- 172.21.15.79:0/1554121698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd494b0 0x56110bd42d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.974+0000 7f3befe00700 1 -- 172.21.15.79:0/1554121698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd43240 msgr2=0x56110bd43680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:35.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.974+0000 7f3befe00700 1 --2- 172.21.15.79:0/1554121698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd43240 0x56110bd43680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.974+0000 7f3befe00700 1 -- 172.21.15.79:0/1554121698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110bc3cf90 con 0x56110bd48b90 2024-06-03T02:21:35.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.974+0000 7f3bf4c00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3bd40a2d70 0x7f3bd40a5230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3be8012970 tx=0x7f3be8012240 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:35.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.975+0000 7f3befe00700 1 --2- 172.21.15.79:0/1554121698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bd48f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3be000bd70 tx=0x7f3be000f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8381557f32250625 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:35.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.975+0000 7f3be5600700 1 -- 172.21.15.79:0/1554121698 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0015070 con 0x56110bd48b90 2024-06-03T02:21:35.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.975+0000 7f3be5600700 1 -- 172.21.15.79:0/1554121698 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be000fdd0 con 0x56110bd48b90 2024-06-03T02:21:35.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.975+0000 7f3be5600700 1 -- 172.21.15.79:0/1554121698 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0011860 con 0x56110bd48b90 2024-06-03T02:21:35.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.976+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1554121698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 msgr2=0x56110bd48f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:35.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.976+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1554121698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bd48f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3be000bd70 tx=0x7f3be000f640 comp rx=0 tx=0).stop 2024-06-03T02:21:35.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.976+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1554121698 shutdown_connections 2024-06-03T02:21:35.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.976+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1554121698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd43240 0x56110bd43680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.976+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1554121698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bd48f70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.976+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1554121698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd494b0 0x56110bd42d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.976+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1554121698 >> 172.21.15.79:0/1554121698 conn(0x56110bc6f170 msgr2=0x56110bd3dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:35.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.977+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1554121698 shutdown_connections 2024-06-03T02:21:35.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.977+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1554121698 wait complete. 2024-06-03T02:21:35.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.978+0000 7f3c014a1b80 1 -- start start 2024-06-03T02:21:35.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.979+0000 7f3c014a1b80 1 Processor -- start 2024-06-03T02:21:35.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.979+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd4f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 0x56110bd4c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bd4ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3befe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd4f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3bef400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 0x56110bd4c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3befe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd4f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40874/0 (socket says 172.21.15.79:40874) 2024-06-03T02:21:35.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3bef400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 0x56110bd4c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59674/0 (socket says 172.21.15.79:59674) 2024-06-03T02:21:35.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3befe00700 1 -- 172.21.15.79:0/1651646038 learned_addr learned my addr 172.21.15.79:0/1651646038 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:35.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56110bc30f60 con 0x56110bd494b0 2024-06-03T02:21:35.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56110bc90ac0 con 0x56110bd43240 2024-06-03T02:21:35.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56110bd4fe90 con 0x56110bd48b90 2024-06-03T02:21:35.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.980+0000 7f3c014a1b80 10 client.0 _ll_register_callbacks cb 0x56110bc2a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:35.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.981+0000 7f3befe00700 1 -- 172.21.15.79:0/1651646038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 msgr2=0x56110bd4c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:35.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.981+0000 7f3befe00700 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 0x56110bd4c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.981+0000 7f3bf4c00700 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bd4ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.981+0000 7f3befe00700 1 -- 172.21.15.79:0/1651646038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 msgr2=0x56110bd4ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:35.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.981+0000 7f3befe00700 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bd4ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.981+0000 7f3befe00700 1 -- 172.21.15.79:0/1651646038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110bbe38d0 con 0x56110bd43240 2024-06-03T02:21:35.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.981+0000 7f3bef400700 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 0x56110bd4c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:21:35.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.981+0000 7f3bf4c00700 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bd4ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.982+0000 7f3befe00700 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd4f200 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3be001af40 tx=0x7f3be00057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6447e57bc8be1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.982+0000 7f3bde000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:21:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.982+0000 7f3bde000700 1 -- 172.21.15.79:0/1651646038 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0028020 con 0x56110bd43240 2024-06-03T02:21:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.982+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be000fdd0 con 0x56110bd43240 2024-06-03T02:21:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.982+0000 7f3bde000700 1 -- 172.21.15.79:0/1651646038 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3be000fbd0 con 0x56110bd43240 2024-06-03T02:21:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.983+0000 7f3bde000700 1 -- 172.21.15.79:0/1651646038 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3be0028020 con 0x56110bd43240 2024-06-03T02:21:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.983+0000 7f3c014a1b80 10 client.25487 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.983+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56110bd6b5e0 con 0x56110bd43240 2024-06-03T02:21:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.983+0000 7f3bde000700 1 -- 172.21.15.79:0/1651646038 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be000fdd0 con 0x56110bd43240 2024-06-03T02:21:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.984+0000 7f3bde000700 1 -- 172.21.15.79:0/1651646038 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3be0023070 con 0x56110bd43240 2024-06-03T02:21:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.984+0000 7f3c014a1b80 10 client.25487 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.984+0000 7f3c014a1b80 10 client.25487 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:21:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.984+0000 7f3c014a1b80 20 client.25487 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:35.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.984+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56110bd6dab0 0x56110bd6fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:35.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.984+0000 7f3c014a1b80 4 client.25487 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:21:35.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.984+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56110bca2000 con 0x56110bd6dab0 2024-06-03T02:21:35.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.985+0000 7f3bef400700 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56110bd6dab0 0x56110bd6fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:35.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.986+0000 7f3bef400700 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56110bd6dab0 0x56110bd6fea0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa737ffc7fe50786 server_cookie=50968eaed68a24d9 in_seq=0 out_seq=0 2024-06-03T02:21:35.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.986+0000 7f3bde000700 10 client.25487 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:21:35.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.986+0000 7f3bde000700 1 -- 172.21.15.79:0/1651646038 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56110bca2000 con 0x56110bd6dab0 2024-06-03T02:21:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.986+0000 7f3bde000700 10 client.25487 handle_command_reply: tid=0 2024-06-03T02:21:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.986+0000 7f3c014a1b80 1 client.25487 shutdown 2024-06-03T02:21:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 msgr2=0x56110bd4f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd4f200 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3be001af40 tx=0x7f3be00057d0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 shutdown_connections 2024-06-03T02:21:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bd4ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56110bd6dab0 0x56110bd6fea0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd4f200 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/1651646038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 0x56110bd4c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3bde000700 0 client.25487 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:21:35.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 >> 172.21.15.79:0/1651646038 conn(0x56110bc6f170 msgr2=0x56110bd37790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:35.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.987+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 shutdown_connections 2024-06-03T02:21:35.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.988+0000 7f3c014a1b80 1 -- 172.21.15.79:0/1651646038 wait complete. 2024-06-03T02:21:35.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:35.988+0000 7f3c014a1b80 20 client.25487 trim_cache size 0 max 16384 2024-06-03T02:21:36.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.013+0000 7f3c014a1b80 1 Processor -- start 2024-06-03T02:21:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.014+0000 7f3c014a1b80 1 -- start start 2024-06-03T02:21:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.014+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd42d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.014+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 0x56110bd38130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.014+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bd4b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.014+0000 7f3c014a1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01ac150 con 0x56110bd494b0 2024-06-03T02:21:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.014+0000 7f3c014a1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01adc90 con 0x56110bd43240 2024-06-03T02:21:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.014+0000 7f3c014a1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01ac820 con 0x56110bd48b90 2024-06-03T02:21:36.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.015+0000 7f3befe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd42d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:36.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.015+0000 7f3befe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd42d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40888/0 (socket says 172.21.15.79:40888) 2024-06-03T02:21:36.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.015+0000 7f3befe00700 1 -- 172.21.15.79:0/2971004017 learned_addr learned my addr 172.21.15.79:0/2971004017 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:36.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.015+0000 7f3befe00700 1 -- 172.21.15.79:0/2971004017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 msgr2=0x56110bd38130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:36.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.015+0000 7f3befe00700 1 --2- 172.21.15.79:0/2971004017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 0x56110bd38130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.015+0000 7f3befe00700 1 -- 172.21.15.79:0/2971004017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 msgr2=0x56110bd4b680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:21:36.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.016+0000 7f3befe00700 1 --2- 172.21.15.79:0/2971004017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bd4b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.016+0000 7f3befe00700 1 -- 172.21.15.79:0/2971004017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110bd6a1f0 con 0x56110bd43240 2024-06-03T02:21:36.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.016+0000 7f3befe00700 1 --2- 172.21.15.79:0/2971004017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd42d00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3be0010010 tx=0x7f3be000f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc5f9a23df91530 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:36.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.017+0000 7f3be4c00700 1 -- 172.21.15.79:0/2971004017 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0015050 con 0x56110bd43240 2024-06-03T02:21:36.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.017+0000 7f3be4c00700 1 -- 172.21.15.79:0/2971004017 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be000fdd0 con 0x56110bd43240 2024-06-03T02:21:36.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.017+0000 7f3be4c00700 1 -- 172.21.15.79:0/2971004017 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0023390 con 0x56110bd43240 2024-06-03T02:21:36.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.019+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2971004017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 msgr2=0x56110bd42d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:36.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.019+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2971004017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd42d00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3be0010010 tx=0x7f3be000f640 comp rx=0 tx=0).stop 2024-06-03T02:21:36.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.019+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2971004017 shutdown_connections 2024-06-03T02:21:36.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.019+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2971004017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bd4b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.020+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2971004017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd43240 0x56110bd42d00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.020+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2971004017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd48b90 0x56110bd38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.020+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2971004017 >> 172.21.15.79:0/2971004017 conn(0x56110bc6f170 msgr2=0x56110bd49f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:36.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.020+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2971004017 shutdown_connections 2024-06-03T02:21:36.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.020+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2971004017 wait complete. 2024-06-03T02:21:36.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.021+0000 7f3c014a1b80 1 -- start start 2024-06-03T02:21:36.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.021+0000 7f3c014a1b80 1 Processor -- start 2024-06-03T02:21:36.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.022+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bdd0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:36.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.022+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bdd10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:36.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.022+0000 7f3c014a1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd71ed0 0x56110bd72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:36.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.022+0000 7f3c014a1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01ad5c0 con 0x56110bd494b0 2024-06-03T02:21:36.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.022+0000 7f3c014a1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01acef0 con 0x56110bd48b90 2024-06-03T02:21:36.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.022+0000 7f3c014a1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf010d190 con 0x56110bd71ed0 2024-06-03T02:21:36.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.022+0000 7f3c014a1b80 10 client.0 _ll_register_callbacks cb 0x56110bc2a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:36.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.022+0000 7f3befe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bdd0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:36.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.023+0000 7f3befe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bdd0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40902/0 (socket says 172.21.15.79:40902) 2024-06-03T02:21:36.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.023+0000 7f3befe00700 1 -- 172.21.15.79:0/2521023715 learned_addr learned my addr 172.21.15.79:0/2521023715 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:36.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.023+0000 7f3bf4c00700 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd71ed0 0x56110bd72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:36.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.023+0000 7f3bef400700 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bdd10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:36.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.023+0000 7f3befe00700 1 -- 172.21.15.79:0/2521023715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd71ed0 msgr2=0x56110bd72350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:36.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.023+0000 7f3befe00700 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd71ed0 0x56110bd72350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.023+0000 7f3befe00700 1 -- 172.21.15.79:0/2521023715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 msgr2=0x56110bdd10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:36.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.023+0000 7f3befe00700 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bdd10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.023+0000 7f3befe00700 1 -- 172.21.15.79:0/2521023715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110bd72890 con 0x56110bd48b90 2024-06-03T02:21:36.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.024+0000 7f3befe00700 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bdd0b60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3be000ffe0 tx=0x7f3be000b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330bfb9ee339d09f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:36.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.025+0000 7f3bde000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:21:36.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.025+0000 7f3bde000700 1 -- 172.21.15.79:0/2521023715 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0043060 con 0x56110bd48b90 2024-06-03T02:21:36.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.025+0000 7f3bde000700 1 -- 172.21.15.79:0/2521023715 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3be0018870 con 0x56110bd48b90 2024-06-03T02:21:36.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.025+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2521023715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be000fdd0 con 0x56110bd48b90 2024-06-03T02:21:36.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.026+0000 7f3bde000700 1 -- 172.21.15.79:0/2521023715 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3be0015070 con 0x56110bd48b90 2024-06-03T02:21:36.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.026+0000 7f3c014a1b80 10 client.25493 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:36.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.026+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2521023715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56110bde93f0 con 0x56110bd48b90 2024-06-03T02:21:36.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.027+0000 7f3bde000700 1 -- 172.21.15.79:0/2521023715 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be000fdd0 con 0x56110bd48b90 2024-06-03T02:21:36.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.028+0000 7f3bde000700 1 -- 172.21.15.79:0/2521023715 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3be0023070 con 0x56110bd48b90 2024-06-03T02:21:36.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.028+0000 7f3c014a1b80 10 client.25493 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:36.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.028+0000 7f3c014a1b80 10 client.25493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:21:36.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.028+0000 7f3c014a1b80 20 client.25493 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:36.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.028+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56110bde94e0 0x56110bdeb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:36.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.029+0000 7f3c014a1b80 4 client.25493 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:21:36.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.029+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2521023715 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56110bc9c580 con 0x56110bde94e0 2024-06-03T02:21:36.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.033+0000 7f3bef400700 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56110bde94e0 0x56110bdeb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.034+0000 7f3bef400700 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56110bde94e0 0x56110bdeb8d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccf91246345c7cf9 server_cookie=94e96f1c7f581807 in_seq=0 out_seq=0 2024-06-03T02:21:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.034+0000 7f3bde000700 10 client.25493 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:21:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.034+0000 7f3bde000700 1 -- 172.21.15.79:0/2521023715 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56110bc9c580 con 0x56110bde94e0 2024-06-03T02:21:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.034+0000 7f3bde000700 10 client.25493 handle_command_reply: tid=0 2024-06-03T02:21:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.034+0000 7f3c014a1b80 1 client.25493 shutdown 2024-06-03T02:21:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.036+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2521023715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 msgr2=0x56110bdd0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.036+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bdd0b60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3be000ffe0 tx=0x7f3be000b770 comp rx=0 tx=0).stop 2024-06-03T02:21:36.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.038+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2521023715 shutdown_connections 2024-06-03T02:21:36.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.038+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56110bd71ed0 0x56110bd72350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.038+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56110bde94e0 0x56110bdeb8d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.038+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56110bd48b90 0x56110bdd0b60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.038+0000 7f3c014a1b80 1 --2- 172.21.15.79:0/2521023715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56110bd494b0 0x56110bdd10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.038+0000 7f3bde000700 0 client.25493 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:21:36.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.038+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2521023715 >> 172.21.15.79:0/2521023715 conn(0x56110bc6f170 msgr2=0x56110bd691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:36.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.038+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2521023715 shutdown_connections 2024-06-03T02:21:36.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.040+0000 7f3c014a1b80 1 -- 172.21.15.79:0/2521023715 wait complete. 2024-06-03T02:21:36.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.040+0000 7f3c014a1b80 20 client.25493 trim_cache size 0 max 16384 2024-06-03T02:21:36.041 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:21:36.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 -- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3bd40a2d70 msgr2=0x7f3bd40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:36.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3bd40a2d70 0x7f3bd40a5230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3be8012970 tx=0x7f3be8012240 comp rx=0 tx=0).stop 2024-06-03T02:21:36.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 -- 172.21.15.79:0/2778756321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 msgr2=0x7f3bf019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:36.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 0x7f3bf019f7f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c010 tx=0x7f3bd8010fe0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 -- 172.21.15.79:0/2778756321 shutdown_connections 2024-06-03T02:21:36.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bf0113660 0x7f3bf01a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3bd40a2d70 0x7f3bd40a5230 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3bf00746c0 0x7f3bf019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 --2- 172.21.15.79:0/2778756321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bf0074fe0 0x7f3bf019f7f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:36.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.041+0000 7f3bf6c00700 1 -- 172.21.15.79:0/2778756321 >> 172.21.15.79:0/2778756321 conn(0x7f3bf0079640 msgr2=0x7f3bf0112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:36.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.042+0000 7f3bf6c00700 1 -- 172.21.15.79:0/2778756321 shutdown_connections 2024-06-03T02:21:36.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:36.043+0000 7f3bf6c00700 1 -- 172.21.15.79:0/2778756321 wait complete. 2024-06-03T02:21:37.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:37 smithi123 ceph-mon[36640]: pgmap v563: 129 pgs: 129 active+clean; 1011 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 907 op/s 2024-06-03T02:21:37.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:37 smithi079 ceph-mon[31819]: pgmap v563: 129 pgs: 129 active+clean; 1011 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 907 op/s 2024-06-03T02:21:37.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:37 smithi135 ceph-mon[36653]: pgmap v563: 129 pgs: 129 active+clean; 1011 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 907 op/s 2024-06-03T02:21:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 90.0 s, 1571.4 tps, lat 62.499 ms stddev 92.929 2024-06-03T02:21:39.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:39 smithi123 ceph-mon[36640]: pgmap v564: 129 pgs: 129 active+clean; 1019 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 804 op/s 2024-06-03T02:21:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:39 smithi079 ceph-mon[31819]: pgmap v564: 129 pgs: 129 active+clean; 1019 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 804 op/s 2024-06-03T02:21:39.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:39 smithi135 ceph-mon[36653]: pgmap v564: 129 pgs: 129 active+clean; 1019 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 804 op/s 2024-06-03T02:21:40.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:21:41.151 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:41 smithi079 ceph-mon[31819]: pgmap v565: 129 pgs: 129 active+clean; 1019 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 556 op/s 2024-06-03T02:21:41.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:41 smithi123 ceph-mon[36640]: pgmap v565: 129 pgs: 129 active+clean; 1019 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 556 op/s 2024-06-03T02:21:41.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:41 smithi135 ceph-mon[36653]: pgmap v565: 129 pgs: 129 active+clean; 1019 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 556 op/s 2024-06-03T02:21:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 95.0 s, 1558.4 tps, lat 65.974 ms stddev 105.692 2024-06-03T02:21:43.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.495+0000 7f51caa00700 1 -- 172.21.15.79:0/46587668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51c41073c0 msgr2=0x7f51c406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:43.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.495+0000 7f51caa00700 1 --2- 172.21.15.79:0/46587668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51c41073c0 0x7f51c406a1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009990 tx=0x7f51b0009ca0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.495+0000 7f51caa00700 1 -- 172.21.15.79:0/46587668 shutdown_connections 2024-06-03T02:21:43.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.495+0000 7f51caa00700 1 --2- 172.21.15.79:0/46587668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51c4104ed0 0x7f51c4105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.495+0000 7f51caa00700 1 --2- 172.21.15.79:0/46587668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51c41073c0 0x7f51c406a1a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.495+0000 7f51caa00700 1 --2- 172.21.15.79:0/46587668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c406a6e0 0x7f51c406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.495+0000 7f51caa00700 1 -- 172.21.15.79:0/46587668 >> 172.21.15.79:0/46587668 conn(0x7f51c4078340 msgr2=0x7f51c4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:43.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.495+0000 7f51caa00700 1 -- 172.21.15.79:0/46587668 shutdown_connections 2024-06-03T02:21:43.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.495+0000 7f51caa00700 1 -- 172.21.15.79:0/46587668 wait complete. 2024-06-03T02:21:43.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.497+0000 7f51caa00700 1 Processor -- start 2024-06-03T02:21:43.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.498+0000 7f51caa00700 1 -- start start 2024-06-03T02:21:43.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51caa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51c406a6e0 0x7f51c4073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51caa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4104ed0 0x7f51c4074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51caa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51c41073c0 0x7f51c406cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51caa00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51c4118d50 con 0x7f51c41073c0 2024-06-03T02:21:43.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51caa00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51c4118bd0 con 0x7f51c4104ed0 2024-06-03T02:21:43.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51caa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51c4118ed0 con 0x7f51c406a6e0 2024-06-03T02:21:43.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51c3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51c406a6e0 0x7f51c4073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51c3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4104ed0 0x7f51c4074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51c3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4104ed0 0x7f51c4074150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40916/0 (socket says 172.21.15.79:40916) 2024-06-03T02:21:43.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51c3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51c406a6e0 0x7f51c4073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59702/0 (socket says 172.21.15.79:59702) 2024-06-03T02:21:43.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51c3400700 1 -- 172.21.15.79:0/142992468 learned_addr learned my addr 172.21.15.79:0/142992468 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:43.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.499+0000 7f51c8a00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51c41073c0 0x7f51c406cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.500+0000 7f51c3400700 1 -- 172.21.15.79:0/142992468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51c406a6e0 msgr2=0x7f51c4073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:43.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.500+0000 7f51c3400700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51c406a6e0 0x7f51c4073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.500+0000 7f51c3400700 1 -- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51c41073c0 msgr2=0x7f51c406cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:43.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.500+0000 7f51c3400700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51c41073c0 0x7f51c406cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.500+0000 7f51c3400700 1 -- 172.21.15.79:0/142992468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c406d7c0 con 0x7f51c4104ed0 2024-06-03T02:21:43.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.500+0000 7f51c3400700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4104ed0 0x7f51c4074150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51b400ef10 tx=0x7f51b400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:43.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.500+0000 7f51c0c00700 1 -- 172.21.15.79:0/142992468 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b400c9c0 con 0x7f51c4104ed0 2024-06-03T02:21:43.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.500+0000 7f51c0c00700 1 -- 172.21.15.79:0/142992468 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b4019070 con 0x7f51c4104ed0 2024-06-03T02:21:43.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.500+0000 7f51c0c00700 1 -- 172.21.15.79:0/142992468 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b4007e60 con 0x7f51c4104ed0 2024-06-03T02:21:43.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.501+0000 7f51caa00700 1 -- 172.21.15.79:0/142992468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c406da50 con 0x7f51c4104ed0 2024-06-03T02:21:43.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.501+0000 7f51caa00700 1 -- 172.21.15.79:0/142992468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c4071970 con 0x7f51c4104ed0 2024-06-03T02:21:43.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.507+0000 7f51d53c0b80 1 Processor -- start 2024-06-03T02:21:43.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.508+0000 7f51d53c0b80 1 -- start start 2024-06-03T02:21:43.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.509+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.509+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 0x557f456a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.509+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456a4240 0x557f456a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.509+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557f455b2a10 con 0x557f456a9b90 2024-06-03T02:21:43.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.509+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f455f5b00 con 0x557f456a4240 2024-06-03T02:21:43.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.509+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557f455b08f0 con 0x557f456aa4b0 2024-06-03T02:21:43.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.509+0000 7f51c8a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.510+0000 7f51c8a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46088/0 (socket says 172.21.15.79:46088) 2024-06-03T02:21:43.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.510+0000 7f51c8a00700 1 -- 172.21.15.79:0/1397981527 learned_addr learned my addr 172.21.15.79:0/1397981527 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:43.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.510+0000 7f51c0c00700 1 -- 172.21.15.79:0/142992468 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f51b4012050 con 0x7f51c4104ed0 2024-06-03T02:21:43.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.510+0000 7f51c8a00700 1 -- 172.21.15.79:0/1397981527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 msgr2=0x557f456a3d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:21:43.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.510+0000 7f51c8a00700 1 --2- 172.21.15.79:0/1397981527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 0x557f456a3d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.511+0000 7f51c8a00700 1 -- 172.21.15.79:0/1397981527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456a4240 msgr2=0x557f456a4680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:21:43.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.511+0000 7f51c8a00700 1 --2- 172.21.15.79:0/1397981527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456a4240 0x557f456a4680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.511+0000 7f51c8a00700 1 -- 172.21.15.79:0/1397981527 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f4559df90 con 0x557f456a9b90 2024-06-03T02:21:43.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.511+0000 7f51c8a00700 1 --2- 172.21.15.79:0/1397981527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456a9f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0046a0 tx=0x7f51bc00b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3a64c1b6722bb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:43.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.511+0000 7f51b9600700 1 -- 172.21.15.79:0/1397981527 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc004a30 con 0x557f456a9b90 2024-06-03T02:21:43.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.512+0000 7f51b9600700 1 -- 172.21.15.79:0/1397981527 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc015070 con 0x557f456a9b90 2024-06-03T02:21:43.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.512+0000 7f51b9600700 1 -- 172.21.15.79:0/1397981527 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc003b30 con 0x557f456a9b90 2024-06-03T02:21:43.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.512+0000 7f51c0c00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a80a2ca0 0x7f51a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.513+0000 7f51c0c00700 1 -- 172.21.15.79:0/142992468 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51b40ced40 con 0x7f51c4104ed0 2024-06-03T02:21:43.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.513+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1397981527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 msgr2=0x557f456a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:43.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.513+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1397981527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456a9f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0046a0 tx=0x7f51bc00b970 comp rx=0 tx=0).stop 2024-06-03T02:21:43.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.513+0000 7f51c3e00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a80a2ca0 0x7f51a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.514+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1397981527 shutdown_connections 2024-06-03T02:21:43.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.514+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1397981527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456a4240 0x557f456a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.514+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1397981527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456a9f70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.514+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1397981527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 0x557f456a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.514+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1397981527 >> 172.21.15.79:0/1397981527 conn(0x557f455d0170 msgr2=0x557f4569eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:43.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.515+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1397981527 shutdown_connections 2024-06-03T02:21:43.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.515+0000 7f51c3e00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a80a2ca0 0x7f51a80a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f51b00098e0 tx=0x7f51b0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:43.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.515+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1397981527 wait complete. 2024-06-03T02:21:43.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.515+0000 7f51d53c0b80 1 -- start start 2024-06-03T02:21:43.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51d53c0b80 1 Processor -- start 2024-06-03T02:21:43.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456a4240 0x557f456b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 0x557f456adac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557f45591f60 con 0x557f456a9b90 2024-06-03T02:21:43.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f455f1ac0 con 0x557f456a4240 2024-06-03T02:21:43.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557f456b0e90 con 0x557f456aa4b0 2024-06-03T02:21:43.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51d53c0b80 10 client.0 _ll_register_callbacks cb 0x557f4558b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:43.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51c8a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 0x557f456adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51c8a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 0x557f456adac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59718/0 (socket says 172.21.15.79:59718) 2024-06-03T02:21:43.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.516+0000 7f51c8a00700 1 -- 172.21.15.79:0/1332146176 learned_addr learned my addr 172.21.15.79:0/1332146176 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:43.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.517+0000 7f51c8a00700 1 -- 172.21.15.79:0/1332146176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456a4240 msgr2=0x557f456b0280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:21:43.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.517+0000 7f51c3400700 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.518+0000 7f51c8a00700 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456a4240 0x557f456b0280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.518+0000 7f51c8a00700 1 -- 172.21.15.79:0/1332146176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 msgr2=0x557f456ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:43.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.518+0000 7f51c8a00700 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.518+0000 7f51c8a00700 1 -- 172.21.15.79:0/1332146176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f455448d0 con 0x557f456aa4b0 2024-06-03T02:21:43.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.519+0000 7f51c8a00700 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 0x557f456adac0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000c00 tx=0x7f51bc004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ccc6035084f9f92 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:43.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.519+0000 7f51ae000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:21:43.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.520+0000 7f51ae000700 1 -- 172.21.15.79:0/1332146176 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc028060 con 0x557f456aa4b0 2024-06-03T02:21:43.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.521+0000 7f51ae000700 1 -- 172.21.15.79:0/1332146176 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51bc005510 con 0x557f456aa4b0 2024-06-03T02:21:43.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.521+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1332146176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51bc015070 con 0x557f456aa4b0 2024-06-03T02:21:43.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.522+0000 7f51ae000700 1 -- 172.21.15.79:0/1332146176 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51bc003740 con 0x557f456aa4b0 2024-06-03T02:21:43.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.522+0000 7f51d53c0b80 10 client.25486 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:43.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.522+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1332146176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f456bb6a0 con 0x557f456aa4b0 2024-06-03T02:21:43.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.522+0000 7f51ae000700 1 -- 172.21.15.79:0/1332146176 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc015070 con 0x557f456aa4b0 2024-06-03T02:21:43.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.522+0000 7f51ae000700 1 -- 172.21.15.79:0/1332146176 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f51bc0119b0 con 0x557f456aa4b0 2024-06-03T02:21:43.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.522+0000 7f51d53c0b80 10 client.25486 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:43.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.522+0000 7f51d53c0b80 10 client.25486 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:21:43.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.522+0000 7f51d53c0b80 20 client.25486 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:43.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.523+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x557f456bdb70 0x557f456bff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.523+0000 7f51d53c0b80 4 client.25486 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:21:43.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.523+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1332146176 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f45603000 con 0x557f456bdb70 2024-06-03T02:21:43.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.523+0000 7f51c3e00700 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x557f456bdb70 0x557f456bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.524+0000 7f51ae000700 10 client.25486 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:21:43.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.524+0000 7f51c3e00700 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x557f456bdb70 0x557f456bff60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f953df55fd8db1a server_cookie=9770f09a91c809b in_seq=0 out_seq=0 2024-06-03T02:21:43.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51ae000700 1 -- 172.21.15.79:0/1332146176 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f45603000 con 0x557f456bdb70 2024-06-03T02:21:43.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51ae000700 10 client.25486 handle_command_reply: tid=0 2024-06-03T02:21:43.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51d53c0b80 1 client.25486 shutdown 2024-06-03T02:21:43.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1332146176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 msgr2=0x557f456adac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 0x557f456adac0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000c00 tx=0x7f51bc004d50 comp rx=0 tx=0).stop 2024-06-03T02:21:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1332146176 shutdown_connections 2024-06-03T02:21:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x557f456bdb70 0x557f456bff60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456aa4b0 0x557f456adac0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456a4240 0x557f456b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/1332146176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1332146176 >> 172.21.15.79:0/1332146176 conn(0x557f455d0170 msgr2=0x557f45698790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.527+0000 7f51ae000700 0 client.25486 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:21:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.528+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1332146176 shutdown_connections 2024-06-03T02:21:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.528+0000 7f51d53c0b80 1 -- 172.21.15.79:0/1332146176 wait complete. 2024-06-03T02:21:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.528+0000 7f51d53c0b80 20 client.25486 trim_cache size 0 max 16384 2024-06-03T02:21:43.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:43 smithi079 ceph-mon[31819]: pgmap v566: 129 pgs: 129 active+clean; 1020 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 MiB/s wr, 788 op/s 2024-06-03T02:21:43.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.558+0000 7f51d53c0b80 1 Processor -- start 2024-06-03T02:21:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.559+0000 7f51d53c0b80 1 -- start start 2024-06-03T02:21:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.559+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a4240 0x557f456a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.559+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a9b90 0x557f456adc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.559+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456aa4b0 0x557f45699130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.559+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41bd2e0 con 0x557f456a4240 2024-06-03T02:21:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.559+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41bee20 con 0x557f456aa4b0 2024-06-03T02:21:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.559+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41bd9b0 con 0x557f456a9b90 2024-06-03T02:21:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.560+0000 7f51c3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a9b90 0x557f456adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.560+0000 7f51c3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a9b90 0x557f456adc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59732/0 (socket says 172.21.15.79:59732) 2024-06-03T02:21:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.560+0000 7f51c3400700 1 -- 172.21.15.79:0/335578148 learned_addr learned my addr 172.21.15.79:0/335578148 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.560+0000 7f51c8a00700 1 --2- 172.21.15.79:0/335578148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456aa4b0 0x557f45699130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.561+0000 7f51c3400700 1 -- 172.21.15.79:0/335578148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456aa4b0 msgr2=0x557f45699130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.561+0000 7f51c3400700 1 --2- 172.21.15.79:0/335578148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456aa4b0 0x557f45699130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.561+0000 7f51c3400700 1 -- 172.21.15.79:0/335578148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a4240 msgr2=0x557f456a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.561+0000 7f51c3400700 1 --2- 172.21.15.79:0/335578148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a4240 0x557f456a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.561+0000 7f51c3400700 1 -- 172.21.15.79:0/335578148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f456ad350 con 0x557f456a9b90 2024-06-03T02:21:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.562+0000 7f51c3400700 1 --2- 172.21.15.79:0/335578148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a9b90 0x557f456adc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f51b40090a0 tx=0x7f51b4097630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d3d9e7cbc3ace server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.562+0000 7f51b8c00700 1 -- 172.21.15.79:0/335578148 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b409b280 con 0x557f456a9b90 2024-06-03T02:21:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.562+0000 7f51b8c00700 1 -- 172.21.15.79:0/335578148 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b409f630 con 0x557f456a9b90 2024-06-03T02:21:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.562+0000 7f51b8c00700 1 -- 172.21.15.79:0/335578148 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b4099610 con 0x557f456a9b90 2024-06-03T02:21:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.563+0000 7f51d53c0b80 1 -- 172.21.15.79:0/335578148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a9b90 msgr2=0x557f456adc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.563+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/335578148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a9b90 0x557f456adc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f51b40090a0 tx=0x7f51b4097630 comp rx=0 tx=0).stop 2024-06-03T02:21:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.564+0000 7f51d53c0b80 1 -- 172.21.15.79:0/335578148 shutdown_connections 2024-06-03T02:21:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.564+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/335578148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456aa4b0 0x557f45699130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.564+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/335578148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a4240 0x557f456a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.564+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/335578148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a9b90 0x557f456adc50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.564+0000 7f51d53c0b80 1 -- 172.21.15.79:0/335578148 >> 172.21.15.79:0/335578148 conn(0x557f455d0170 msgr2=0x557f456ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:43.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.564+0000 7f51d53c0b80 1 -- 172.21.15.79:0/335578148 shutdown_connections 2024-06-03T02:21:43.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.564+0000 7f51d53c0b80 1 -- 172.21.15.79:0/335578148 wait complete. 2024-06-03T02:21:43.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.565+0000 7f51d53c0b80 1 -- start start 2024-06-03T02:21:43.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.565+0000 7f51d53c0b80 1 Processor -- start 2024-06-03T02:21:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.566+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a4240 0x557f456c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.566+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456c0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.566+0000 7f51d53c0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456aa4b0 0x557f456c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.567+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41be750 con 0x557f456a9b90 2024-06-03T02:21:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.567+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41be080 con 0x557f456aa4b0 2024-06-03T02:21:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.567+0000 7f51d53c0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41bb240 con 0x557f456a4240 2024-06-03T02:21:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.567+0000 7f51d53c0b80 10 client.0 _ll_register_callbacks cb 0x557f4558b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.567+0000 7f51c3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.567+0000 7f51c3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456c0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46104/0 (socket says 172.21.15.79:46104) 2024-06-03T02:21:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.567+0000 7f51c3400700 1 -- 172.21.15.79:0/279048118 learned_addr learned my addr 172.21.15.79:0/279048118 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.568+0000 7f51c3400700 1 -- 172.21.15.79:0/279048118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a4240 msgr2=0x557f456c0020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:21:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.568+0000 7f51c3400700 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a4240 0x557f456c0020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.568+0000 7f51c3400700 1 -- 172.21.15.79:0/279048118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456aa4b0 msgr2=0x557f456c0ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:21:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.568+0000 7f51c3400700 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456aa4b0 0x557f456c0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.568+0000 7f51c3400700 1 -- 172.21.15.79:0/279048118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f457267f0 con 0x557f456a9b90 2024-06-03T02:21:43.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.569+0000 7f51c3400700 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456c0560 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51b40cc800 tx=0x7f51b409baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ffa8996d30a789 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:43.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.569+0000 7f51ae000700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:21:43.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.570+0000 7f51ae000700 1 -- 172.21.15.79:0/279048118 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b40a3e60 con 0x557f456a9b90 2024-06-03T02:21:43.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.570+0000 7f51ae000700 1 -- 172.21.15.79:0/279048118 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51b4099580 con 0x557f456a9b90 2024-06-03T02:21:43.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.570+0000 7f51d53c0b80 1 -- 172.21.15.79:0/279048118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b409f630 con 0x557f456a9b90 2024-06-03T02:21:43.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.570+0000 7f51ae000700 1 -- 172.21.15.79:0/279048118 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51b40bb020 con 0x557f456a9b90 2024-06-03T02:21:43.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.571+0000 7f51d53c0b80 10 client.15342 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:43.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.571+0000 7f51d53c0b80 1 -- 172.21.15.79:0/279048118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f45727a90 con 0x557f456a9b90 2024-06-03T02:21:43.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.571+0000 7f51ae000700 1 -- 172.21.15.79:0/279048118 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b409f630 con 0x557f456a9b90 2024-06-03T02:21:43.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.572+0000 7f51ae000700 1 -- 172.21.15.79:0/279048118 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f51b409a6d0 con 0x557f456a9b90 2024-06-03T02:21:43.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.572+0000 7f51d53c0b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:43.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.572+0000 7f51d53c0b80 10 client.15342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:21:43.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.572+0000 7f51d53c0b80 20 client.15342 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:43.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.572+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x557f45728190 0x557f4572a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:43.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.573+0000 7f51d53c0b80 4 client.15342 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:21:43.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.573+0000 7f51d53c0b80 1 -- 172.21.15.79:0/279048118 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f455fd580 con 0x557f45728190 2024-06-03T02:21:43.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.573+0000 7f51c3e00700 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x557f45728190 0x557f4572a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:43.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.573+0000 7f51c3e00700 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x557f45728190 0x557f4572a5b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1c3186177e25856 server_cookie=4b5357be6bff2c6e in_seq=0 out_seq=0 2024-06-03T02:21:43.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.573+0000 7f51ae000700 10 client.15342 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:21:43.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.574+0000 7f51ae000700 1 -- 172.21.15.79:0/279048118 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f455fd580 con 0x557f45728190 2024-06-03T02:21:43.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.574+0000 7f51ae000700 10 client.15342 handle_command_reply: tid=0 2024-06-03T02:21:43.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.574+0000 7f51d53c0b80 1 client.15342 shutdown 2024-06-03T02:21:43.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.574+0000 7f51d53c0b80 1 -- 172.21.15.79:0/279048118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 msgr2=0x557f456c0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:43.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.574+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456c0560 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51b40cc800 tx=0x7f51b409baf0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51d53c0b80 1 -- 172.21.15.79:0/279048118 shutdown_connections 2024-06-03T02:21:43.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f456aa4b0 0x557f456c0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x557f45728190 0x557f4572a5b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f456a4240 0x557f456c0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51d53c0b80 1 --2- 172.21.15.79:0/279048118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f456a9b90 0x557f456c0560 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51d53c0b80 1 -- 172.21.15.79:0/279048118 >> 172.21.15.79:0/279048118 conn(0x557f455d0170 msgr2=0x557f456aa910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:43.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51ae000700 0 client.15342 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:21:43.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51d53c0b80 1 -- 172.21.15.79:0/279048118 shutdown_connections 2024-06-03T02:21:43.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51d53c0b80 1 -- 172.21.15.79:0/279048118 wait complete. 2024-06-03T02:21:43.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.575+0000 7f51d53c0b80 20 client.15342 trim_cache size 0 max 16384 2024-06-03T02:21:43.580 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:21:43.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.576+0000 7f51caa00700 1 -- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a80a2ca0 msgr2=0x7f51a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:43.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.576+0000 7f51caa00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a80a2ca0 0x7f51a80a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f51b00098e0 tx=0x7f51b0005960 comp rx=0 tx=0).stop 2024-06-03T02:21:43.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.576+0000 7f51caa00700 1 -- 172.21.15.79:0/142992468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4104ed0 msgr2=0x7f51c4074150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:43.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.576+0000 7f51caa00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4104ed0 0x7f51c4074150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51b400ef10 tx=0x7f51b400c3a0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.577+0000 7f51caa00700 1 -- 172.21.15.79:0/142992468 shutdown_connections 2024-06-03T02:21:43.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.577+0000 7f51caa00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a80a2ca0 0x7f51a80a5160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.577+0000 7f51caa00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51c41073c0 0x7f51c406cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.577+0000 7f51caa00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51c406a6e0 0x7f51c4073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.577+0000 7f51caa00700 1 --2- 172.21.15.79:0/142992468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51c4104ed0 0x7f51c4074150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:43.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.577+0000 7f51caa00700 1 -- 172.21.15.79:0/142992468 >> 172.21.15.79:0/142992468 conn(0x7f51c4078340 msgr2=0x7f51c4100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:43.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.577+0000 7f51caa00700 1 -- 172.21.15.79:0/142992468 shutdown_connections 2024-06-03T02:21:43.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:43.577+0000 7f51caa00700 1 -- 172.21.15.79:0/142992468 wait complete. 2024-06-03T02:21:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:43 smithi123 ceph-mon[36640]: pgmap v566: 129 pgs: 129 active+clean; 1020 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 MiB/s wr, 788 op/s 2024-06-03T02:21:43.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:43 smithi135 ceph-mon[36653]: pgmap v566: 129 pgs: 129 active+clean; 1020 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 MiB/s wr, 788 op/s 2024-06-03T02:21:44.851 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:21:45.485 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:45.689 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:45 smithi079 ceph-mon[31819]: pgmap v567: 129 pgs: 129 active+clean; 1020 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.3 MiB/s wr, 682 op/s 2024-06-03T02:21:45.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:45 smithi123 ceph-mon[36640]: pgmap v567: 129 pgs: 129 active+clean; 1020 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.3 MiB/s wr, 682 op/s 2024-06-03T02:21:45.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:45 smithi135 ceph-mon[36653]: pgmap v567: 129 pgs: 129 active+clean; 1020 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.3 MiB/s wr, 682 op/s 2024-06-03T02:21:47.697 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:47 smithi079 ceph-mon[31819]: pgmap v568: 129 pgs: 129 active+clean; 1020 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.6 MiB/s wr, 712 op/s 2024-06-03T02:21:47.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:47 smithi123 ceph-mon[36640]: pgmap v568: 129 pgs: 129 active+clean; 1020 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.6 MiB/s wr, 712 op/s 2024-06-03T02:21:47.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:47 smithi135 ceph-mon[36653]: pgmap v568: 129 pgs: 129 active+clean; 1020 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.6 MiB/s wr, 712 op/s 2024-06-03T02:21:48.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.045+0000 7ff635e00700 1 -- 172.21.15.79:0/971572459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6300772e0 msgr2=0x7ff630112e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.045+0000 7ff635e00700 1 --2- 172.21.15.79:0/971572459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6300772e0 0x7ff630112e30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0098f0 tx=0x7ff61c009c00 comp rx=0 tx=0).stop 2024-06-03T02:21:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.046+0000 7ff635e00700 1 -- 172.21.15.79:0/971572459 shutdown_connections 2024-06-03T02:21:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.046+0000 7ff635e00700 1 --2- 172.21.15.79:0/971572459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff630113370 0x7ff630115760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.046+0000 7ff635e00700 1 --2- 172.21.15.79:0/971572459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6300769c0 0x7ff630076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.046+0000 7ff635e00700 1 --2- 172.21.15.79:0/971572459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6300772e0 0x7ff630112e30 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0098f0 tx=0x7ff61c009c00 comp rx=0 tx=0).stop 2024-06-03T02:21:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.046+0000 7ff635e00700 1 -- 172.21.15.79:0/971572459 >> 172.21.15.79:0/971572459 conn(0x7ff6301001c0 msgr2=0x7ff6301025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.047+0000 7ff635e00700 1 -- 172.21.15.79:0/971572459 shutdown_connections 2024-06-03T02:21:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.047+0000 7ff635e00700 1 -- 172.21.15.79:0/971572459 wait complete. 2024-06-03T02:21:48.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.049+0000 7ff635e00700 1 Processor -- start 2024-06-03T02:21:48.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.049+0000 7ff635e00700 1 -- start start 2024-06-03T02:21:48.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.050+0000 7ff635e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6300769c0 0x7ff630112a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.050+0000 7ff635e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff630113370 0x7ff63010bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.050+0000 7ff635e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63010c020 0x7ff63010c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.051+0000 7ff635e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff630118e80 con 0x7ff630113370 2024-06-03T02:21:48.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.051+0000 7ff635e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff630118d00 con 0x7ff63010c020 2024-06-03T02:21:48.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.051+0000 7ff635e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff630119000 con 0x7ff6300769c0 2024-06-03T02:21:48.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.051+0000 7ff62f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63010c020 0x7ff63010c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.051+0000 7ff62f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63010c020 0x7ff63010c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:39818/0 (socket says 172.21.15.79:39818) 2024-06-03T02:21:48.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.051+0000 7ff62f400700 1 -- 172.21.15.79:0/3498848841 learned_addr learned my addr 172.21.15.79:0/3498848841 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:48.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.051+0000 7ff62f400700 1 -- 172.21.15.79:0/3498848841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6300769c0 msgr2=0x7ff630112a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:21:48.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.051+0000 7ff62f400700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6300769c0 0x7ff630112a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.051+0000 7ff62f400700 1 -- 172.21.15.79:0/3498848841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff630113370 msgr2=0x7ff63010bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:21:48.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.052+0000 7ff62f400700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff630113370 0x7ff63010bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.052+0000 7ff62f400700 1 -- 172.21.15.79:0/3498848841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6301105d0 con 0x7ff63010c020 2024-06-03T02:21:48.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.053+0000 7ff62f400700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63010c020 0x7ff63010c4a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff628009fd0 tx=0x7ff62800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:48.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.053+0000 7ff634a00700 1 -- 172.21.15.79:0/3498848841 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280092e0 con 0x7ff63010c020 2024-06-03T02:21:48.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.053+0000 7ff634a00700 1 -- 172.21.15.79:0/3498848841 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff628004bb0 con 0x7ff63010c020 2024-06-03T02:21:48.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.054+0000 7ff635e00700 1 -- 172.21.15.79:0/3498848841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff630110860 con 0x7ff63010c020 2024-06-03T02:21:48.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.054+0000 7ff634a00700 1 -- 172.21.15.79:0/3498848841 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628004280 con 0x7ff63010c020 2024-06-03T02:21:48.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.054+0000 7ff635e00700 1 -- 172.21.15.79:0/3498848841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff630110da0 con 0x7ff63010c020 2024-06-03T02:21:48.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.059+0000 7ff634a00700 1 -- 172.21.15.79:0/3498848841 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff62801b030 con 0x7ff63010c020 2024-06-03T02:21:48.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.061+0000 7ff634a00700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff6140a2ca0 0x7ff6140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.061+0000 7ff634a00700 1 -- 172.21.15.79:0/3498848841 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff6280cedc0 con 0x7ff63010c020 2024-06-03T02:21:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.061+0000 7ff6406cdb80 1 Processor -- start 2024-06-03T02:21:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.061+0000 7ff62ea00700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff6140a2ca0 0x7ff6140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.062+0000 7ff6406cdb80 1 -- start start 2024-06-03T02:21:48.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.063+0000 7ff62ea00700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff6140a2ca0 0x7ff6140a5160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff62000a9b0 tx=0x7ff620005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:48.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.064+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0eb90 0x55846ca0ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.064+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0f4b0 0x55846ca08d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.064+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 0x55846ca09680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.064+0000 7ff62ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0eb90 0x55846ca0ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.064+0000 7ff62ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0eb90 0x55846ca0ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:39564/0 (socket says 172.21.15.79:39564) 2024-06-03T02:21:48.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.064+0000 7ff62e000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0f4b0 0x55846ca08d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.064+0000 7ff62e000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0f4b0 0x55846ca08d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:39834/0 (socket says 172.21.15.79:39834) 2024-06-03T02:21:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.065+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1685586558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55846c917a10 con 0x55846ca09240 2024-06-03T02:21:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.065+0000 7ff62ea00700 1 -- 172.21.15.79:0/1685586558 learned_addr learned my addr 172.21.15.79:0/1685586558 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.065+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1685586558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55846c95ab00 con 0x55846ca0f4b0 2024-06-03T02:21:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.065+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1685586558 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55846c9158f0 con 0x55846ca0eb90 2024-06-03T02:21:48.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.065+0000 7ff62ea00700 1 -- 172.21.15.79:0/1685586558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0f4b0 msgr2=0x55846ca08d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:48.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.065+0000 7ff62ea00700 1 --2- 172.21.15.79:0/1685586558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0f4b0 0x55846ca08d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.065+0000 7ff62ea00700 1 -- 172.21.15.79:0/1685586558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 msgr2=0x55846ca09680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.065+0000 7ff62ea00700 1 --2- 172.21.15.79:0/1685586558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 0x55846ca09680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.065+0000 7ff62ea00700 1 -- 172.21.15.79:0/1685586558 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846c902f90 con 0x55846ca0eb90 2024-06-03T02:21:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.066+0000 7ff62ea00700 1 --2- 172.21.15.79:0/1685586558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0eb90 0x55846ca0ef70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff62000cbc0 tx=0x7ff62000f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=876b352e81368164 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.066+0000 7ff624c00700 1 -- 172.21.15.79:0/1685586558 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620004020 con 0x55846ca0eb90 2024-06-03T02:21:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.066+0000 7ff624c00700 1 -- 172.21.15.79:0/1685586558 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff620015030 con 0x55846ca0eb90 2024-06-03T02:21:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.066+0000 7ff624c00700 1 -- 172.21.15.79:0/1685586558 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620003ac0 con 0x55846ca0eb90 2024-06-03T02:21:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.068+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1685586558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0eb90 msgr2=0x55846ca0ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.068+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/1685586558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0eb90 0x55846ca0ef70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff62000cbc0 tx=0x7ff62000f000 comp rx=0 tx=0).stop 2024-06-03T02:21:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.070+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1685586558 shutdown_connections 2024-06-03T02:21:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.070+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/1685586558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 0x55846ca09680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.070+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/1685586558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0eb90 0x55846ca0ef70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.070+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/1685586558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0f4b0 0x55846ca08d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.071+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1685586558 >> 172.21.15.79:0/1685586558 conn(0x55846c935170 msgr2=0x55846ca03ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:48.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.072+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1685586558 shutdown_connections 2024-06-03T02:21:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.072+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1685586558 wait complete. 2024-06-03T02:21:48.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.073+0000 7ff6406cdb80 1 -- start start 2024-06-03T02:21:48.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.074+0000 7ff6406cdb80 1 Processor -- start 2024-06-03T02:21:48.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.075+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 0x55846ca152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.076+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846ca125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.076+0000 7ff62ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 0x55846ca152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.076+0000 7ff62ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 0x55846ca152a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50114/0 (socket says 172.21.15.79:50114) 2024-06-03T02:21:48.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.077+0000 7ff62ea00700 1 -- 172.21.15.79:0/2004780186 learned_addr learned my addr 172.21.15.79:0/2004780186 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:48.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.077+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0f4b0 0x55846ca12ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.077+0000 7ff62e000700 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846ca125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.077+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55846c8f6f60 con 0x55846ca09240 2024-06-03T02:21:48.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.078+0000 7ff62f400700 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0f4b0 0x55846ca12ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.078+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55846c956ac0 con 0x55846ca0eb90 2024-06-03T02:21:48.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.078+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55846ca15e30 con 0x55846ca0f4b0 2024-06-03T02:21:48.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.079+0000 7ff6406cdb80 10 client.0 _ll_register_callbacks cb 0x55846c8f0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:48.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.080+0000 7ff62f400700 1 -- 172.21.15.79:0/2004780186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 msgr2=0x55846ca125a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:48.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.080+0000 7ff62f400700 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846ca125a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.080+0000 7ff62f400700 1 -- 172.21.15.79:0/2004780186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 msgr2=0x55846ca152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:48.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.080+0000 7ff62f400700 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 0x55846ca152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.080+0000 7ff62f400700 1 -- 172.21.15.79:0/2004780186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846c8a98d0 con 0x55846ca0f4b0 2024-06-03T02:21:48.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.080+0000 7ff62f400700 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0f4b0 0x55846ca12ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff62809afa0 tx=0x7ff62800cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=756b6ce2e9c7634e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:48.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.080+0000 7ff619600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:21:48.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.081+0000 7ff619600700 1 -- 172.21.15.79:0/2004780186 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62809ac00 con 0x55846ca0f4b0 2024-06-03T02:21:48.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.081+0000 7ff619600700 1 -- 172.21.15.79:0/2004780186 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff6280a39f0 con 0x55846ca0f4b0 2024-06-03T02:21:48.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.081+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff620015030 con 0x55846ca0f4b0 2024-06-03T02:21:48.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.082+0000 7ff619600700 1 -- 172.21.15.79:0/2004780186 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6280b4200 con 0x55846ca0f4b0 2024-06-03T02:21:48.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.082+0000 7ff6406cdb80 10 client.25498 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:48.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.082+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55846ca20630 con 0x55846ca0f4b0 2024-06-03T02:21:48.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.082+0000 7ff619600700 1 -- 172.21.15.79:0/2004780186 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff620015030 con 0x55846ca0f4b0 2024-06-03T02:21:48.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.082+0000 7ff619600700 1 -- 172.21.15.79:0/2004780186 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff62809f120 con 0x55846ca0f4b0 2024-06-03T02:21:48.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.082+0000 7ff6406cdb80 10 client.25498 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:48.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.082+0000 7ff6406cdb80 10 client.25498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:48.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.083+0000 7ff6406cdb80 20 client.25498 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:48.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.083+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55846ca22bc0 0x55846ca24fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.083+0000 7ff6406cdb80 4 client.25498 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:21:48.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.083+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55846c968000 con 0x55846ca22bc0 2024-06-03T02:21:48.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.083+0000 7ff62ea00700 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55846ca22bc0 0x55846ca24fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.084+0000 7ff62ea00700 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55846ca22bc0 0x55846ca24fb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe0d72abd1baf47 server_cookie=3fa1cbefb02187f2 in_seq=0 out_seq=0 2024-06-03T02:21:48.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.084+0000 7ff619600700 10 client.25498 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:48.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.085+0000 7ff619600700 1 -- 172.21.15.79:0/2004780186 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55846c968000 con 0x55846ca22bc0 2024-06-03T02:21:48.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.085+0000 7ff619600700 10 client.25498 handle_command_reply: tid=0 2024-06-03T02:21:48.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.085+0000 7ff6406cdb80 1 client.25498 shutdown 2024-06-03T02:21:48.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.086+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0f4b0 msgr2=0x55846ca12ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:48.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.086+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0f4b0 0x55846ca12ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff62809afa0 tx=0x7ff62800cd70 comp rx=0 tx=0).stop 2024-06-03T02:21:48.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.086+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 shutdown_connections 2024-06-03T02:21:48.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.086+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca0f4b0 0x55846ca12ae0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.086+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55846ca22bc0 0x55846ca24fb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.086+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca09240 0x55846ca152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.086+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/2004780186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846ca125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.086+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 >> 172.21.15.79:0/2004780186 conn(0x55846c935170 msgr2=0x55846c9fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:48.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.087+0000 7ff619600700 0 client.25498 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:48.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.087+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 shutdown_connections 2024-06-03T02:21:48.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.087+0000 7ff6406cdb80 1 -- 172.21.15.79:0/2004780186 wait complete. 2024-06-03T02:21:48.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.087+0000 7ff6406cdb80 20 client.25498 trim_cache size 0 max 16384 2024-06-03T02:21:48.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.123+0000 7ff6406cdb80 1 Processor -- start 2024-06-03T02:21:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.123+0000 7ff6406cdb80 1 -- start start 2024-06-03T02:21:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.124+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 0x55846ca08d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.124+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846c9fe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.124+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.124+0000 7ff6406cdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301bd880 con 0x55846ca0f4b0 2024-06-03T02:21:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.124+0000 7ff6406cdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301bf3c0 con 0x55846ca0eb90 2024-06-03T02:21:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.124+0000 7ff6406cdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301bdf50 con 0x55846ca09240 2024-06-03T02:21:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.125+0000 7ff62f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.125+0000 7ff62f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50126/0 (socket says 172.21.15.79:50126) 2024-06-03T02:21:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.125+0000 7ff62f400700 1 -- 172.21.15.79:0/1753840060 learned_addr learned my addr 172.21.15.79:0/1753840060 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.126+0000 7ff62e000700 1 --2- 172.21.15.79:0/1753840060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846c9fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.126+0000 7ff62f400700 1 -- 172.21.15.79:0/1753840060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 msgr2=0x55846ca08d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:21:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.126+0000 7ff62f400700 1 --2- 172.21.15.79:0/1753840060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 0x55846ca08d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.126+0000 7ff62f400700 1 -- 172.21.15.79:0/1753840060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 msgr2=0x55846c9fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:48.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.126+0000 7ff62f400700 1 --2- 172.21.15.79:0/1753840060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846c9fe130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.126+0000 7ff62f400700 1 -- 172.21.15.79:0/1753840060 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846ca10b30 con 0x55846ca0f4b0 2024-06-03T02:21:48.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.126+0000 7ff62f400700 1 --2- 172.21.15.79:0/1753840060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca115c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff6280090a0 tx=0x7ff6280060d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96be5b48fbf332f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:48.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.127+0000 7ff61be00700 1 -- 172.21.15.79:0/1753840060 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62809b070 con 0x55846ca0f4b0 2024-06-03T02:21:48.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.127+0000 7ff61be00700 1 -- 172.21.15.79:0/1753840060 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6280b9e50 con 0x55846ca0f4b0 2024-06-03T02:21:48.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.127+0000 7ff61be00700 1 -- 172.21.15.79:0/1753840060 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62809f650 con 0x55846ca0f4b0 2024-06-03T02:21:48.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.128+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1753840060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 msgr2=0x55846ca115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:48.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.128+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/1753840060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca115c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff6280090a0 tx=0x7ff6280060d0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.128+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1753840060 shutdown_connections 2024-06-03T02:21:48.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.128+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/1753840060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca115c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.128+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/1753840060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 0x55846ca08d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.128+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/1753840060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846c9fe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.129+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1753840060 >> 172.21.15.79:0/1753840060 conn(0x55846c935170 msgr2=0x55846ca10eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:48.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.129+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1753840060 shutdown_connections 2024-06-03T02:21:48.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.129+0000 7ff6406cdb80 1 -- 172.21.15.79:0/1753840060 wait complete. 2024-06-03T02:21:48.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.130+0000 7ff6406cdb80 1 -- start start 2024-06-03T02:21:48.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.130+0000 7ff6406cdb80 1 Processor -- start 2024-06-03T02:21:48.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.131+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 0x55846ca16870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.131+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846ca16db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.131+0000 7ff6406cdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca85820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.131+0000 7ff6406cdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301becf0 con 0x55846ca0f4b0 2024-06-03T02:21:48.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.131+0000 7ff62ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 0x55846ca16870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.132+0000 7ff62ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 0x55846ca16870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:39592/0 (socket says 172.21.15.79:39592) 2024-06-03T02:21:48.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.132+0000 7ff62ea00700 1 -- 172.21.15.79:0/222577593 learned_addr learned my addr 172.21.15.79:0/222577593 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:48.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.132+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301be620 con 0x55846ca0eb90 2024-06-03T02:21:48.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.132+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301bb820 con 0x55846ca09240 2024-06-03T02:21:48.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.132+0000 7ff62f400700 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca85820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.133+0000 7ff62e000700 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846ca16db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.133+0000 7ff6406cdb80 10 client.0 _ll_register_callbacks cb 0x55846c8f0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.134+0000 7ff62ea00700 1 -- 172.21.15.79:0/222577593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 msgr2=0x55846ca16db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.134+0000 7ff62ea00700 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846ca16db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.134+0000 7ff62ea00700 1 -- 172.21.15.79:0/222577593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 msgr2=0x55846ca85820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.134+0000 7ff62ea00700 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca85820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.134+0000 7ff62ea00700 1 -- 172.21.15.79:0/222577593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846ca85e90 con 0x55846ca09240 2024-06-03T02:21:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.135+0000 7ff62e000700 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846ca16db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:21:48.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.136+0000 7ff62ea00700 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 0x55846ca16870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff62000c870 tx=0x7ff62000f950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59f18b189a35302 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:48.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.136+0000 7ff619600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:21:48.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.136+0000 7ff619600700 1 -- 172.21.15.79:0/222577593 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620034060 con 0x55846ca09240 2024-06-03T02:21:48.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.137+0000 7ff619600700 1 -- 172.21.15.79:0/222577593 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff620005250 con 0x55846ca09240 2024-06-03T02:21:48.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.137+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6280b9e50 con 0x55846ca09240 2024-06-03T02:21:48.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.138+0000 7ff619600700 1 -- 172.21.15.79:0/222577593 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff620005450 con 0x55846ca09240 2024-06-03T02:21:48.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.138+0000 7ff6406cdb80 10 client.25501 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:48.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.138+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55846ca8d1d0 con 0x55846ca09240 2024-06-03T02:21:48.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.139+0000 7ff619600700 1 -- 172.21.15.79:0/222577593 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6280b9e50 con 0x55846ca09240 2024-06-03T02:21:48.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.139+0000 7ff619600700 1 -- 172.21.15.79:0/222577593 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff62002e030 con 0x55846ca09240 2024-06-03T02:21:48.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.140+0000 7ff6406cdb80 10 client.25501 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:48.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.140+0000 7ff6406cdb80 10 client.25501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:48.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.140+0000 7ff6406cdb80 20 client.25501 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:48.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.140+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55846ca8d6d0 0x55846ca8faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:48.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.140+0000 7ff6406cdb80 4 client.25501 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:21:48.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.140+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55846c962580 con 0x55846ca8d6d0 2024-06-03T02:21:48.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.140+0000 7ff62e000700 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55846ca8d6d0 0x55846ca8faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:48.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.141+0000 7ff62e000700 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55846ca8d6d0 0x55846ca8faf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62d185e5c8480f0 server_cookie=be877377634057ab in_seq=0 out_seq=0 2024-06-03T02:21:48.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.141+0000 7ff619600700 10 client.25501 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:48.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.142+0000 7ff619600700 1 -- 172.21.15.79:0/222577593 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55846c962580 con 0x55846ca8d6d0 2024-06-03T02:21:48.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.143+0000 7ff619600700 10 client.25501 handle_command_reply: tid=0 2024-06-03T02:21:48.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.143+0000 7ff6406cdb80 1 client.25501 shutdown 2024-06-03T02:21:48.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.143+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 msgr2=0x55846ca16870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:48.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.143+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 0x55846ca16870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff62000c870 tx=0x7ff62000f950 comp rx=0 tx=0).stop 2024-06-03T02:21:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.144+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 shutdown_connections 2024-06-03T02:21:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.144+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55846ca8d6d0 0x55846ca8faf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.144+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55846ca0f4b0 0x55846ca85820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.144+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846ca09240 0x55846ca16870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.144+0000 7ff6406cdb80 1 --2- 172.21.15.79:0/222577593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55846ca0eb90 0x55846ca16db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.144+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 >> 172.21.15.79:0/222577593 conn(0x55846c935170 msgr2=0x55846ca14660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:48.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.144+0000 7ff619600700 0 client.25501 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:48.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.144+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 shutdown_connections 2024-06-03T02:21:48.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.145+0000 7ff6406cdb80 1 -- 172.21.15.79:0/222577593 wait complete. 2024-06-03T02:21:48.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.146+0000 7ff6406cdb80 20 client.25501 trim_cache size 0 max 16384 2024-06-03T02:21:48.146 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:21:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.147+0000 7ff635e00700 1 -- 172.21.15.79:0/3498848841 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff6140a2ca0 msgr2=0x7ff6140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.147+0000 7ff635e00700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff6140a2ca0 0x7ff6140a5160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff62000a9b0 tx=0x7ff620005c90 comp rx=0 tx=0).stop 2024-06-03T02:21:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.147+0000 7ff635e00700 1 -- 172.21.15.79:0/3498848841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63010c020 msgr2=0x7ff63010c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:48.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.147+0000 7ff635e00700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63010c020 0x7ff63010c4a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff628009fd0 tx=0x7ff62800ef90 comp rx=0 tx=0).stop 2024-06-03T02:21:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.149+0000 7ff635e00700 1 -- 172.21.15.79:0/3498848841 shutdown_connections 2024-06-03T02:21:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.149+0000 7ff635e00700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63010c020 0x7ff63010c4a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.149+0000 7ff635e00700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6300769c0 0x7ff630112a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.149+0000 7ff635e00700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff6140a2ca0 0x7ff6140a5160 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.149+0000 7ff635e00700 1 --2- 172.21.15.79:0/3498848841 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff630113370 0x7ff63010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.149+0000 7ff635e00700 1 -- 172.21.15.79:0/3498848841 >> 172.21.15.79:0/3498848841 conn(0x7ff6301001c0 msgr2=0x7ff6301025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.149+0000 7ff635e00700 1 -- 172.21.15.79:0/3498848841 shutdown_connections 2024-06-03T02:21:48.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:48.149+0000 7ff635e00700 1 -- 172.21.15.79:0/3498848841 wait complete. 2024-06-03T02:21:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 100.0 s, 1793.0 tps, lat 55.991 ms stddev 71.949 2024-06-03T02:21:49.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:49 smithi123 ceph-mon[36640]: pgmap v569: 129 pgs: 129 active+clean; 1020 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 760 op/s 2024-06-03T02:21:49.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:49 smithi079 ceph-mon[31819]: pgmap v569: 129 pgs: 129 active+clean; 1020 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 760 op/s 2024-06-03T02:21:49.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:49 smithi135 ceph-mon[36653]: pgmap v569: 129 pgs: 129 active+clean; 1020 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 760 op/s 2024-06-03T02:21:50.153 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:21:50.153 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:21:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:51.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:51 smithi079 ceph-mon[31819]: pgmap v570: 129 pgs: 129 active+clean; 1021 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.5 MiB/s wr, 638 op/s 2024-06-03T02:21:51.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:51 smithi123 ceph-mon[36640]: pgmap v570: 129 pgs: 129 active+clean; 1021 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.5 MiB/s wr, 638 op/s 2024-06-03T02:21:51.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:51 smithi135 ceph-mon[36653]: pgmap v570: 129 pgs: 129 active+clean; 1021 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.5 MiB/s wr, 638 op/s 2024-06-03T02:21:53.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 105.0 s, 2612.9 tps, lat 38.225 ms stddev 42.354 2024-06-03T02:21:53.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.420+0000 7fc5aa400700 1 -- 172.21.15.79:0/1887077795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a4100870 msgr2=0x7fc5a4100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:53.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.420+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1887077795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a4100870 0x7fc5a4100c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5940099c0 tx=0x7fc594009cd0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.420+0000 7fc5aa400700 1 -- 172.21.15.79:0/1887077795 shutdown_connections 2024-06-03T02:21:53.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.420+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1887077795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a4108950 0x7fc5a410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.420+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1887077795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a4100870 0x7fc5a4100c50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.420+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1887077795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a4101220 0x7fc5a4108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.420+0000 7fc5aa400700 1 -- 172.21.15.79:0/1887077795 >> 172.21.15.79:0/1887077795 conn(0x7fc5a4077590 msgr2=0x7fc5a40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:53.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.422+0000 7fc5aa400700 1 -- 172.21.15.79:0/1887077795 shutdown_connections 2024-06-03T02:21:53.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.422+0000 7fc5aa400700 1 -- 172.21.15.79:0/1887077795 wait complete. 2024-06-03T02:21:53.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.423+0000 7fc5aa400700 1 Processor -- start 2024-06-03T02:21:53.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.424+0000 7fc5aa400700 1 -- start start 2024-06-03T02:21:53.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.424+0000 7fc5aa400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a4100870 0x7fc5a410e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.424+0000 7fc5aa400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a4101220 0x7fc5a410ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.424+0000 7fc5aa400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a4108950 0x7fc5a41a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.424+0000 7fc5aa400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41127e0 con 0x7fc5a4100870 2024-06-03T02:21:53.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.424+0000 7fc5aa400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4112660 con 0x7fc5a4108950 2024-06-03T02:21:53.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.424+0000 7fc5aa400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4112960 con 0x7fc5a4101220 2024-06-03T02:21:53.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.425+0000 7fc5a3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a4108950 0x7fc5a41a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.425+0000 7fc5a3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a4108950 0x7fc5a41a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:39880/0 (socket says 172.21.15.79:39880) 2024-06-03T02:21:53.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.425+0000 7fc5a3e00700 1 -- 172.21.15.79:0/1593464307 learned_addr learned my addr 172.21.15.79:0/1593464307 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:53.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.425+0000 7fc5a3e00700 1 -- 172.21.15.79:0/1593464307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a4101220 msgr2=0x7fc5a410ebb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:21:53.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.425+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a4101220 0x7fc5a410ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.425+0000 7fc5a3e00700 1 -- 172.21.15.79:0/1593464307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a4100870 msgr2=0x7fc5a410e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:53.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.425+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a4100870 0x7fc5a410e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.425+0000 7fc5a3e00700 1 -- 172.21.15.79:0/1593464307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a41a4280 con 0x7fc5a4108950 2024-06-03T02:21:53.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.427+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a4108950 0x7fc5a41a3b40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc59c007ee0 tx=0x7fc59c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:53.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.428+0000 7fc5a0c00700 1 -- 172.21.15.79:0/1593464307 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00dbd0 con 0x7fc5a4108950 2024-06-03T02:21:53.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.428+0000 7fc5a0c00700 1 -- 172.21.15.79:0/1593464307 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c00dd70 con 0x7fc5a4108950 2024-06-03T02:21:53.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.428+0000 7fc5a0c00700 1 -- 172.21.15.79:0/1593464307 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0133f0 con 0x7fc5a4108950 2024-06-03T02:21:53.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.430+0000 7fc5aa400700 1 -- 172.21.15.79:0/1593464307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a41a4570 con 0x7fc5a4108950 2024-06-03T02:21:53.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.430+0000 7fc5aa400700 1 -- 172.21.15.79:0/1593464307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41abdf0 con 0x7fc5a4108950 2024-06-03T02:21:53.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.437+0000 7fc5a0c00700 1 -- 172.21.15.79:0/1593464307 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc59c01a020 con 0x7fc5a4108950 2024-06-03T02:21:53.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.441+0000 7fc5a0c00700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5880a2d70 0x7fc5880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.442+0000 7fc5a0c00700 1 -- 172.21.15.79:0/1593464307 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc59c0ce270 con 0x7fc5a4108950 2024-06-03T02:21:53.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.443+0000 7fc5a3400700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5880a2d70 0x7fc5880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.445+0000 7fc5a3400700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5880a2d70 0x7fc5880a5230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc594009750 tx=0x7fc594005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:53.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.448+0000 7fc5b4e1bb80 1 Processor -- start 2024-06-03T02:21:53.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.452+0000 7fc5b4e1bb80 1 -- start start 2024-06-03T02:21:53.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.453+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea7d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.453+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.453+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.453+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fdea6dba90 con 0x55fdea7d2c10 2024-06-03T02:21:53.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.453+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fdea71eb80 con 0x55fdea7d3530 2024-06-03T02:21:53.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.453+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fdea6d9970 con 0x55fdea7cd2c0 2024-06-03T02:21:53.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.454+0000 7fc59be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.454+0000 7fc59be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:39882/0 (socket says 172.21.15.79:39882) 2024-06-03T02:21:53.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.454+0000 7fc59be00700 1 -- 172.21.15.79:0/4086078166 learned_addr learned my addr 172.21.15.79:0/4086078166 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:53.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.454+0000 7fc5a3400700 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea7d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.454+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.455+0000 7fc59be00700 1 -- 172.21.15.79:0/4086078166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 msgr2=0x55fdea7cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:53.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.455+0000 7fc59be00700 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.455+0000 7fc59be00700 1 -- 172.21.15.79:0/4086078166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 msgr2=0x55fdea7d2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:53.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.455+0000 7fc59be00700 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea7d2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.455+0000 7fc59be00700 1 -- 172.21.15.79:0/4086078166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdea6c7010 con 0x55fdea7d3530 2024-06-03T02:21:53.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.456+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:21:53.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.456+0000 7fc59be00700 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7ccd80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00bab0 tx=0x7fc58c00bdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc28991a6a18437a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:53.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.460+0000 7fc598c00700 1 -- 172.21.15.79:0/4086078166 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c016070 con 0x55fdea7d3530 2024-06-03T02:21:53.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.463+0000 7fc598c00700 1 -- 172.21.15.79:0/4086078166 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc58c00d7d0 con 0x55fdea7d3530 2024-06-03T02:21:53.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.463+0000 7fc598c00700 1 -- 172.21.15.79:0/4086078166 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c005480 con 0x55fdea7d3530 2024-06-03T02:21:53.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.464+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/4086078166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 msgr2=0x55fdea7ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:53.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.464+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7ccd80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00bab0 tx=0x7fc58c00bdc0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.464+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/4086078166 shutdown_connections 2024-06-03T02:21:53.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.464+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.464+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea7d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.464+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/4086078166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7ccd80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.464+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/4086078166 >> 172.21.15.79:0/4086078166 conn(0x55fdea6f91f0 msgr2=0x55fdea7c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:53.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.465+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/4086078166 shutdown_connections 2024-06-03T02:21:53.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.465+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/4086078166 wait complete. 2024-06-03T02:21:53.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.466+0000 7fc5b4e1bb80 1 -- start start 2024-06-03T02:21:53.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.467+0000 7fc5b4e1bb80 1 Processor -- start 2024-06-03T02:21:53.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.467+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7d92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.467+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea7d65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.467+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.467+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fdea6bafe0 con 0x55fdea7d2c10 2024-06-03T02:21:53.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.467+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fdea71ab40 con 0x55fdea7d3530 2024-06-03T02:21:53.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.467+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fdea7d9f10 con 0x55fdea7cd2c0 2024-06-03T02:21:53.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.467+0000 7fc5b4e1bb80 10 client.0 _ll_register_callbacks cb 0x55fdea6b4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:53.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.468+0000 7fc59be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea7d65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.468+0000 7fc5a3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7d92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.468+0000 7fc59be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea7d65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50184/0 (socket says 172.21.15.79:50184) 2024-06-03T02:21:53.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.468+0000 7fc59be00700 1 -- 172.21.15.79:0/188688865 learned_addr learned my addr 172.21.15.79:0/188688865 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:53.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.469+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.470+0000 7fc5a3400700 1 -- 172.21.15.79:0/188688865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 msgr2=0x55fdea7d6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:53.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.470+0000 7fc5a3400700 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.470+0000 7fc5a3400700 1 -- 172.21.15.79:0/188688865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 msgr2=0x55fdea7d65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:53.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.470+0000 7fc5a3400700 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea7d65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.470+0000 7fc5a3400700 1 -- 172.21.15.79:0/188688865 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdea66d950 con 0x55fdea7cd2c0 2024-06-03T02:21:53.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.471+0000 7fc5a3400700 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7d92a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc594009fd0 tx=0x7fc594009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d42dcdd84596b437 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:53.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.471+0000 7fc591600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:21:53.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.472+0000 7fc591600700 1 -- 172.21.15.79:0/188688865 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59400b9a0 con 0x55fdea7cd2c0 2024-06-03T02:21:53.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.472+0000 7fc591600700 1 -- 172.21.15.79:0/188688865 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc59401e070 con 0x55fdea7cd2c0 2024-06-03T02:21:53.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.472+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/188688865 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc58c00d7d0 con 0x55fdea7cd2c0 2024-06-03T02:21:53.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.473+0000 7fc591600700 1 -- 172.21.15.79:0/188688865 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc59400bb40 con 0x55fdea7cd2c0 2024-06-03T02:21:53.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.473+0000 7fc5b4e1bb80 10 client.25510 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:53.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.473+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/188688865 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdea7e4500 con 0x55fdea7cd2c0 2024-06-03T02:21:53.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.475+0000 7fc591600700 1 -- 172.21.15.79:0/188688865 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc58c00d7d0 con 0x55fdea7cd2c0 2024-06-03T02:21:53.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.475+0000 7fc591600700 1 -- 172.21.15.79:0/188688865 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc594022070 con 0x55fdea7cd2c0 2024-06-03T02:21:53.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.476+0000 7fc5b4e1bb80 10 client.25510 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:53.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.476+0000 7fc5b4e1bb80 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:53.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.476+0000 7fc5b4e1bb80 20 client.25510 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:53.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.476+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fdea7e6b40 0x55fdea7e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.476+0000 7fc5b4e1bb80 4 client.25510 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:21:53.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.476+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/188688865 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdea72c080 con 0x55fdea7e6b40 2024-06-03T02:21:53.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.476+0000 7fc59be00700 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fdea7e6b40 0x55fdea7e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.478+0000 7fc59be00700 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fdea7e6b40 0x55fdea7e8f30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1129d3e25d847a3b server_cookie=efa75b35ea10c792 in_seq=0 out_seq=0 2024-06-03T02:21:53.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.478+0000 7fc591600700 10 client.25510 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:53.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.479+0000 7fc591600700 1 -- 172.21.15.79:0/188688865 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fdea72c080 con 0x55fdea7e6b40 2024-06-03T02:21:53.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.479+0000 7fc591600700 10 client.25510 handle_command_reply: tid=0 2024-06-03T02:21:53.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.480+0000 7fc5b4e1bb80 1 client.25510 shutdown 2024-06-03T02:21:53.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.482+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/188688865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 msgr2=0x55fdea7d92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:53.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.482+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7d92a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc594009fd0 tx=0x7fc594009310 comp rx=0 tx=0).stop 2024-06-03T02:21:53.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.482+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/188688865 shutdown_connections 2024-06-03T02:21:53.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.482+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.482+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea7d92a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.482+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fdea7e6b40 0x55fdea7e8f30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.482+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/188688865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea7d65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.482+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/188688865 >> 172.21.15.79:0/188688865 conn(0x55fdea6f91f0 msgr2=0x55fdea7c1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:53.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.483+0000 7fc591600700 0 client.25510 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:53.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.483+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/188688865 shutdown_connections 2024-06-03T02:21:53.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.483+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/188688865 wait complete. 2024-06-03T02:21:53.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.484+0000 7fc5b4e1bb80 20 client.25510 trim_cache size 0 max 16384 2024-06-03T02:21:53.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.533+0000 7fc5b4e1bb80 1 Processor -- start 2024-06-03T02:21:53.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.533+0000 7fc5b4e1bb80 1 -- start start 2024-06-03T02:21:53.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.534+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7cd2c0 0x55fdea7c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.534+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7d2c10 0x55fdea7d8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.534+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.534+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b4920 con 0x55fdea7cd2c0 2024-06-03T02:21:53.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.534+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b6460 con 0x55fdea7d3530 2024-06-03T02:21:53.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.534+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b4ff0 con 0x55fdea7d2c10 2024-06-03T02:21:53.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.534+0000 7fc5a3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.534+0000 7fc5a3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:39910/0 (socket says 172.21.15.79:39910) 2024-06-03T02:21:53.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.534+0000 7fc5a3e00700 1 -- 172.21.15.79:0/1787763461 learned_addr learned my addr 172.21.15.79:0/1787763461 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:53.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.535+0000 7fc5a3e00700 1 -- 172.21.15.79:0/1787763461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7d2c10 msgr2=0x55fdea7d8de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:21:53.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.535+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/1787763461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7d2c10 0x55fdea7d8de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.535+0000 7fc5a3e00700 1 -- 172.21.15.79:0/1787763461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7cd2c0 msgr2=0x55fdea7c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:53.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.535+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/1787763461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7cd2c0 0x55fdea7c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.535+0000 7fc5a3e00700 1 -- 172.21.15.79:0/1787763461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdea7d74a0 con 0x55fdea7d3530 2024-06-03T02:21:53.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.535+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/1787763461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d1d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00d7b0 tx=0x7fc59c0cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d6d2b05be93961b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:53.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.538+0000 7fc593e00700 1 -- 172.21.15.79:0/1787763461 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c09ead0 con 0x55fdea7d3530 2024-06-03T02:21:53.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.538+0000 7fc593e00700 1 -- 172.21.15.79:0/1787763461 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c0a3390 con 0x55fdea7d3530 2024-06-03T02:21:53.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.538+0000 7fc593e00700 1 -- 172.21.15.79:0/1787763461 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0a2f00 con 0x55fdea7d3530 2024-06-03T02:21:53.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.539+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1787763461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 msgr2=0x55fdea7d1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:53.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.539+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1787763461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d1d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00d7b0 tx=0x7fc59c0cd040 comp rx=0 tx=0).stop 2024-06-03T02:21:53.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.539+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1787763461 shutdown_connections 2024-06-03T02:21:53.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.539+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1787763461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea7d1d80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.539+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1787763461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7cd2c0 0x55fdea7c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.539+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1787763461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7d2c10 0x55fdea7d8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.539+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1787763461 >> 172.21.15.79:0/1787763461 conn(0x55fdea6f91f0 msgr2=0x55fdea7d1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:53.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.539+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1787763461 shutdown_connections 2024-06-03T02:21:53.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.539+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1787763461 wait complete. 2024-06-03T02:21:53.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.540+0000 7fc5b4e1bb80 1 -- start start 2024-06-03T02:21:53.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.540+0000 7fc5b4e1bb80 1 Processor -- start 2024-06-03T02:21:53.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.541+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea84b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.541+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea84bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.542+0000 7fc5b4e1bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea84c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.542+0000 7fc59be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea84bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.542+0000 7fc59be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea84bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50216/0 (socket says 172.21.15.79:50216) 2024-06-03T02:21:53.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.542+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b5d90 con 0x55fdea7d2c10 2024-06-03T02:21:53.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.542+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b56c0 con 0x55fdea7d3530 2024-06-03T02:21:53.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.542+0000 7fc5b4e1bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b28c0 con 0x55fdea7cd2c0 2024-06-03T02:21:53.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.542+0000 7fc5a3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea84b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.542+0000 7fc59be00700 1 -- 172.21.15.79:0/1727966293 learned_addr learned my addr 172.21.15.79:0/1727966293 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:53.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.542+0000 7fc5b4e1bb80 10 client.0 _ll_register_callbacks cb 0x55fdea6b4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:53.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.543+0000 7fc59be00700 1 -- 172.21.15.79:0/1727966293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 msgr2=0x55fdea84b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:53.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.544+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea84c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.544+0000 7fc59be00700 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea84b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.544+0000 7fc59be00700 1 -- 172.21.15.79:0/1727966293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 msgr2=0x55fdea84c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.544+0000 7fc59be00700 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea84c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.544+0000 7fc59be00700 1 -- 172.21.15.79:0/1727966293 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdea7de4b0 con 0x55fdea7d2c10 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.544+0000 7fc5a3e00700 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea84c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.544+0000 7fc5a3400700 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea84b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.544+0000 7fc59be00700 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea84bdd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00b2e0 tx=0x7fc58c00daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a906bdd5519408 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.544+0000 7fc591600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.545+0000 7fc591600700 1 -- 172.21.15.79:0/1727966293 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c016040 con 0x55fdea7d2c10 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.545+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1727966293 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59c0a3390 con 0x55fdea7d2c10 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.545+0000 7fc591600700 1 -- 172.21.15.79:0/1727966293 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc58c0054a0 con 0x55fdea7d2c10 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.546+0000 7fc591600700 1 -- 172.21.15.79:0/1727966293 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc58c033050 con 0x55fdea7d2c10 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.546+0000 7fc5b4e1bb80 10 client.15366 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:53.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.546+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1727966293 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdea7df850 con 0x55fdea7d2c10 2024-06-03T02:21:53.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.547+0000 7fc591600700 1 -- 172.21.15.79:0/1727966293 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c0a3390 con 0x55fdea7d2c10 2024-06-03T02:21:53.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.547+0000 7fc591600700 1 -- 172.21.15.79:0/1727966293 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc58c023070 con 0x55fdea7d2c10 2024-06-03T02:21:53.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.547+0000 7fc5b4e1bb80 10 client.15366 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:53.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.547+0000 7fc5b4e1bb80 10 client.15366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:53.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.547+0000 7fc5b4e1bb80 20 client.15366 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:53.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.547+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fdea7dfc60 0x55fdea8667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:53.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.548+0000 7fc5b4e1bb80 4 client.15366 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:21:53.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.548+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1727966293 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fdea7d5930 con 0x55fdea7dfc60 2024-06-03T02:21:53.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.548+0000 7fc5a3400700 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fdea7dfc60 0x55fdea8667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:53.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.548+0000 7fc591600700 10 client.15366 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:53.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.549+0000 7fc5a3400700 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fdea7dfc60 0x55fdea8667b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=facda522baf67884 server_cookie=92c638bc5c573e16 in_seq=0 out_seq=0 2024-06-03T02:21:53.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.549+0000 7fc591600700 1 -- 172.21.15.79:0/1727966293 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fdea7d5930 con 0x55fdea7dfc60 2024-06-03T02:21:53.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.549+0000 7fc591600700 10 client.15366 handle_command_reply: tid=0 2024-06-03T02:21:53.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.549+0000 7fc5b4e1bb80 1 client.15366 shutdown 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.550+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1727966293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 msgr2=0x55fdea84bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.550+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea84bdd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00b2e0 tx=0x7fc58c00daa0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.550+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1727966293 shutdown_connections 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.550+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdea7d3530 0x55fdea84c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.550+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fdea7cd2c0 0x55fdea84b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.550+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fdea7dfc60 0x55fdea8667b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.550+0000 7fc5b4e1bb80 1 --2- 172.21.15.79:0/1727966293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fdea7d2c10 0x55fdea84bdd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.550+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1727966293 >> 172.21.15.79:0/1727966293 conn(0x55fdea6f91f0 msgr2=0x55fdea7d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.550+0000 7fc591600700 0 client.15366 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.551+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1727966293 shutdown_connections 2024-06-03T02:21:53.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.551+0000 7fc5b4e1bb80 1 -- 172.21.15.79:0/1727966293 wait complete. 2024-06-03T02:21:53.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.551+0000 7fc5b4e1bb80 20 client.15366 trim_cache size 0 max 16384 2024-06-03T02:21:53.552 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:21:53.552 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:21:53.552 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "f8428187-2fa3-49ac-af29-d7912fe01936", 2024-06-03T02:21:53.552 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:21:53.552 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:21:53.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 -- 172.21.15.79:0/1593464307 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5880a2d70 msgr2=0x7fc5880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:53.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5880a2d70 0x7fc5880a5230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc594009750 tx=0x7fc594005fb0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 -- 172.21.15.79:0/1593464307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a4108950 msgr2=0x7fc5a41a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:53.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a4108950 0x7fc5a41a3b40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc59c007ee0 tx=0x7fc59c00d3b0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 -- 172.21.15.79:0/1593464307 shutdown_connections 2024-06-03T02:21:53.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a4108950 0x7fc5a41a3b40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc5880a2d70 0x7fc5880a5230 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a4100870 0x7fc5a410e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 --2- 172.21.15.79:0/1593464307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a4101220 0x7fc5a410ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 -- 172.21.15.79:0/1593464307 >> 172.21.15.79:0/1593464307 conn(0x7fc5a4077590 msgr2=0x7fc5a40ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 -- 172.21.15.79:0/1593464307 shutdown_connections 2024-06-03T02:21:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:53.552+0000 7fc5aa400700 1 -- 172.21.15.79:0/1593464307 wait complete. 2024-06-03T02:21:53.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:53 smithi123 ceph-mon[36640]: pgmap v571: 129 pgs: 129 active+clean; 1021 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.6 MiB/s wr, 841 op/s 2024-06-03T02:21:53.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:53 smithi079 ceph-mon[31819]: pgmap v571: 129 pgs: 129 active+clean; 1021 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.6 MiB/s wr, 841 op/s 2024-06-03T02:21:54.269 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:53 smithi135 ceph-mon[36653]: pgmap v571: 129 pgs: 129 active+clean; 1021 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.6 MiB/s wr, 841 op/s 2024-06-03T02:21:54.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:54 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:21:54.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:54 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:54.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:54 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:21:54.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:54 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:54.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:54 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:21:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:54 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:21:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:54 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:54 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:21:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:54 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:54.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:54 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:21:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:54 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:21:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:54 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:54 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:21:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:54 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:21:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:54 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:21:55.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:21:55.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:55 smithi123 ceph-mon[36640]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 629 op/s 2024-06-03T02:21:55.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:55 smithi079 ceph-mon[31819]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 629 op/s 2024-06-03T02:21:55.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:55 smithi135 ceph-mon[36653]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 629 op/s 2024-06-03T02:21:55.851 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:21:57.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:57 smithi079 ceph-mon[31819]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.2 MiB/s wr, 611 op/s 2024-06-03T02:21:57.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:57 smithi135 ceph-mon[36653]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.2 MiB/s wr, 611 op/s 2024-06-03T02:21:57.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:57 smithi123 ceph-mon[36640]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.2 MiB/s wr, 611 op/s 2024-06-03T02:21:58.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 110.0 s, 1580.4 tps, lat 63.125 ms stddev 77.656 2024-06-03T02:21:58.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.346+0000 7f32f0800700 1 -- 172.21.15.79:0/3856073826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e806a6e0 msgr2=0x7f32e806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:58.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.346+0000 7f32f0800700 1 --2- 172.21.15.79:0/3856073826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e806a6e0 0x7f32e806ab40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009990 tx=0x7f32d8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.346+0000 7f32f0800700 1 -- 172.21.15.79:0/3856073826 shutdown_connections 2024-06-03T02:21:58.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.346+0000 7f32f0800700 1 --2- 172.21.15.79:0/3856073826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32e8104ed0 0x7f32e8105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.346+0000 7f32f0800700 1 --2- 172.21.15.79:0/3856073826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e81073c0 0x7f32e806a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.346+0000 7f32f0800700 1 --2- 172.21.15.79:0/3856073826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e806a6e0 0x7f32e806ab40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.346+0000 7f32f0800700 1 -- 172.21.15.79:0/3856073826 >> 172.21.15.79:0/3856073826 conn(0x7f32e8078340 msgr2=0x7f32e8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:58.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.346+0000 7f32f0800700 1 -- 172.21.15.79:0/3856073826 shutdown_connections 2024-06-03T02:21:58.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.346+0000 7f32f0800700 1 -- 172.21.15.79:0/3856073826 wait complete. 2024-06-03T02:21:58.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.349+0000 7f32f0800700 1 Processor -- start 2024-06-03T02:21:58.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.350+0000 7f32f0800700 1 -- start start 2024-06-03T02:21:58.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.351+0000 7f32f0800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e806a6e0 0x7f32e81159b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.351+0000 7f32f0800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32e8104ed0 0x7f32e810fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.351+0000 7f32f0800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e81073c0 0x7f32e810ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.351+0000 7f32f0800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8108400 con 0x7f32e8104ed0 2024-06-03T02:21:58.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.351+0000 7f32f0800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8108280 con 0x7f32e806a6e0 2024-06-03T02:21:58.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.351+0000 7f32f0800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8108580 con 0x7f32e81073c0 2024-06-03T02:21:58.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.351+0000 7f32ed400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32e8104ed0 0x7f32e810fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.352+0000 7f32ed400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32e8104ed0 0x7f32e810fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37006/0 (socket says 172.21.15.79:37006) 2024-06-03T02:21:58.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.352+0000 7f32ed400700 1 -- 172.21.15.79:0/3559930715 learned_addr learned my addr 172.21.15.79:0/3559930715 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:58.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.352+0000 7f32ed400700 1 -- 172.21.15.79:0/3559930715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e81073c0 msgr2=0x7f32e810ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:58.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.352+0000 7f32ee800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e81073c0 0x7f32e810ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.352+0000 7f32ede00700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e806a6e0 0x7f32e81159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.352+0000 7f32ed400700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e81073c0 0x7f32e810ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.352+0000 7f32ed400700 1 -- 172.21.15.79:0/3559930715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e806a6e0 msgr2=0x7f32e81159b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:58.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.352+0000 7f32ed400700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e806a6e0 0x7f32e81159b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.352+0000 7f32ed400700 1 -- 172.21.15.79:0/3559930715 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e8110730 con 0x7f32e8104ed0 2024-06-03T02:21:58.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.353+0000 7f32ee800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e81073c0 0x7f32e810ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:58.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.354+0000 7f32ede00700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e806a6e0 0x7f32e81159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:58.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.354+0000 7f32ed400700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32e8104ed0 0x7f32e810fa10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32d80057f0 tx=0x7f32d8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:58.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.355+0000 7f32e2a00700 1 -- 172.21.15.79:0/3559930715 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8027070 con 0x7f32e8104ed0 2024-06-03T02:21:58.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.355+0000 7f32f0800700 1 -- 172.21.15.79:0/3559930715 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e806beb0 con 0x7f32e8104ed0 2024-06-03T02:21:58.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.355+0000 7f32f0800700 1 -- 172.21.15.79:0/3559930715 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e806c380 con 0x7f32e8104ed0 2024-06-03T02:21:58.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.360+0000 7f32e2a00700 1 -- 172.21.15.79:0/3559930715 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32d80095d0 con 0x7f32e8104ed0 2024-06-03T02:21:58.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.360+0000 7f32e2a00700 1 -- 172.21.15.79:0/3559930715 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d800f880 con 0x7f32e8104ed0 2024-06-03T02:21:58.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.362+0000 7f32e2a00700 1 -- 172.21.15.79:0/3559930715 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f32d8005080 con 0x7f32e8104ed0 2024-06-03T02:21:58.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.366+0000 7f32fb204b80 1 Processor -- start 2024-06-03T02:21:58.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.367+0000 7f32e2a00700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f32d00a2ca0 0x7f32d00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.367+0000 7f32e2a00700 1 -- 172.21.15.79:0/3559930715 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f32d80d0db0 con 0x7f32e8104ed0 2024-06-03T02:21:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.368+0000 7f32fb204b80 1 -- start start 2024-06-03T02:21:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.368+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be24f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.368+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be1ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.368+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be1f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.368+0000 7f32ee800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f32d00a2ca0 0x7f32d00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.368+0000 7f32fb204b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56190bd2da10 con 0x56190be1f240 2024-06-03T02:21:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.368+0000 7f32fb204b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56190bd70b00 con 0x56190be24b90 2024-06-03T02:21:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.368+0000 7f32fb204b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56190bd2b8f0 con 0x56190be254b0 2024-06-03T02:21:58.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.369+0000 7f32ed400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be1ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.370+0000 7f32ede00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be24f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.370+0000 7f32ede00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be24f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:45882/0 (socket says 172.21.15.79:45882) 2024-06-03T02:21:58.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.370+0000 7f32ede00700 1 -- 172.21.15.79:0/1476519145 learned_addr learned my addr 172.21.15.79:0/1476519145 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:58.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.370+0000 7f32ee800700 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be1f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.370+0000 7f32ede00700 1 -- 172.21.15.79:0/1476519145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 msgr2=0x56190be1ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:58.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.370+0000 7f32ede00700 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be1ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.370+0000 7f32ede00700 1 -- 172.21.15.79:0/1476519145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 msgr2=0x56190be1f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:58.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.370+0000 7f32ede00700 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be1f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.370+0000 7f32ede00700 1 -- 172.21.15.79:0/1476519145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190bd18f90 con 0x56190be24b90 2024-06-03T02:21:58.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.371+0000 7f32ee800700 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be1f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:58.372 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.371+0000 7f32ed400700 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be1ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:58.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.372+0000 7f32ede00700 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be24f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007fb0 tx=0x7f32dc00c5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f4213fdd9926d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:58.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.372+0000 7f32ee800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f32d00a2ca0 0x7f32d00a5160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f32e4006fd0 tx=0x7f32e400a500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:58.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.372+0000 7f32d7400700 1 -- 172.21.15.79:0/1476519145 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc014070 con 0x56190be24b90 2024-06-03T02:21:58.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.372+0000 7f32d7400700 1 -- 172.21.15.79:0/1476519145 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32dc00f920 con 0x56190be24b90 2024-06-03T02:21:58.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.372+0000 7f32d7400700 1 -- 172.21.15.79:0/1476519145 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc00e960 con 0x56190be24b90 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.373+0000 7f32fb204b80 1 -- 172.21.15.79:0/1476519145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 msgr2=0x56190be24f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.373+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be24f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007fb0 tx=0x7f32dc00c5a0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.374+0000 7f32fb204b80 1 -- 172.21.15.79:0/1476519145 shutdown_connections 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.374+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be1f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.374+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be24f70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.374+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1476519145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be1ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.374+0000 7f32fb204b80 1 -- 172.21.15.79:0/1476519145 >> 172.21.15.79:0/1476519145 conn(0x56190bd4b170 msgr2=0x56190be19ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.374+0000 7f32fb204b80 1 -- 172.21.15.79:0/1476519145 shutdown_connections 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.374+0000 7f32fb204b80 1 -- 172.21.15.79:0/1476519145 wait complete. 2024-06-03T02:21:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.375+0000 7f32fb204b80 1 -- start start 2024-06-03T02:21:58.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.375+0000 7f32fb204b80 1 Processor -- start 2024-06-03T02:21:58.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.376+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be2b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.376+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be24b90 0x56190be283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.376+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be254b0 0x56190be28930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.376+0000 7f32ede00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be2b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.377+0000 7f32fb204b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56190bd0cf60 con 0x56190be1f240 2024-06-03T02:21:58.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.377+0000 7f32fb204b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56190bd6cac0 con 0x56190be254b0 2024-06-03T02:21:58.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.377+0000 7f32fb204b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56190be2be90 con 0x56190be24b90 2024-06-03T02:21:58.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.377+0000 7f32fb204b80 10 client.0 _ll_register_callbacks cb 0x56190bd06380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:58.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.377+0000 7f32ed400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be24b90 0x56190be283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.377+0000 7f32ed400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be24b90 0x56190be283f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60958/0 (socket says 172.21.15.79:60958) 2024-06-03T02:21:58.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.377+0000 7f32ed400700 1 -- 172.21.15.79:0/3712584258 learned_addr learned my addr 172.21.15.79:0/3712584258 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.378+0000 7f32ed400700 1 -- 172.21.15.79:0/3712584258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be254b0 msgr2=0x56190be28930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:21:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.378+0000 7f32ed400700 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be254b0 0x56190be28930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.378+0000 7f32ed400700 1 -- 172.21.15.79:0/3712584258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 msgr2=0x56190be2b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.378+0000 7f32ed400700 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be2b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.378+0000 7f32ed400700 1 -- 172.21.15.79:0/3712584258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190bcbf8d0 con 0x56190be24b90 2024-06-03T02:21:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.380+0000 7f32ed400700 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be24b90 0x56190be283f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32d8005050 tx=0x7f32d809de60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a40771aa39538e95 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.380+0000 7f32cbe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:21:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.380+0000 7f32cbe00700 1 -- 172.21.15.79:0/3712584258 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d80bd0f0 con 0x56190be24b90 2024-06-03T02:21:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.380+0000 7f32cbe00700 1 -- 172.21.15.79:0/3712584258 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f32d80bd290 con 0x56190be24b90 2024-06-03T02:21:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.381+0000 7f32fb204b80 1 -- 172.21.15.79:0/3712584258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32dc00f920 con 0x56190be24b90 2024-06-03T02:21:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.381+0000 7f32cbe00700 1 -- 172.21.15.79:0/3712584258 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32d80b61a0 con 0x56190be24b90 2024-06-03T02:21:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.381+0000 7f32fb204b80 10 client.25516 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.381+0000 7f32fb204b80 1 -- 172.21.15.79:0/3712584258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56190be364b0 con 0x56190be24b90 2024-06-03T02:21:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.382+0000 7f32cbe00700 1 -- 172.21.15.79:0/3712584258 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32dc00f920 con 0x56190be24b90 2024-06-03T02:21:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.382+0000 7f32cbe00700 1 -- 172.21.15.79:0/3712584258 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f32d809d2b0 con 0x56190be24b90 2024-06-03T02:21:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.382+0000 7f32fb204b80 10 client.25516 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:58.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.382+0000 7f32fb204b80 10 client.25516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:58.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.382+0000 7f32fb204b80 20 client.25516 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:58.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.382+0000 7f32fb204b80 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56190be389d0 0x56190be3ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.382+0000 7f32fb204b80 4 client.25516 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:21:58.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.382+0000 7f32fb204b80 1 -- 172.21.15.79:0/3712584258 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56190bd7e000 con 0x56190be389d0 2024-06-03T02:21:58.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.383+0000 7f32ede00700 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56190be389d0 0x56190be3ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.383+0000 7f32ede00700 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56190be389d0 0x56190be3ade0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adcbb09b78435ee9 server_cookie=584e62e62345eb4f in_seq=0 out_seq=0 2024-06-03T02:21:58.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.384+0000 7f32cbe00700 10 client.25516 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:58.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.384+0000 7f32cbe00700 1 -- 172.21.15.79:0/3712584258 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56190bd7e000 con 0x56190be389d0 2024-06-03T02:21:58.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.384+0000 7f32cbe00700 10 client.25516 handle_command_reply: tid=0 2024-06-03T02:21:58.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.385+0000 7f32fb204b80 1 client.25516 shutdown 2024-06-03T02:21:58.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 -- 172.21.15.79:0/3712584258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be24b90 msgr2=0x56190be283f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:58.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be24b90 0x56190be283f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32d8005050 tx=0x7f32d809de60 comp rx=0 tx=0).stop 2024-06-03T02:21:58.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 -- 172.21.15.79:0/3712584258 shutdown_connections 2024-06-03T02:21:58.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be254b0 0x56190be28930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56190be389d0 0x56190be3ade0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be2b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 --2- 172.21.15.79:0/3712584258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be24b90 0x56190be283f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 -- 172.21.15.79:0/3712584258 >> 172.21.15.79:0/3712584258 conn(0x56190bd4b170 msgr2=0x56190be13790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:58.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32cbe00700 0 client.25516 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:58.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 -- 172.21.15.79:0/3712584258 shutdown_connections 2024-06-03T02:21:58.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 1 -- 172.21.15.79:0/3712584258 wait complete. 2024-06-03T02:21:58.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.387+0000 7f32fb204b80 20 client.25516 trim_cache size 0 max 16384 2024-06-03T02:21:58.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.428+0000 7f32fb204b80 1 Processor -- start 2024-06-03T02:21:58.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.429+0000 7f32fb204b80 1 -- start start 2024-06-03T02:21:58.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.429+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be1f240 0x56190be1ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.429+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be24b90 0x56190be28980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.429+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be14130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.429+0000 7f32fb204b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc00ebd0 con 0x56190be24b90 2024-06-03T02:21:58.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.429+0000 7f32fb204b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc010710 con 0x56190be1f240 2024-06-03T02:21:58.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.429+0000 7f32fb204b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc00f2a0 con 0x56190be254b0 2024-06-03T02:21:58.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.430+0000 7f32ede00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be1f240 0x56190be1ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.430+0000 7f32ede00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be1f240 0x56190be1ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:45892/0 (socket says 172.21.15.79:45892) 2024-06-03T02:21:58.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.430+0000 7f32ede00700 1 -- 172.21.15.79:0/853529819 learned_addr learned my addr 172.21.15.79:0/853529819 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:58.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.430+0000 7f32ee800700 1 --2- 172.21.15.79:0/853529819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be14130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.430+0000 7f32ede00700 1 -- 172.21.15.79:0/853529819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 msgr2=0x56190be14130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:58.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.430+0000 7f32ede00700 1 --2- 172.21.15.79:0/853529819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be14130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.430+0000 7f32ede00700 1 -- 172.21.15.79:0/853529819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be24b90 msgr2=0x56190be28980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:58.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.430+0000 7f32ede00700 1 --2- 172.21.15.79:0/853529819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be24b90 0x56190be28980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.430+0000 7f32ede00700 1 -- 172.21.15.79:0/853529819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190be24f70 con 0x56190be1f240 2024-06-03T02:21:58.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.433+0000 7f32ede00700 1 --2- 172.21.15.79:0/853529819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be1f240 0x56190be1ed00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007bb0 tx=0x7f32dc003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f830651c8359f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:58.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.433+0000 7f32d6a00700 1 -- 172.21.15.79:0/853529819 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc02d070 con 0x56190be1f240 2024-06-03T02:21:58.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.433+0000 7f32d6a00700 1 -- 172.21.15.79:0/853529819 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32dc0268a0 con 0x56190be1f240 2024-06-03T02:21:58.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.433+0000 7f32d6a00700 1 -- 172.21.15.79:0/853529819 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc014040 con 0x56190be1f240 2024-06-03T02:21:58.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.434+0000 7f32fb204b80 1 -- 172.21.15.79:0/853529819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be1f240 msgr2=0x56190be1ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:58.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.434+0000 7f32fb204b80 1 --2- 172.21.15.79:0/853529819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be1f240 0x56190be1ed00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007bb0 tx=0x7f32dc003c30 comp rx=0 tx=0).stop 2024-06-03T02:21:58.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.434+0000 7f32fb204b80 1 -- 172.21.15.79:0/853529819 shutdown_connections 2024-06-03T02:21:58.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.434+0000 7f32fb204b80 1 --2- 172.21.15.79:0/853529819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be14130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.434+0000 7f32fb204b80 1 --2- 172.21.15.79:0/853529819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be1f240 0x56190be1ed00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.434+0000 7f32fb204b80 1 --2- 172.21.15.79:0/853529819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be24b90 0x56190be28980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.434+0000 7f32fb204b80 1 -- 172.21.15.79:0/853529819 >> 172.21.15.79:0/853529819 conn(0x56190bd4b170 msgr2=0x56190be25890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:58.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.435+0000 7f32fb204b80 1 -- 172.21.15.79:0/853529819 shutdown_connections 2024-06-03T02:21:58.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.436+0000 7f32fb204b80 1 -- 172.21.15.79:0/853529819 wait complete. 2024-06-03T02:21:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.436+0000 7f32fb204b80 1 -- start start 2024-06-03T02:21:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.436+0000 7f32fb204b80 1 Processor -- start 2024-06-03T02:21:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.437+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be31710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.437+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be3fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.437+0000 7f32fb204b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.437+0000 7f32fb204b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc010040 con 0x56190be1f240 2024-06-03T02:21:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.437+0000 7f32fb204b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc00f970 con 0x56190be24b90 2024-06-03T02:21:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.437+0000 7f32fb204b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc00cb70 con 0x56190be254b0 2024-06-03T02:21:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.437+0000 7f32fb204b80 10 client.0 _ll_register_callbacks cb 0x56190bd06380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:21:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.438+0000 7f32ed400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be3fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.438+0000 7f32ee800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.438+0000 7f32ede00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be31710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.438+0000 7f32ed400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be3fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:45904/0 (socket says 172.21.15.79:45904) 2024-06-03T02:21:58.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.438+0000 7f32ed400700 1 -- 172.21.15.79:0/1190744393 learned_addr learned my addr 172.21.15.79:0/1190744393 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:21:58.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.438+0000 7f32ed400700 1 -- 172.21.15.79:0/1190744393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 msgr2=0x56190be402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:58.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.438+0000 7f32ee800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60966/0 (socket says 172.21.15.79:60966) 2024-06-03T02:21:58.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.439+0000 7f32ed400700 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.439+0000 7f32ed400700 1 -- 172.21.15.79:0/1190744393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 msgr2=0x56190be31710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:21:58.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.439+0000 7f32ed400700 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be31710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.439+0000 7f32ed400700 1 -- 172.21.15.79:0/1190744393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190be40820 con 0x56190be24b90 2024-06-03T02:21:58.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.439+0000 7f32ee800700 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.439+0000 7f32ede00700 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be31710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.440+0000 7f32ed400700 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be3fda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32d809a830 tx=0x7f32d809dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29d657dec7238f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.440+0000 7f32cbe00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.440+0000 7f32cbe00700 1 -- 172.21.15.79:0/1190744393 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d80a1290 con 0x56190be24b90 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.440+0000 7f32cbe00700 1 -- 172.21.15.79:0/1190744393 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f32d80a1430 con 0x56190be24b90 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.440+0000 7f32fb204b80 1 -- 172.21.15.79:0/1190744393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32dc0268a0 con 0x56190be24b90 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.441+0000 7f32cbe00700 1 -- 172.21.15.79:0/1190744393 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32d80bd0b0 con 0x56190be24b90 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.441+0000 7f32fb204b80 10 client.25538 fetch_fsmap learned FSMap version 30 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.441+0000 7f32fb204b80 1 -- 172.21.15.79:0/1190744393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56190bea3420 con 0x56190be24b90 2024-06-03T02:21:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.441+0000 7f32cbe00700 1 -- 172.21.15.79:0/1190744393 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32dc0268a0 con 0x56190be24b90 2024-06-03T02:21:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.441+0000 7f32cbe00700 1 -- 172.21.15.79:0/1190744393 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f32d800fbb0 con 0x56190be24b90 2024-06-03T02:21:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.441+0000 7f32fb204b80 10 client.25538 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:21:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.441+0000 7f32fb204b80 10 client.25538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:21:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.441+0000 7f32fb204b80 20 client.25538 populate_metadata read hostname 'smithi079' 2024-06-03T02:21:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.441+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56190bea3950 0x56190bea5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:21:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.442+0000 7f32fb204b80 4 client.25538 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:21:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.442+0000 7f32fb204b80 1 -- 172.21.15.79:0/1190744393 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56190bd78580 con 0x56190bea3950 2024-06-03T02:21:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.442+0000 7f32ede00700 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56190bea3950 0x56190bea5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:21:58.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32ede00700 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56190bea3950 0x56190bea5d70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ab3a7f8a709086 server_cookie=abd30e8eab77de21 in_seq=0 out_seq=0 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32cbe00700 10 client.25538 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32cbe00700 1 -- 172.21.15.79:0/1190744393 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56190bd78580 con 0x56190bea3950 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32cbe00700 10 client.25538 handle_command_reply: tid=0 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32fb204b80 1 client.25538 shutdown 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32fb204b80 1 -- 172.21.15.79:0/1190744393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 msgr2=0x56190be3fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be3fda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32d809a830 tx=0x7f32d809dd50 comp rx=0 tx=0).stop 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32fb204b80 1 -- 172.21.15.79:0/1190744393 shutdown_connections 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190be254b0 0x56190be402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56190bea3950 0x56190bea5d70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56190be1f240 0x56190be31710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32fb204b80 1 --2- 172.21.15.79:0/1190744393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190be24b90 0x56190be3fda0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.443+0000 7f32fb204b80 1 -- 172.21.15.79:0/1190744393 >> 172.21.15.79:0/1190744393 conn(0x56190bd4b170 msgr2=0x56190be29c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:58.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.444+0000 7f32cbe00700 0 client.25538 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:21:58.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.444+0000 7f32fb204b80 1 -- 172.21.15.79:0/1190744393 shutdown_connections 2024-06-03T02:21:58.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.444+0000 7f32fb204b80 1 -- 172.21.15.79:0/1190744393 wait complete. 2024-06-03T02:21:58.446 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:21:58.446 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:21:58.446 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:21:58.446 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.444+0000 7f32fb204b80 20 client.25538 trim_cache size 0 max 16384 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 -- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f32d00a2ca0 msgr2=0x7f32d00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f32d00a2ca0 0x7f32d00a5160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f32e4006fd0 tx=0x7f32e400a500 comp rx=0 tx=0).stop 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 -- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32e8104ed0 msgr2=0x7f32e810fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32e8104ed0 0x7f32e810fa10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32d80057f0 tx=0x7f32d8003680 comp rx=0 tx=0).stop 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 -- 172.21.15.79:0/3559930715 shutdown_connections 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e81073c0 0x7f32e810ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32e806a6e0 0x7f32e81159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f32d00a2ca0 0x7f32d00a5160 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 --2- 172.21.15.79:0/3559930715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32e8104ed0 0x7f32e810fa10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.445+0000 7f32f0800700 1 -- 172.21.15.79:0/3559930715 >> 172.21.15.79:0/3559930715 conn(0x7f32e8078340 msgr2=0x7f32e8103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.446+0000 7f32f0800700 1 -- 172.21.15.79:0/3559930715 shutdown_connections 2024-06-03T02:21:58.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:21:58.446+0000 7f32f0800700 1 -- 172.21.15.79:0/3559930715 wait complete. 2024-06-03T02:21:59.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:21:59 smithi079 ceph-mon[31819]: pgmap v574: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 688 op/s 2024-06-03T02:21:59.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:21:59 smithi135 ceph-mon[36653]: pgmap v574: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 688 op/s 2024-06-03T02:21:59.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:21:59 smithi123 ceph-mon[36640]: pgmap v574: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 688 op/s 2024-06-03T02:22:01.624 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:22:01.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:22:01.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:01 smithi079 ceph-mon[31819]: pgmap v575: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.1 MiB/s wr, 597 op/s 2024-06-03T02:22:01.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:01 smithi135 ceph-mon[36653]: pgmap v575: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.1 MiB/s wr, 597 op/s 2024-06-03T02:22:01.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:01 smithi123 ceph-mon[36640]: pgmap v575: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.1 MiB/s wr, 597 op/s 2024-06-03T02:22:02.318 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 115.0 s, 1366.0 tps, lat 72.723 ms stddev 101.253 2024-06-03T02:22:03.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:03 smithi079 ceph-mon[31819]: pgmap v576: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 745 op/s 2024-06-03T02:22:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:03 smithi135 ceph-mon[36653]: pgmap v576: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 745 op/s 2024-06-03T02:22:03.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:03 smithi123 ceph-mon[36640]: pgmap v576: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 745 op/s 2024-06-03T02:22:05.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:05 smithi079 ceph-mon[31819]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 575 op/s 2024-06-03T02:22:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.795+0000 7f27c2800700 1 -- 172.21.15.79:0/1391334245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27bc06ab30 msgr2=0x7f27bc10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.795+0000 7f27c2800700 1 --2- 172.21.15.79:0/1391334245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27bc06ab30 0x7f27bc10f320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f27b400b0a0 tx=0x7f27b400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:22:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.795+0000 7f27c2800700 1 -- 172.21.15.79:0/1391334245 shutdown_connections 2024-06-03T02:22:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.795+0000 7f27c2800700 1 --2- 172.21.15.79:0/1391334245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27bc06ab30 0x7f27bc10f320 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.795+0000 7f27c2800700 1 --2- 172.21.15.79:0/1391334245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27bc108500 0x7f27bc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.795+0000 7f27c2800700 1 --2- 172.21.15.79:0/1391334245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc06a170 0x7f27bc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.795+0000 7f27c2800700 1 -- 172.21.15.79:0/1391334245 >> 172.21.15.79:0/1391334245 conn(0x7f27bc078340 msgr2=0x7f27bc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.795+0000 7f27c2800700 1 -- 172.21.15.79:0/1391334245 shutdown_connections 2024-06-03T02:22:05.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.796+0000 7f27c2800700 1 -- 172.21.15.79:0/1391334245 wait complete. 2024-06-03T02:22:05.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.797+0000 7f27c2800700 1 Processor -- start 2024-06-03T02:22:05.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.799+0000 7f27c2800700 1 -- start start 2024-06-03T02:22:05.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.799+0000 7f27c2800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc06a170 0x7f27bc19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:05.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.799+0000 7f27c2800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27bc06ab30 0x7f27bc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:05.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.799+0000 7f27c2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27bc108500 0x7f27bc19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:05.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.799+0000 7f27c2800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc112830 con 0x7f27bc06ab30 2024-06-03T02:22:05.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.799+0000 7f27c2800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1126b0 con 0x7f27bc06a170 2024-06-03T02:22:05.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.799+0000 7f27c2800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1129b0 con 0x7f27bc108500 2024-06-03T02:22:05.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.800+0000 7f27bb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc06a170 0x7f27bc19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:05.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.800+0000 7f27bb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc06a170 0x7f27bc19aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53180/0 (socket says 172.21.15.79:53180) 2024-06-03T02:22:05.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.800+0000 7f27bb400700 1 -- 172.21.15.79:0/3927732122 learned_addr learned my addr 172.21.15.79:0/3927732122 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:05.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.801+0000 7f27bbe00700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27bc108500 0x7f27bc19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:05.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.801+0000 7f27baa00700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27bc06ab30 0x7f27bc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:05.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.801+0000 7f27bb400700 1 -- 172.21.15.79:0/3927732122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27bc108500 msgr2=0x7f27bc19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:05.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.801+0000 7f27bb400700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27bc108500 0x7f27bc19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:05.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.801+0000 7f27bb400700 1 -- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27bc06ab30 msgr2=0x7f27bc19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:05.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.801+0000 7f27bb400700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27bc06ab30 0x7f27bc19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:05.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.802+0000 7f27bb400700 1 -- 172.21.15.79:0/3927732122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc19ff00 con 0x7f27bc06a170 2024-06-03T02:22:05.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.802+0000 7f27bb400700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc06a170 0x7f27bc19aef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00b530 tx=0x7f27ac00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:05.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.802+0000 7f27b3e00700 1 -- 172.21.15.79:0/3927732122 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac015020 con 0x7f27bc06a170 2024-06-03T02:22:05.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.802+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc1a01f0 con 0x7f27bc06a170 2024-06-03T02:22:05.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.802+0000 7f27b3e00700 1 -- 172.21.15.79:0/3927732122 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac011d10 con 0x7f27bc06a170 2024-06-03T02:22:05.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.803+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc06beb0 con 0x7f27bc06a170 2024-06-03T02:22:05.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.803+0000 7f27b3e00700 1 -- 172.21.15.79:0/3927732122 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00f2e0 con 0x7f27bc06a170 2024-06-03T02:22:05.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.807+0000 7f27b3e00700 1 -- 172.21.15.79:0/3927732122 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f27ac018020 con 0x7f27bc06a170 2024-06-03T02:22:05.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.808+0000 7f27b3e00700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27a00a2e20 0x7f27a00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:05.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.808+0000 7f27b3e00700 1 -- 172.21.15.79:0/3927732122 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f27ac010d10 con 0x7f27bc06a170 2024-06-03T02:22:05.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.809+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc103bc0 con 0x7f27bc06a170 2024-06-03T02:22:05.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.810+0000 7f27baa00700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27a00a2e20 0x7f27a00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:05.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.814+0000 7f27baa00700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27a00a2e20 0x7f27a00a52e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f27bc103330 tx=0x7f27a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:05.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:05.816+0000 7f27b3e00700 1 -- 172.21.15.79:0/3927732122 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f27ac01f020 con 0x7f27bc06a170 2024-06-03T02:22:05.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:05 smithi135 ceph-mon[36653]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 575 op/s 2024-06-03T02:22:05.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:05 smithi123 ceph-mon[36640]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 575 op/s 2024-06-03T02:22:06.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.269+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27bc19c250 con 0x7f27bc06a170 2024-06-03T02:22:06.270 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:22:06.270 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:22:06.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.270+0000 7f27b3e00700 1 -- 172.21.15.79:0/3927732122 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f27ac09b600 con 0x7f27bc06a170 2024-06-03T02:22:06.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27a00a2e20 msgr2=0x7f27a00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27a00a2e20 0x7f27a00a52e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f27bc103330 tx=0x7f27a8009260 comp rx=0 tx=0).stop 2024-06-03T02:22:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc06a170 msgr2=0x7f27bc19aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc06a170 0x7f27bc19aef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00b530 tx=0x7f27ac00b840 comp rx=0 tx=0).stop 2024-06-03T02:22:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 shutdown_connections 2024-06-03T02:22:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27bc108500 0x7f27bc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc06a170 0x7f27bc19aef0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27a00a2e20 0x7f27a00a52e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 --2- 172.21.15.79:0/3927732122 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27bc06ab30 0x7f27bc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:06.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.273+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 >> 172.21.15.79:0/3927732122 conn(0x7f27bc078340 msgr2=0x7f27bc10d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:06.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.274+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 shutdown_connections 2024-06-03T02:22:06.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:06.275+0000 7f27c2800700 1 -- 172.21.15.79:0/3927732122 wait complete. 2024-06-03T02:22:06.275 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:22:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:06 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3927732122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:22:06.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:06 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3927732122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:22:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:06 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3927732122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:22:07.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:07 smithi079 ceph-mon[31819]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.2 MiB/s wr, 675 op/s 2024-06-03T02:22:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:07 smithi135 ceph-mon[36653]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.2 MiB/s wr, 675 op/s 2024-06-03T02:22:07.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:07 smithi123 ceph-mon[36640]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.2 MiB/s wr, 675 op/s 2024-06-03T02:22:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 120.0 s, 1193.4 tps, lat 84.069 ms stddev 151.831 2024-06-03T02:22:08.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:22:09.421 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:09.817 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:09 smithi079 ceph-mon[31819]: pgmap v579: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-06-03T02:22:09.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:09 smithi135 ceph-mon[36653]: pgmap v579: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-06-03T02:22:09.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:09 smithi123 ceph-mon[36640]: pgmap v579: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-06-03T02:22:11.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:11 smithi079 ceph-mon[31819]: pgmap v580: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 916 op/s 2024-06-03T02:22:11.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:11 smithi135 ceph-mon[36653]: pgmap v580: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 916 op/s 2024-06-03T02:22:11.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.952+0000 7f4875000700 1 -- 172.21.15.79:0/2723976866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4870109a60 msgr2=0x7f4870109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:11.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.953+0000 7f4875000700 1 --2- 172.21.15.79:0/2723976866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4870109a60 0x7f4870109ee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f485c009990 tx=0x7f485c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.953+0000 7f4875000700 1 -- 172.21.15.79:0/2723976866 shutdown_connections 2024-06-03T02:22:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.953+0000 7f4875000700 1 --2- 172.21.15.79:0/2723976866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48701175c0 0x7f48701199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.953+0000 7f4875000700 1 --2- 172.21.15.79:0/2723976866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f487010cc10 0x7f4870109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.954+0000 7f4875000700 1 --2- 172.21.15.79:0/2723976866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4870109a60 0x7f4870109ee0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.954+0000 7f4875000700 1 -- 172.21.15.79:0/2723976866 >> 172.21.15.79:0/2723976866 conn(0x7f487006d600 msgr2=0x7f487006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:11.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.956+0000 7f4875000700 1 -- 172.21.15.79:0/2723976866 shutdown_connections 2024-06-03T02:22:11.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.956+0000 7f4875000700 1 -- 172.21.15.79:0/2723976866 wait complete. 2024-06-03T02:22:11.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.958+0000 7f4875000700 1 Processor -- start 2024-06-03T02:22:11.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.958+0000 7f4875000700 1 -- start start 2024-06-03T02:22:11.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.959+0000 7f4875000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4870109a60 0x7f48701a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.959+0000 7f4875000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487010cc10 0x7f48701a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.959+0000 7f4875000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48701175c0 0x7f48701a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.959+0000 7f4875000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f487011d2b0 con 0x7f48701175c0 2024-06-03T02:22:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.959+0000 7f4875000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f487011d130 con 0x7f4870109a60 2024-06-03T02:22:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.959+0000 7f4875000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f487011d430 con 0x7f487010cc10 2024-06-03T02:22:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.960+0000 7f486d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487010cc10 0x7f48701a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.960+0000 7f486e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48701175c0 0x7f48701a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.960+0000 7f486d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487010cc10 0x7f48701a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:38750/0 (socket says 172.21.15.79:38750) 2024-06-03T02:22:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.960+0000 7f486d400700 1 -- 172.21.15.79:0/470150499 learned_addr learned my addr 172.21.15.79:0/470150499 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:11.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.960+0000 7f486de00700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4870109a60 0x7f48701a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.962+0000 7f486d400700 1 -- 172.21.15.79:0/470150499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4870109a60 msgr2=0x7f48701a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:11.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.962+0000 7f486d400700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4870109a60 0x7f48701a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.962+0000 7f486d400700 1 -- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48701175c0 msgr2=0x7f48701a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:11.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.962+0000 7f486d400700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48701175c0 0x7f48701a7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.962+0000 7f486d400700 1 -- 172.21.15.79:0/470150499 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48701a86c0 con 0x7f487010cc10 2024-06-03T02:22:11.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.962+0000 7f486de00700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4870109a60 0x7f48701a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:22:11.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.962+0000 7f486e800700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48701175c0 0x7f48701a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:22:11.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.962+0000 7f486d400700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487010cc10 0x7f48701a3c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f485c0092e0 tx=0x7f485c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:11.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.963+0000 7f4866a00700 1 -- 172.21.15.79:0/470150499 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c003c10 con 0x7f487010cc10 2024-06-03T02:22:11.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.963+0000 7f4866a00700 1 -- 172.21.15.79:0/470150499 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485c004830 con 0x7f487010cc10 2024-06-03T02:22:11.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.963+0000 7f4866a00700 1 -- 172.21.15.79:0/470150499 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c005420 con 0x7f487010cc10 2024-06-03T02:22:11.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.964+0000 7f4875000700 1 -- 172.21.15.79:0/470150499 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48701a8950 con 0x7f487010cc10 2024-06-03T02:22:11.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.964+0000 7f4875000700 1 -- 172.21.15.79:0/470150499 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48701a8d80 con 0x7f487010cc10 2024-06-03T02:22:11.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.970+0000 7f4866a00700 1 -- 172.21.15.79:0/470150499 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f485c022470 con 0x7f487010cc10 2024-06-03T02:22:11.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.972+0000 7f4866a00700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48540a2ca0 0x7f48540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.972+0000 7f4866a00700 1 -- 172.21.15.79:0/470150499 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f485c0d0390 con 0x7f487010cc10 2024-06-03T02:22:11.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:11 smithi123 ceph-mon[36640]: pgmap v580: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 916 op/s 2024-06-03T02:22:11.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.972+0000 7f487fa01b80 1 Processor -- start 2024-06-03T02:22:11.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.973+0000 7f487fa01b80 1 -- start start 2024-06-03T02:22:11.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.974+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 0x56072da29f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.974+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da23d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.974+0000 7f486de00700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48540a2ca0 0x7f48540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.974+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.975+0000 7f486e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.975+0000 7f486e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da24680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:38754/0 (socket says 172.21.15.79:38754) 2024-06-03T02:22:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.975+0000 7f487fa01b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56072d932a10 con 0x56072da2a4b0 2024-06-03T02:22:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.975+0000 7f487fa01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56072d975b00 con 0x56072da29b90 2024-06-03T02:22:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.975+0000 7f487fa01b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56072d9308f0 con 0x56072da24240 2024-06-03T02:22:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.975+0000 7f486e800700 1 -- 172.21.15.79:0/1660847355 learned_addr learned my addr 172.21.15.79:0/1660847355 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.975+0000 7f486d400700 1 --2- 172.21.15.79:0/1660847355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da23d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.976+0000 7f486de00700 1 --2- 172.21.15.79:0/1660847355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 0x56072da29f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.976+0000 7f486d400700 1 -- 172.21.15.79:0/1660847355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 msgr2=0x56072da24680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.976+0000 7f486d400700 1 --2- 172.21.15.79:0/1660847355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da24680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.976+0000 7f486d400700 1 -- 172.21.15.79:0/1660847355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 msgr2=0x56072da29f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.976+0000 7f486d400700 1 --2- 172.21.15.79:0/1660847355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 0x56072da29f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.976+0000 7f486d400700 1 -- 172.21.15.79:0/1660847355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56072d91df90 con 0x56072da2a4b0 2024-06-03T02:22:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.976+0000 7f486d400700 1 --2- 172.21.15.79:0/1660847355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da23d00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f485c003730 tx=0x7f485c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c76be509db7ec45 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.976+0000 7f486de00700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48540a2ca0 0x7f48540a5160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4860005eb0 tx=0x7f486000a640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:11.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.977+0000 7f485b400700 1 -- 172.21.15.79:0/1660847355 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0a54b0 con 0x56072da2a4b0 2024-06-03T02:22:11.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.977+0000 7f485b400700 1 -- 172.21.15.79:0/1660847355 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485c086e50 con 0x56072da2a4b0 2024-06-03T02:22:11.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.977+0000 7f485b400700 1 -- 172.21.15.79:0/1660847355 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0a31e0 con 0x56072da2a4b0 2024-06-03T02:22:11.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.978+0000 7f487fa01b80 1 -- 172.21.15.79:0/1660847355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 msgr2=0x56072da23d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:11.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.978+0000 7f487fa01b80 1 --2- 172.21.15.79:0/1660847355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da23d00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f485c003730 tx=0x7f485c0038a0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.978+0000 7f487fa01b80 1 -- 172.21.15.79:0/1660847355 shutdown_connections 2024-06-03T02:22:11.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.978+0000 7f487fa01b80 1 --2- 172.21.15.79:0/1660847355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da24680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.978+0000 7f487fa01b80 1 --2- 172.21.15.79:0/1660847355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 0x56072da29f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.978+0000 7f487fa01b80 1 --2- 172.21.15.79:0/1660847355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da23d00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.978+0000 7f487fa01b80 1 -- 172.21.15.79:0/1660847355 >> 172.21.15.79:0/1660847355 conn(0x56072d950170 msgr2=0x56072da1eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:11.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.979+0000 7f487fa01b80 1 -- 172.21.15.79:0/1660847355 shutdown_connections 2024-06-03T02:22:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.979+0000 7f487fa01b80 1 -- 172.21.15.79:0/1660847355 wait complete. 2024-06-03T02:22:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.980+0000 7f487fa01b80 1 -- start start 2024-06-03T02:22:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.980+0000 7f487fa01b80 1 Processor -- start 2024-06-03T02:22:11.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.983+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da24240 0x56072da30160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.983+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da2d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.983+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da2a4b0 0x56072da2da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.984+0000 7f486e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da2a4b0 0x56072da2da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.984+0000 7f486de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da24240 0x56072da30160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.984+0000 7f486e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da2a4b0 0x56072da2da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:38762/0 (socket says 172.21.15.79:38762) 2024-06-03T02:22:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.984+0000 7f486e800700 1 -- 172.21.15.79:0/602010718 learned_addr learned my addr 172.21.15.79:0/602010718 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.984+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56072d911f60 con 0x56072da29b90 2024-06-03T02:22:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.984+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56072d971ac0 con 0x56072da24240 2024-06-03T02:22:11.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.984+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56072da30c40 con 0x56072da2a4b0 2024-06-03T02:22:11.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.984+0000 7f486d400700 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da2d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.984+0000 7f487fa01b80 10 client.0 _ll_register_callbacks cb 0x56072d90b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:11.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.985+0000 7f486e800700 1 -- 172.21.15.79:0/602010718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da24240 msgr2=0x56072da30160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:11.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.985+0000 7f486e800700 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da24240 0x56072da30160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.985+0000 7f486e800700 1 -- 172.21.15.79:0/602010718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 msgr2=0x56072da2d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:11.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.985+0000 7f486e800700 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da2d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.985+0000 7f486e800700 1 -- 172.21.15.79:0/602010718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56072d8c48d0 con 0x56072da2a4b0 2024-06-03T02:22:11.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.985+0000 7f486d400700 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da2d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:11.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.985+0000 7f486e800700 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da2a4b0 0x56072da2da40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f486800ce20 tx=0x7f486800ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a66cca7fc97922 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:11.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.985+0000 7f484fe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:22:11.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.986+0000 7f484fe00700 1 -- 172.21.15.79:0/602010718 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486801c060 con 0x56072da2a4b0 2024-06-03T02:22:11.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.986+0000 7f484fe00700 1 -- 172.21.15.79:0/602010718 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4868007820 con 0x56072da2a4b0 2024-06-03T02:22:11.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.986+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f485c086e50 con 0x56072da2a4b0 2024-06-03T02:22:11.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.986+0000 7f484fe00700 1 -- 172.21.15.79:0/602010718 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4868023050 con 0x56072da2a4b0 2024-06-03T02:22:11.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.986+0000 7f487fa01b80 10 client.25537 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:11.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.986+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56072da43b10 con 0x56072da2a4b0 2024-06-03T02:22:11.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.987+0000 7f484fe00700 1 -- 172.21.15.79:0/602010718 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485c086e50 con 0x56072da2a4b0 2024-06-03T02:22:11.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.987+0000 7f484fe00700 1 -- 172.21.15.79:0/602010718 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f486800aa00 con 0x56072da2a4b0 2024-06-03T02:22:11.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.987+0000 7f487fa01b80 10 client.25537 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:11.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.987+0000 7f487fa01b80 10 client.25537 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:22:11.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.987+0000 7f487fa01b80 20 client.25537 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:11.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.987+0000 7f487fa01b80 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x56072da46150 0x56072da48540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:11.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.987+0000 7f487fa01b80 4 client.25537 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:22:11.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.987+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56072d983000 con 0x56072da46150 2024-06-03T02:22:11.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.988+0000 7f486de00700 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x56072da46150 0x56072da48540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:11.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.988+0000 7f484fe00700 10 client.25537 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:22:11.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.989+0000 7f486de00700 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x56072da46150 0x56072da48540 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2ae6377f34f2b703 server_cookie=4a1be63ab2e1cf06 in_seq=0 out_seq=0 2024-06-03T02:22:11.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.991+0000 7f484fe00700 1 -- 172.21.15.79:0/602010718 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56072d983000 con 0x56072da46150 2024-06-03T02:22:11.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.991+0000 7f484fe00700 10 client.25537 handle_command_reply: tid=0 2024-06-03T02:22:11.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.992+0000 7f487fa01b80 1 client.25537 shutdown 2024-06-03T02:22:11.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.993+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da2a4b0 msgr2=0x56072da2da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:11.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.993+0000 7f487fa01b80 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da2a4b0 0x56072da2da40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f486800ce20 tx=0x7f486800ce50 comp rx=0 tx=0).stop 2024-06-03T02:22:11.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.993+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 shutdown_connections 2024-06-03T02:22:11.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.993+0000 7f487fa01b80 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x56072da46150 0x56072da48540 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.993+0000 7f487fa01b80 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da2a4b0 0x56072da2da40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.993+0000 7f487fa01b80 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da24240 0x56072da30160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.993+0000 7f487fa01b80 1 --2- 172.21.15.79:0/602010718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da2d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:11.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.993+0000 7f484fe00700 0 client.25537 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:22:11.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.993+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 >> 172.21.15.79:0/602010718 conn(0x56072d950170 msgr2=0x56072da18790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:11.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.995+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 shutdown_connections 2024-06-03T02:22:11.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.995+0000 7f487fa01b80 1 -- 172.21.15.79:0/602010718 wait complete. 2024-06-03T02:22:11.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:11.995+0000 7f487fa01b80 20 client.25537 trim_cache size 0 max 16384 2024-06-03T02:22:12.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.020+0000 7f487fa01b80 1 Processor -- start 2024-06-03T02:22:12.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.020+0000 7f487fa01b80 1 -- start start 2024-06-03T02:22:12.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.020+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da23d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.020+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 0x56072da1f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.020+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da19130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.020+0000 7f487fa01b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f48701b4960 con 0x56072da2a4b0 2024-06-03T02:22:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.020+0000 7f487fa01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48701b64a0 con 0x56072da29b90 2024-06-03T02:22:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.020+0000 7f487fa01b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48701b5030 con 0x56072da24240 2024-06-03T02:22:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da19130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da19130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:59432/0 (socket says 172.21.15.79:59432) 2024-06-03T02:22:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486e800700 1 -- 172.21.15.79:0/60221131 learned_addr learned my addr 172.21.15.79:0/60221131 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486d400700 1 --2- 172.21.15.79:0/60221131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 0x56072da1f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486de00700 1 --2- 172.21.15.79:0/60221131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da23d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486e800700 1 -- 172.21.15.79:0/60221131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 msgr2=0x56072da23d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486e800700 1 --2- 172.21.15.79:0/60221131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da23d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486e800700 1 -- 172.21.15.79:0/60221131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 msgr2=0x56072da1f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486e800700 1 --2- 172.21.15.79:0/60221131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 0x56072da1f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.021+0000 7f486e800700 1 -- 172.21.15.79:0/60221131 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56072da2ba70 con 0x56072da2a4b0 2024-06-03T02:22:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.022+0000 7f486e800700 1 --2- 172.21.15.79:0/60221131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da19130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4868009440 tx=0x7f48680093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b4f3bbdd091a58 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.023+0000 7f485aa00700 1 -- 172.21.15.79:0/60221131 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48680169f0 con 0x56072da2a4b0 2024-06-03T02:22:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.023+0000 7f485aa00700 1 -- 172.21.15.79:0/60221131 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4868009960 con 0x56072da2a4b0 2024-06-03T02:22:12.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.023+0000 7f485aa00700 1 -- 172.21.15.79:0/60221131 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486800ab50 con 0x56072da2a4b0 2024-06-03T02:22:12.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.025+0000 7f487fa01b80 1 -- 172.21.15.79:0/60221131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 msgr2=0x56072da19130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:12.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.025+0000 7f487fa01b80 1 --2- 172.21.15.79:0/60221131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da19130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4868009440 tx=0x7f48680093d0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.025+0000 7f487fa01b80 1 -- 172.21.15.79:0/60221131 shutdown_connections 2024-06-03T02:22:12.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.026+0000 7f487fa01b80 1 --2- 172.21.15.79:0/60221131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da2a4b0 0x56072da19130 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.026+0000 7f487fa01b80 1 --2- 172.21.15.79:0/60221131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da23d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.026+0000 7f487fa01b80 1 --2- 172.21.15.79:0/60221131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da29b90 0x56072da1f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.026+0000 7f487fa01b80 1 -- 172.21.15.79:0/60221131 >> 172.21.15.79:0/60221131 conn(0x56072d950170 msgr2=0x56072da1ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.027+0000 7f487fa01b80 1 -- 172.21.15.79:0/60221131 shutdown_connections 2024-06-03T02:22:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.027+0000 7f487fa01b80 1 -- 172.21.15.79:0/60221131 wait complete. 2024-06-03T02:22:12.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.028+0000 7f487fa01b80 1 -- start start 2024-06-03T02:22:12.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.029+0000 7f487fa01b80 1 Processor -- start 2024-06-03T02:22:12.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da3e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:12.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:12.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f487fa01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da2a4b0 0x56072da4d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:12.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f487fa01b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f48701b5dd0 con 0x56072da29b90 2024-06-03T02:22:12.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f487fa01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48701b5700 con 0x56072da2a4b0 2024-06-03T02:22:12.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f487fa01b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48701b2900 con 0x56072da24240 2024-06-03T02:22:12.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f487fa01b80 10 client.0 _ll_register_callbacks cb 0x56072d90b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:12.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f486d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:12.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f486d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da3ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:59438/0 (socket says 172.21.15.79:59438) 2024-06-03T02:22:12.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f486d400700 1 -- 172.21.15.79:0/3298859342 learned_addr learned my addr 172.21.15.79:0/3298859342 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:12.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.031+0000 7f486e800700 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da2a4b0 0x56072da4d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:12.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.030+0000 7f486de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da3e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:12.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.032+0000 7f486e800700 1 -- 172.21.15.79:0/3298859342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 msgr2=0x56072da3e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:12.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.032+0000 7f486e800700 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da3e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.032+0000 7f486e800700 1 -- 172.21.15.79:0/3298859342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 msgr2=0x56072da3ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:12.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.032+0000 7f486e800700 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da3ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.032+0000 7f486e800700 1 -- 172.21.15.79:0/3298859342 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56072da4dc60 con 0x56072da2a4b0 2024-06-03T02:22:12.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.033+0000 7f486de00700 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da3e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:22:12.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.033+0000 7f486e800700 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da2a4b0 0x56072da4d420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4868009530 tx=0x7f486800a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba4f692f7dca8287 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:12.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.033+0000 7f484fe00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:22:12.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.034+0000 7f484fe00700 1 -- 172.21.15.79:0/3298859342 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868033070 con 0x56072da2a4b0 2024-06-03T02:22:12.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.034+0000 7f484fe00700 1 -- 172.21.15.79:0/3298859342 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f48680167a0 con 0x56072da2a4b0 2024-06-03T02:22:12.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.034+0000 7f487fa01b80 1 -- 172.21.15.79:0/3298859342 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56072d97d580 con 0x56072da2a4b0 2024-06-03T02:22:12.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.035+0000 7f484fe00700 1 -- 172.21.15.79:0/3298859342 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f486800abe0 con 0x56072da2a4b0 2024-06-03T02:22:12.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.035+0000 7f487fa01b80 10 client.25559 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:12.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.035+0000 7f487fa01b80 1 -- 172.21.15.79:0/3298859342 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56072da31a90 con 0x56072da2a4b0 2024-06-03T02:22:12.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.035+0000 7f484fe00700 1 -- 172.21.15.79:0/3298859342 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56072d97d580 con 0x56072da2a4b0 2024-06-03T02:22:12.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.035+0000 7f484fe00700 1 -- 172.21.15.79:0/3298859342 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4868007a10 con 0x56072da2a4b0 2024-06-03T02:22:12.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.036+0000 7f487fa01b80 10 client.25559 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:12.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.036+0000 7f487fa01b80 10 client.25559 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:22:12.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.036+0000 7f487fa01b80 20 client.25559 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:12.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.036+0000 7f487fa01b80 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x56072da320f0 0x56072da34510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:12.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.036+0000 7f487fa01b80 4 client.25559 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:22:12.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.036+0000 7f487fa01b80 1 -- 172.21.15.79:0/3298859342 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56072d7f7250 con 0x56072da320f0 2024-06-03T02:22:12.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.036+0000 7f486de00700 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x56072da320f0 0x56072da34510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:12.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.037+0000 7f484fe00700 10 client.25559 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:22:12.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.037+0000 7f486de00700 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x56072da320f0 0x56072da34510 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=90efa61987477015 server_cookie=8d9593e7f7eed5d6 in_seq=0 out_seq=0 2024-06-03T02:22:12.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f484fe00700 1 -- 172.21.15.79:0/3298859342 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56072d7f7250 con 0x56072da320f0 2024-06-03T02:22:12.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f484fe00700 10 client.25559 handle_command_reply: tid=0 2024-06-03T02:22:12.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f487fa01b80 1 client.25559 shutdown 2024-06-03T02:22:12.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f487fa01b80 1 -- 172.21.15.79:0/3298859342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da2a4b0 msgr2=0x56072da4d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:12.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f487fa01b80 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da2a4b0 0x56072da4d420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4868009530 tx=0x7f486800a630 comp rx=0 tx=0).stop 2024-06-03T02:22:12.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f487fa01b80 1 -- 172.21.15.79:0/3298859342 shutdown_connections 2024-06-03T02:22:12.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f487fa01b80 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x56072da320f0 0x56072da34510 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f487fa01b80 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56072da2a4b0 0x56072da4d420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f487fa01b80 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56072da24240 0x56072da3e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f487fa01b80 1 --2- 172.21.15.79:0/3298859342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56072da29b90 0x56072da3ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f487fa01b80 1 -- 172.21.15.79:0/3298859342 >> 172.21.15.79:0/3298859342 conn(0x56072d950170 msgr2=0x56072da2a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:12.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.038+0000 7f484fe00700 0 client.25559 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:22:12.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.039+0000 7f487fa01b80 1 -- 172.21.15.79:0/3298859342 shutdown_connections 2024-06-03T02:22:12.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.039+0000 7f487fa01b80 1 -- 172.21.15.79:0/3298859342 wait complete. 2024-06-03T02:22:12.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.039+0000 7f487fa01b80 20 client.25559 trim_cache size 0 max 16384 2024-06-03T02:22:12.040 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 -- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48540a2ca0 msgr2=0x7f48540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48540a2ca0 0x7f48540a5160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4860005eb0 tx=0x7f486000a640 comp rx=0 tx=0).stop 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 -- 172.21.15.79:0/470150499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487010cc10 msgr2=0x7f48701a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487010cc10 0x7f48701a3c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f485c0092e0 tx=0x7f485c0047b0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 -- 172.21.15.79:0/470150499 shutdown_connections 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48540a2ca0 0x7f48540a5160 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48701175c0 0x7f48701a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4870109a60 0x7f48701a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 --2- 172.21.15.79:0/470150499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487010cc10 0x7f48701a3c40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:12.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.040+0000 7f4875000700 1 -- 172.21.15.79:0/470150499 >> 172.21.15.79:0/470150499 conn(0x7f487006d600 msgr2=0x7f4870118c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:12.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.041+0000 7f4875000700 1 -- 172.21.15.79:0/470150499 shutdown_connections 2024-06-03T02:22:12.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:12.041+0000 7f4875000700 1 -- 172.21.15.79:0/470150499 wait complete. 2024-06-03T02:22:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 125.0 s, 1833.8 tps, lat 54.022 ms stddev 65.987 2024-06-03T02:22:13.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:13 smithi079 ceph-mon[31819]: pgmap v581: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-06-03T02:22:13.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:13 smithi135 ceph-mon[36653]: pgmap v581: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-06-03T02:22:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:13 smithi123 ceph-mon[36640]: pgmap v581: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-06-03T02:22:14.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:22:15.402 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:15.792 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:15 smithi079 ceph-mon[31819]: pgmap v582: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 957 op/s 2024-06-03T02:22:15.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:15 smithi135 ceph-mon[36653]: pgmap v582: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 957 op/s 2024-06-03T02:22:15.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:15 smithi123 ceph-mon[36640]: pgmap v582: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 957 op/s 2024-06-03T02:22:17.791 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:17 smithi079 ceph-mon[31819]: pgmap v583: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 928 op/s 2024-06-03T02:22:17.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:17 smithi135 ceph-mon[36653]: pgmap v583: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 928 op/s 2024-06-03T02:22:17.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.903+0000 7f5228c00700 1 -- 172.21.15.79:0/3516697402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 msgr2=0x7f522406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:17.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.903+0000 7f5228c00700 1 --2- 172.21.15.79:0/3516697402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 0x7f522406a5f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5210009990 tx=0x7f5210009ca0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.903+0000 7f5228c00700 1 -- 172.21.15.79:0/3516697402 shutdown_connections 2024-06-03T02:22:17.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.903+0000 7f5228c00700 1 --2- 172.21.15.79:0/3516697402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f522406ab30 0x7f522410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.903+0000 7f5228c00700 1 --2- 172.21.15.79:0/3516697402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5224108500 0x7f52241088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.903+0000 7f5228c00700 1 --2- 172.21.15.79:0/3516697402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 0x7f522406a5f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.903+0000 7f5228c00700 1 -- 172.21.15.79:0/3516697402 >> 172.21.15.79:0/3516697402 conn(0x7f5224078340 msgr2=0x7f5224078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:17.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.903+0000 7f5228c00700 1 -- 172.21.15.79:0/3516697402 shutdown_connections 2024-06-03T02:22:17.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.903+0000 7f5228c00700 1 -- 172.21.15.79:0/3516697402 wait complete. 2024-06-03T02:22:17.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.904+0000 7f5228c00700 1 Processor -- start 2024-06-03T02:22:17.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.905+0000 7f5228c00700 1 -- start start 2024-06-03T02:22:17.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.905+0000 7f5228c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 0x7f522410eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5228c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f522406ab30 0x7f522410eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5228c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5224108500 0x7f522410bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5228c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5224102800 con 0x7f5224108500 2024-06-03T02:22:17.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5228c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5224102680 con 0x7f522406a170 2024-06-03T02:22:17.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5228c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5224102980 con 0x7f522406ab30 2024-06-03T02:22:17.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5221400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f522406ab30 0x7f522410eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5221e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 0x7f522410eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5221400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f522406ab30 0x7f522410eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:50228/0 (socket says 172.21.15.79:50228) 2024-06-03T02:22:17.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5221e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 0x7f522410eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34626/0 (socket says 172.21.15.79:34626) 2024-06-03T02:22:17.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5221400700 1 -- 172.21.15.79:0/453997449 learned_addr learned my addr 172.21.15.79:0/453997449 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:17.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5222800700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5224108500 0x7f522410bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.906+0000 7f5221e00700 1 -- 172.21.15.79:0/453997449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f522406ab30 msgr2=0x7f522410eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:17.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f5221e00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f522406ab30 0x7f522410eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f5221e00700 1 -- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5224108500 msgr2=0x7f522410bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:17.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f5221e00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5224108500 0x7f522410bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f5221e00700 1 -- 172.21.15.79:0/453997449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522410c560 con 0x7f522406a170 2024-06-03T02:22:17.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f5221400700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f522406ab30 0x7f522410eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:22:17.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f5221e00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 0x7f522410eab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f521400c970 tx=0x7f521400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:17.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f521aa00700 1 -- 172.21.15.79:0/453997449 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521400ce60 con 0x7f522406a170 2024-06-03T02:22:17.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f521aa00700 1 -- 172.21.15.79:0/453997449 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5214004830 con 0x7f522406a170 2024-06-03T02:22:17.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f521aa00700 1 -- 172.21.15.79:0/453997449 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214005530 con 0x7f522406a170 2024-06-03T02:22:17.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f5228c00700 1 -- 172.21.15.79:0/453997449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522410c850 con 0x7f522406a170 2024-06-03T02:22:17.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.907+0000 7f5228c00700 1 -- 172.21.15.79:0/453997449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52241abea0 con 0x7f522406a170 2024-06-03T02:22:17.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.911+0000 7f521aa00700 1 -- 172.21.15.79:0/453997449 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f52140049e0 con 0x7f522406a170 2024-06-03T02:22:17.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.912+0000 7f521aa00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52080a2df0 0x7f52080a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.912+0000 7f521aa00700 1 -- 172.21.15.79:0/453997449 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f52140ceea0 con 0x7f522406a170 2024-06-03T02:22:17.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.913+0000 7f5221400700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52080a2df0 0x7f52080a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.914+0000 7f5221400700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52080a2df0 0x7f52080a52b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f522410d420 tx=0x7f521000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:17.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.915+0000 7f5233522b80 1 Processor -- start 2024-06-03T02:22:17.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.916+0000 7f5233522b80 1 -- start start 2024-06-03T02:22:17.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.916+0000 7f5233522b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x56168957d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.916+0000 7f5233522b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x56168957da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.916+0000 7f5233522b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561689588300 0x56168958a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.916+0000 7f5233522b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56168948ca10 con 0x5616894f9e40 2024-06-03T02:22:17.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.916+0000 7f5233522b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5616894cfb00 con 0x561689588300 2024-06-03T02:22:17.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.916+0000 7f5233522b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56168948a8f0 con 0x56168957d5b0 2024-06-03T02:22:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x56168957da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x56168957da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:50230/0 (socket says 172.21.15.79:50230) 2024-06-03T02:22:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221400700 1 -- 172.21.15.79:0/964949952 learned_addr learned my addr 172.21.15.79:0/964949952 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5222800700 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561689588300 0x56168958a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221e00700 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x56168957d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221400700 1 -- 172.21.15.79:0/964949952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561689588300 msgr2=0x56168958a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221400700 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561689588300 0x56168958a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221400700 1 -- 172.21.15.79:0/964949952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 msgr2=0x56168957d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221400700 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x56168957d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221400700 1 -- 172.21.15.79:0/964949952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561689477f90 con 0x56168957d5b0 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221e00700 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x56168957d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.917+0000 7f5221400700 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x56168957da30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5210005910 tx=0x7f5210020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e087d9e0ebc694d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.918+0000 7f520f400700 1 -- 172.21.15.79:0/964949952 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521000bea0 con 0x56168957d5b0 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.918+0000 7f520f400700 1 -- 172.21.15.79:0/964949952 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5210005050 con 0x56168957d5b0 2024-06-03T02:22:17.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.918+0000 7f520f400700 1 -- 172.21.15.79:0/964949952 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210004c80 con 0x56168957d5b0 2024-06-03T02:22:17.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.920+0000 7f5233522b80 1 -- 172.21.15.79:0/964949952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 msgr2=0x56168957da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:17.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.920+0000 7f5233522b80 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x56168957da30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5210005910 tx=0x7f5210020000 comp rx=0 tx=0).stop 2024-06-03T02:22:17.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.921+0000 7f5233522b80 1 -- 172.21.15.79:0/964949952 shutdown_connections 2024-06-03T02:22:17.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.921+0000 7f5233522b80 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561689588300 0x56168958a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.921+0000 7f5233522b80 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x56168957d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.921+0000 7f5233522b80 1 --2- 172.21.15.79:0/964949952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x56168957da30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.921+0000 7f5233522b80 1 -- 172.21.15.79:0/964949952 >> 172.21.15.79:0/964949952 conn(0x5616894aa170 msgr2=0x5616895794b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:17.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.921+0000 7f5233522b80 1 -- 172.21.15.79:0/964949952 shutdown_connections 2024-06-03T02:22:17.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.921+0000 7f5233522b80 1 -- 172.21.15.79:0/964949952 wait complete. 2024-06-03T02:22:17.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.922+0000 7f5233522b80 1 -- start start 2024-06-03T02:22:17.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.922+0000 7f5233522b80 1 Processor -- start 2024-06-03T02:22:17.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.922+0000 7f5233522b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616894f9e40 0x561689587ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.922+0000 7f5233522b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x561689584dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.922+0000 7f5233522b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561689588300 0x561689585310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5233522b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56168946bf60 con 0x561689588300 2024-06-03T02:22:17.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5233522b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5616894cbac0 con 0x5616894f9e40 2024-06-03T02:22:17.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5233522b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56168959bf70 con 0x56168957d5b0 2024-06-03T02:22:17.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5233522b80 10 client.0 _ll_register_callbacks cb 0x561689469720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:17.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x561689584dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x561689584dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:50246/0 (socket says 172.21.15.79:50246) 2024-06-03T02:22:17.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221400700 1 -- 172.21.15.79:0/4019726906 learned_addr learned my addr 172.21.15.79:0/4019726906 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:17.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221e00700 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616894f9e40 0x561689587ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5222800700 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561689588300 0x561689585310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221400700 1 -- 172.21.15.79:0/4019726906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616894f9e40 msgr2=0x561689587ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:17.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221400700 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616894f9e40 0x561689587ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221400700 1 -- 172.21.15.79:0/4019726906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561689588300 msgr2=0x561689585310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:17.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221400700 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561689588300 0x561689585310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221400700 1 -- 172.21.15.79:0/4019726906 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168941e8d0 con 0x56168957d5b0 2024-06-03T02:22:17.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5221400700 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x561689584dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5210005850 tx=0x7f5210003d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e3dd57ac21fc25b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:17.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.923+0000 7f5203e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:22:17.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.924+0000 7f5203e00700 1 -- 172.21.15.79:0/4019726906 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521000bea0 con 0x56168957d5b0 2024-06-03T02:22:17.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.924+0000 7f5203e00700 1 -- 172.21.15.79:0/4019726906 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5210004ab0 con 0x56168957d5b0 2024-06-03T02:22:17.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.924+0000 7f5233522b80 1 -- 172.21.15.79:0/4019726906 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5210005050 con 0x56168957d5b0 2024-06-03T02:22:17.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.924+0000 7f5203e00700 1 -- 172.21.15.79:0/4019726906 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f521003e050 con 0x56168957d5b0 2024-06-03T02:22:17.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.924+0000 7f5233522b80 10 client.25555 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:17.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.924+0000 7f5233522b80 1 -- 172.21.15.79:0/4019726906 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5616895a6660 con 0x56168957d5b0 2024-06-03T02:22:17.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.925+0000 7f5203e00700 1 -- 172.21.15.79:0/4019726906 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5210005050 con 0x56168957d5b0 2024-06-03T02:22:17.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.926+0000 7f5203e00700 1 -- 172.21.15.79:0/4019726906 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f521000f460 con 0x56168957d5b0 2024-06-03T02:22:17.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.926+0000 7f5233522b80 10 client.25555 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:17.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.926+0000 7f5233522b80 10 client.25555 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:22:17.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.926+0000 7f5233522b80 20 client.25555 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:17.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.926+0000 7f5233522b80 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5616895a8b30 0x5616895aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.926+0000 7f5233522b80 4 client.25555 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:22:17.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.926+0000 7f5233522b80 1 -- 172.21.15.79:0/4019726906 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616894dd000 con 0x5616895a8b30 2024-06-03T02:22:17.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.926+0000 7f5221e00700 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5616895a8b30 0x5616895aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.928+0000 7f5221e00700 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5616895a8b30 0x5616895aaf20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bff86fda9f103869 server_cookie=b3aa1e3e7ca97f0c in_seq=0 out_seq=0 2024-06-03T02:22:17.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.928+0000 7f5203e00700 10 client.25555 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:22:17.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.930+0000 7f5203e00700 1 -- 172.21.15.79:0/4019726906 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5616894dd000 con 0x5616895a8b30 2024-06-03T02:22:17.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.930+0000 7f5203e00700 10 client.25555 handle_command_reply: tid=0 2024-06-03T02:22:17.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.930+0000 7f5233522b80 1 client.25555 shutdown 2024-06-03T02:22:17.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.933+0000 7f5233522b80 1 -- 172.21.15.79:0/4019726906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 msgr2=0x561689584dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:17.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.933+0000 7f5233522b80 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x561689584dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5210005850 tx=0x7f5210003d70 comp rx=0 tx=0).stop 2024-06-03T02:22:17.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.933+0000 7f5233522b80 1 -- 172.21.15.79:0/4019726906 shutdown_connections 2024-06-03T02:22:17.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.933+0000 7f5233522b80 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5616895a8b30 0x5616895aaf20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.933+0000 7f5233522b80 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561689588300 0x561689585310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.933+0000 7f5233522b80 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616894f9e40 0x561689587ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.933+0000 7f5233522b80 1 --2- 172.21.15.79:0/4019726906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56168957d5b0 0x561689584dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.933+0000 7f5233522b80 1 -- 172.21.15.79:0/4019726906 >> 172.21.15.79:0/4019726906 conn(0x5616894aa170 msgr2=0x561689578740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:17.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.933+0000 7f5203e00700 0 client.25555 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:22:17.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.934+0000 7f5233522b80 1 -- 172.21.15.79:0/4019726906 shutdown_connections 2024-06-03T02:22:17.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.934+0000 7f5233522b80 1 -- 172.21.15.79:0/4019726906 wait complete. 2024-06-03T02:22:17.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.934+0000 7f5233522b80 20 client.25555 trim_cache size 0 max 16384 2024-06-03T02:22:17.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.962+0000 7f5233522b80 1 Processor -- start 2024-06-03T02:22:17.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.962+0000 7f5233522b80 1 -- start start 2024-06-03T02:22:17.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.963+0000 7f5233522b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616894f9e40 0x561689583d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.963+0000 7f5233522b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x561689587610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.963+0000 7f5233522b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561689588300 0x5616895896f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.963+0000 7f5233522b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52241ac0f0 con 0x561689588300 2024-06-03T02:22:17.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.963+0000 7f5233522b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52241adc30 con 0x56168957d5b0 2024-06-03T02:22:17.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.963+0000 7f5233522b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52241ac7c0 con 0x5616894f9e40 2024-06-03T02:22:17.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.963+0000 7f5221e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616894f9e40 0x561689583d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.963+0000 7f5221e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616894f9e40 0x561689583d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:50254/0 (socket says 172.21.15.79:50254) 2024-06-03T02:22:17.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.963+0000 7f5221e00700 1 -- 172.21.15.79:0/3433501084 learned_addr learned my addr 172.21.15.79:0/3433501084 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:17.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.964+0000 7f5221400700 1 --2- 172.21.15.79:0/3433501084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x561689587610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.964+0000 7f5221e00700 1 -- 172.21.15.79:0/3433501084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 msgr2=0x561689587610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:17.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.964+0000 7f5221e00700 1 --2- 172.21.15.79:0/3433501084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x561689587610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.964+0000 7f5221e00700 1 -- 172.21.15.79:0/3433501084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561689588300 msgr2=0x5616895896f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:17.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.964+0000 7f5221e00700 1 --2- 172.21.15.79:0/3433501084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561689588300 0x5616895896f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.964+0000 7f5221e00700 1 -- 172.21.15.79:0/3433501084 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168957d3e0 con 0x5616894f9e40 2024-06-03T02:22:17.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.965+0000 7f5221e00700 1 --2- 172.21.15.79:0/3433501084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616894f9e40 0x561689583d30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5214097cc0 tx=0x7f52140ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69dec7019a670e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:17.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.965+0000 7f520ea00700 1 -- 172.21.15.79:0/3433501084 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52140b4070 con 0x5616894f9e40 2024-06-03T02:22:17.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.965+0000 7f520ea00700 1 -- 172.21.15.79:0/3433501084 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52140a3f60 con 0x5616894f9e40 2024-06-03T02:22:17.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.965+0000 7f520ea00700 1 -- 172.21.15.79:0/3433501084 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521401d070 con 0x5616894f9e40 2024-06-03T02:22:17.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.967+0000 7f5233522b80 1 -- 172.21.15.79:0/3433501084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616894f9e40 msgr2=0x561689583d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:17.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.967+0000 7f5233522b80 1 --2- 172.21.15.79:0/3433501084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616894f9e40 0x561689583d30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5214097cc0 tx=0x7f52140ce040 comp rx=0 tx=0).stop 2024-06-03T02:22:17.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.967+0000 7f5233522b80 1 -- 172.21.15.79:0/3433501084 shutdown_connections 2024-06-03T02:22:17.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.967+0000 7f5233522b80 1 --2- 172.21.15.79:0/3433501084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561689588300 0x5616895896f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.967+0000 7f5233522b80 1 --2- 172.21.15.79:0/3433501084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616894f9e40 0x561689583d30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.967+0000 7f5233522b80 1 --2- 172.21.15.79:0/3433501084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x561689587610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.967+0000 7f5233522b80 1 -- 172.21.15.79:0/3433501084 >> 172.21.15.79:0/3433501084 conn(0x5616894aa170 msgr2=0x56168957f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:17.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.968+0000 7f5233522b80 1 -- 172.21.15.79:0/3433501084 shutdown_connections 2024-06-03T02:22:17.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.968+0000 7f5233522b80 1 -- 172.21.15.79:0/3433501084 wait complete. 2024-06-03T02:22:17.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.969+0000 7f5233522b80 1 -- start start 2024-06-03T02:22:17.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.970+0000 7f5233522b80 1 Processor -- start 2024-06-03T02:22:17.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.971+0000 7f5233522b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x5616895ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.971+0000 7f5233522b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x5616895a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.971+0000 7f5233522b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561689588300 0x5616895a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.971+0000 7f5233522b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52241ad560 con 0x5616894f9e40 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.971+0000 7f5233522b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52241ace90 con 0x56168957d5b0 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.971+0000 7f5233522b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f522411bf70 con 0x561689588300 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.971+0000 7f5233522b80 10 client.0 _ll_register_callbacks cb 0x561689469720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.971+0000 7f5221400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x5616895a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.972+0000 7f5221400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x5616895a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34648/0 (socket says 172.21.15.79:34648) 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.972+0000 7f5221400700 1 -- 172.21.15.79:0/1071027437 learned_addr learned my addr 172.21.15.79:0/1071027437 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.972+0000 7f5221e00700 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x5616895ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:17 smithi123 ceph-mon[36640]: pgmap v583: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 928 op/s 2024-06-03T02:22:17.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.972+0000 7f5222800700 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561689588300 0x5616895a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.973+0000 7f5221400700 1 -- 172.21.15.79:0/1071027437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561689588300 msgr2=0x5616895a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:17.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.974+0000 7f5221400700 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561689588300 0x5616895a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.974+0000 7f5221400700 1 -- 172.21.15.79:0/1071027437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 msgr2=0x5616895ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:17.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.974+0000 7f5221400700 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x5616895ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.974+0000 7f5221400700 1 -- 172.21.15.79:0/1071027437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616895aa3d0 con 0x56168957d5b0 2024-06-03T02:22:17.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.976+0000 7f5222800700 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561689588300 0x5616895a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:22:17.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.976+0000 7f5221e00700 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x5616895ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:22:17.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.978+0000 7f5221400700 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x5616895a9850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f521000f9f0 tx=0x7f521000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70353b262ca94653 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:17.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.978+0000 7f5203e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:22:17.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.979+0000 7f5203e00700 1 -- 172.21.15.79:0/1071027437 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521000bea0 con 0x56168957d5b0 2024-06-03T02:22:17.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.980+0000 7f5203e00700 1 -- 172.21.15.79:0/1071027437 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f521000fa70 con 0x56168957d5b0 2024-06-03T02:22:17.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.980+0000 7f5233522b80 1 -- 172.21.15.79:0/1071027437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52140a3f60 con 0x56168957d5b0 2024-06-03T02:22:17.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.981+0000 7f5203e00700 1 -- 172.21.15.79:0/1071027437 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f521000fc70 con 0x56168957d5b0 2024-06-03T02:22:17.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.981+0000 7f5233522b80 10 client.25574 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:17.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.981+0000 7f5233522b80 1 -- 172.21.15.79:0/1071027437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5616896242c0 con 0x56168957d5b0 2024-06-03T02:22:17.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.982+0000 7f5203e00700 1 -- 172.21.15.79:0/1071027437 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52140a3f60 con 0x56168957d5b0 2024-06-03T02:22:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.982+0000 7f5203e00700 1 -- 172.21.15.79:0/1071027437 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5210030780 con 0x56168957d5b0 2024-06-03T02:22:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.983+0000 7f5233522b80 10 client.25574 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.983+0000 7f5233522b80 10 client.25574 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:22:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.983+0000 7f5233522b80 20 client.25574 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.983+0000 7f5233522b80 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x561689624860 0x561689626c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:17.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.983+0000 7f5233522b80 4 client.25574 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:22:17.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.983+0000 7f5233522b80 1 -- 172.21.15.79:0/1071027437 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616894d7580 con 0x561689624860 2024-06-03T02:22:17.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.985+0000 7f5221e00700 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x561689624860 0x561689626c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:17.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.986+0000 7f5221e00700 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x561689624860 0x561689626c80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=75bd8ebe08d83df0 server_cookie=a312b1a7345283bc in_seq=0 out_seq=0 2024-06-03T02:22:17.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.986+0000 7f5203e00700 10 client.25574 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:22:17.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.987+0000 7f5203e00700 1 -- 172.21.15.79:0/1071027437 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616894d7580 con 0x561689624860 2024-06-03T02:22:17.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.987+0000 7f5203e00700 10 client.25574 handle_command_reply: tid=0 2024-06-03T02:22:17.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.987+0000 7f5233522b80 1 client.25574 shutdown 2024-06-03T02:22:17.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.987+0000 7f5233522b80 1 -- 172.21.15.79:0/1071027437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 msgr2=0x5616895a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:17.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.987+0000 7f5233522b80 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x5616895a9850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f521000f9f0 tx=0x7f521000b690 comp rx=0 tx=0).stop 2024-06-03T02:22:17.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.988+0000 7f5233522b80 1 -- 172.21.15.79:0/1071027437 shutdown_connections 2024-06-03T02:22:17.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.988+0000 7f5233522b80 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561689588300 0x5616895a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.988+0000 7f5233522b80 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x561689624860 0x561689626c80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.988+0000 7f5233522b80 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616894f9e40 0x5616895ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.988+0000 7f5233522b80 1 --2- 172.21.15.79:0/1071027437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168957d5b0 0x5616895a9850 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.988+0000 7f5233522b80 1 -- 172.21.15.79:0/1071027437 >> 172.21.15.79:0/1071027437 conn(0x5616894aa170 msgr2=0x561689584580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:17.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.988+0000 7f5203e00700 0 client.25574 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:22:17.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.989+0000 7f5233522b80 1 -- 172.21.15.79:0/1071027437 shutdown_connections 2024-06-03T02:22:17.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.989+0000 7f5233522b80 1 -- 172.21.15.79:0/1071027437 wait complete. 2024-06-03T02:22:17.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.989+0000 7f5233522b80 20 client.25574 trim_cache size 0 max 16384 2024-06-03T02:22:17.990 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:22:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.991+0000 7f5228c00700 1 -- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52080a2df0 msgr2=0x7f52080a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.991+0000 7f5228c00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52080a2df0 0x7f52080a52b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f522410d420 tx=0x7f521000b260 comp rx=0 tx=0).stop 2024-06-03T02:22:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.991+0000 7f5228c00700 1 -- 172.21.15.79:0/453997449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 msgr2=0x7f522410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.991+0000 7f5228c00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 0x7f522410eab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f521400c970 tx=0x7f521400cc80 comp rx=0 tx=0).stop 2024-06-03T02:22:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.992+0000 7f5228c00700 1 -- 172.21.15.79:0/453997449 shutdown_connections 2024-06-03T02:22:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.992+0000 7f5228c00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f52080a2df0 0x7f52080a52b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.992+0000 7f5228c00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5224108500 0x7f522410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.992+0000 7f5228c00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f522406a170 0x7f522410eab0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.992+0000 7f5228c00700 1 --2- 172.21.15.79:0/453997449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f522406ab30 0x7f522410eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.992+0000 7f5228c00700 1 -- 172.21.15.79:0/453997449 >> 172.21.15.79:0/453997449 conn(0x7f5224078340 msgr2=0x7f5224106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.993+0000 7f5228c00700 1 -- 172.21.15.79:0/453997449 shutdown_connections 2024-06-03T02:22:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:17.993+0000 7f5228c00700 1 -- 172.21.15.79:0/453997449 wait complete. 2024-06-03T02:22:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 130.0 s, 1458.6 tps, lat 69.646 ms stddev 95.742 2024-06-03T02:22:19.587 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:19 smithi079 ceph-mon[31819]: pgmap v584: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 943 op/s 2024-06-03T02:22:19.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:19 smithi135 ceph-mon[36653]: pgmap v584: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 943 op/s 2024-06-03T02:22:19.887 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:22:19.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:19 smithi123 ceph-mon[36640]: pgmap v584: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 943 op/s 2024-06-03T02:22:20.512 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:21.553 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:21 smithi079 ceph-mon[31819]: pgmap v585: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.4 MiB/s wr, 595 op/s 2024-06-03T02:22:21.553 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:21 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:22:21.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:21 smithi135 ceph-mon[36653]: pgmap v585: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.4 MiB/s wr, 595 op/s 2024-06-03T02:22:21.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:21 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:22:21.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:21 smithi123 ceph-mon[36640]: pgmap v585: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.4 MiB/s wr, 595 op/s 2024-06-03T02:22:21.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:21 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:22:23.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.116+0000 7f836d600700 1 -- 172.21.15.79:0/2785310475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836810da90 msgr2=0x7f8368111ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:23.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.116+0000 7f836d600700 1 --2- 172.21.15.79:0/2785310475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836810da90 0x7f8368111ae0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8350009920 tx=0x7f8350009c30 comp rx=0 tx=0).stop 2024-06-03T02:22:23.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.118+0000 7f836d600700 1 -- 172.21.15.79:0/2785310475 shutdown_connections 2024-06-03T02:22:23.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.118+0000 7f836d600700 1 --2- 172.21.15.79:0/2785310475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83681121c0 0x7f8368119d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.118+0000 7f836d600700 1 --2- 172.21.15.79:0/2785310475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836810d0e0 0x7f836810d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.118+0000 7f836d600700 1 --2- 172.21.15.79:0/2785310475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836810da90 0x7f8368111ae0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.118+0000 7f836d600700 1 -- 172.21.15.79:0/2785310475 >> 172.21.15.79:0/2785310475 conn(0x7f836806d6f0 msgr2=0x7f836810b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:23.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.118+0000 7f836d600700 1 -- 172.21.15.79:0/2785310475 shutdown_connections 2024-06-03T02:22:23.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.119+0000 7f836d600700 1 -- 172.21.15.79:0/2785310475 wait complete. 2024-06-03T02:22:23.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.120+0000 7f836d600700 1 Processor -- start 2024-06-03T02:22:23.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.121+0000 7f836d600700 1 -- start start 2024-06-03T02:22:23.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.121+0000 7f836d600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836810d0e0 0x7f8368119cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.121+0000 7f836d600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836810da90 0x7f8368116dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.121+0000 7f836d600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83681121c0 0x7f83681173c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.121+0000 7f836d600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f836811d1d0 con 0x7f836810d0e0 2024-06-03T02:22:23.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.121+0000 7f836d600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f836811d050 con 0x7f836810da90 2024-06-03T02:22:23.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.121+0000 7f836d600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f836811d350 con 0x7f83681121c0 2024-06-03T02:22:23.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.123+0000 7f835e000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836810da90 0x7f8368116dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.123+0000 7f835e000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836810da90 0x7f8368116dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34660/0 (socket says 172.21.15.79:34660) 2024-06-03T02:22:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.123+0000 7f835e000700 1 -- 172.21.15.79:0/1211003615 learned_addr learned my addr 172.21.15.79:0/1211003615 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.124+0000 7f8367400700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83681121c0 0x7f83681173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.125+0000 7f8366a00700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836810d0e0 0x7f8368119cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.125+0000 7f835e000700 1 -- 172.21.15.79:0/1211003615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83681121c0 msgr2=0x7f83681173c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.125+0000 7f835e000700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83681121c0 0x7f83681173c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.125+0000 7f835e000700 1 -- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836810d0e0 msgr2=0x7f8368119cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:23.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.125+0000 7f835e000700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836810d0e0 0x7f8368119cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.125+0000 7f835e000700 1 -- 172.21.15.79:0/1211003615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83681a96f0 con 0x7f836810da90 2024-06-03T02:22:23.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.127+0000 7f835e000700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836810da90 0x7f8368116dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f835000a010 tx=0x7f8350005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:23.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.127+0000 7f835fe00700 1 -- 172.21.15.79:0/1211003615 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83500053a0 con 0x7f836810da90 2024-06-03T02:22:23.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.128+0000 7f835fe00700 1 -- 172.21.15.79:0/1211003615 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8350005540 con 0x7f836810da90 2024-06-03T02:22:23.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.128+0000 7f835fe00700 1 -- 172.21.15.79:0/1211003615 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350003c40 con 0x7f836810da90 2024-06-03T02:22:23.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.128+0000 7f836d600700 1 -- 172.21.15.79:0/1211003615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83681a9980 con 0x7f836810da90 2024-06-03T02:22:23.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.129+0000 7f836d600700 1 -- 172.21.15.79:0/1211003615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681a9e10 con 0x7f836810da90 2024-06-03T02:22:23.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.136+0000 7f835fe00700 1 -- 172.21.15.79:0/1211003615 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f835000bad0 con 0x7f836810da90 2024-06-03T02:22:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.138+0000 7f835fe00700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f834c0a2d70 0x7f834c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.138+0000 7f8377f6cb80 1 Processor -- start 2024-06-03T02:22:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.138+0000 7f835fe00700 1 -- 172.21.15.79:0/1211003615 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f83500d0420 con 0x7f836810da90 2024-06-03T02:22:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.139+0000 7f8377f6cb80 1 -- start start 2024-06-03T02:22:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.139+0000 7f8366a00700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f834c0a2d70 0x7f834c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.140+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x562129681070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.140+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621296815b0 0x562129681a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.140+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212968c300 0x56212968e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.140+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562129590a10 con 0x5621296815b0 2024-06-03T02:22:23.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.140+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5621295d3b00 con 0x56212968c300 2024-06-03T02:22:23.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.140+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56212958e8f0 con 0x5621295fde40 2024-06-03T02:22:23.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.141+0000 7f8366a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x562129681070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.141+0000 7f8366a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x562129681070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:50296/0 (socket says 172.21.15.79:50296) 2024-06-03T02:22:23.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.141+0000 7f8367400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212968c300 0x56212968e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.141+0000 7f8366a00700 1 -- 172.21.15.79:0/2659930489 learned_addr learned my addr 172.21.15.79:0/2659930489 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:23.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.141+0000 7f8366a00700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f834c0a2d70 0x7f834c0a5230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8358009750 tx=0x7f83580116c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:23.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.141+0000 7f835e000700 1 --2- 172.21.15.79:0/2659930489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621296815b0 0x562129681a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.142+0000 7f8366a00700 1 -- 172.21.15.79:0/2659930489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212968c300 msgr2=0x56212968e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:23.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.142+0000 7f8366a00700 1 --2- 172.21.15.79:0/2659930489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212968c300 0x56212968e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.142+0000 7f8366a00700 1 -- 172.21.15.79:0/2659930489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621296815b0 msgr2=0x562129681a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:23.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.142+0000 7f8366a00700 1 --2- 172.21.15.79:0/2659930489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621296815b0 0x562129681a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.142+0000 7f8366a00700 1 -- 172.21.15.79:0/2659930489 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56212957bf90 con 0x5621295fde40 2024-06-03T02:22:23.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.142+0000 7f8366a00700 1 --2- 172.21.15.79:0/2659930489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x562129681070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f83580144a0 tx=0x7f835801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b00edf288080fcfd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:23.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.142+0000 7f8357e00700 1 -- 172.21.15.79:0/2659930489 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580142d0 con 0x5621295fde40 2024-06-03T02:22:23.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.142+0000 7f8357e00700 1 -- 172.21.15.79:0/2659930489 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8358014c40 con 0x5621295fde40 2024-06-03T02:22:23.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.142+0000 7f8357e00700 1 -- 172.21.15.79:0/2659930489 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835801f710 con 0x5621295fde40 2024-06-03T02:22:23.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.144+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2659930489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 msgr2=0x562129681070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:23.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.144+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/2659930489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x562129681070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f83580144a0 tx=0x7f835801c000 comp rx=0 tx=0).stop 2024-06-03T02:22:23.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.145+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2659930489 shutdown_connections 2024-06-03T02:22:23.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.145+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/2659930489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212968c300 0x56212968e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.145+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/2659930489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x562129681070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.145+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/2659930489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621296815b0 0x562129681a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.145+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2659930489 >> 172.21.15.79:0/2659930489 conn(0x5621295ae170 msgr2=0x56212967d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:23.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.146+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2659930489 shutdown_connections 2024-06-03T02:22:23.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.147+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2659930489 wait complete. 2024-06-03T02:22:23.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.147+0000 7f8377f6cb80 1 -- start start 2024-06-03T02:22:23.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.148+0000 7f8377f6cb80 1 Processor -- start 2024-06-03T02:22:23.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x56212968bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621296815b0 0x562129688dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 0x562129689300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56212956ff60 con 0x56212968c300 2024-06-03T02:22:23.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5621295cfac0 con 0x5621296815b0 2024-06-03T02:22:23.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56212968efb0 con 0x5621295fde40 2024-06-03T02:22:23.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8377f6cb80 10 client.0 _ll_register_callbacks cb 0x56212956d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:23.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8366a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x56212968bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8366a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x56212968bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:50312/0 (socket says 172.21.15.79:50312) 2024-06-03T02:22:23.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f8366a00700 1 -- 172.21.15.79:0/1815866522 learned_addr learned my addr 172.21.15.79:0/1815866522 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:23.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.149+0000 7f835e000700 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621296815b0 0x562129688dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.150+0000 7f8367400700 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 0x562129689300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.150+0000 7f8366a00700 1 -- 172.21.15.79:0/1815866522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621296815b0 msgr2=0x562129688dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:23.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.150+0000 7f8366a00700 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621296815b0 0x562129688dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.150+0000 7f8366a00700 1 -- 172.21.15.79:0/1815866522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 msgr2=0x562129689300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.150+0000 7f8366a00700 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 0x562129689300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.150+0000 7f8366a00700 1 -- 172.21.15.79:0/1815866522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621295228d0 con 0x5621295fde40 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.150+0000 7f8367400700 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 0x562129689300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.150+0000 7f8366a00700 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x56212968bac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f83580052d0 tx=0x7f83580053b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f541ee2da1f3ef5c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.150+0000 7f8354c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.151+0000 7f8354c00700 1 -- 172.21.15.79:0/1815866522 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358004b20 con 0x5621295fde40 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.151+0000 7f8354c00700 1 -- 172.21.15.79:0/1815866522 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8358004cc0 con 0x5621295fde40 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.151+0000 7f8377f6cb80 1 -- 172.21.15.79:0/1815866522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8358014c40 con 0x5621295fde40 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.152+0000 7f8354c00700 1 -- 172.21.15.79:0/1815866522 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8358005680 con 0x5621295fde40 2024-06-03T02:22:23.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.152+0000 7f8377f6cb80 10 client.25573 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:23.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.152+0000 7f8377f6cb80 1 -- 172.21.15.79:0/1815866522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562129699730 con 0x5621295fde40 2024-06-03T02:22:23.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.152+0000 7f8354c00700 1 -- 172.21.15.79:0/1815866522 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8358014c40 con 0x5621295fde40 2024-06-03T02:22:23.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.152+0000 7f8354c00700 1 -- 172.21.15.79:0/1815866522 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f835801f420 con 0x5621295fde40 2024-06-03T02:22:23.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.153+0000 7f8377f6cb80 10 client.25573 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:23.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.153+0000 7f8377f6cb80 10 client.25573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:22:23.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.153+0000 7f8377f6cb80 20 client.25573 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:23.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.153+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56212969bc60 0x56212969e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.153+0000 7f8377f6cb80 4 client.25573 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:22:23.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.153+0000 7f8377f6cb80 1 -- 172.21.15.79:0/1815866522 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621295e1000 con 0x56212969bc60 2024-06-03T02:22:23.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.153+0000 7f835e000700 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56212969bc60 0x56212969e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.155+0000 7f835e000700 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56212969bc60 0x56212969e050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91702be9f0e7d05e server_cookie=30ed744aa7267285 in_seq=0 out_seq=0 2024-06-03T02:22:23.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.156+0000 7f8354c00700 10 client.25573 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:22:23.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.156+0000 7f8354c00700 1 -- 172.21.15.79:0/1815866522 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5621295e1000 con 0x56212969bc60 2024-06-03T02:22:23.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.156+0000 7f8354c00700 10 client.25573 handle_command_reply: tid=0 2024-06-03T02:22:23.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.156+0000 7f8377f6cb80 1 client.25573 shutdown 2024-06-03T02:22:23.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.157+0000 7f8377f6cb80 1 -- 172.21.15.79:0/1815866522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 msgr2=0x56212968bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:23.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.157+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x56212968bac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f83580052d0 tx=0x7f83580053b0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.157+0000 7f8377f6cb80 1 -- 172.21.15.79:0/1815866522 shutdown_connections 2024-06-03T02:22:23.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.157+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 0x562129689300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.157+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621295fde40 0x56212968bac0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.157+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56212969bc60 0x56212969e050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.157+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/1815866522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621296815b0 0x562129688dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.157+0000 7f8377f6cb80 1 -- 172.21.15.79:0/1815866522 >> 172.21.15.79:0/1815866522 conn(0x5621295ae170 msgr2=0x56212967c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:23.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.157+0000 7f8354c00700 0 client.25573 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:22:23.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.158+0000 7f8377f6cb80 1 -- 172.21.15.79:0/1815866522 shutdown_connections 2024-06-03T02:22:23.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.158+0000 7f8377f6cb80 1 -- 172.21.15.79:0/1815866522 wait complete. 2024-06-03T02:22:23.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.158+0000 7f8377f6cb80 20 client.25573 trim_cache size 0 max 16384 2024-06-03T02:22:23.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.183+0000 7f8377f6cb80 1 Processor -- start 2024-06-03T02:22:23.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.183+0000 7f8377f6cb80 1 -- start start 2024-06-03T02:22:23.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x562129687dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621296815b0 0x56212968d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56212968c300 0x56212968dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83681c5980 con 0x5621296815b0 2024-06-03T02:22:23.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83681c74c0 con 0x5621295fde40 2024-06-03T02:22:23.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83681c6050 con 0x56212968c300 2024-06-03T02:22:23.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8366a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x562129687dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8366a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x562129687dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34684/0 (socket says 172.21.15.79:34684) 2024-06-03T02:22:23.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8366a00700 1 -- 172.21.15.79:0/2474262863 learned_addr learned my addr 172.21.15.79:0/2474262863 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:23.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8367400700 1 --2- 172.21.15.79:0/2474262863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56212968c300 0x56212968dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8366a00700 1 -- 172.21.15.79:0/2474262863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56212968c300 msgr2=0x56212968dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:23.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8366a00700 1 --2- 172.21.15.79:0/2474262863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56212968c300 0x56212968dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8366a00700 1 -- 172.21.15.79:0/2474262863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621296815b0 msgr2=0x56212968d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:23.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8366a00700 1 --2- 172.21.15.79:0/2474262863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621296815b0 0x56212968d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.184+0000 7f8366a00700 1 -- 172.21.15.79:0/2474262863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562129689cf0 con 0x5621295fde40 2024-06-03T02:22:23.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.185+0000 7f8366a00700 1 --2- 172.21.15.79:0/2474262863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x562129687dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f83580098f0 tx=0x7f8358004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f3f143b4f97b3ff server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:23.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.185+0000 7f8357400700 1 -- 172.21.15.79:0/2474262863 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358005680 con 0x5621295fde40 2024-06-03T02:22:23.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.185+0000 7f8357400700 1 -- 172.21.15.79:0/2474262863 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8358040070 con 0x5621295fde40 2024-06-03T02:22:23.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.185+0000 7f8357400700 1 -- 172.21.15.79:0/2474262863 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358012450 con 0x5621295fde40 2024-06-03T02:22:23.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.186+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2474262863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 msgr2=0x562129687dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:23.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.186+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/2474262863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x562129687dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f83580098f0 tx=0x7f8358004ab0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.186+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2474262863 shutdown_connections 2024-06-03T02:22:23.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.186+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/2474262863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56212968c300 0x56212968dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.186+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/2474262863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x562129687dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.186+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/2474262863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621296815b0 0x56212968d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.186+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2474262863 >> 172.21.15.79:0/2474262863 conn(0x5621295ae170 msgr2=0x5621296876b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:23.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.187+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2474262863 shutdown_connections 2024-06-03T02:22:23.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.187+0000 7f8377f6cb80 1 -- 172.21.15.79:0/2474262863 wait complete. 2024-06-03T02:22:23.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.187+0000 7f8377f6cb80 1 -- start start 2024-06-03T02:22:23.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.188+0000 7f8377f6cb80 1 Processor -- start 2024-06-03T02:22:23.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.189+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x5621296880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.189+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621296815b0 0x562129688610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.189+0000 7f8377f6cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 0x562129693240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.189+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83681c6df0 con 0x56212968c300 2024-06-03T02:22:23.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.189+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83681c6720 con 0x5621295fde40 2024-06-03T02:22:23.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.189+0000 7f8377f6cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83681c38e0 con 0x5621296815b0 2024-06-03T02:22:23.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.189+0000 7f8377f6cb80 10 client.0 _ll_register_callbacks cb 0x56212956d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:23.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.190+0000 7f8366a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x5621296880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.190+0000 7f8366a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x5621296880d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34690/0 (socket says 172.21.15.79:34690) 2024-06-03T02:22:23.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.190+0000 7f8366a00700 1 -- 172.21.15.79:0/3179697034 learned_addr learned my addr 172.21.15.79:0/3179697034 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:23.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.190+0000 7f8366a00700 1 -- 172.21.15.79:0/3179697034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621296815b0 msgr2=0x562129688610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:22:23.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.190+0000 7f8366a00700 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621296815b0 0x562129688610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.190+0000 7f8366a00700 1 -- 172.21.15.79:0/3179697034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 msgr2=0x562129693240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:23.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.190+0000 7f8366a00700 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 0x562129693240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.190+0000 7f8366a00700 1 -- 172.21.15.79:0/3179697034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562129693800 con 0x5621295fde40 2024-06-03T02:22:23.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.190+0000 7f8377f6cb80 1 -- 172.21.15.79:0/3179697034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8358040070 con 0x5621295fde40 2024-06-03T02:22:23.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.191+0000 7f8366a00700 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x5621296880d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f835801f870 tx=0x7f83580053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d01535c7033c50 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:23.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.191+0000 7f8354c00700 10 client.25592 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:22:23.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.191+0000 7f8354c00700 1 -- 172.21.15.79:0/3179697034 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358014a90 con 0x5621295fde40 2024-06-03T02:22:23.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.192+0000 7f8354c00700 1 -- 172.21.15.79:0/3179697034 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8358014c30 con 0x5621295fde40 2024-06-03T02:22:23.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.192+0000 7f8354c00700 1 -- 172.21.15.79:0/3179697034 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8358014e30 con 0x5621295fde40 2024-06-03T02:22:23.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.192+0000 7f8377f6cb80 10 client.25592 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:23.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.192+0000 7f8377f6cb80 1 -- 172.21.15.79:0/3179697034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x562129693800 con 0x5621295fde40 2024-06-03T02:22:23.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.193+0000 7f8354c00700 1 -- 172.21.15.79:0/3179697034 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8358012450 con 0x5621295fde40 2024-06-03T02:22:23.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.193+0000 7f8377f6cb80 10 client.25592 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:23.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.193+0000 7f8377f6cb80 10 client.25592 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:22:23.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.193+0000 7f8377f6cb80 20 client.25592 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:23.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.193+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562129694b50 0x5621297085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:23.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.193+0000 7f8377f6cb80 4 client.25592 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:22:23.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.193+0000 7f8377f6cb80 1 -- 172.21.15.79:0/3179697034 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621295db580 con 0x562129694b50 2024-06-03T02:22:23.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.194+0000 7f835e000700 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562129694b50 0x5621297085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.195+0000 7f835e000700 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562129694b50 0x5621297085d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=227de021e21494dc server_cookie=5a5b1ac3bd096bf0 in_seq=0 out_seq=0 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.195+0000 7f8354c00700 10 client.25592 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.195+0000 7f8354c00700 1 -- 172.21.15.79:0/3179697034 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621295db580 con 0x562129694b50 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.195+0000 7f8354c00700 10 client.25592 handle_command_reply: tid=0 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.195+0000 7f8377f6cb80 1 client.25592 shutdown 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 -- 172.21.15.79:0/3179697034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 msgr2=0x5621296880d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x5621296880d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f835801f870 tx=0x7f83580053b0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 -- 172.21.15.79:0/3179697034 shutdown_connections 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56212968c300 0x562129693240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562129694b50 0x5621297085d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621295fde40 0x5621296880d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 --2- 172.21.15.79:0/3179697034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621296815b0 0x562129688610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 -- 172.21.15.79:0/3179697034 >> 172.21.15.79:0/3179697034 conn(0x5621295ae170 msgr2=0x56212968ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:23.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8354c00700 0 client.25592 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:22:23.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 -- 172.21.15.79:0/3179697034 shutdown_connections 2024-06-03T02:22:23.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 1 -- 172.21.15.79:0/3179697034 wait complete. 2024-06-03T02:22:23.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.196+0000 7f8377f6cb80 20 client.25592 trim_cache size 0 max 16384 2024-06-03T02:22:23.198 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:22:23.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.197+0000 7f836d600700 1 -- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f834c0a2d70 msgr2=0x7f834c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:23.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.197+0000 7f836d600700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f834c0a2d70 0x7f834c0a5230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8358009750 tx=0x7f83580116c0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.197+0000 7f836d600700 1 -- 172.21.15.79:0/1211003615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836810da90 msgr2=0x7f8368116dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:23.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.197+0000 7f836d600700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836810da90 0x7f8368116dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f835000a010 tx=0x7f8350005040 comp rx=0 tx=0).stop 2024-06-03T02:22:23.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.197+0000 7f836d600700 1 -- 172.21.15.79:0/1211003615 shutdown_connections 2024-06-03T02:22:23.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.198+0000 7f836d600700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83681121c0 0x7f83681173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.198+0000 7f836d600700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f834c0a2d70 0x7f834c0a5230 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.198+0000 7f836d600700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f836810d0e0 0x7f8368119cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.198+0000 7f836d600700 1 --2- 172.21.15.79:0/1211003615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836810da90 0x7f8368116dd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:23.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.198+0000 7f836d600700 1 -- 172.21.15.79:0/1211003615 >> 172.21.15.79:0/1211003615 conn(0x7f836806d6f0 msgr2=0x7f836810b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:23.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.199+0000 7f836d600700 1 -- 172.21.15.79:0/1211003615 shutdown_connections 2024-06-03T02:22:23.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:23.199+0000 7f836d600700 1 -- 172.21.15.79:0/1211003615 wait complete. 2024-06-03T02:22:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 135.0 s, 1515.7 tps, lat 65.024 ms stddev 82.511 2024-06-03T02:22:24.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: pgmap v586: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 651 op/s 2024-06-03T02:22:24.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi079 to 176.4M 2024-06-03T02:22:24.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: Unable to set osd_memory_target on smithi079 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-06-03T02:22:24.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:22:24.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:22:24.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:23 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:22:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: pgmap v586: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 651 op/s 2024-06-03T02:22:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi079 to 176.4M 2024-06-03T02:22:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: Unable to set osd_memory_target on smithi079 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-06-03T02:22:24.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:22:24.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:22:24.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:23 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:22:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: pgmap v586: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 651 op/s 2024-06-03T02:22:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:22:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi079 to 176.4M 2024-06-03T02:22:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: Unable to set osd_memory_target on smithi079 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-06-03T02:22:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:22:24.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:22:24.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:23 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:22:26.052 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:25 smithi079 ceph-mon[31819]: pgmap v587: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 MiB/s wr, 590 op/s 2024-06-03T02:22:26.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:25 smithi135 ceph-mon[36653]: pgmap v587: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 MiB/s wr, 590 op/s 2024-06-03T02:22:26.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:22:26.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:25 smithi123 ceph-mon[36640]: pgmap v587: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 MiB/s wr, 590 op/s 2024-06-03T02:22:26.721 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:27.784 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:27 smithi079 ceph-mon[31819]: pgmap v588: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 MiB/s wr, 544 op/s 2024-06-03T02:22:28.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:27 smithi135 ceph-mon[36653]: pgmap v588: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 MiB/s wr, 544 op/s 2024-06-03T02:22:28.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:27 smithi123 ceph-mon[36640]: pgmap v588: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 MiB/s wr, 544 op/s 2024-06-03T02:22:28.313 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 140.0 s, 2037.1 tps, lat 48.758 ms stddev 61.865 2024-06-03T02:22:29.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.248+0000 7f520bc00700 1 -- 172.21.15.79:0/2876286966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 msgr2=0x7f520406a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:29.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.248+0000 7f520bc00700 1 --2- 172.21.15.79:0/2876286966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 0x7f520406a580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009990 tx=0x7f51f8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.248+0000 7f520bc00700 1 -- 172.21.15.79:0/2876286966 shutdown_connections 2024-06-03T02:22:29.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.248+0000 7f520bc00700 1 --2- 172.21.15.79:0/2876286966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f520410ef40 0x7f5204111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.248+0000 7f520bc00700 1 --2- 172.21.15.79:0/2876286966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 0x7f520406a580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.248+0000 7f520bc00700 1 --2- 172.21.15.79:0/2876286966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520406aac0 0x7f520410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.248+0000 7f520bc00700 1 -- 172.21.15.79:0/2876286966 >> 172.21.15.79:0/2876286966 conn(0x7f5204078340 msgr2=0x7f5204078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:29.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.249+0000 7f520bc00700 1 -- 172.21.15.79:0/2876286966 shutdown_connections 2024-06-03T02:22:29.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.249+0000 7f520bc00700 1 -- 172.21.15.79:0/2876286966 wait complete. 2024-06-03T02:22:29.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.250+0000 7f520bc00700 1 Processor -- start 2024-06-03T02:22:29.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.251+0000 7f520bc00700 1 -- start start 2024-06-03T02:22:29.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.251+0000 7f520bc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 0x7f520410e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.252+0000 7f520bc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f520406aac0 0x7f520410b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.252+0000 7f520bc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520410ef40 0x7f520410be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.252+0000 7f520bc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52041149f0 con 0x7f520410ef40 2024-06-03T02:22:29.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.252+0000 7f520bc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5204114870 con 0x7f520406aac0 2024-06-03T02:22:29.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.252+0000 7f520bc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5204114b70 con 0x7f520406a1a0 2024-06-03T02:22:29.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.252+0000 7f5209200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 0x7f520410e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.252+0000 7f5209200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 0x7f520410e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41830/0 (socket says 172.21.15.79:41830) 2024-06-03T02:22:29.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.252+0000 7f5209200700 1 -- 172.21.15.79:0/2868447065 learned_addr learned my addr 172.21.15.79:0/2868447065 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:29.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.253+0000 7f5203e00700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f520406aac0 0x7f520410b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.257+0000 7f5209200700 1 -- 172.21.15.79:0/2868447065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f520406aac0 msgr2=0x7f520410b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:29.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.257+0000 7f5209200700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f520406aac0 0x7f520410b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.257+0000 7f5209200700 1 -- 172.21.15.79:0/2868447065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520410ef40 msgr2=0x7f520410be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:29.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.257+0000 7f5209200700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520410ef40 0x7f520410be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.257+0000 7f5209200700 1 -- 172.21.15.79:0/2868447065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520410c710 con 0x7f520406a1a0 2024-06-03T02:22:29.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.257+0000 7f5209200700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 0x7f520410e820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51f8006010 tx=0x7f51f80038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:29.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.258+0000 7f5201600700 1 -- 172.21.15.79:0/2868447065 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8027070 con 0x7f520406a1a0 2024-06-03T02:22:29.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.258+0000 7f5201600700 1 -- 172.21.15.79:0/2868447065 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f8004050 con 0x7f520406a1a0 2024-06-03T02:22:29.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.258+0000 7f5201600700 1 -- 172.21.15.79:0/2868447065 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8022d20 con 0x7f520406a1a0 2024-06-03T02:22:29.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.259+0000 7f520bc00700 1 -- 172.21.15.79:0/2868447065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52041ab790 con 0x7f520406a1a0 2024-06-03T02:22:29.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.259+0000 7f520bc00700 1 -- 172.21.15.79:0/2868447065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52041abbf0 con 0x7f520406a1a0 2024-06-03T02:22:29.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.262+0000 7f5201600700 1 -- 172.21.15.79:0/2868447065 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f51f802a070 con 0x7f520406a1a0 2024-06-03T02:22:29.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.264+0000 7f5201600700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51ec0a2df0 0x7f51ec0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.264+0000 7f5201600700 1 -- 172.21.15.79:0/2868447065 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51f80d0400 con 0x7f520406a1a0 2024-06-03T02:22:29.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.265+0000 7f5216519b80 1 Processor -- start 2024-06-03T02:22:29.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.265+0000 7f5203e00700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51ec0a2df0 0x7f51ec0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.266+0000 7f5216519b80 1 -- start start 2024-06-03T02:22:29.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.266+0000 7f5216519b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9ab90 0x5629ded9af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.266+0000 7f5216519b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9b4b0 0x5629ded94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.267+0000 7f5216519b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629ded95680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.267+0000 7f5216519b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5629deca3a10 con 0x5629ded95240 2024-06-03T02:22:29.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.267+0000 7f5216519b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5629dece6b00 con 0x5629ded9b4b0 2024-06-03T02:22:29.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.267+0000 7f5216519b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5629deca18f0 con 0x5629ded9ab90 2024-06-03T02:22:29.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.267+0000 7f5209c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629ded95680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.267+0000 7f5209c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629ded95680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47288/0 (socket says 172.21.15.79:47288) 2024-06-03T02:22:29.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.267+0000 7f5209c00700 1 -- 172.21.15.79:0/192099500 learned_addr learned my addr 172.21.15.79:0/192099500 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:29.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.267+0000 7f5209200700 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9ab90 0x5629ded9af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.268+0000 7f5203e00700 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9b4b0 0x5629ded94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.268+0000 7f5209c00700 1 -- 172.21.15.79:0/192099500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9ab90 msgr2=0x5629ded9af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:29.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.268+0000 7f5209c00700 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9ab90 0x5629ded9af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.268+0000 7f5209c00700 1 -- 172.21.15.79:0/192099500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9b4b0 msgr2=0x5629ded94d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:29.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.268+0000 7f5209c00700 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9b4b0 0x5629ded94d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.268+0000 7f5209c00700 1 -- 172.21.15.79:0/192099500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629dec8ef90 con 0x5629ded95240 2024-06-03T02:22:29.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.269+0000 7f5203e00700 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9b4b0 0x5629ded94d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:29.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.269+0000 7f5203e00700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51ec0a2df0 0x7f51ec0a52b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f520410d090 tx=0x7f51f000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:29.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.269+0000 7f5209c00700 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629ded95680 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009c00 tx=0x7f51fc009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4580b148f274d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:29.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.269+0000 7f51f6000700 1 -- 172.21.15.79:0/192099500 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc007cc0 con 0x5629ded95240 2024-06-03T02:22:29.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.270+0000 7f51f6000700 1 -- 172.21.15.79:0/192099500 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc004020 con 0x5629ded95240 2024-06-03T02:22:29.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.270+0000 7f51f6000700 1 -- 172.21.15.79:0/192099500 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc007e60 con 0x5629ded95240 2024-06-03T02:22:29.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.271+0000 7f5216519b80 1 -- 172.21.15.79:0/192099500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 msgr2=0x5629ded95680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:29.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.271+0000 7f5216519b80 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629ded95680 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009c00 tx=0x7f51fc009510 comp rx=0 tx=0).stop 2024-06-03T02:22:29.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.271+0000 7f5216519b80 1 -- 172.21.15.79:0/192099500 shutdown_connections 2024-06-03T02:22:29.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.271+0000 7f5216519b80 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629ded95680 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.271+0000 7f5216519b80 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9ab90 0x5629ded9af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.271+0000 7f5216519b80 1 --2- 172.21.15.79:0/192099500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9b4b0 0x5629ded94d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.271+0000 7f5216519b80 1 -- 172.21.15.79:0/192099500 >> 172.21.15.79:0/192099500 conn(0x5629decc1170 msgr2=0x5629ded8fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:29.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.272+0000 7f5216519b80 1 -- 172.21.15.79:0/192099500 shutdown_connections 2024-06-03T02:22:29.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.272+0000 7f5216519b80 1 -- 172.21.15.79:0/192099500 wait complete. 2024-06-03T02:22:29.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.274+0000 7f5216519b80 1 -- start start 2024-06-03T02:22:29.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.274+0000 7f5216519b80 1 Processor -- start 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.276+0000 7f5216519b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629deda1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.276+0000 7f5216519b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded9e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.277+0000 7f5216519b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 0x5629ded9ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.277+0000 7f5203e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded9e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.277+0000 7f5203e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded9e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43254/0 (socket says 172.21.15.79:43254) 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.277+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5629dec82f60 con 0x5629ded9b4b0 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.277+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5629dece2ac0 con 0x5629ded9ab90 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.277+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5629deda1e90 con 0x5629ded95240 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.277+0000 7f5203e00700 1 -- 172.21.15.79:0/1841949409 learned_addr learned my addr 172.21.15.79:0/1841949409 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.277+0000 7f5209200700 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629deda1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.277+0000 7f5216519b80 10 client.0 _ll_register_callbacks cb 0x5629dec7c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:29.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.278+0000 7f5209200700 1 -- 172.21.15.79:0/1841949409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 msgr2=0x5629ded9e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:29.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.278+0000 7f5209200700 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded9e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.278+0000 7f5209200700 1 -- 172.21.15.79:0/1841949409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 msgr2=0x5629ded9ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:29.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.278+0000 7f5209200700 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 0x5629ded9ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.278+0000 7f5209200700 1 -- 172.21.15.79:0/1841949409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629dec358d0 con 0x5629ded95240 2024-06-03T02:22:29.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.278+0000 7f5209200700 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629deda1200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51f80cdaf0 tx=0x7f51f8022770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f9d0e0e61be865 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:29.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.278+0000 7f51eaa00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:22:29.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.278+0000 7f51eaa00700 1 -- 172.21.15.79:0/1841949409 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f80b4070 con 0x5629ded95240 2024-06-03T02:22:29.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.279+0000 7f51eaa00700 1 -- 172.21.15.79:0/1841949409 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51f809cb40 con 0x5629ded95240 2024-06-03T02:22:29.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.279+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51fc004020 con 0x5629ded95240 2024-06-03T02:22:29.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.279+0000 7f51eaa00700 1 -- 172.21.15.79:0/1841949409 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51f80057a0 con 0x5629ded95240 2024-06-03T02:22:29.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f5216519b80 10 client.25588 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:29.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629dedac560 con 0x5629ded95240 2024-06-03T02:22:29.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f51eaa00700 1 -- 172.21.15.79:0/1841949409 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc004020 con 0x5629ded95240 2024-06-03T02:22:29.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f51eaa00700 1 -- 172.21.15.79:0/1841949409 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f51f80a5030 con 0x5629ded95240 2024-06-03T02:22:29.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f5216519b80 10 client.25588 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:29.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f5216519b80 10 client.25588 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:22:29.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f5216519b80 20 client.25588 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:29.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f5216519b80 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5629dedaeb20 0x5629dedb0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f5216519b80 4 client.25588 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:22:29.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.280+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629decf4000 con 0x5629dedaeb20 2024-06-03T02:22:29.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.281+0000 7f5203e00700 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5629dedaeb20 0x5629dedb0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.283+0000 7f5203e00700 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5629dedaeb20 0x5629dedb0f80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f34af2a82dd9e841 server_cookie=6a0733c1d592af6f in_seq=0 out_seq=0 2024-06-03T02:22:29.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.283+0000 7f51eaa00700 10 client.25588 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:22:29.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.284+0000 7f51eaa00700 1 -- 172.21.15.79:0/1841949409 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5629decf4000 con 0x5629dedaeb20 2024-06-03T02:22:29.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.284+0000 7f51eaa00700 10 client.25588 handle_command_reply: tid=0 2024-06-03T02:22:29.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.285+0000 7f5216519b80 1 client.25588 shutdown 2024-06-03T02:22:29.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.285+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 msgr2=0x5629deda1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:29.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.285+0000 7f5216519b80 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629deda1200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51f80cdaf0 tx=0x7f51f8022770 comp rx=0 tx=0).stop 2024-06-03T02:22:29.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 shutdown_connections 2024-06-03T02:22:29.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f5216519b80 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 0x5629ded9ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f5216519b80 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5629dedaeb20 0x5629dedb0f80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f5216519b80 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629deda1200 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f5216519b80 1 --2- 172.21.15.79:0/1841949409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded9e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 >> 172.21.15.79:0/1841949409 conn(0x5629decc1170 msgr2=0x5629ded89790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:29.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f51eaa00700 0 client.25588 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:22:29.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 shutdown_connections 2024-06-03T02:22:29.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f5216519b80 1 -- 172.21.15.79:0/1841949409 wait complete. 2024-06-03T02:22:29.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.286+0000 7f5216519b80 20 client.25588 trim_cache size 0 max 16384 2024-06-03T02:22:29.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.322+0000 7f5216519b80 1 Processor -- start 2024-06-03T02:22:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.322+0000 7f5216519b80 1 -- start start 2024-06-03T02:22:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.323+0000 7f5216519b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629ded94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.323+0000 7f5216519b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded8a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.323+0000 7f5216519b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 0x5629ded8fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.323+0000 7f5216519b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52041b4910 con 0x5629ded9b4b0 2024-06-03T02:22:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.323+0000 7f5216519b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52041b6450 con 0x5629ded9ab90 2024-06-03T02:22:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.323+0000 7f5216519b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52041b4fe0 con 0x5629ded95240 2024-06-03T02:22:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.325+0000 7f5203e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded8a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.325+0000 7f5209c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 0x5629ded8fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.325+0000 7f5203e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded8a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43264/0 (socket says 172.21.15.79:43264) 2024-06-03T02:22:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.325+0000 7f5209200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629ded94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.325+0000 7f5209c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 0x5629ded8fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47308/0 (socket says 172.21.15.79:47308) 2024-06-03T02:22:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.325+0000 7f5209c00700 1 -- 172.21.15.79:0/2835357457 learned_addr learned my addr 172.21.15.79:0/2835357457 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:29.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.327+0000 7f5203e00700 1 -- 172.21.15.79:0/2835357457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 msgr2=0x5629ded94d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:29.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.329+0000 7f5203e00700 1 --2- 172.21.15.79:0/2835357457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629ded94d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.329+0000 7f5203e00700 1 -- 172.21.15.79:0/2835357457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 msgr2=0x5629ded8fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:29.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.329+0000 7f5203e00700 1 --2- 172.21.15.79:0/2835357457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 0x5629ded8fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.329+0000 7f5203e00700 1 -- 172.21.15.79:0/2835357457 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ded9e2d0 con 0x5629ded9ab90 2024-06-03T02:22:29.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.329+0000 7f5209200700 1 --2- 172.21.15.79:0/2835357457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629ded94d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:22:29.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.329+0000 7f5203e00700 1 --2- 172.21.15.79:0/2835357457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded8a130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5629dedb17c0 tx=0x7f51f0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48bc2f7552e1a7e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:29.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.330+0000 7f51f5600700 1 -- 172.21.15.79:0/2835357457 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0016070 con 0x5629ded9ab90 2024-06-03T02:22:29.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.330+0000 7f51f5600700 1 -- 172.21.15.79:0/2835357457 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f0004420 con 0x5629ded9ab90 2024-06-03T02:22:29.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.330+0000 7f51f5600700 1 -- 172.21.15.79:0/2835357457 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f001c950 con 0x5629ded9ab90 2024-06-03T02:22:29.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.331+0000 7f5216519b80 1 -- 172.21.15.79:0/2835357457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 msgr2=0x5629ded8a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:29.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.331+0000 7f5216519b80 1 --2- 172.21.15.79:0/2835357457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded8a130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5629dedb17c0 tx=0x7f51f0017000 comp rx=0 tx=0).stop 2024-06-03T02:22:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.333+0000 7f5216519b80 1 -- 172.21.15.79:0/2835357457 shutdown_connections 2024-06-03T02:22:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.333+0000 7f5216519b80 1 --2- 172.21.15.79:0/2835357457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded9b4b0 0x5629ded8fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.333+0000 7f5216519b80 1 --2- 172.21.15.79:0/2835357457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded95240 0x5629ded94d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.333+0000 7f5216519b80 1 --2- 172.21.15.79:0/2835357457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629ded8a130 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.333+0000 7f5216519b80 1 -- 172.21.15.79:0/2835357457 >> 172.21.15.79:0/2835357457 conn(0x5629decc1170 msgr2=0x5629ded9ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.334+0000 7f5216519b80 1 -- 172.21.15.79:0/2835357457 shutdown_connections 2024-06-03T02:22:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.334+0000 7f5216519b80 1 -- 172.21.15.79:0/2835357457 wait complete. 2024-06-03T02:22:29.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.334+0000 7f5216519b80 1 -- start start 2024-06-03T02:22:29.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.335+0000 7f5216519b80 1 Processor -- start 2024-06-03T02:22:29.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.335+0000 7f5216519b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629dee14a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.336+0000 7f5216519b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629dee14fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.336+0000 7f5216519b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9b4b0 0x5629deda7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.336+0000 7f5209200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629dee14a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.336+0000 7f5209200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629dee14a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47316/0 (socket says 172.21.15.79:47316) 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.338+0000 7f5209c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9b4b0 0x5629deda7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.338+0000 7f5209c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9b4b0 0x5629deda7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41872/0 (socket says 172.21.15.79:41872) 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.338+0000 7f5209c00700 1 -- 172.21.15.79:0/3366737037 learned_addr learned my addr 172.21.15.79:0/3366737037 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.338+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52041b5d80 con 0x5629ded95240 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.338+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52041b56b0 con 0x5629ded9ab90 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.338+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52041b28b0 con 0x5629ded9b4b0 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.338+0000 7f5216519b80 10 client.0 _ll_register_callbacks cb 0x5629dec7c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:29.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.338+0000 7f5203e00700 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629dee14fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.339+0000 7f5209c00700 1 -- 172.21.15.79:0/3366737037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 msgr2=0x5629dee14fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:29.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.339+0000 7f5209c00700 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629dee14fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.339+0000 7f5209c00700 1 -- 172.21.15.79:0/3366737037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 msgr2=0x5629dee14a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:29.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.339+0000 7f5209c00700 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629dee14a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.339+0000 7f5209c00700 1 -- 172.21.15.79:0/3366737037 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629deda77c0 con 0x5629ded9b4b0 2024-06-03T02:22:29.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.339+0000 7f5203e00700 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629dee14fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:22:29.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.340+0000 7f5209c00700 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9b4b0 0x5629deda7150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009b50 tx=0x7f51fc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e47e5e94b6f110f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:29.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.340+0000 7f51eaa00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:22:29.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.341+0000 7f51eaa00700 1 -- 172.21.15.79:0/3366737037 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0095e0 con 0x5629ded9b4b0 2024-06-03T02:22:29.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.341+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f0004420 con 0x5629ded9b4b0 2024-06-03T02:22:29.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.341+0000 7f51eaa00700 1 -- 172.21.15.79:0/3366737037 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51fc0045f0 con 0x5629ded9b4b0 2024-06-03T02:22:29.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.342+0000 7f51eaa00700 1 -- 172.21.15.79:0/3366737037 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51fc02e050 con 0x5629ded9b4b0 2024-06-03T02:22:29.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.342+0000 7f5216519b80 10 client.25594 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:29.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.342+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629dedb6950 con 0x5629ded9b4b0 2024-06-03T02:22:29.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.342+0000 7f51eaa00700 1 -- 172.21.15.79:0/3366737037 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc0047f0 con 0x5629ded9b4b0 2024-06-03T02:22:29.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.342+0000 7f51eaa00700 1 -- 172.21.15.79:0/3366737037 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f51fc0254a0 con 0x5629ded9b4b0 2024-06-03T02:22:29.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.343+0000 7f5216519b80 10 client.25594 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:29.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.343+0000 7f5216519b80 10 client.25594 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:22:29.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.343+0000 7f5216519b80 20 client.25594 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:29.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.343+0000 7f5216519b80 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5629dee2a830 0x5629dee2cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:29.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.343+0000 7f5216519b80 4 client.25594 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:22:29.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.343+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f51f0004420 con 0x5629dee2a830 2024-06-03T02:22:29.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.343+0000 7f5209200700 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5629dee2a830 0x5629dee2cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:29.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.345+0000 7f5209200700 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5629dee2a830 0x5629dee2cc20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c51c0205cf24c0c3 server_cookie=81c1d70bb315dfd in_seq=0 out_seq=0 2024-06-03T02:22:29.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.345+0000 7f51eaa00700 10 client.25594 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:22:29.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.345+0000 7f51eaa00700 1 -- 172.21.15.79:0/3366737037 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f51f0004420 con 0x5629dee2a830 2024-06-03T02:22:29.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.345+0000 7f51eaa00700 10 client.25594 handle_command_reply: tid=0 2024-06-03T02:22:29.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.345+0000 7f5216519b80 1 client.25594 shutdown 2024-06-03T02:22:29.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.346+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9b4b0 msgr2=0x5629deda7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:29.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.346+0000 7f5216519b80 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9b4b0 0x5629deda7150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009b50 tx=0x7f51fc004270 comp rx=0 tx=0).stop 2024-06-03T02:22:29.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.347+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 shutdown_connections 2024-06-03T02:22:29.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.347+0000 7f5216519b80 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5629dee2a830 0x5629dee2cc20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.347+0000 7f5216519b80 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629ded9b4b0 0x5629deda7150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.347+0000 7f5216519b80 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629ded95240 0x5629dee14a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.347+0000 7f5216519b80 1 --2- 172.21.15.79:0/3366737037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629ded9ab90 0x5629dee14fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.347+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 >> 172.21.15.79:0/3366737037 conn(0x5629decc1170 msgr2=0x5629ded8fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:29.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.347+0000 7f51eaa00700 0 client.25594 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:22:29.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.348+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 shutdown_connections 2024-06-03T02:22:29.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.348+0000 7f5216519b80 1 -- 172.21.15.79:0/3366737037 wait complete. 2024-06-03T02:22:29.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.348+0000 7f5216519b80 20 client.25594 trim_cache size 0 max 16384 2024-06-03T02:22:29.349 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:22:29.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.349+0000 7f520bc00700 1 -- 172.21.15.79:0/2868447065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51ec0a2df0 msgr2=0x7f51ec0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:29.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.349+0000 7f520bc00700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51ec0a2df0 0x7f51ec0a52b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f520410d090 tx=0x7f51f000a380 comp rx=0 tx=0).stop 2024-06-03T02:22:29.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.349+0000 7f520bc00700 1 -- 172.21.15.79:0/2868447065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 msgr2=0x7f520410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:29.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.349+0000 7f520bc00700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 0x7f520410e820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51f8006010 tx=0x7f51f80038b0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.350+0000 7f520bc00700 1 -- 172.21.15.79:0/2868447065 shutdown_connections 2024-06-03T02:22:29.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.350+0000 7f520bc00700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51ec0a2df0 0x7f51ec0a52b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.350+0000 7f520bc00700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520410ef40 0x7f520410be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.350+0000 7f520bc00700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520406a1a0 0x7f520410e820 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.350+0000 7f520bc00700 1 --2- 172.21.15.79:0/2868447065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f520406aac0 0x7f520410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:29.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.350+0000 7f520bc00700 1 -- 172.21.15.79:0/2868447065 >> 172.21.15.79:0/2868447065 conn(0x7f5204078340 msgr2=0x7f5204110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:29.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.350+0000 7f520bc00700 1 -- 172.21.15.79:0/2868447065 shutdown_connections 2024-06-03T02:22:29.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:29.350+0000 7f520bc00700 1 -- 172.21.15.79:0/2868447065 wait complete. 2024-06-03T02:22:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:29 smithi079 ceph-mon[31819]: pgmap v589: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.8 MiB/s wr, 688 op/s 2024-06-03T02:22:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:29 smithi135 ceph-mon[36653]: pgmap v589: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.8 MiB/s wr, 688 op/s 2024-06-03T02:22:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:29 smithi123 ceph-mon[36640]: pgmap v589: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.8 MiB/s wr, 688 op/s 2024-06-03T02:22:30.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:22:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:32.050 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:31 smithi079 ceph-mon[31819]: pgmap v590: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 MiB/s wr, 575 op/s 2024-06-03T02:22:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:31 smithi135 ceph-mon[36653]: pgmap v590: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 MiB/s wr, 575 op/s 2024-06-03T02:22:32.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:31 smithi123 ceph-mon[36640]: pgmap v590: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 MiB/s wr, 575 op/s 2024-06-03T02:22:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 145.0 s, 1005.5 tps, lat 99.678 ms stddev 123.401 2024-06-03T02:22:33.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:33 smithi079 ceph-mon[31819]: pgmap v591: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-06-03T02:22:34.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:33 smithi135 ceph-mon[36653]: pgmap v591: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-06-03T02:22:34.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.145+0000 7f0a62c00700 1 -- 172.21.15.79:0/2730177464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a5c102510 msgr2=0x7f0a5c102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:34.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.145+0000 7f0a62c00700 1 --2- 172.21.15.79:0/2730177464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a5c102510 0x7f0a5c102990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a480098f0 tx=0x7f0a48009c00 comp rx=0 tx=0).stop 2024-06-03T02:22:34.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.145+0000 7f0a62c00700 1 -- 172.21.15.79:0/2730177464 shutdown_connections 2024-06-03T02:22:34.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.145+0000 7f0a62c00700 1 --2- 172.21.15.79:0/2730177464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a5c102ed0 0x7f0a5c10f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.145+0000 7f0a62c00700 1 --2- 172.21.15.79:0/2730177464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a5c108520 0x7f0a5c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.145+0000 7f0a62c00700 1 --2- 172.21.15.79:0/2730177464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a5c102510 0x7f0a5c102990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.145+0000 7f0a62c00700 1 -- 172.21.15.79:0/2730177464 >> 172.21.15.79:0/2730177464 conn(0x7f0a5c079560 msgr2=0x7f0a5c079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:34.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.145+0000 7f0a62c00700 1 -- 172.21.15.79:0/2730177464 shutdown_connections 2024-06-03T02:22:34.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.145+0000 7f0a62c00700 1 -- 172.21.15.79:0/2730177464 wait complete. 2024-06-03T02:22:34.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.146+0000 7f0a62c00700 1 Processor -- start 2024-06-03T02:22:34.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.146+0000 7f0a62c00700 1 -- start start 2024-06-03T02:22:34.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.147+0000 7f0a62c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a5c102510 0x7f0a5c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.147+0000 7f0a62c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a5c102ed0 0x7f0a5c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.148+0000 7f0a5b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a5c102ed0 0x7f0a5c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.148+0000 7f0a5be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a5c102510 0x7f0a5c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.148+0000 7f0a5b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a5c102ed0 0x7f0a5c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43292/0 (socket says 172.21.15.79:43292) 2024-06-03T02:22:34.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.148+0000 7f0a62c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a5c108520 0x7f0a5c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.149+0000 7f0a5b400700 1 -- 172.21.15.79:0/1673502887 learned_addr learned my addr 172.21.15.79:0/1673502887 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:34.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.149+0000 7f0a60c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a5c108520 0x7f0a5c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.150+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1127f0 con 0x7f0a5c102510 2024-06-03T02:22:34.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.150+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c112670 con 0x7f0a5c102ed0 2024-06-03T02:22:34.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.150+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c112970 con 0x7f0a5c108520 2024-06-03T02:22:34.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.151+0000 7f0a60c00700 1 -- 172.21.15.79:0/1673502887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a5c102ed0 msgr2=0x7f0a5c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:34.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.151+0000 7f0a60c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a5c102ed0 0x7f0a5c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.151+0000 7f0a60c00700 1 -- 172.21.15.79:0/1673502887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a5c102510 msgr2=0x7f0a5c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:34.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.151+0000 7f0a60c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a5c102510 0x7f0a5c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.151+0000 7f0a60c00700 1 -- 172.21.15.79:0/1673502887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c1a7180 con 0x7f0a5c108520 2024-06-03T02:22:34.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.151+0000 7f0a5b400700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a5c102ed0 0x7f0a5c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:22:34.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.152+0000 7f0a60c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a5c108520 0x7f0a5c075210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400efc0 tx=0x7f0a5400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:34.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.152+0000 7f0a58c00700 1 -- 172.21.15.79:0/1673502887 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540092e0 con 0x7f0a5c108520 2024-06-03T02:22:34.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.152+0000 7f0a58c00700 1 -- 172.21.15.79:0/1673502887 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a54004bb0 con 0x7f0a5c108520 2024-06-03T02:22:34.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.152+0000 7f0a58c00700 1 -- 172.21.15.79:0/1673502887 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54004280 con 0x7f0a5c108520 2024-06-03T02:22:34.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.152+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c1a7410 con 0x7f0a5c108520 2024-06-03T02:22:34.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.153+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c1a78d0 con 0x7f0a5c108520 2024-06-03T02:22:34.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.158+0000 7f0a58c00700 1 -- 172.21.15.79:0/1673502887 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0a5401b030 con 0x7f0a5c108520 2024-06-03T02:22:34.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.159+0000 7f0a6d678b80 1 Processor -- start 2024-06-03T02:22:34.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.159+0000 7f0a6d678b80 1 -- start start 2024-06-03T02:22:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea60fe40 0x5588ea693070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea693a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5588ea5a2a10 con 0x5588ea60fe40 2024-06-03T02:22:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5588ea5e5b00 con 0x5588ea6a6b10 2024-06-03T02:22:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5588ea5a08f0 con 0x5588ea6935b0 2024-06-03T02:22:34.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a5be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea693a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a5be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea693a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41892/0 (socket says 172.21.15.79:41892) 2024-06-03T02:22:34.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a5be00700 1 -- 172.21.15.79:0/3585040183 learned_addr learned my addr 172.21.15.79:0/3585040183 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:34.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a5b400700 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.160+0000 7f0a5b400700 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea60fe40 0x5588ea693070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.161+0000 7f0a58c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a400a2df0 0x7f0a400a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.161+0000 7f0a5b400700 1 -- 172.21.15.79:0/3585040183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 msgr2=0x5588ea693a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:34.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.161+0000 7f0a5b400700 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea693a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.162+0000 7f0a5b400700 1 -- 172.21.15.79:0/3585040183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea60fe40 msgr2=0x5588ea693070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:34.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.162+0000 7f0a5b400700 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea60fe40 0x5588ea693070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.162+0000 7f0a5b400700 1 -- 172.21.15.79:0/3585040183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ea58df90 con 0x5588ea6a6b10 2024-06-03T02:22:34.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.162+0000 7f0a5be00700 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea693a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:22:34.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.162+0000 7f0a58c00700 1 -- 172.21.15.79:0/1673502887 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0a540ced90 con 0x7f0a5c108520 2024-06-03T02:22:34.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.163+0000 7f0a5b400700 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a8f00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009810 tx=0x7f0a48019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4d5efee5faa374 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:34.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.163+0000 7f0a60c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a400a2df0 0x7f0a400a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.164+0000 7f0a51600700 1 -- 172.21.15.79:0/3585040183 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800f850 con 0x5588ea6a6b10 2024-06-03T02:22:34.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.164+0000 7f0a51600700 1 -- 172.21.15.79:0/3585040183 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4800f9f0 con 0x5588ea6a6b10 2024-06-03T02:22:34.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.164+0000 7f0a51600700 1 -- 172.21.15.79:0/3585040183 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48003c80 con 0x5588ea6a6b10 2024-06-03T02:22:34.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.165+0000 7f0a6d678b80 1 -- 172.21.15.79:0/3585040183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 msgr2=0x5588ea6a8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:34.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.165+0000 7f0a60c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a400a2df0 0x7f0a400a52b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009fb0 tx=0x7f0a540cefe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:34.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.165+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a8f00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009810 tx=0x7f0a48019040 comp rx=0 tx=0).stop 2024-06-03T02:22:34.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.166+0000 7f0a6d678b80 1 -- 172.21.15.79:0/3585040183 shutdown_connections 2024-06-03T02:22:34.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.166+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a8f00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.166+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea60fe40 0x5588ea693070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.166+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/3585040183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea693a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.166+0000 7f0a6d678b80 1 -- 172.21.15.79:0/3585040183 >> 172.21.15.79:0/3585040183 conn(0x5588ea5c0170 msgr2=0x5588ea68f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:34.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.166+0000 7f0a6d678b80 1 -- 172.21.15.79:0/3585040183 shutdown_connections 2024-06-03T02:22:34.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.166+0000 7f0a6d678b80 1 -- 172.21.15.79:0/3585040183 wait complete. 2024-06-03T02:22:34.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.167+0000 7f0a6d678b80 1 -- start start 2024-06-03T02:22:34.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.167+0000 7f0a6d678b80 1 Processor -- start 2024-06-03T02:22:34.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea60fe40 0x5588ea6a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea6a3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5588ea581f60 con 0x5588ea60fe40 2024-06-03T02:22:34.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5588ea5e1ac0 con 0x5588ea6a6b10 2024-06-03T02:22:34.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5588ea6a9770 con 0x5588ea6935b0 2024-06-03T02:22:34.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a6d678b80 10 client.0 _ll_register_callbacks cb 0x5588ea57f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:34.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a5be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea6a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a5be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea6a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41904/0 (socket says 172.21.15.79:41904) 2024-06-03T02:22:34.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a5be00700 1 -- 172.21.15.79:0/2772487113 learned_addr learned my addr 172.21.15.79:0/2772487113 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:34.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.168+0000 7f0a5b400700 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.169+0000 7f0a5be00700 1 -- 172.21.15.79:0/2772487113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 msgr2=0x5588ea6a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:34.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.169+0000 7f0a5be00700 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.169+0000 7f0a5be00700 1 -- 172.21.15.79:0/2772487113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea60fe40 msgr2=0x5588ea6a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:34.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.169+0000 7f0a5be00700 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea60fe40 0x5588ea6a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.169+0000 7f0a5be00700 1 -- 172.21.15.79:0/2772487113 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ea5348d0 con 0x5588ea6935b0 2024-06-03T02:22:34.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.170+0000 7f0a5be00700 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea6a3620 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c011830 tx=0x7f0a4c011b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2fc3e04d0aa47d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:34.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.171+0000 7f0a46000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:22:34.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.171+0000 7f0a46000700 1 -- 172.21.15.79:0/2772487113 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c01f060 con 0x5588ea6935b0 2024-06-03T02:22:34.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.171+0000 7f0a46000700 1 -- 172.21.15.79:0/2772487113 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0a4c004260 con 0x5588ea6935b0 2024-06-03T02:22:34.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.171+0000 7f0a6d678b80 1 -- 172.21.15.79:0/2772487113 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a4800f9f0 con 0x5588ea6935b0 2024-06-03T02:22:34.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.172+0000 7f0a46000700 1 -- 172.21.15.79:0/2772487113 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c004460 con 0x5588ea6935b0 2024-06-03T02:22:34.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.172+0000 7f0a6d678b80 10 client.25606 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:34.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.172+0000 7f0a6d678b80 1 -- 172.21.15.79:0/2772487113 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5588ea6b3ff0 con 0x5588ea6935b0 2024-06-03T02:22:34.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.173+0000 7f0a46000700 1 -- 172.21.15.79:0/2772487113 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4800f9f0 con 0x5588ea6935b0 2024-06-03T02:22:34.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.173+0000 7f0a46000700 1 -- 172.21.15.79:0/2772487113 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0a4c00f9b0 con 0x5588ea6935b0 2024-06-03T02:22:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.173+0000 7f0a6d678b80 10 client.25606 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.173+0000 7f0a6d678b80 10 client.25606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:22:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.173+0000 7f0a6d678b80 20 client.25606 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.173+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5588ea6b6580 0x5588ea6b8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.173+0000 7f0a6d678b80 4 client.25606 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:22:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.173+0000 7f0a6d678b80 1 -- 172.21.15.79:0/2772487113 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588ea5f3000 con 0x5588ea6b6580 2024-06-03T02:22:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.173+0000 7f0a5b400700 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5588ea6b6580 0x5588ea6b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.174+0000 7f0a46000700 10 client.25606 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.177+0000 7f0a5b400700 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5588ea6b6580 0x5588ea6b8970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=430b841f45f15954 server_cookie=104022e284644f4b in_seq=0 out_seq=0 2024-06-03T02:22:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.178+0000 7f0a46000700 1 -- 172.21.15.79:0/2772487113 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588ea5f3000 con 0x5588ea6b6580 2024-06-03T02:22:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.178+0000 7f0a46000700 10 client.25606 handle_command_reply: tid=0 2024-06-03T02:22:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.178+0000 7f0a6d678b80 1 client.25606 shutdown 2024-06-03T02:22:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 -- 172.21.15.79:0/2772487113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 msgr2=0x5588ea6a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea6a3620 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c011830 tx=0x7f0a4c011b40 comp rx=0 tx=0).stop 2024-06-03T02:22:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 -- 172.21.15.79:0/2772487113 shutdown_connections 2024-06-03T02:22:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5588ea6b6580 0x5588ea6b8970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea60fe40 0x5588ea6a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/2772487113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea6a3620 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 -- 172.21.15.79:0/2772487113 >> 172.21.15.79:0/2772487113 conn(0x5588ea5c0170 msgr2=0x5588ea68e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a46000700 0 client.25606 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 -- 172.21.15.79:0/2772487113 shutdown_connections 2024-06-03T02:22:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 1 -- 172.21.15.79:0/2772487113 wait complete. 2024-06-03T02:22:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.179+0000 7f0a6d678b80 20 client.25606 trim_cache size 0 max 16384 2024-06-03T02:22:34.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.212+0000 7f0a6d678b80 1 Processor -- start 2024-06-03T02:22:34.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.212+0000 7f0a6d678b80 1 -- start start 2024-06-03T02:22:34.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.213+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea60fe40 0x5588ea6a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.213+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea6a7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.213+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6a6b10 0x5588ea6a8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.213+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c070560 con 0x5588ea6a6b10 2024-06-03T02:22:34.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.213+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c0720a0 con 0x5588ea60fe40 2024-06-03T02:22:34.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.213+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c070c30 con 0x5588ea6935b0 2024-06-03T02:22:34.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.213+0000 7f0a5b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea60fe40 0x5588ea6a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.214+0000 7f0a5b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea60fe40 0x5588ea6a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43312/0 (socket says 172.21.15.79:43312) 2024-06-03T02:22:34.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.214+0000 7f0a5b400700 1 -- 172.21.15.79:0/1626176012 learned_addr learned my addr 172.21.15.79:0/1626176012 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:34.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.214+0000 7f0a5b400700 1 -- 172.21.15.79:0/1626176012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 msgr2=0x5588ea6a7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:34.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.214+0000 7f0a5b400700 1 --2- 172.21.15.79:0/1626176012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea6a7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.214+0000 7f0a5b400700 1 -- 172.21.15.79:0/1626176012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6a6b10 msgr2=0x5588ea6a8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:34.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.214+0000 7f0a5b400700 1 --2- 172.21.15.79:0/1626176012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6a6b10 0x5588ea6a8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.214+0000 7f0a5b400700 1 -- 172.21.15.79:0/1626176012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ea6a33f0 con 0x5588ea60fe40 2024-06-03T02:22:34.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.214+0000 7f0a5b400700 1 --2- 172.21.15.79:0/1626176012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea60fe40 0x5588ea6a5e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0a48005e90 tx=0x7f0a4800fba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=621af657a7dba8ab server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:34.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.219+0000 7f0a50c00700 1 -- 172.21.15.79:0/1626176012 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800f970 con 0x5588ea60fe40 2024-06-03T02:22:34.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.219+0000 7f0a50c00700 1 -- 172.21.15.79:0/1626176012 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a480055c0 con 0x5588ea60fe40 2024-06-03T02:22:34.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.219+0000 7f0a50c00700 1 -- 172.21.15.79:0/1626176012 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800f970 con 0x5588ea60fe40 2024-06-03T02:22:34.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.220+0000 7f0a6d678b80 1 -- 172.21.15.79:0/1626176012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea60fe40 msgr2=0x5588ea6a5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:34.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.220+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/1626176012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea60fe40 0x5588ea6a5e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0a48005e90 tx=0x7f0a4800fba0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.220+0000 7f0a6d678b80 1 -- 172.21.15.79:0/1626176012 shutdown_connections 2024-06-03T02:22:34.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.220+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/1626176012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6a6b10 0x5588ea6a8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.220+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/1626176012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea60fe40 0x5588ea6a5e60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.220+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/1626176012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea6935b0 0x5588ea6a7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.220+0000 7f0a6d678b80 1 -- 172.21.15.79:0/1626176012 >> 172.21.15.79:0/1626176012 conn(0x5588ea5c0170 msgr2=0x5588ea6a1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:34.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.221+0000 7f0a6d678b80 1 -- 172.21.15.79:0/1626176012 shutdown_connections 2024-06-03T02:22:34.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.221+0000 7f0a6d678b80 1 -- 172.21.15.79:0/1626176012 wait complete. 2024-06-03T02:22:34.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.221+0000 7f0a6d678b80 1 -- start start 2024-06-03T02:22:34.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.222+0000 7f0a6d678b80 1 Processor -- start 2024-06-03T02:22:34.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:33 smithi123 ceph-mon[36640]: pgmap v591: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-06-03T02:22:34.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.223+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea60fe40 0x5588ea6aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.223+0000 7f0a5b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea60fe40 0x5588ea6aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.224+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6935b0 0x5588ea6aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.224+0000 7f0a6d678b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.224+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c0719d0 con 0x5588ea6935b0 2024-06-03T02:22:34.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.224+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c071300 con 0x5588ea6a6b10 2024-06-03T02:22:34.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.224+0000 7f0a6d678b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c06e500 con 0x5588ea60fe40 2024-06-03T02:22:34.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.224+0000 7f0a5b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea60fe40 0x5588ea6aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41936/0 (socket says 172.21.15.79:41936) 2024-06-03T02:22:34.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.224+0000 7f0a5b400700 1 -- 172.21.15.79:0/938935 learned_addr learned my addr 172.21.15.79:0/938935 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:34.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.224+0000 7f0a6d678b80 10 client.0 _ll_register_callbacks cb 0x5588ea57f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:34.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.225+0000 7f0a5be00700 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6935b0 0x5588ea6aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.225+0000 7f0a5b400700 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.225+0000 7f0a5be00700 1 -- 172.21.15.79:0/938935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea60fe40 msgr2=0x5588ea6aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:34.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.225+0000 7f0a5be00700 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea60fe40 0x5588ea6aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.226+0000 7f0a5be00700 1 -- 172.21.15.79:0/938935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 msgr2=0x5588ea6a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:34.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.226+0000 7f0a5be00700 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.226+0000 7f0a5be00700 1 -- 172.21.15.79:0/938935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ea6af580 con 0x5588ea6935b0 2024-06-03T02:22:34.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.226+0000 7f0a5b400700 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:34.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.228+0000 7f0a5be00700 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6935b0 0x5588ea6aef40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5588ea6b31a0 tx=0x7f0a4c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15899e59a5faff1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:34.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.228+0000 7f0a46000700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:22:34.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.229+0000 7f0a46000700 1 -- 172.21.15.79:0/938935 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c029080 con 0x5588ea6935b0 2024-06-03T02:22:34.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.229+0000 7f0a46000700 1 -- 172.21.15.79:0/938935 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0a4c0114f0 con 0x5588ea6935b0 2024-06-03T02:22:34.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.229+0000 7f0a6d678b80 1 -- 172.21.15.79:0/938935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a480055c0 con 0x5588ea6935b0 2024-06-03T02:22:34.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.231+0000 7f0a46000700 1 -- 172.21.15.79:0/938935 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c00f5c0 con 0x5588ea6935b0 2024-06-03T02:22:34.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.231+0000 7f0a6d678b80 10 client.15423 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:34.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.231+0000 7f0a6d678b80 1 -- 172.21.15.79:0/938935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5588ea6bd790 con 0x5588ea6935b0 2024-06-03T02:22:34.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.232+0000 7f0a46000700 1 -- 172.21.15.79:0/938935 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a480055c0 con 0x5588ea6935b0 2024-06-03T02:22:34.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.234+0000 7f0a46000700 1 -- 172.21.15.79:0/938935 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0a4c023650 con 0x5588ea6935b0 2024-06-03T02:22:34.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.234+0000 7f0a6d678b80 10 client.15423 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:34.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.234+0000 7f0a6d678b80 10 client.15423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:22:34.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.234+0000 7f0a6d678b80 20 client.15423 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:34.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.234+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5588ea6bdda0 0x5588ea7231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:34.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.234+0000 7f0a6d678b80 4 client.15423 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:22:34.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.234+0000 7f0a6d678b80 1 -- 172.21.15.79:0/938935 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588ea5ed580 con 0x5588ea6bdda0 2024-06-03T02:22:34.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.234+0000 7f0a5b400700 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5588ea6bdda0 0x5588ea7231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:34.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.235+0000 7f0a5b400700 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5588ea6bdda0 0x5588ea7231f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7041221dc9977a server_cookie=fa5a82312eccc9bf in_seq=0 out_seq=0 2024-06-03T02:22:34.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.235+0000 7f0a46000700 10 client.15423 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:34.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.235+0000 7f0a46000700 1 -- 172.21.15.79:0/938935 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588ea5ed580 con 0x5588ea6bdda0 2024-06-03T02:22:34.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.235+0000 7f0a46000700 10 client.15423 handle_command_reply: tid=0 2024-06-03T02:22:34.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a6d678b80 1 client.15423 shutdown 2024-06-03T02:22:34.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a6d678b80 1 -- 172.21.15.79:0/938935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6935b0 msgr2=0x5588ea6aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:34.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6935b0 0x5588ea6aef40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5588ea6b31a0 tx=0x7f0a4c005a00 comp rx=0 tx=0).stop 2024-06-03T02:22:34.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a6d678b80 1 -- 172.21.15.79:0/938935 shutdown_connections 2024-06-03T02:22:34.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588ea6a6b10 0x5588ea6a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588ea60fe40 0x5588ea6aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5588ea6bdda0 0x5588ea7231f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a6d678b80 1 --2- 172.21.15.79:0/938935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588ea6935b0 0x5588ea6aef40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a6d678b80 1 -- 172.21.15.79:0/938935 >> 172.21.15.79:0/938935 conn(0x5588ea5c0170 msgr2=0x5588ea6a1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:34.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.236+0000 7f0a46000700 0 client.15423 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:34.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.237+0000 7f0a6d678b80 1 -- 172.21.15.79:0/938935 shutdown_connections 2024-06-03T02:22:34.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.237+0000 7f0a6d678b80 1 -- 172.21.15.79:0/938935 wait complete. 2024-06-03T02:22:34.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.237+0000 7f0a6d678b80 20 client.15423 trim_cache size 0 max 16384 2024-06-03T02:22:34.238 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:22:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.238+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a400a2df0 msgr2=0x7f0a400a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.238+0000 7f0a62c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a400a2df0 0x7f0a400a52b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009fb0 tx=0x7f0a540cefe0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.238+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a5c108520 msgr2=0x7f0a5c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.238+0000 7f0a62c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a5c108520 0x7f0a5c075210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400efc0 tx=0x7f0a5400c3a0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.239+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 shutdown_connections 2024-06-03T02:22:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.239+0000 7f0a62c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a5c108520 0x7f0a5c075210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.239+0000 7f0a62c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a400a2df0 0x7f0a400a52b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.239+0000 7f0a62c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a5c102510 0x7f0a5c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.239+0000 7f0a62c00700 1 --2- 172.21.15.79:0/1673502887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a5c102ed0 0x7f0a5c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:34.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.240+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 >> 172.21.15.79:0/1673502887 conn(0x7f0a5c079560 msgr2=0x7f0a5c101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:34.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.240+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 shutdown_connections 2024-06-03T02:22:34.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:34.240+0000 7f0a62c00700 1 -- 172.21.15.79:0/1673502887 wait complete. 2024-06-03T02:22:36.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:35 smithi079 ceph-mon[31819]: pgmap v592: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.6 MiB/s wr, 594 op/s 2024-06-03T02:22:36.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:35 smithi135 ceph-mon[36653]: pgmap v592: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.6 MiB/s wr, 594 op/s 2024-06-03T02:22:36.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:35 smithi123 ceph-mon[36640]: pgmap v592: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.6 MiB/s wr, 594 op/s 2024-06-03T02:22:36.250 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:22:36.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:22:36.897 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:37.992 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:37 smithi079 ceph-mon[31819]: pgmap v593: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.2 MiB/s wr, 562 op/s 2024-06-03T02:22:38.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:37 smithi135 ceph-mon[36653]: pgmap v593: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.2 MiB/s wr, 562 op/s 2024-06-03T02:22:38.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:37 smithi123 ceph-mon[36640]: pgmap v593: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.2 MiB/s wr, 562 op/s 2024-06-03T02:22:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 150.0 s, 1017.4 tps, lat 98.335 ms stddev 124.040 2024-06-03T02:22:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.466+0000 7f3754800700 1 -- 172.21.15.79:0/2767444455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f374c104ed0 msgr2=0x7f374c105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.466+0000 7f3754800700 1 --2- 172.21.15.79:0/2767444455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f374c104ed0 0x7f374c105360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3748009f90 tx=0x7f374800b330 comp rx=0 tx=0).stop 2024-06-03T02:22:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.466+0000 7f3754800700 1 -- 172.21.15.79:0/2767444455 shutdown_connections 2024-06-03T02:22:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.466+0000 7f3754800700 1 --2- 172.21.15.79:0/2767444455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f374c104ed0 0x7f374c105360 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.466+0000 7f3754800700 1 --2- 172.21.15.79:0/2767444455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f374c1073c0 0x7f374c06a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.466+0000 7f3754800700 1 --2- 172.21.15.79:0/2767444455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f374c06a6e0 0x7f374c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.466+0000 7f3754800700 1 -- 172.21.15.79:0/2767444455 >> 172.21.15.79:0/2767444455 conn(0x7f374c078340 msgr2=0x7f374c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.466+0000 7f3754800700 1 -- 172.21.15.79:0/2767444455 shutdown_connections 2024-06-03T02:22:39.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.466+0000 7f3754800700 1 -- 172.21.15.79:0/2767444455 wait complete. 2024-06-03T02:22:39.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.468+0000 7f3754800700 1 Processor -- start 2024-06-03T02:22:39.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.468+0000 7f3754800700 1 -- start start 2024-06-03T02:22:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.469+0000 7f3754800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f374c06a6e0 0x7f374c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.469+0000 7f3754800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f374c104ed0 0x7f374c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.469+0000 7f3754800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f374c1073c0 0x7f374c1a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.469+0000 7f3754800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f374c118da0 con 0x7f374c104ed0 2024-06-03T02:22:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.469+0000 7f3754800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f374c118c20 con 0x7f374c06a6e0 2024-06-03T02:22:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.469+0000 7f3754800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f374c118f20 con 0x7f374c1073c0 2024-06-03T02:22:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.469+0000 7f3752800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f374c1073c0 0x7f374c1a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.470+0000 7f3752800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f374c1073c0 0x7f374c1a3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:34420/0 (socket says 172.21.15.79:34420) 2024-06-03T02:22:39.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.470+0000 7f3752800700 1 -- 172.21.15.79:0/3218344887 learned_addr learned my addr 172.21.15.79:0/3218344887 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:39.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.470+0000 7f3751400700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f374c104ed0 0x7f374c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.470+0000 7f3752800700 1 -- 172.21.15.79:0/3218344887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f374c06a6e0 msgr2=0x7f374c19f230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:22:39.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.471+0000 7f3751e00700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f374c06a6e0 0x7f374c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.471+0000 7f3752800700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f374c06a6e0 0x7f374c19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.471+0000 7f3752800700 1 -- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f374c104ed0 msgr2=0x7f374c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:39.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.471+0000 7f3752800700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f374c104ed0 0x7f374c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.471+0000 7f3752800700 1 -- 172.21.15.79:0/3218344887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c1a41f0 con 0x7f374c1073c0 2024-06-03T02:22:39.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.471+0000 7f3751400700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f374c104ed0 0x7f374c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:22:39.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.471+0000 7f3751e00700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f374c06a6e0 0x7f374c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:39.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.472+0000 7f3752800700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f374c1073c0 0x7f374c1a3ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3748009550 tx=0x7f3748009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:39.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.475+0000 7f3746a00700 1 -- 172.21.15.79:0/3218344887 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748021070 con 0x7f374c1073c0 2024-06-03T02:22:39.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.475+0000 7f3746a00700 1 -- 172.21.15.79:0/3218344887 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3748007b40 con 0x7f374c1073c0 2024-06-03T02:22:39.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.475+0000 7f3746a00700 1 -- 172.21.15.79:0/3218344887 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480052a0 con 0x7f374c1073c0 2024-06-03T02:22:39.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.475+0000 7f3754800700 1 -- 172.21.15.79:0/3218344887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c1a4480 con 0x7f374c1073c0 2024-06-03T02:22:39.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.475+0000 7f3754800700 1 -- 172.21.15.79:0/3218344887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c1abd60 con 0x7f374c1073c0 2024-06-03T02:22:39.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.480+0000 7f3746a00700 1 -- 172.21.15.79:0/3218344887 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3748024070 con 0x7f374c1073c0 2024-06-03T02:22:39.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.482+0000 7f3746a00700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37340a2ca0 0x7f37340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.482+0000 7f3746a00700 1 -- 172.21.15.79:0/3218344887 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f374800f020 con 0x7f374c1073c0 2024-06-03T02:22:39.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.482+0000 7f3751e00700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37340a2ca0 0x7f37340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.483+0000 7f375f0c3b80 1 Processor -- start 2024-06-03T02:22:39.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f3751e00700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37340a2ca0 0x7f37340a5160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3740009bc0 tx=0x7f3740009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:39.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f375f0c3b80 1 -- start start 2024-06-03T02:22:39.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9bd9630 0x5629b9bd9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9be4380 0x5629b9be6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5629b9ae8a90 con 0x5629b9bd9630 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5629b9b2bb80 con 0x5629b9b55ec0 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5629b9ae6970 con 0x5629b9be4380 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f3752800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9be4380 0x5629b9be6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f3752800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9be4380 0x5629b9be6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:34424/0 (socket says 172.21.15.79:34424) 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.484+0000 7f3752800700 1 -- 172.21.15.79:0/3388982240 learned_addr learned my addr 172.21.15.79:0/3388982240 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.485+0000 7f3751e00700 1 --2- 172.21.15.79:0/3388982240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.485+0000 7f3752800700 1 -- 172.21.15.79:0/3388982240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 msgr2=0x5629b9bd90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.485+0000 7f3752800700 1 --2- 172.21.15.79:0/3388982240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.485+0000 7f3752800700 1 -- 172.21.15.79:0/3388982240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9bd9630 msgr2=0x5629b9bd9ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:22:39.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.485+0000 7f3752800700 1 --2- 172.21.15.79:0/3388982240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9bd9630 0x5629b9bd9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.485+0000 7f3752800700 1 -- 172.21.15.79:0/3388982240 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b9ad4010 con 0x5629b9be4380 2024-06-03T02:22:39.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.485+0000 7f3752800700 1 --2- 172.21.15.79:0/3388982240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9be4380 0x5629b9be6770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f374809c9e0 tx=0x7f3748006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35ba285fca798629 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:39.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.485+0000 7f373b400700 1 -- 172.21.15.79:0/3388982240 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480b2020 con 0x5629b9be4380 2024-06-03T02:22:39.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.486+0000 7f373b400700 1 -- 172.21.15.79:0/3388982240 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37480a1090 con 0x5629b9be4380 2024-06-03T02:22:39.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.486+0000 7f373b400700 1 -- 172.21.15.79:0/3388982240 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374809cf40 con 0x5629b9be4380 2024-06-03T02:22:39.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.486+0000 7f375f0c3b80 1 -- 172.21.15.79:0/3388982240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9be4380 msgr2=0x5629b9be6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:39.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.486+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/3388982240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9be4380 0x5629b9be6770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f374809c9e0 tx=0x7f3748006000 comp rx=0 tx=0).stop 2024-06-03T02:22:39.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.487+0000 7f375f0c3b80 1 -- 172.21.15.79:0/3388982240 shutdown_connections 2024-06-03T02:22:39.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.487+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/3388982240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9be4380 0x5629b9be6770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.487+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/3388982240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.487+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/3388982240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9bd9630 0x5629b9bd9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.487+0000 7f375f0c3b80 1 -- 172.21.15.79:0/3388982240 >> 172.21.15.79:0/3388982240 conn(0x5629b9b061f0 msgr2=0x5629b9bd5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:39.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.490+0000 7f375f0c3b80 1 -- 172.21.15.79:0/3388982240 shutdown_connections 2024-06-03T02:22:39.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.490+0000 7f375f0c3b80 1 -- 172.21.15.79:0/3388982240 wait complete. 2024-06-03T02:22:39.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.491+0000 7f375f0c3b80 1 -- start start 2024-06-03T02:22:39.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.492+0000 7f375f0c3b80 1 Processor -- start 2024-06-03T02:22:39.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.492+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9b55ec0 0x5629b9be3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.492+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9bd9630 0x5629b9be0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.492+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9be4380 0x5629b9be1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.492+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5629b9ac7fe0 con 0x5629b9be4380 2024-06-03T02:22:39.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.492+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5629b9b27b40 con 0x5629b9bd9630 2024-06-03T02:22:39.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.492+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5629b9be6df0 con 0x5629b9b55ec0 2024-06-03T02:22:39.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.492+0000 7f375f0c3b80 10 client.0 _ll_register_callbacks cb 0x5629b9ac57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:39.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.493+0000 7f3751400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9bd9630 0x5629b9be0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.493+0000 7f3751400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9bd9630 0x5629b9be0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:46650/0 (socket says 172.21.15.79:46650) 2024-06-03T02:22:39.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.493+0000 7f3751400700 1 -- 172.21.15.79:0/1840897672 learned_addr learned my addr 172.21.15.79:0/1840897672 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:39.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.493+0000 7f3751400700 1 -- 172.21.15.79:0/1840897672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9b55ec0 msgr2=0x5629b9be3bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:22:39.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.493+0000 7f3751400700 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9b55ec0 0x5629b9be3bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.493+0000 7f3751400700 1 -- 172.21.15.79:0/1840897672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9be4380 msgr2=0x5629b9be1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:39.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.493+0000 7f3751400700 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9be4380 0x5629b9be1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.493+0000 7f3751400700 1 -- 172.21.15.79:0/1840897672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b9a7a950 con 0x5629b9bd9630 2024-06-03T02:22:39.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.494+0000 7f3751400700 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9bd9630 0x5629b9be0ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f373c009fd0 tx=0x7f373c011d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa266d6ad91c1da7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:39.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.494+0000 7f372fe00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:22:39.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.494+0000 7f372fe00700 1 -- 172.21.15.79:0/1840897672 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c00fc80 con 0x5629b9bd9630 2024-06-03T02:22:39.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.494+0000 7f372fe00700 1 -- 172.21.15.79:0/1840897672 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f373c00fe20 con 0x5629b9bd9630 2024-06-03T02:22:39.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.494+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1840897672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37480a1090 con 0x5629b9bd9630 2024-06-03T02:22:39.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.495+0000 7f372fe00700 1 -- 172.21.15.79:0/1840897672 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f373c024050 con 0x5629b9bd9630 2024-06-03T02:22:39.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.495+0000 7f375f0c3b80 10 client.25631 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:39.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.495+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1840897672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629b9bf1600 con 0x5629b9bd9630 2024-06-03T02:22:39.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.495+0000 7f372fe00700 1 -- 172.21.15.79:0/1840897672 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37480a1090 con 0x5629b9bd9630 2024-06-03T02:22:39.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.496+0000 7f372fe00700 1 -- 172.21.15.79:0/1840897672 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f373c00e5b0 con 0x5629b9bd9630 2024-06-03T02:22:39.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.496+0000 7f375f0c3b80 10 client.25631 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:39.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.496+0000 7f375f0c3b80 10 client.25631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:22:39.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.496+0000 7f375f0c3b80 20 client.25631 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:39.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.496+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5629b9bf3ce0 0x5629b9bf6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.496+0000 7f375f0c3b80 4 client.25631 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:22:39.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.496+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1840897672 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629b9b39080 con 0x5629b9bf3ce0 2024-06-03T02:22:39.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.497+0000 7f3751e00700 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5629b9bf3ce0 0x5629b9bf6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.498+0000 7f3751e00700 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5629b9bf3ce0 0x5629b9bf6180 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64b5c3cc9fbdee29 server_cookie=b8b175bc328e48a1 in_seq=0 out_seq=0 2024-06-03T02:22:39.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.498+0000 7f372fe00700 10 client.25631 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:39.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.498+0000 7f372fe00700 1 -- 172.21.15.79:0/1840897672 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5629b9b39080 con 0x5629b9bf3ce0 2024-06-03T02:22:39.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.498+0000 7f372fe00700 10 client.25631 handle_command_reply: tid=0 2024-06-03T02:22:39.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.498+0000 7f375f0c3b80 1 client.25631 shutdown 2024-06-03T02:22:39.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.499+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1840897672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9bd9630 msgr2=0x5629b9be0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:39.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.499+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9bd9630 0x5629b9be0ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f373c009fd0 tx=0x7f373c011d40 comp rx=0 tx=0).stop 2024-06-03T02:22:39.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1840897672 shutdown_connections 2024-06-03T02:22:39.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5629b9bf3ce0 0x5629b9bf6180 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9be4380 0x5629b9be1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9b55ec0 0x5629b9be3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1840897672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9bd9630 0x5629b9be0ef0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1840897672 >> 172.21.15.79:0/1840897672 conn(0x5629b9b061f0 msgr2=0x5629b9bd47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:39.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f372fe00700 0 client.25631 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:39.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1840897672 shutdown_connections 2024-06-03T02:22:39.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1840897672 wait complete. 2024-06-03T02:22:39.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.500+0000 7f375f0c3b80 20 client.25631 trim_cache size 0 max 16384 2024-06-03T02:22:39.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.541+0000 7f375f0c3b80 1 Processor -- start 2024-06-03T02:22:39.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.541+0000 7f375f0c3b80 1 -- start start 2024-06-03T02:22:39.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.542+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.542+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 0x5629b9bd4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.542+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9be4380 0x5629b9be3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.542+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b4930 con 0x5629b9be4380 2024-06-03T02:22:39.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.542+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b6470 con 0x5629b9b55ec0 2024-06-03T02:22:39.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.542+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b5000 con 0x5629b9bd9630 2024-06-03T02:22:39.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.542+0000 7f3751e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.542+0000 7f3751e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:46666/0 (socket says 172.21.15.79:46666) 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.542+0000 7f3751e00700 1 -- 172.21.15.79:0/2819282323 learned_addr learned my addr 172.21.15.79:0/2819282323 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.543+0000 7f3751400700 1 --2- 172.21.15.79:0/2819282323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 0x5629b9bd4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.543+0000 7f3752800700 1 --2- 172.21.15.79:0/2819282323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9be4380 0x5629b9be3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.543+0000 7f3751e00700 1 -- 172.21.15.79:0/2819282323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 msgr2=0x5629b9bd4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.543+0000 7f3751e00700 1 --2- 172.21.15.79:0/2819282323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 0x5629b9bd4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.543+0000 7f3751e00700 1 -- 172.21.15.79:0/2819282323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9be4380 msgr2=0x5629b9be3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.543+0000 7f3751e00700 1 --2- 172.21.15.79:0/2819282323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9be4380 0x5629b9be3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.543+0000 7f3751e00700 1 -- 172.21.15.79:0/2819282323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b9be1a30 con 0x5629b9b55ec0 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.543+0000 7f3751e00700 1 --2- 172.21.15.79:0/2819282323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd56a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f374000f6f0 tx=0x7f3740015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e979298bfd4c6817 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:39.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.544+0000 7f373aa00700 1 -- 172.21.15.79:0/2819282323 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740011bc0 con 0x5629b9b55ec0 2024-06-03T02:22:39.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.544+0000 7f373aa00700 1 -- 172.21.15.79:0/2819282323 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5629b9b39080 con 0x5629b9b55ec0 2024-06-03T02:22:39.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.544+0000 7f373aa00700 1 -- 172.21.15.79:0/2819282323 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374001f720 con 0x5629b9b55ec0 2024-06-03T02:22:39.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.545+0000 7f375f0c3b80 1 -- 172.21.15.79:0/2819282323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 msgr2=0x5629b9bd56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:39.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.545+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/2819282323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd56a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f374000f6f0 tx=0x7f3740015000 comp rx=0 tx=0).stop 2024-06-03T02:22:39.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.545+0000 7f375f0c3b80 1 -- 172.21.15.79:0/2819282323 shutdown_connections 2024-06-03T02:22:39.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.545+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/2819282323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9be4380 0x5629b9be3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.545+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/2819282323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9b55ec0 0x5629b9bd56a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.545+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/2819282323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 0x5629b9bd4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.545+0000 7f375f0c3b80 1 -- 172.21.15.79:0/2819282323 >> 172.21.15.79:0/2819282323 conn(0x5629b9b061f0 msgr2=0x5629b9bd3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:39.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.545+0000 7f375f0c3b80 1 -- 172.21.15.79:0/2819282323 shutdown_connections 2024-06-03T02:22:39.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.545+0000 7f375f0c3b80 1 -- 172.21.15.79:0/2819282323 wait complete. 2024-06-03T02:22:39.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.546+0000 7f375f0c3b80 1 -- start start 2024-06-03T02:22:39.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.546+0000 7f375f0c3b80 1 Processor -- start 2024-06-03T02:22:39.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.547+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9b55ec0 0x5629b9bec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.548+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 0x5629b9be7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.548+0000 7f375f0c3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9be4380 0x5629b9be7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.548+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b5da0 con 0x5629b9b55ec0 2024-06-03T02:22:39.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.548+0000 7f3752800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9be4380 0x5629b9be7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.548+0000 7f3752800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9be4380 0x5629b9be7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:46670/0 (socket says 172.21.15.79:46670) 2024-06-03T02:22:39.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.548+0000 7f3752800700 1 -- 172.21.15.79:0/1540858593 learned_addr learned my addr 172.21.15.79:0/1540858593 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:39.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.548+0000 7f375f0c3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b56d0 con 0x5629b9be4380 2024-06-03T02:22:39.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.549+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1540858593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b28d0 con 0x5629b9bd9630 2024-06-03T02:22:39.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.549+0000 7f3751400700 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 0x5629b9be7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.549+0000 7f3752800700 1 -- 172.21.15.79:0/1540858593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 msgr2=0x5629b9be7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:39.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.549+0000 7f3752800700 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 0x5629b9be7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.549+0000 7f3752800700 1 -- 172.21.15.79:0/1540858593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9b55ec0 msgr2=0x5629b9bec700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:39.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.549+0000 7f3751e00700 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9b55ec0 0x5629b9bec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.550+0000 7f3752800700 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9b55ec0 0x5629b9bec700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.550+0000 7f375f0c3b80 10 client.0 _ll_register_callbacks cb 0x5629b9ac57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:39.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.550+0000 7f3752800700 1 -- 172.21.15.79:0/1540858593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b9be7c50 con 0x5629b9be4380 2024-06-03T02:22:39.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.550+0000 7f3751e00700 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9b55ec0 0x5629b9bec700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:39.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.550+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1540858593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629b9b39080 con 0x5629b9be4380 2024-06-03T02:22:39.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.551+0000 7f3752800700 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9be4380 0x5629b9be7540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37480cdf40 tx=0x7f374801db20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f35daf95c9f24f21 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:39.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.551+0000 7f372fe00700 10 client.25637 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:22:39.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.551+0000 7f372fe00700 1 -- 172.21.15.79:0/1540858593 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480ba060 con 0x5629b9be4380 2024-06-03T02:22:39.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.551+0000 7f372fe00700 1 -- 172.21.15.79:0/1540858593 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f374802edb0 con 0x5629b9be4380 2024-06-03T02:22:39.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.551+0000 7f372fe00700 1 -- 172.21.15.79:0/1540858593 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37480b2040 con 0x5629b9be4380 2024-06-03T02:22:39.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.551+0000 7f375f0c3b80 10 client.25637 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:39.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.551+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1540858593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5629b9c5f830 con 0x5629b9be4380 2024-06-03T02:22:39.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.552+0000 7f372fe00700 1 -- 172.21.15.79:0/1540858593 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3748011070 con 0x5629b9be4380 2024-06-03T02:22:39.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.554+0000 7f375f0c3b80 10 client.25637 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:39.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.554+0000 7f375f0c3b80 10 client.25637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:22:39.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.554+0000 7f375f0c3b80 20 client.25637 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:39.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.554+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5629b9c5faa0 0x5629b9c61e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:39.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.554+0000 7f375f0c3b80 4 client.25637 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:22:39.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.554+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1540858593 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5629b9b33600 con 0x5629b9c5faa0 2024-06-03T02:22:39.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.555+0000 7f3751e00700 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5629b9c5faa0 0x5629b9c61e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:39.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.556+0000 7f3751e00700 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5629b9c5faa0 0x5629b9c61e90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37aab93474216b6 server_cookie=b1b3c5965c15e479 in_seq=0 out_seq=0 2024-06-03T02:22:39.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.556+0000 7f372fe00700 10 client.25637 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:39.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f372fe00700 1 -- 172.21.15.79:0/1540858593 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629b9b33600 con 0x5629b9c5faa0 2024-06-03T02:22:39.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f372fe00700 10 client.25637 handle_command_reply: tid=0 2024-06-03T02:22:39.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f375f0c3b80 1 client.25637 shutdown 2024-06-03T02:22:39.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1540858593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9be4380 msgr2=0x5629b9be7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:39.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9be4380 0x5629b9be7540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37480cdf40 tx=0x7f374801db20 comp rx=0 tx=0).stop 2024-06-03T02:22:39.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1540858593 shutdown_connections 2024-06-03T02:22:39.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629b9be4380 0x5629b9be7540 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5629b9c5faa0 0x5629b9c61e90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5629b9b55ec0 0x5629b9bec700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f375f0c3b80 1 --2- 172.21.15.79:0/1540858593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629b9bd9630 0x5629b9be7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f372fe00700 0 client.25637 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:39.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.557+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1540858593 >> 172.21.15.79:0/1540858593 conn(0x5629b9b061f0 msgr2=0x5629b9bd47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:39.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.558+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1540858593 shutdown_connections 2024-06-03T02:22:39.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.558+0000 7f375f0c3b80 1 -- 172.21.15.79:0/1540858593 wait complete. 2024-06-03T02:22:39.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.558+0000 7f375f0c3b80 20 client.25637 trim_cache size 0 max 16384 2024-06-03T02:22:39.559 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:22:39.559 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:22:39.560 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "83cc8837-928c-49cf-b839-ad5b2270bc9b", 2024-06-03T02:22:39.560 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:22:39.560 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:22:39.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 -- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37340a2ca0 msgr2=0x7f37340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:39.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37340a2ca0 0x7f37340a5160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3740009bc0 tx=0x7f3740009260 comp rx=0 tx=0).stop 2024-06-03T02:22:39.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 -- 172.21.15.79:0/3218344887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f374c1073c0 msgr2=0x7f374c1a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:39.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f374c1073c0 0x7f374c1a3ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3748009550 tx=0x7f3748009ba0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 -- 172.21.15.79:0/3218344887 shutdown_connections 2024-06-03T02:22:39.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f374c1073c0 0x7f374c1a3ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f374c06a6e0 0x7f374c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37340a2ca0 0x7f37340a5160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 --2- 172.21.15.79:0/3218344887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f374c104ed0 0x7f374c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:39.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.560+0000 7f3754800700 1 -- 172.21.15.79:0/3218344887 >> 172.21.15.79:0/3218344887 conn(0x7f374c078340 msgr2=0x7f374c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:39.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.561+0000 7f3754800700 1 -- 172.21.15.79:0/3218344887 shutdown_connections 2024-06-03T02:22:39.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:39.561+0000 7f3754800700 1 -- 172.21.15.79:0/3218344887 wait complete. 2024-06-03T02:22:40.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:39 smithi079 ceph-mon[31819]: pgmap v594: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 MiB/s wr, 698 op/s 2024-06-03T02:22:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:39 smithi135 ceph-mon[36653]: pgmap v594: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 MiB/s wr, 698 op/s 2024-06-03T02:22:40.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:39 smithi123 ceph-mon[36640]: pgmap v594: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 MiB/s wr, 698 op/s 2024-06-03T02:22:40.578 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:22:41.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:40 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:22:41.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:40 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:22:41.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:40 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:22:41.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:40 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:22:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:40 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:22:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:40 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:22:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:40 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:22:41.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:40 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:22:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:40 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:22:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:40 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:22:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:40 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:22:41.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:40 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:22:41.248 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:41 smithi079 ceph-mon[31819]: pgmap v595: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-06-03T02:22:42.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:41 smithi135 ceph-mon[36653]: pgmap v595: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-06-03T02:22:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:41 smithi123 ceph-mon[36640]: pgmap v595: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-06-03T02:22:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 155.0 s, 1051.0 tps, lat 95.496 ms stddev 106.177 2024-06-03T02:22:44.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:43 smithi123 ceph-mon[36640]: pgmap v596: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 MiB/s wr, 683 op/s 2024-06-03T02:22:44.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:43 smithi079 ceph-mon[31819]: pgmap v596: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 MiB/s wr, 683 op/s 2024-06-03T02:22:44.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:43 smithi135 ceph-mon[36653]: pgmap v596: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 MiB/s wr, 683 op/s 2024-06-03T02:22:45.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.030+0000 7fa991200700 1 -- 172.21.15.79:0/1119922657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c10ee90 msgr2=0x7fa98c111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:45.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.030+0000 7fa991200700 1 --2- 172.21.15.79:0/1119922657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c10ee90 0x7fa98c111280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa98400b0a0 tx=0x7fa98400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.032 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:44 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:22:45.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.032+0000 7fa991200700 1 -- 172.21.15.79:0/1119922657 shutdown_connections 2024-06-03T02:22:45.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.032+0000 7fa991200700 1 --2- 172.21.15.79:0/1119922657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c10ee90 0x7fa98c111280 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.032+0000 7fa991200700 1 --2- 172.21.15.79:0/1119922657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0746c0 0x7fa98c074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.032+0000 7fa991200700 1 --2- 172.21.15.79:0/1119922657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa98c074fe0 0x7fa98c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.032+0000 7fa991200700 1 -- 172.21.15.79:0/1119922657 >> 172.21.15.79:0/1119922657 conn(0x7fa98c0795f0 msgr2=0x7fa98c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:45.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.032+0000 7fa991200700 1 -- 172.21.15.79:0/1119922657 shutdown_connections 2024-06-03T02:22:45.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.032+0000 7fa991200700 1 -- 172.21.15.79:0/1119922657 wait complete. 2024-06-03T02:22:45.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.034+0000 7fa991200700 1 Processor -- start 2024-06-03T02:22:45.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa991200700 1 -- start start 2024-06-03T02:22:45.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa991200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c0746c0 0x7fa98c10e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa991200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa98c074fe0 0x7fa98c1053e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa991200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c10ee90 0x7fa98c105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa991200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c104130 con 0x7fa98c10ee90 2024-06-03T02:22:45.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa991200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c103fb0 con 0x7fa98c0746c0 2024-06-03T02:22:45.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa991200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1042b0 con 0x7fa98c074fe0 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa989e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c0746c0 0x7fa98c10e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa989e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c0746c0 0x7fa98c10e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36778/0 (socket says 172.21.15.79:36778) 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.035+0000 7fa989e00700 1 -- 172.21.15.79:0/1164732332 learned_addr learned my addr 172.21.15.79:0/1164732332 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.036+0000 7fa989e00700 1 -- 172.21.15.79:0/1164732332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa98c074fe0 msgr2=0x7fa98c1053e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.036+0000 7fa989e00700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa98c074fe0 0x7fa98c1053e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.036+0000 7fa989e00700 1 -- 172.21.15.79:0/1164732332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c10ee90 msgr2=0x7fa98c105920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.036+0000 7fa989e00700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c10ee90 0x7fa98c105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.036+0000 7fa989e00700 1 -- 172.21.15.79:0/1164732332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c105ee0 con 0x7fa98c0746c0 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.037+0000 7fa989e00700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c0746c0 0x7fa98c10e420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00e9a0 tx=0x7fa97c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.037+0000 7fa982a00700 1 -- 172.21.15.79:0/1164732332 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00cbd0 con 0x7fa98c0746c0 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.037+0000 7fa982a00700 1 -- 172.21.15.79:0/1164732332 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa97c00cd70 con 0x7fa98c0746c0 2024-06-03T02:22:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.037+0000 7fa982a00700 1 -- 172.21.15.79:0/1164732332 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0187c0 con 0x7fa98c0746c0 2024-06-03T02:22:45.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.037+0000 7fa991200700 1 -- 172.21.15.79:0/1164732332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c1061d0 con 0x7fa98c0746c0 2024-06-03T02:22:45.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.037+0000 7fa991200700 1 -- 172.21.15.79:0/1164732332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c10a190 con 0x7fa98c0746c0 2024-06-03T02:22:45.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.047+0000 7fa982a00700 1 -- 172.21.15.79:0/1164732332 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa97c010430 con 0x7fa98c0746c0 2024-06-03T02:22:45.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.048+0000 7fa99bb5bb80 1 Processor -- start 2024-06-03T02:22:45.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.048+0000 7fa99bb5bb80 1 -- start start 2024-06-03T02:22:45.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.049+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 0x559d0d13b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.049+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 0x559d0d13ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.049+0000 7fa982a00700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9700a2ca0 0x7fa9700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.050+0000 7fa982a00700 1 -- 172.21.15.79:0/1164732332 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa97c0cf660 con 0x7fa98c0746c0 2024-06-03T02:22:45.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.050+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d146300 0x559d0d1486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.050+0000 7fa99bb5bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559d0d04aa10 con 0x559d0d0b7e40 2024-06-03T02:22:45.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.050+0000 7fa99bb5bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559d0d08db00 con 0x559d0d146300 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.050+0000 7fa99bb5bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559d0d0488f0 con 0x559d0d13b5b0 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.050+0000 7fa98a800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d146300 0x559d0d1486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.050+0000 7fa98a800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d146300 0x559d0d1486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36794/0 (socket says 172.21.15.79:36794) 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.050+0000 7fa98a800700 1 -- 172.21.15.79:0/374384628 learned_addr learned my addr 172.21.15.79:0/374384628 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.051+0000 7fa98a800700 1 --2- 172.21.15.79:0/374384628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 0x559d0d13b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.051+0000 7fa98a800700 1 -- 172.21.15.79:0/374384628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 msgr2=0x559d0d13ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.051+0000 7fa98a800700 1 --2- 172.21.15.79:0/374384628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 0x559d0d13ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.051+0000 7fa98a800700 1 -- 172.21.15.79:0/374384628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 msgr2=0x559d0d13b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.051+0000 7fa98a800700 1 --2- 172.21.15.79:0/374384628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 0x559d0d13b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.051+0000 7fa98a800700 1 -- 172.21.15.79:0/374384628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0d035f90 con 0x559d0d146300 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.052+0000 7fa98a800700 1 --2- 172.21.15.79:0/374384628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d146300 0x559d0d1486f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa984009e60 tx=0x7fa984009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d758477aec1c07ef server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.052+0000 7fa977400700 1 -- 172.21.15.79:0/374384628 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98400cb40 con 0x559d0d146300 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.052+0000 7fa977400700 1 -- 172.21.15.79:0/374384628 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa984023030 con 0x559d0d146300 2024-06-03T02:22:45.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.052+0000 7fa977400700 1 -- 172.21.15.79:0/374384628 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984007bb0 con 0x559d0d146300 2024-06-03T02:22:45.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.053+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/374384628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d146300 msgr2=0x559d0d1486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:45.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.053+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/374384628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d146300 0x559d0d1486f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa984009e60 tx=0x7fa984009ba0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.054+0000 7fa989400700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9700a2ca0 0x7fa9700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.055+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/374384628 shutdown_connections 2024-06-03T02:22:45.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.055+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/374384628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d146300 0x559d0d1486f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.055+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/374384628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 0x559d0d13b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.055+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/374384628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 0x559d0d13ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.055+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/374384628 >> 172.21.15.79:0/374384628 conn(0x559d0d068170 msgr2=0x559d0d1374b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:45.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.055+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/374384628 shutdown_connections 2024-06-03T02:22:45.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.055+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/374384628 wait complete. 2024-06-03T02:22:45.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.056+0000 7fa99bb5bb80 1 -- start start 2024-06-03T02:22:45.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.056+0000 7fa99bb5bb80 1 Processor -- start 2024-06-03T02:22:45.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.057+0000 7fa989400700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9700a2ca0 0x7fa9700a5160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa98c106ba0 tx=0x7fa978009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:45.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.057+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 0x559d0d145a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.057+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d142d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.057+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d146300 0x559d0d143280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.058+0000 7fa99bb5bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559d0d029f60 con 0x559d0d0b7e40 2024-06-03T02:22:45.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.058+0000 7fa99bb5bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559d0d089ac0 con 0x559d0d13b5b0 2024-06-03T02:22:45.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.058+0000 7fa99bb5bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559d0d148f60 con 0x559d0d146300 2024-06-03T02:22:45.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.058+0000 7fa98a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d146300 0x559d0d143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.058+0000 7fa98a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d146300 0x559d0d143280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:57702/0 (socket says 172.21.15.79:57702) 2024-06-03T02:22:45.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.058+0000 7fa98a800700 1 -- 172.21.15.79:0/3622334723 learned_addr learned my addr 172.21.15.79:0/3622334723 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:45.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.058+0000 7fa989400700 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.059+0000 7fa99bb5bb80 10 client.0 _ll_register_callbacks cb 0x559d0d027720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:45.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.059+0000 7fa989e00700 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 0x559d0d145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.059+0000 7fa989400700 1 -- 172.21.15.79:0/3622334723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d146300 msgr2=0x559d0d143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:45.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.059+0000 7fa989400700 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d146300 0x559d0d143280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.059+0000 7fa989400700 1 -- 172.21.15.79:0/3622334723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 msgr2=0x559d0d145a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:45.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.060+0000 7fa989400700 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 0x559d0d145a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.060+0000 7fa989400700 1 -- 172.21.15.79:0/3622334723 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0cfdc8d0 con 0x559d0d13b5b0 2024-06-03T02:22:45.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.061+0000 7fa989e00700 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 0x559d0d145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:45.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.061+0000 7fa989400700 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d142d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa97800ef50 tx=0x7fa9780047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e27a6308183494a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:45.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.061+0000 7fa96be00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:22:45.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.061+0000 7fa96be00700 1 -- 172.21.15.79:0/3622334723 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978020070 con 0x559d0d13b5b0 2024-06-03T02:22:45.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.061+0000 7fa96be00700 1 -- 172.21.15.79:0/3622334723 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa978025660 con 0x559d0d13b5b0 2024-06-03T02:22:45.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.061+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3622334723 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa984023030 con 0x559d0d13b5b0 2024-06-03T02:22:45.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.062+0000 7fa96be00700 1 -- 172.21.15.79:0/3622334723 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa978010e60 con 0x559d0d13b5b0 2024-06-03T02:22:45.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.062+0000 7fa99bb5bb80 10 client.25649 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.062+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3622334723 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d0d1536c0 con 0x559d0d13b5b0 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.062+0000 7fa96be00700 1 -- 172.21.15.79:0/3622334723 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa984023030 con 0x559d0d13b5b0 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.062+0000 7fa96be00700 1 -- 172.21.15.79:0/3622334723 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa97801cac0 con 0x559d0d13b5b0 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.062+0000 7fa99bb5bb80 10 client.25649 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.062+0000 7fa99bb5bb80 10 client.25649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.063+0000 7fa99bb5bb80 20 client.25649 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.063+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559d0d155d00 0x559d0d1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.063+0000 7fa99bb5bb80 4 client.25649 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.063+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3622334723 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d0d09b000 con 0x559d0d155d00 2024-06-03T02:22:45.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.063+0000 7fa989e00700 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559d0d155d00 0x559d0d1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.065+0000 7fa989e00700 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559d0d155d00 0x559d0d1580f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4b7a22b953a0ee server_cookie=bacad07f47545026 in_seq=0 out_seq=0 2024-06-03T02:22:45.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.066+0000 7fa96be00700 10 client.25649 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:45.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.066+0000 7fa96be00700 1 -- 172.21.15.79:0/3622334723 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559d0d09b000 con 0x559d0d155d00 2024-06-03T02:22:45.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.066+0000 7fa96be00700 10 client.25649 handle_command_reply: tid=0 2024-06-03T02:22:45.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.066+0000 7fa99bb5bb80 1 client.25649 shutdown 2024-06-03T02:22:45.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.070+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3622334723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 msgr2=0x559d0d142d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:45.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.070+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d142d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa97800ef50 tx=0x7fa9780047c0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.070+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3622334723 shutdown_connections 2024-06-03T02:22:45.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.070+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d146300 0x559d0d143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.070+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559d0d155d00 0x559d0d1580f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.070+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d0b7e40 0x559d0d145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.070+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3622334723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d142d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.070+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3622334723 >> 172.21.15.79:0/3622334723 conn(0x559d0d068170 msgr2=0x559d0d136740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:45.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.070+0000 7fa96be00700 0 client.25649 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:45.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.072+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3622334723 shutdown_connections 2024-06-03T02:22:45.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.072+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3622334723 wait complete. 2024-06-03T02:22:45.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.072+0000 7fa99bb5bb80 20 client.25649 trim_cache size 0 max 16384 2024-06-03T02:22:45.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.113+0000 7fa99bb5bb80 1 Processor -- start 2024-06-03T02:22:45.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.113+0000 7fa99bb5bb80 1 -- start start 2024-06-03T02:22:45.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d0b7e40 0x559d0d145580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d1476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 0x559d0d1520b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa989400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d1476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa989400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d1476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36816/0 (socket says 172.21.15.79:36816) 2024-06-03T02:22:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa989400700 1 -- 172.21.15.79:0/3061944162 learned_addr learned my addr 172.21.15.79:0/3061944162 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa989e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d0b7e40 0x559d0d145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3061944162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1bd370 con 0x559d0d146300 2024-06-03T02:22:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3061944162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1beeb0 con 0x559d0d13b5b0 2024-06-03T02:22:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.114+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3061944162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1bda40 con 0x559d0d0b7e40 2024-06-03T02:22:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.115+0000 7fa98a800700 1 --2- 172.21.15.79:0/3061944162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 0x559d0d1520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.115+0000 7fa989e00700 1 -- 172.21.15.79:0/3061944162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 msgr2=0x559d0d1476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.115+0000 7fa989e00700 1 --2- 172.21.15.79:0/3061944162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d1476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.115+0000 7fa989e00700 1 -- 172.21.15.79:0/3061944162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 msgr2=0x559d0d1520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.115+0000 7fa989e00700 1 --2- 172.21.15.79:0/3061944162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 0x559d0d1520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.115+0000 7fa989e00700 1 -- 172.21.15.79:0/3061944162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0d138080 con 0x559d0d0b7e40 2024-06-03T02:22:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.116+0000 7fa989e00700 1 --2- 172.21.15.79:0/3061944162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d0b7e40 0x559d0d145580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x559d0d138920 tx=0x7fa97c010e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfe390858ac24f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.116+0000 7fa976a00700 1 -- 172.21.15.79:0/3061944162 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0b3030 con 0x559d0d0b7e40 2024-06-03T02:22:45.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.116+0000 7fa976a00700 1 -- 172.21.15.79:0/3061944162 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa97c09bb10 con 0x559d0d0b7e40 2024-06-03T02:22:45.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.116+0000 7fa976a00700 1 -- 172.21.15.79:0/3061944162 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c098180 con 0x559d0d0b7e40 2024-06-03T02:22:45.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.117+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3061944162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d0b7e40 msgr2=0x559d0d145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:45.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.118+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3061944162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d0b7e40 0x559d0d145580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x559d0d138920 tx=0x7fa97c010e30 comp rx=0 tx=0).stop 2024-06-03T02:22:45.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.120+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3061944162 shutdown_connections 2024-06-03T02:22:45.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.120+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3061944162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 0x559d0d1520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.120+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3061944162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d0b7e40 0x559d0d145580 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.120+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3061944162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d13b5b0 0x559d0d1476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.120+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3061944162 >> 172.21.15.79:0/3061944162 conn(0x559d0d068170 msgr2=0x559d0d136fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:45.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.120+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3061944162 shutdown_connections 2024-06-03T02:22:45.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.120+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3061944162 wait complete. 2024-06-03T02:22:45.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.120+0000 7fa99bb5bb80 1 -- start start 2024-06-03T02:22:45.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.121+0000 7fa99bb5bb80 1 Processor -- start 2024-06-03T02:22:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.121+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d0b7e40 0x559d0d15cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.121+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 0x559d0d15d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.121+0000 7fa99bb5bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 0x559d0d14d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.121+0000 7fa99bb5bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1be7e0 con 0x559d0d146300 2024-06-03T02:22:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.121+0000 7fa99bb5bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1be110 con 0x559d0d0b7e40 2024-06-03T02:22:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.121+0000 7fa99bb5bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1bb310 con 0x559d0d13b5b0 2024-06-03T02:22:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.121+0000 7fa99bb5bb80 10 client.0 _ll_register_callbacks cb 0x559d0d027720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 0x559d0d15d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 0x559d0d15d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:57728/0 (socket says 172.21.15.79:57728) 2024-06-03T02:22:45.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989400700 1 -- 172.21.15.79:0/3874908863 learned_addr learned my addr 172.21.15.79:0/3874908863 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989e00700 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d0b7e40 0x559d0d15cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989400700 1 -- 172.21.15.79:0/3874908863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d0b7e40 msgr2=0x559d0d15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989400700 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d0b7e40 0x559d0d15cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989400700 1 -- 172.21.15.79:0/3874908863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 msgr2=0x559d0d14d220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa98a800700 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 0x559d0d14d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989400700 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 0x559d0d14d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989400700 1 -- 172.21.15.79:0/3874908863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0d14d760 con 0x559d0d13b5b0 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa989e00700 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d0b7e40 0x559d0d15cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.122+0000 7fa98a800700 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 0x559d0d14d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.123+0000 7fa989400700 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 0x559d0d15d0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa97801cee0 tx=0x7fa97801cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92af28ee089600cd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.123+0000 7fa96be00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:22:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.123+0000 7fa96be00700 1 -- 172.21.15.79:0/3874908863 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978020040 con 0x559d0d13b5b0 2024-06-03T02:22:45.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.123+0000 7fa96be00700 1 -- 172.21.15.79:0/3874908863 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa978004a30 con 0x559d0d13b5b0 2024-06-03T02:22:45.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.123+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3874908863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa97c09bb10 con 0x559d0d13b5b0 2024-06-03T02:22:45.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa96be00700 1 -- 172.21.15.79:0/3874908863 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa978009350 con 0x559d0d13b5b0 2024-06-03T02:22:45.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa99bb5bb80 10 client.25636 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:45.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3874908863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d0d14eb00 con 0x559d0d13b5b0 2024-06-03T02:22:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa96be00700 1 -- 172.21.15.79:0/3874908863 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa978005500 con 0x559d0d13b5b0 2024-06-03T02:22:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa96be00700 1 -- 172.21.15.79:0/3874908863 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa97800fa40 con 0x559d0d13b5b0 2024-06-03T02:22:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa99bb5bb80 10 client.25636 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa99bb5bb80 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:22:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa99bb5bb80 20 client.25636 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559d0d1c0a20 0x559d0d1c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa99bb5bb80 4 client.25636 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:22:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.124+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3874908863 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa97c09bb10 con 0x559d0d1c0a20 2024-06-03T02:22:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.125+0000 7fa989e00700 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559d0d1c0a20 0x559d0d1c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:45.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.125+0000 7fa96be00700 10 client.25636 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:45.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.125+0000 7fa989e00700 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559d0d1c0a20 0x559d0d1c2e10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4110b466db25e277 server_cookie=c527c00854c4f508 in_seq=0 out_seq=0 2024-06-03T02:22:45.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.126+0000 7fa96be00700 1 -- 172.21.15.79:0/3874908863 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fa97c09bb10 con 0x559d0d1c0a20 2024-06-03T02:22:45.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.126+0000 7fa96be00700 10 client.25636 handle_command_reply: tid=0 2024-06-03T02:22:45.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.126+0000 7fa99bb5bb80 1 client.25636 shutdown 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.126+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3874908863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 msgr2=0x559d0d15d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.126+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 0x559d0d15d0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa97801cee0 tx=0x7fa97801cd10 comp rx=0 tx=0).stop 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.126+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3874908863 shutdown_connections 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.126+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559d0d1c0a20 0x559d0d1c2e10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.127+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559d0d146300 0x559d0d14d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.127+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d0d0b7e40 0x559d0d15cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.127+0000 7fa99bb5bb80 1 --2- 172.21.15.79:0/3874908863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d0d13b5b0 0x559d0d15d0d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.127+0000 7fa96be00700 0 client.25636 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.127+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3874908863 >> 172.21.15.79:0/3874908863 conn(0x559d0d068170 msgr2=0x559d0d1415c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.127+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3874908863 shutdown_connections 2024-06-03T02:22:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.127+0000 7fa99bb5bb80 1 -- 172.21.15.79:0/3874908863 wait complete. 2024-06-03T02:22:45.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.127+0000 7fa99bb5bb80 20 client.25636 trim_cache size 0 max 16384 2024-06-03T02:22:45.129 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:22:45.129 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:22:45.129 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:22:45.129 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 -- 172.21.15.79:0/1164732332 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9700a2ca0 msgr2=0x7fa9700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9700a2ca0 0x7fa9700a5160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa98c106ba0 tx=0x7fa978009260 comp rx=0 tx=0).stop 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 -- 172.21.15.79:0/1164732332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c0746c0 msgr2=0x7fa98c10e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c0746c0 0x7fa98c10e420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00e9a0 tx=0x7fa97c00ed60 comp rx=0 tx=0).stop 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 -- 172.21.15.79:0/1164732332 shutdown_connections 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa9700a2ca0 0x7fa9700a5160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c10ee90 0x7fa98c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c0746c0 0x7fa98c10e420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 --2- 172.21.15.79:0/1164732332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa98c074fe0 0x7fa98c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.128+0000 7fa991200700 1 -- 172.21.15.79:0/1164732332 >> 172.21.15.79:0/1164732332 conn(0x7fa98c0795f0 msgr2=0x7fa98c0ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.129+0000 7fa991200700 1 -- 172.21.15.79:0/1164732332 shutdown_connections 2024-06-03T02:22:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:45.129+0000 7fa991200700 1 -- 172.21.15.79:0/1164732332 wait complete. 2024-06-03T02:22:45.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:44 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:22:45.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:44 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:22:46.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:45 smithi123 ceph-mon[36640]: pgmap v597: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-06-03T02:22:46.331 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:45 smithi079 ceph-mon[31819]: pgmap v597: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-06-03T02:22:46.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:45 smithi135 ceph-mon[36653]: pgmap v597: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-06-03T02:22:48.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:47 smithi123 ceph-mon[36640]: pgmap v598: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 MiB/s wr, 570 op/s 2024-06-03T02:22:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 160.0 s, 928.8 tps, lat 106.715 ms stddev 143.170 2024-06-03T02:22:48.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:47 smithi079 ceph-mon[31819]: pgmap v598: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 MiB/s wr, 570 op/s 2024-06-03T02:22:48.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:47 smithi135 ceph-mon[36653]: pgmap v598: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 MiB/s wr, 570 op/s 2024-06-03T02:22:48.548 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:22:48.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:22:49.217 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:50.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:49 smithi079 ceph-mon[31819]: pgmap v599: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 648 op/s 2024-06-03T02:22:50.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:49 smithi123 ceph-mon[36640]: pgmap v599: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 648 op/s 2024-06-03T02:22:50.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:49 smithi135 ceph-mon[36653]: pgmap v599: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 648 op/s 2024-06-03T02:22:51.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.673+0000 7f1630400700 1 -- 172.21.15.79:0/3125320490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 msgr2=0x7f16281088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:51.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.673+0000 7f1630400700 1 --2- 172.21.15.79:0/3125320490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 0x7f16281088e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f161c0098f0 tx=0x7f161c009c00 comp rx=0 tx=0).stop 2024-06-03T02:22:51.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.674+0000 7f1630400700 1 -- 172.21.15.79:0/3125320490 shutdown_connections 2024-06-03T02:22:51.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.674+0000 7f1630400700 1 --2- 172.21.15.79:0/3125320490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f162806ab30 0x7f162810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:51.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.674+0000 7f1630400700 1 --2- 172.21.15.79:0/3125320490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 0x7f16281088e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:51.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.674+0000 7f1630400700 1 --2- 172.21.15.79:0/3125320490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f162806a170 0x7f162806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:51.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.674+0000 7f1630400700 1 -- 172.21.15.79:0/3125320490 >> 172.21.15.79:0/3125320490 conn(0x7f1628078340 msgr2=0x7f1628078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:51.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.674+0000 7f1630400700 1 -- 172.21.15.79:0/3125320490 shutdown_connections 2024-06-03T02:22:51.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.674+0000 7f1630400700 1 -- 172.21.15.79:0/3125320490 wait complete. 2024-06-03T02:22:51.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.677+0000 7f1630400700 1 Processor -- start 2024-06-03T02:22:51.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.678+0000 7f1630400700 1 -- start start 2024-06-03T02:22:51.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.679+0000 7f1630400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f162806a170 0x7f1628101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:51.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.679+0000 7f1630400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f162806ab30 0x7f16281019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:51.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.680+0000 7f1630400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 0x7f1628101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:51.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.680+0000 7f1630400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1628112800 con 0x7f162806a170 2024-06-03T02:22:51.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.680+0000 7f1630400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1628112680 con 0x7f1628108500 2024-06-03T02:22:51.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.680+0000 7f1630400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1628112980 con 0x7f162806ab30 2024-06-03T02:22:51.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.680+0000 7f162d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f162806ab30 0x7f16281019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:51.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.680+0000 7f162e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 0x7f1628101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.680+0000 7f162e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 0x7f1628101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36844/0 (socket says 172.21.15.79:36844) 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.680+0000 7f162e400700 1 -- 172.21.15.79:0/1116588144 learned_addr learned my addr 172.21.15.79:0/1116588144 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.681+0000 7f162e400700 1 -- 172.21.15.79:0/1116588144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f162806ab30 msgr2=0x7f16281019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.681+0000 7f162e400700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f162806ab30 0x7f16281019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.681+0000 7f162e400700 1 -- 172.21.15.79:0/1116588144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f162806a170 msgr2=0x7f1628101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.681+0000 7f162e400700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f162806a170 0x7f1628101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.681+0000 7f162e400700 1 -- 172.21.15.79:0/1116588144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16281a7250 con 0x7f1628108500 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.682+0000 7f162e400700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 0x7f1628101ee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f162000c110 tx=0x7f162000c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.682+0000 7f1626000700 1 -- 172.21.15.79:0/1116588144 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162000cd90 con 0x7f1628108500 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.682+0000 7f1626000700 1 -- 172.21.15.79:0/1116588144 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1620004bb0 con 0x7f1628108500 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.682+0000 7f1626000700 1 -- 172.21.15.79:0/1116588144 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620007e00 con 0x7f1628108500 2024-06-03T02:22:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.682+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16281a74e0 con 0x7f1628108500 2024-06-03T02:22:51.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.682+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16281a79d0 con 0x7f1628108500 2024-06-03T02:22:51.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.684+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1628069180 con 0x7f1628108500 2024-06-03T02:22:51.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.691+0000 7f1626000700 1 -- 172.21.15.79:0/1116588144 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f162001a020 con 0x7f1628108500 2024-06-03T02:22:51.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.693+0000 7f1626000700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16100a2e20 0x7f16100a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:51.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.693+0000 7f1626000700 1 -- 172.21.15.79:0/1116588144 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f16200ceee0 con 0x7f1628108500 2024-06-03T02:22:51.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.694+0000 7f162da00700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16100a2e20 0x7f16100a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:51.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.695+0000 7f162da00700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16100a2e20 0x7f16100a52e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f161c009230 tx=0x7f161c009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:51.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:51.697+0000 7f1626000700 1 -- 172.21.15.79:0/1116588144 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1620010040 con 0x7f1628108500 2024-06-03T02:22:52.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:51 smithi079 ceph-mon[31819]: pgmap v600: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 512 op/s 2024-06-03T02:22:52.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.171+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1628102940 con 0x7f1628108500 2024-06-03T02:22:52.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.172+0000 7f1626000700 1 -- 172.21.15.79:0/1116588144 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f162009ba30 con 0x7f1628108500 2024-06-03T02:22:52.174 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:22:52.174 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:22:52.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16100a2e20 msgr2=0x7f16100a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:52.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16100a2e20 0x7f16100a52e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f161c009230 tx=0x7f161c009540 comp rx=0 tx=0).stop 2024-06-03T02:22:52.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 msgr2=0x7f1628101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:52.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 0x7f1628101ee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f162000c110 tx=0x7f162000c4d0 comp rx=0 tx=0).stop 2024-06-03T02:22:52.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 shutdown_connections 2024-06-03T02:22:52.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1628108500 0x7f1628101ee0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:52.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f16100a2e20 0x7f16100a52e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:52.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f162806a170 0x7f1628101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:52.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 --2- 172.21.15.79:0/1116588144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f162806ab30 0x7f16281019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:52.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.176+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 >> 172.21.15.79:0/1116588144 conn(0x7f1628078340 msgr2=0x7f16280fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:52.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.177+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 shutdown_connections 2024-06-03T02:22:52.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:52.177+0000 7f1630400700 1 -- 172.21.15.79:0/1116588144 wait complete. 2024-06-03T02:22:52.178 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:22:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:51 smithi123 ceph-mon[36640]: pgmap v600: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 512 op/s 2024-06-03T02:22:52.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:51 smithi135 ceph-mon[36653]: pgmap v600: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 512 op/s 2024-06-03T02:22:53.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:52 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1116588144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:22:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 165.0 s, 1207.6 tps, lat 82.886 ms stddev 102.435 2024-06-03T02:22:53.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:52 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1116588144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:22:53.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:52 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1116588144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:22:54.213 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:53 smithi079 ceph-mon[31819]: pgmap v601: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 MiB/s wr, 660 op/s 2024-06-03T02:22:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:53 smithi123 ceph-mon[36640]: pgmap v601: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 MiB/s wr, 660 op/s 2024-06-03T02:22:54.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:22:54.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:53 smithi135 ceph-mon[36653]: pgmap v601: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 MiB/s wr, 660 op/s 2024-06-03T02:22:54.896 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:22:56.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:55 smithi123 ceph-mon[36640]: pgmap v602: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.9 MiB/s wr, 524 op/s 2024-06-03T02:22:56.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:55 smithi079 ceph-mon[31819]: pgmap v602: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.9 MiB/s wr, 524 op/s 2024-06-03T02:22:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:55 smithi135 ceph-mon[36653]: pgmap v602: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.9 MiB/s wr, 524 op/s 2024-06-03T02:22:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.646+0000 7fed4c200700 1 -- 172.21.15.79:0/1061951743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 msgr2=0x7fed44104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.646+0000 7fed4c200700 1 --2- 172.21.15.79:0/1061951743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 0x7fed44104b20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed30009990 tx=0x7fed30009ca0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.647+0000 7fed4c200700 1 -- 172.21.15.79:0/1061951743 shutdown_connections 2024-06-03T02:22:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.647+0000 7fed4c200700 1 --2- 172.21.15.79:0/1061951743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed44105060 0x7fed44111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.647+0000 7fed4c200700 1 --2- 172.21.15.79:0/1061951743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed4410a6a0 0x7fed4410aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.647+0000 7fed4c200700 1 --2- 172.21.15.79:0/1061951743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 0x7fed44104b20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.647+0000 7fed4c200700 1 -- 172.21.15.79:0/1061951743 >> 172.21.15.79:0/1061951743 conn(0x7fed44100170 msgr2=0x7fed44102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.647+0000 7fed4c200700 1 -- 172.21.15.79:0/1061951743 shutdown_connections 2024-06-03T02:22:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.647+0000 7fed4c200700 1 -- 172.21.15.79:0/1061951743 wait complete. 2024-06-03T02:22:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.648+0000 7fed4c200700 1 Processor -- start 2024-06-03T02:22:57.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.649+0000 7fed4c200700 1 -- start start 2024-06-03T02:22:57.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.650+0000 7fed4c200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 0x7fed4419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.650+0000 7fed4c200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed44105060 0x7fed4419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.650+0000 7fed4c200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed4410a6a0 0x7fed4419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.650+0000 7fed4c200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fed44114a30 con 0x7fed44105060 2024-06-03T02:22:57.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.650+0000 7fed4c200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed441148b0 con 0x7fed441046a0 2024-06-03T02:22:57.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.650+0000 7fed4c200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed44114bb0 con 0x7fed4410a6a0 2024-06-03T02:22:57.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.650+0000 7fed49800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 0x7fed4419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.650+0000 7fed49800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 0x7fed4419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60232/0 (socket says 172.21.15.79:60232) 2024-06-03T02:22:57.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.650+0000 7fed49800700 1 -- 172.21.15.79:0/2367005730 learned_addr learned my addr 172.21.15.79:0/2367005730 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:57.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.651+0000 7fed4a200700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed4410a6a0 0x7fed4419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.651+0000 7fed49800700 1 -- 172.21.15.79:0/2367005730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed4410a6a0 msgr2=0x7fed4419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:57.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.651+0000 7fed49800700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed4410a6a0 0x7fed4419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.651+0000 7fed49800700 1 -- 172.21.15.79:0/2367005730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed44105060 msgr2=0x7fed4419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:57.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.651+0000 7fed49800700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed44105060 0x7fed4419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.651+0000 7fed49800700 1 -- 172.21.15.79:0/2367005730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4419fe50 con 0x7fed441046a0 2024-06-03T02:22:57.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.651+0000 7fed4a200700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed4410a6a0 0x7fed4419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:22:57.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.652+0000 7fed49800700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 0x7fed4419ae40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed3800d980 tx=0x7fed3800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:57.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.652+0000 7fed42000700 1 -- 172.21.15.79:0/2367005730 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38016070 con 0x7fed441046a0 2024-06-03T02:22:57.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.652+0000 7fed42000700 1 -- 172.21.15.79:0/2367005730 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed38004500 con 0x7fed441046a0 2024-06-03T02:22:57.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.652+0000 7fed42000700 1 -- 172.21.15.79:0/2367005730 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38004d00 con 0x7fed441046a0 2024-06-03T02:22:57.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.652+0000 7fed4c200700 1 -- 172.21.15.79:0/2367005730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed441a0140 con 0x7fed441046a0 2024-06-03T02:22:57.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.653+0000 7fed4c200700 1 -- 172.21.15.79:0/2367005730 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed440768d0 con 0x7fed441046a0 2024-06-03T02:22:57.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.658+0000 7fed42000700 1 -- 172.21.15.79:0/2367005730 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fed3800bbf0 con 0x7fed441046a0 2024-06-03T02:22:57.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.663+0000 7fed42000700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fed2c0a2d70 0x7fed2c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.663+0000 7fed42000700 1 -- 172.21.15.79:0/2367005730 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fed380cf0d0 con 0x7fed441046a0 2024-06-03T02:22:57.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.663+0000 7fed56bb7b80 1 Processor -- start 2024-06-03T02:22:57.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.663+0000 7fed56bb7b80 1 -- start start 2024-06-03T02:22:57.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.663+0000 7fed48e00700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fed2c0a2d70 0x7fed2c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce29b90 0x55ef1ce29f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce23d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce24240 0x55ef1ce24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed56bb7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ef1cd32a10 con 0x55ef1ce29b90 2024-06-03T02:22:57.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed56bb7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef1cd75b00 con 0x55ef1ce24240 2024-06-03T02:22:57.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed56bb7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef1cd308f0 con 0x55ef1ce2a4b0 2024-06-03T02:22:57.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed48e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce23d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed4a200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce24240 0x55ef1ce24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed49800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce29b90 0x55ef1ce29f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed4a200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce24240 0x55ef1ce24680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60238/0 (socket says 172.21.15.79:60238) 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.664+0000 7fed4a200700 1 -- 172.21.15.79:0/2480880189 learned_addr learned my addr 172.21.15.79:0/2480880189 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.665+0000 7fed4a200700 1 -- 172.21.15.79:0/2480880189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 msgr2=0x55ef1ce23d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.665+0000 7fed4a200700 1 --2- 172.21.15.79:0/2480880189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce23d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.665+0000 7fed4a200700 1 -- 172.21.15.79:0/2480880189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce29b90 msgr2=0x55ef1ce29f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.665+0000 7fed4a200700 1 --2- 172.21.15.79:0/2480880189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce29b90 0x55ef1ce29f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.665+0000 7fed4a200700 1 -- 172.21.15.79:0/2480880189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef1cd1df90 con 0x55ef1ce24240 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.665+0000 7fed48e00700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fed2c0a2d70 0x7fed2c0a5230 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fed300098e0 tx=0x7fed30009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:57.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.665+0000 7fed4a200700 1 --2- 172.21.15.79:0/2480880189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce24240 0x55ef1ce24680 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009400 tx=0x7fed3c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a69bb295d57054d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:57.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.666+0000 7fed36a00700 1 -- 172.21.15.79:0/2480880189 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c007e60 con 0x55ef1ce24240 2024-06-03T02:22:57.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.666+0000 7fed36a00700 1 -- 172.21.15.79:0/2480880189 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c019070 con 0x55ef1ce24240 2024-06-03T02:22:57.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.666+0000 7fed36a00700 1 -- 172.21.15.79:0/2480880189 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0139a0 con 0x55ef1ce24240 2024-06-03T02:22:57.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.668+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2480880189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce24240 msgr2=0x55ef1ce24680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:57.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.668+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/2480880189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce24240 0x55ef1ce24680 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009400 tx=0x7fed3c009920 comp rx=0 tx=0).stop 2024-06-03T02:22:57.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.668+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2480880189 shutdown_connections 2024-06-03T02:22:57.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.668+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/2480880189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce24240 0x55ef1ce24680 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.668+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/2480880189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce29b90 0x55ef1ce29f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.668+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/2480880189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce23d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.668+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2480880189 >> 172.21.15.79:0/2480880189 conn(0x55ef1cd50170 msgr2=0x55ef1ce1eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:57.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.669+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2480880189 shutdown_connections 2024-06-03T02:22:57.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.669+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2480880189 wait complete. 2024-06-03T02:22:57.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.670+0000 7fed56bb7b80 1 -- start start 2024-06-03T02:22:57.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.670+0000 7fed56bb7b80 1 Processor -- start 2024-06-03T02:22:57.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.671+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 0x55ef1ce30220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.671+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce29b90 0x55ef1ce2d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce2da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed48e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce29b90 0x55ef1ce2d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed48e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce29b90 0x55ef1ce2d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60250/0 (socket says 172.21.15.79:60250) 2024-06-03T02:22:57.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed56bb7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ef1cd11f60 con 0x55ef1ce24240 2024-06-03T02:22:57.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed48e00700 1 -- 172.21.15.79:0/531250253 learned_addr learned my addr 172.21.15.79:0/531250253 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:57.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef1cd71ac0 con 0x55ef1ce29b90 2024-06-03T02:22:57.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef1ce30e90 con 0x55ef1ce2a4b0 2024-06-03T02:22:57.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed56bb7b80 10 client.0 _ll_register_callbacks cb 0x55ef1cd0b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:57.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed4a200700 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce2da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed49800700 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 0x55ef1ce30220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed48e00700 1 -- 172.21.15.79:0/531250253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 msgr2=0x55ef1ce2da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:57.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed48e00700 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce2da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed48e00700 1 -- 172.21.15.79:0/531250253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 msgr2=0x55ef1ce30220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:57.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed48e00700 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 0x55ef1ce30220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.672+0000 7fed48e00700 1 -- 172.21.15.79:0/531250253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef1ccc48d0 con 0x55ef1ce29b90 2024-06-03T02:22:57.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.673+0000 7fed48e00700 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce29b90 0x55ef1ce2d520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fed30015000 tx=0x7fed30000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1fa3a813e4895cf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:57.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.673+0000 7fed2b400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:22:57.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.673+0000 7fed2b400700 1 -- 172.21.15.79:0/531250253 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30021040 con 0x55ef1ce29b90 2024-06-03T02:22:57.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.673+0000 7fed2b400700 1 -- 172.21.15.79:0/531250253 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fed3000f870 con 0x55ef1ce29b90 2024-06-03T02:22:57.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.673+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3c019070 con 0x55ef1ce29b90 2024-06-03T02:22:57.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.674+0000 7fed2b400700 1 -- 172.21.15.79:0/531250253 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed3000be70 con 0x55ef1ce29b90 2024-06-03T02:22:57.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.674+0000 7fed56bb7b80 10 client.25670 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:57.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.674+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef1ce3b610 con 0x55ef1ce29b90 2024-06-03T02:22:57.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.674+0000 7fed2b400700 1 -- 172.21.15.79:0/531250253 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c019070 con 0x55ef1ce29b90 2024-06-03T02:22:57.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.675+0000 7fed2b400700 1 -- 172.21.15.79:0/531250253 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed3000fa70 con 0x55ef1ce29b90 2024-06-03T02:22:57.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.676+0000 7fed56bb7b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:57.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.676+0000 7fed56bb7b80 10 client.25670 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:22:57.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.676+0000 7fed56bb7b80 20 client.25670 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:57.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.676+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55ef1ce3dc50 0x55ef1ce40040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.676+0000 7fed56bb7b80 4 client.25670 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:22:57.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.676+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef1cd83000 con 0x55ef1ce3dc50 2024-06-03T02:22:57.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.676+0000 7fed49800700 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55ef1ce3dc50 0x55ef1ce40040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.678+0000 7fed2b400700 10 client.25670 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:22:57.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.678+0000 7fed49800700 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55ef1ce3dc50 0x55ef1ce40040 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dcf3545be284e9b9 server_cookie=bb807695efd3d6e1 in_seq=0 out_seq=0 2024-06-03T02:22:57.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.679+0000 7fed2b400700 1 -- 172.21.15.79:0/531250253 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef1cd83000 con 0x55ef1ce3dc50 2024-06-03T02:22:57.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.679+0000 7fed2b400700 10 client.25670 handle_command_reply: tid=0 2024-06-03T02:22:57.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.679+0000 7fed56bb7b80 1 client.25670 shutdown 2024-06-03T02:22:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.679+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce29b90 msgr2=0x55ef1ce2d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:57.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.679+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce29b90 0x55ef1ce2d520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fed30015000 tx=0x7fed30000bc0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.680+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 shutdown_connections 2024-06-03T02:22:57.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.680+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55ef1ce3dc50 0x55ef1ce40040 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.680+0000 7fed2b400700 0 client.25670 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:22:57.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.680+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce2da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.680+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 0x55ef1ce30220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.680+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/531250253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce29b90 0x55ef1ce2d520 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.680+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 >> 172.21.15.79:0/531250253 conn(0x55ef1cd50170 msgr2=0x55ef1ce18790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:57.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.680+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 shutdown_connections 2024-06-03T02:22:57.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.681+0000 7fed56bb7b80 1 -- 172.21.15.79:0/531250253 wait complete. 2024-06-03T02:22:57.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.681+0000 7fed56bb7b80 20 client.25670 trim_cache size 0 max 16384 2024-06-03T02:22:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.712+0000 7fed56bb7b80 1 Processor -- start 2024-06-03T02:22:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.712+0000 7fed56bb7b80 1 -- start start 2024-06-03T02:22:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.712+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce24240 0x55ef1ce23d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.712+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce29b90 0x55ef1ce1f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.712+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce19130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.712+0000 7fed56bb7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b49b0 con 0x55ef1ce29b90 2024-06-03T02:22:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.712+0000 7fed56bb7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b64f0 con 0x55ef1ce2a4b0 2024-06-03T02:22:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.712+0000 7fed56bb7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b5080 con 0x55ef1ce24240 2024-06-03T02:22:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.713+0000 7fed4a200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce19130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.713+0000 7fed4a200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce19130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60260/0 (socket says 172.21.15.79:60260) 2024-06-03T02:22:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.713+0000 7fed4a200700 1 -- 172.21.15.79:0/2872589224 learned_addr learned my addr 172.21.15.79:0/2872589224 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.714+0000 7fed4a200700 1 -- 172.21.15.79:0/2872589224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce24240 msgr2=0x55ef1ce23d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.714+0000 7fed4a200700 1 --2- 172.21.15.79:0/2872589224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce24240 0x55ef1ce23d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.714+0000 7fed4a200700 1 -- 172.21.15.79:0/2872589224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce29b90 msgr2=0x55ef1ce1f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.714+0000 7fed4a200700 1 --2- 172.21.15.79:0/2872589224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce29b90 0x55ef1ce1f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.714+0000 7fed4a200700 1 -- 172.21.15.79:0/2872589224 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef1ce2d2f0 con 0x55ef1ce2a4b0 2024-06-03T02:22:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.714+0000 7fed4a200700 1 --2- 172.21.15.79:0/2872589224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce19130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009400 tx=0x7fed3c013500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e44d094b6a253d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:57.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.715+0000 7fed36000700 1 -- 172.21.15.79:0/2872589224 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c007e60 con 0x55ef1ce2a4b0 2024-06-03T02:22:57.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.715+0000 7fed36000700 1 -- 172.21.15.79:0/2872589224 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c019070 con 0x55ef1ce2a4b0 2024-06-03T02:22:57.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.715+0000 7fed36000700 1 -- 172.21.15.79:0/2872589224 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0277b0 con 0x55ef1ce2a4b0 2024-06-03T02:22:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.716+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2872589224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 msgr2=0x55ef1ce19130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.716+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/2872589224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce19130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009400 tx=0x7fed3c013500 comp rx=0 tx=0).stop 2024-06-03T02:22:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.717+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2872589224 shutdown_connections 2024-06-03T02:22:57.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.717+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/2872589224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce19130 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.718+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/2872589224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce24240 0x55ef1ce23d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.718+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/2872589224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce29b90 0x55ef1ce1f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.718+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2872589224 >> 172.21.15.79:0/2872589224 conn(0x55ef1cd50170 msgr2=0x55ef1ce1ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.719+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2872589224 shutdown_connections 2024-06-03T02:22:57.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.719+0000 7fed56bb7b80 1 -- 172.21.15.79:0/2872589224 wait complete. 2024-06-03T02:22:57.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.720+0000 7fed56bb7b80 1 -- start start 2024-06-03T02:22:57.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.721+0000 7fed56bb7b80 1 Processor -- start 2024-06-03T02:22:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.721+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 0x55ef1ce36330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.722+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce29b90 0x55ef1ce36870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.722+0000 7fed56bb7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce44cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.723+0000 7fed56bb7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b5e20 con 0x55ef1ce24240 2024-06-03T02:22:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.723+0000 7fed4a200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce44cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.723+0000 7fed4a200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce44cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60270/0 (socket says 172.21.15.79:60270) 2024-06-03T02:22:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.723+0000 7fed4a200700 1 -- 172.21.15.79:0/1718890242 learned_addr learned my addr 172.21.15.79:0/1718890242 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:22:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.723+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b5750 con 0x55ef1ce2a4b0 2024-06-03T02:22:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.723+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b2950 con 0x55ef1ce29b90 2024-06-03T02:22:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.723+0000 7fed56bb7b80 10 client.0 _ll_register_callbacks cb 0x55ef1cd0b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:22:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.724+0000 7fed49800700 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 0x55ef1ce36330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.724+0000 7fed48e00700 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce29b90 0x55ef1ce36870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.724+0000 7fed48e00700 1 -- 172.21.15.79:0/1718890242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 msgr2=0x55ef1ce44cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.724+0000 7fed48e00700 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce44cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.724+0000 7fed48e00700 1 -- 172.21.15.79:0/1718890242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 msgr2=0x55ef1ce36330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:22:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.724+0000 7fed48e00700 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 0x55ef1ce36330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.724+0000 7fed48e00700 1 -- 172.21.15.79:0/1718890242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef1ce45420 con 0x55ef1ce29b90 2024-06-03T02:22:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.725+0000 7fed4a200700 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce44cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:22:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.725+0000 7fed48e00700 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce29b90 0x55ef1ce36870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55ef1ce1aed0 tx=0x7fed3000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4753d7d885db4a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:22:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.725+0000 7fed2b400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:22:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.726+0000 7fed2b400700 1 -- 172.21.15.79:0/1718890242 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3000fa80 con 0x55ef1ce29b90 2024-06-03T02:22:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.726+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3c019070 con 0x55ef1ce29b90 2024-06-03T02:22:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.726+0000 7fed2b400700 1 -- 172.21.15.79:0/1718890242 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fed3003c7a0 con 0x55ef1ce29b90 2024-06-03T02:22:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.727+0000 7fed2b400700 1 -- 172.21.15.79:0/1718890242 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed3004c050 con 0x55ef1ce29b90 2024-06-03T02:22:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.727+0000 7fed56bb7b80 10 client.25654 fetch_fsmap learned FSMap version 30 2024-06-03T02:22:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.727+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef1ceb92c0 con 0x55ef1ce29b90 2024-06-03T02:22:57.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.727+0000 7fed2b400700 1 -- 172.21.15.79:0/1718890242 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed30022040 con 0x55ef1ce29b90 2024-06-03T02:22:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.728+0000 7fed2b400700 1 -- 172.21.15.79:0/1718890242 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed3003c9a0 con 0x55ef1ce29b90 2024-06-03T02:22:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.728+0000 7fed56bb7b80 10 client.25654 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:22:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.728+0000 7fed56bb7b80 10 client.25654 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:22:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.728+0000 7fed56bb7b80 20 client.25654 populate_metadata read hostname 'smithi079' 2024-06-03T02:22:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.728+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55ef1ceb9910 0x55ef1cebbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:22:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.728+0000 7fed56bb7b80 4 client.25654 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:22:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.728+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fed3c019070 con 0x55ef1ceb9910 2024-06-03T02:22:57.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.731+0000 7fed49800700 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55ef1ceb9910 0x55ef1cebbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:22:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.732+0000 7fed49800700 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55ef1ceb9910 0x55ef1cebbd00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a9b954c4e5faa8f server_cookie=8b63632fa67e54c0 in_seq=0 out_seq=0 2024-06-03T02:22:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.732+0000 7fed2b400700 10 client.25654 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:22:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.733+0000 7fed2b400700 1 -- 172.21.15.79:0/1718890242 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fed3c019070 con 0x55ef1ceb9910 2024-06-03T02:22:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.733+0000 7fed2b400700 10 client.25654 handle_command_reply: tid=0 2024-06-03T02:22:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.733+0000 7fed56bb7b80 1 client.25654 shutdown 2024-06-03T02:22:57.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce29b90 msgr2=0x55ef1ce36870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:57.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce29b90 0x55ef1ce36870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55ef1ce1aed0 tx=0x7fed3000b390 comp rx=0 tx=0).stop 2024-06-03T02:22:57.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 shutdown_connections 2024-06-03T02:22:57.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55ef1ceb9910 0x55ef1cebbd00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef1ce2a4b0 0x55ef1ce44cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1ce24240 0x55ef1ce36330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed2b400700 0 client.25654 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:22:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed56bb7b80 1 --2- 172.21.15.79:0/1718890242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef1ce29b90 0x55ef1ce36870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 >> 172.21.15.79:0/1718890242 conn(0x55ef1cd50170 msgr2=0x55ef1ce2a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.735+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 shutdown_connections 2024-06-03T02:22:57.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.737+0000 7fed56bb7b80 1 -- 172.21.15.79:0/1718890242 wait complete. 2024-06-03T02:22:57.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.737+0000 7fed56bb7b80 20 client.25654 trim_cache size 0 max 16384 2024-06-03T02:22:57.738 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 -- 172.21.15.79:0/2367005730 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fed2c0a2d70 msgr2=0x7fed2c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fed2c0a2d70 0x7fed2c0a5230 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fed300098e0 tx=0x7fed30009680 comp rx=0 tx=0).stop 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 -- 172.21.15.79:0/2367005730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 msgr2=0x7fed4419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 0x7fed4419ae40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed3800d980 tx=0x7fed3800dd40 comp rx=0 tx=0).stop 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 -- 172.21.15.79:0/2367005730 shutdown_connections 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed4410a6a0 0x7fed4419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed441046a0 0x7fed4419ae40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fed2c0a2d70 0x7fed2c0a5230 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 --2- 172.21.15.79:0/2367005730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fed44105060 0x7fed4419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:22:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.738+0000 7fed4c200700 1 -- 172.21.15.79:0/2367005730 >> 172.21.15.79:0/2367005730 conn(0x7fed44100170 msgr2=0x7fed441006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:22:57.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.739+0000 7fed4c200700 1 -- 172.21.15.79:0/2367005730 shutdown_connections 2024-06-03T02:22:57.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:22:57.739+0000 7fed4c200700 1 -- 172.21.15.79:0/2367005730 wait complete. 2024-06-03T02:22:58.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:57 smithi123 ceph-mon[36640]: pgmap v603: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.8 MiB/s wr, 525 op/s 2024-06-03T02:22:58.301 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:57 smithi079 ceph-mon[31819]: pgmap v603: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.8 MiB/s wr, 525 op/s 2024-06-03T02:22:58.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 170.0 s, 933.6 tps, lat 107.559 ms stddev 125.744 2024-06-03T02:22:58.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:57 smithi135 ceph-mon[36653]: pgmap v603: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.8 MiB/s wr, 525 op/s 2024-06-03T02:22:59.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:23:00.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:22:59 smithi079 ceph-mon[31819]: pgmap v604: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 591 op/s 2024-06-03T02:23:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:22:59 smithi123 ceph-mon[36640]: pgmap v604: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 591 op/s 2024-06-03T02:23:00.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:22:59 smithi135 ceph-mon[36653]: pgmap v604: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 591 op/s 2024-06-03T02:23:00.560 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:01 smithi123 ceph-mon[36640]: pgmap v605: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-06-03T02:23:02.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:01 smithi079 ceph-mon[31819]: pgmap v605: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-06-03T02:23:02.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:01 smithi135 ceph-mon[36653]: pgmap v605: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-06-03T02:23:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 175.0 s, 869.2 tps, lat 115.815 ms stddev 149.284 2024-06-03T02:23:04.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:03 smithi123 ceph-mon[36640]: pgmap v606: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 654 op/s 2024-06-03T02:23:04.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:03 smithi079 ceph-mon[31819]: pgmap v606: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 654 op/s 2024-06-03T02:23:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:03 smithi135 ceph-mon[36653]: pgmap v606: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 654 op/s 2024-06-03T02:23:05.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.054+0000 7f5f1f400700 1 -- 172.21.15.79:0/4288449341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f181046a0 msgr2=0x7f5f18104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:05.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.054+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4288449341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f181046a0 0x7f5f18104b20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009990 tx=0x7f5f04009ca0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.054+0000 7f5f1f400700 1 -- 172.21.15.79:0/4288449341 shutdown_connections 2024-06-03T02:23:05.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.054+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4288449341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f18105060 0x7f5f18111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.054+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4288449341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f1810a6a0 0x7f5f1810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.054+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4288449341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f181046a0 0x7f5f18104b20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.054+0000 7f5f1f400700 1 -- 172.21.15.79:0/4288449341 >> 172.21.15.79:0/4288449341 conn(0x7f5f18100170 msgr2=0x7f5f18102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:05.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.054+0000 7f5f1f400700 1 -- 172.21.15.79:0/4288449341 shutdown_connections 2024-06-03T02:23:05.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.054+0000 7f5f1f400700 1 -- 172.21.15.79:0/4288449341 wait complete. 2024-06-03T02:23:05.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.056+0000 7f5f1f400700 1 Processor -- start 2024-06-03T02:23:05.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.056+0000 7f5f1f400700 1 -- start start 2024-06-03T02:23:05.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.056+0000 7f5f1f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f181046a0 0x7f5f1819aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f18105060 0x7f5f1819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f1810a6a0 0x7f5f1819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1f400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114ad0 con 0x7f5f18105060 2024-06-03T02:23:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1f400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114950 con 0x7f5f1810a6a0 2024-06-03T02:23:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1f400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114c50 con 0x7f5f181046a0 2024-06-03T02:23:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f1810a6a0 0x7f5f1819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f1810a6a0 0x7f5f1819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48470/0 (socket says 172.21.15.79:48470) 2024-06-03T02:23:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1d400700 1 -- 172.21.15.79:0/4248883026 learned_addr learned my addr 172.21.15.79:0/4248883026 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:05.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f17e00700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f18105060 0x7f5f1819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f181046a0 0x7f5f1819aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1d400700 1 -- 172.21.15.79:0/4248883026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f181046a0 msgr2=0x7f5f1819aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:05.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1d400700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f181046a0 0x7f5f1819aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1d400700 1 -- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f18105060 msgr2=0x7f5f1819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:05.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.057+0000 7f5f1d400700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f18105060 0x7f5f1819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.058+0000 7f5f1d400700 1 -- 172.21.15.79:0/4248883026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1819fea0 con 0x7f5f1810a6a0 2024-06-03T02:23:05.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.058+0000 7f5f1d400700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f1810a6a0 0x7f5f1819f760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000b970 tx=0x7f5f1000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:05.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.059+0000 7f5f15600700 1 -- 172.21.15.79:0/4248883026 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10019070 con 0x7f5f1810a6a0 2024-06-03T02:23:05.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.059+0000 7f5f15600700 1 -- 172.21.15.79:0/4248883026 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f10014440 con 0x7f5f1810a6a0 2024-06-03T02:23:05.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.059+0000 7f5f15600700 1 -- 172.21.15.79:0/4248883026 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f100075d0 con 0x7f5f1810a6a0 2024-06-03T02:23:05.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.059+0000 7f5f1f400700 1 -- 172.21.15.79:0/4248883026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f181a0190 con 0x7f5f1810a6a0 2024-06-03T02:23:05.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.060+0000 7f5f1f400700 1 -- 172.21.15.79:0/4248883026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181a7a10 con 0x7f5f1810a6a0 2024-06-03T02:23:05.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.065+0000 7f5f15600700 1 -- 172.21.15.79:0/4248883026 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5f1001c070 con 0x7f5f1810a6a0 2024-06-03T02:23:05.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.067+0000 7f5f15600700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f000a2e20 0x7f5f000a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.067+0000 7f5f15600700 1 -- 172.21.15.79:0/4248883026 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5f100ce780 con 0x7f5f1810a6a0 2024-06-03T02:23:05.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.068+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f000a2e20 0x7f5f000a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.069+0000 7f5f29d28b80 1 Processor -- start 2024-06-03T02:23:05.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.069+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f000a2e20 0x7f5f000a52e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009680 tx=0x7f5f0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.070+0000 7f5f29d28b80 1 -- start start 2024-06-03T02:23:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.070+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445eeb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.070+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 0x559445ee5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.070+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445ee5a50 0x559445ee5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.070+0000 7f5f29d28b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559445deba10 con 0x559445ee5a50 2024-06-03T02:23:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.070+0000 7f5f29d28b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559445e2eb00 con 0x559445eebcc0 2024-06-03T02:23:05.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.070+0000 7f5f29d28b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559445de98f0 con 0x559445eeb3a0 2024-06-03T02:23:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.071+0000 7f5f1ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445eeb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.071+0000 7f5f17e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 0x559445ee5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.071+0000 7f5f1ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445eeb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:32974/0 (socket says 172.21.15.79:32974) 2024-06-03T02:23:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.071+0000 7f5f1ca00700 1 -- 172.21.15.79:0/2306840579 learned_addr learned my addr 172.21.15.79:0/2306840579 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.071+0000 7f5f1d400700 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445ee5a50 0x559445ee5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.072+0000 7f5f17e00700 1 -- 172.21.15.79:0/2306840579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 msgr2=0x559445eeb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.072+0000 7f5f17e00700 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445eeb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.072+0000 7f5f17e00700 1 -- 172.21.15.79:0/2306840579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445ee5a50 msgr2=0x559445ee5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.072+0000 7f5f17e00700 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445ee5a50 0x559445ee5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.072+0000 7f5f17e00700 1 -- 172.21.15.79:0/2306840579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559445dd6f90 con 0x559445eebcc0 2024-06-03T02:23:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.072+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445eeb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:23:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.072+0000 7f5f1d400700 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445ee5a50 0x559445ee5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:23:05.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.073+0000 7f5f17e00700 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 0x559445ee5510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f18109bf0 tx=0x7f5f0400b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45056a8739152d6b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:05.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.073+0000 7f5f0a000700 1 -- 172.21.15.79:0/2306840579 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04009510 con 0x559445eebcc0 2024-06-03T02:23:05.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.073+0000 7f5f0a000700 1 -- 172.21.15.79:0/2306840579 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f040055a0 con 0x559445eebcc0 2024-06-03T02:23:05.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.073+0000 7f5f0a000700 1 -- 172.21.15.79:0/2306840579 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04003c40 con 0x559445eebcc0 2024-06-03T02:23:05.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.074+0000 7f5f29d28b80 1 -- 172.21.15.79:0/2306840579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 msgr2=0x559445ee5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:05.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.074+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 0x559445ee5510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f18109bf0 tx=0x7f5f0400b7a0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.075+0000 7f5f29d28b80 1 -- 172.21.15.79:0/2306840579 shutdown_connections 2024-06-03T02:23:05.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.075+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445ee5a50 0x559445ee5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.075+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445eeb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.075+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/2306840579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 0x559445ee5510 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.075+0000 7f5f29d28b80 1 -- 172.21.15.79:0/2306840579 >> 172.21.15.79:0/2306840579 conn(0x559445e09170 msgr2=0x559445ed7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:05.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.077+0000 7f5f29d28b80 1 -- 172.21.15.79:0/2306840579 shutdown_connections 2024-06-03T02:23:05.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.078+0000 7f5f29d28b80 1 -- 172.21.15.79:0/2306840579 wait complete. 2024-06-03T02:23:05.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.079+0000 7f5f29d28b80 1 -- start start 2024-06-03T02:23:05.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.083+0000 7f5f29d28b80 1 Processor -- start 2024-06-03T02:23:05.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.089+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445ee5a50 0x559445ef1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.090+0000 7f5f1ca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445ee5a50 0x559445ef1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.090+0000 7f5f1ca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445ee5a50 0x559445ef1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48488/0 (socket says 172.21.15.79:48488) 2024-06-03T02:23:05.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.090+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 0x559445eeedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.090+0000 7f5f1ca00700 1 -- 172.21.15.79:0/3357975719 learned_addr learned my addr 172.21.15.79:0/3357975719 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:05.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.091+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eebcc0 0x559445eef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.091+0000 7f5f17e00700 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 0x559445eeedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.091+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559445dcaf60 con 0x559445eeb3a0 2024-06-03T02:23:05.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.091+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559445e2aac0 con 0x559445ee5a50 2024-06-03T02:23:05.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.091+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559445ef2640 con 0x559445eebcc0 2024-06-03T02:23:05.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.091+0000 7f5f29d28b80 10 client.0 _ll_register_callbacks cb 0x559445dc4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.092+0000 7f5f1d400700 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eebcc0 0x559445eef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.092+0000 7f5f1ca00700 1 -- 172.21.15.79:0/3357975719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eebcc0 msgr2=0x559445eef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.092+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eebcc0 0x559445eef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.092+0000 7f5f1ca00700 1 -- 172.21.15.79:0/3357975719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 msgr2=0x559445eeedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.092+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 0x559445eeedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.092+0000 7f5f1ca00700 1 -- 172.21.15.79:0/3357975719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559445d7d8d0 con 0x559445ee5a50 2024-06-03T02:23:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.092+0000 7f5f17e00700 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 0x559445eeedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:23:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.093+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445ee5a50 0x559445ef1ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c012c10 tx=0x7f5f0c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a42759c39670bb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.093+0000 7f5efea00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:23:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.094+0000 7f5efea00700 1 -- 172.21.15.79:0/3357975719 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c004a30 con 0x559445ee5a50 2024-06-03T02:23:05.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.094+0000 7f5efea00700 1 -- 172.21.15.79:0/3357975719 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5f0c004bd0 con 0x559445ee5a50 2024-06-03T02:23:05.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.094+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f040055a0 con 0x559445ee5a50 2024-06-03T02:23:05.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.095+0000 7f5efea00700 1 -- 172.21.15.79:0/3357975719 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f0c011b00 con 0x559445ee5a50 2024-06-03T02:23:05.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.095+0000 7f5f29d28b80 10 client.25688 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:05.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.095+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559445efcef0 con 0x559445ee5a50 2024-06-03T02:23:05.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.096+0000 7f5efea00700 1 -- 172.21.15.79:0/3357975719 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f040055a0 con 0x559445ee5a50 2024-06-03T02:23:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.096+0000 7f5efea00700 1 -- 172.21.15.79:0/3357975719 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5f0c020a50 con 0x559445ee5a50 2024-06-03T02:23:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.096+0000 7f5f29d28b80 10 client.25688 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.096+0000 7f5f29d28b80 10 client.25688 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:23:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.097+0000 7f5f29d28b80 20 client.25688 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.097+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x559445eff480 0x559445f01870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.097+0000 7f5f29d28b80 4 client.25688 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:23:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.097+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559445e3c000 con 0x559445eff480 2024-06-03T02:23:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.097+0000 7f5f17e00700 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x559445eff480 0x559445f01870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.099+0000 7f5f17e00700 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x559445eff480 0x559445f01870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a14174f59bc82210 server_cookie=26ac8422757c8471 in_seq=0 out_seq=0 2024-06-03T02:23:05.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.099+0000 7f5efea00700 10 client.25688 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:23:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.100+0000 7f5efea00700 1 -- 172.21.15.79:0/3357975719 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559445e3c000 con 0x559445eff480 2024-06-03T02:23:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.100+0000 7f5efea00700 10 client.25688 handle_command_reply: tid=0 2024-06-03T02:23:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.100+0000 7f5f29d28b80 1 client.25688 shutdown 2024-06-03T02:23:05.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445ee5a50 msgr2=0x559445ef1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:05.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445ee5a50 0x559445ef1ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c012c10 tx=0x7f5f0c0047c0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 shutdown_connections 2024-06-03T02:23:05.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eebcc0 0x559445eef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445ee5a50 0x559445ef1ab0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c012c10 tx=0x7f5f0c0047c0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x559445eff480 0x559445f01870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/3357975719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 0x559445eeedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5efea00700 0 client.25688 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:23:05.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 >> 172.21.15.79:0/3357975719 conn(0x559445e09170 msgr2=0x559445ed1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:05.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.102+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 shutdown_connections 2024-06-03T02:23:05.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.103+0000 7f5f29d28b80 1 -- 172.21.15.79:0/3357975719 wait complete. 2024-06-03T02:23:05.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.103+0000 7f5f29d28b80 20 client.25688 trim_cache size 0 max 16384 2024-06-03T02:23:05.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.134+0000 7f5f29d28b80 1 Processor -- start 2024-06-03T02:23:05.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.134+0000 7f5f29d28b80 1 -- start start 2024-06-03T02:23:05.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.134+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 0x559445ed2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.134+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 0x559445eedf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.134+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445efb790 0x559445ef15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.134+0000 7f5f29d28b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181ac200 con 0x559445eeb3a0 2024-06-03T02:23:05.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.134+0000 7f5f29d28b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181add40 con 0x559445eebcc0 2024-06-03T02:23:05.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.134+0000 7f5f29d28b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181ac8d0 con 0x559445efb790 2024-06-03T02:23:05.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.135+0000 7f5f17e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 0x559445eedf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.135+0000 7f5f1d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445efb790 0x559445ef15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.135+0000 7f5f1d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445efb790 0x559445ef15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33000/0 (socket says 172.21.15.79:33000) 2024-06-03T02:23:05.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.135+0000 7f5f1d400700 1 -- 172.21.15.79:0/1653003309 learned_addr learned my addr 172.21.15.79:0/1653003309 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:05.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.135+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/1653003309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 0x559445ed2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.136+0000 7f5f1d400700 1 -- 172.21.15.79:0/1653003309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 msgr2=0x559445eedf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:05.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.136+0000 7f5f1d400700 1 --2- 172.21.15.79:0/1653003309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 0x559445eedf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.136+0000 7f5f1d400700 1 -- 172.21.15.79:0/1653003309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 msgr2=0x559445ed2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:05.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.136+0000 7f5f1d400700 1 --2- 172.21.15.79:0/1653003309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 0x559445ed2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.136+0000 7f5f1d400700 1 -- 172.21.15.79:0/1653003309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559445f01ea0 con 0x559445efb790 2024-06-03T02:23:05.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.137+0000 7f5f1d400700 1 --2- 172.21.15.79:0/1653003309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445efb790 0x559445ef15f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f1009a8d0 tx=0x7f5f10014d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de6d6bca49cd3cae server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:05.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.137+0000 7f5f09600700 1 -- 172.21.15.79:0/1653003309 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f100b1070 con 0x559445efb790 2024-06-03T02:23:05.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.137+0000 7f5f09600700 1 -- 172.21.15.79:0/1653003309 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f1009ef80 con 0x559445efb790 2024-06-03T02:23:05.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.137+0000 7f5f09600700 1 -- 172.21.15.79:0/1653003309 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f100a33b0 con 0x559445efb790 2024-06-03T02:23:05.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.138+0000 7f5f29d28b80 1 -- 172.21.15.79:0/1653003309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445efb790 msgr2=0x559445ef15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:05.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.138+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/1653003309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445efb790 0x559445ef15f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f1009a8d0 tx=0x7f5f10014d60 comp rx=0 tx=0).stop 2024-06-03T02:23:05.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.139+0000 7f5f29d28b80 1 -- 172.21.15.79:0/1653003309 shutdown_connections 2024-06-03T02:23:05.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.139+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/1653003309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445efb790 0x559445ef15f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.139+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/1653003309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eeb3a0 0x559445ed2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.139+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/1653003309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445eebcc0 0x559445eedf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.139+0000 7f5f29d28b80 1 -- 172.21.15.79:0/1653003309 >> 172.21.15.79:0/1653003309 conn(0x559445e09170 msgr2=0x559445eec710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:05.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.141+0000 7f5f29d28b80 1 -- 172.21.15.79:0/1653003309 shutdown_connections 2024-06-03T02:23:05.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.141+0000 7f5f29d28b80 1 -- 172.21.15.79:0/1653003309 wait complete. 2024-06-03T02:23:05.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.142+0000 7f5f29d28b80 1 -- start start 2024-06-03T02:23:05.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.143+0000 7f5f29d28b80 1 Processor -- start 2024-06-03T02:23:05.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.145+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445ef7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.145+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eebcc0 0x559445f06870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.145+0000 7f5f1ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445ef7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.145+0000 7f5f1ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445ef7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33006/0 (socket says 172.21.15.79:33006) 2024-06-03T02:23:05.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.146+0000 7f5f29d28b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445efb790 0x559445f06db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.146+0000 7f5f29d28b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181ad670 con 0x559445eebcc0 2024-06-03T02:23:05.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.146+0000 7f5f29d28b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181acfa0 con 0x559445efb790 2024-06-03T02:23:05.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.146+0000 7f5f29d28b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181aa1a0 con 0x559445eeb3a0 2024-06-03T02:23:05.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.146+0000 7f5f29d28b80 10 client.0 _ll_register_callbacks cb 0x559445dc4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:05.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.146+0000 7f5f1d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445efb790 0x559445f06db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.146+0000 7f5f1d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445efb790 0x559445f06db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48508/0 (socket says 172.21.15.79:48508) 2024-06-03T02:23:05.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.146+0000 7f5f1d400700 1 -- 172.21.15.79:0/810880725 learned_addr learned my addr 172.21.15.79:0/810880725 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:05.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.147+0000 7f5f17e00700 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eebcc0 0x559445f06870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.147+0000 7f5f1d400700 1 -- 172.21.15.79:0/810880725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 msgr2=0x559445ef7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:05.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.147+0000 7f5f1d400700 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445ef7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.147+0000 7f5f1d400700 1 -- 172.21.15.79:0/810880725 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eebcc0 msgr2=0x559445f06870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:05.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.147+0000 7f5f1d400700 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eebcc0 0x559445f06870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.147+0000 7f5f1d400700 1 -- 172.21.15.79:0/810880725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559445f073f0 con 0x559445efb790 2024-06-03T02:23:05.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.147+0000 7f5f17e00700 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eebcc0 0x559445f06870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:23:05.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.150+0000 7f5f1d400700 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445efb790 0x559445f06db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5f10020040 tx=0x7f5f10004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cdbc5fbcdd99b49 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:05.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.150+0000 7f5efea00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:23:05.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.150+0000 7f5efea00700 1 -- 172.21.15.79:0/810880725 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f100b7050 con 0x559445efb790 2024-06-03T02:23:05.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.150+0000 7f5efea00700 1 -- 172.21.15.79:0/810880725 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5f10014960 con 0x559445efb790 2024-06-03T02:23:05.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.151+0000 7f5f29d28b80 1 -- 172.21.15.79:0/810880725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1009ef80 con 0x559445efb790 2024-06-03T02:23:05.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.151+0000 7f5efea00700 1 -- 172.21.15.79:0/810880725 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f100b1040 con 0x559445efb790 2024-06-03T02:23:05.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.151+0000 7f5f29d28b80 10 client.25694 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:05.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.151+0000 7f5f29d28b80 1 -- 172.21.15.79:0/810880725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559445f69ed0 con 0x559445efb790 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.151+0000 7f5efea00700 1 -- 172.21.15.79:0/810880725 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f1009ef80 con 0x559445efb790 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.152+0000 7f5efea00700 1 -- 172.21.15.79:0/810880725 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5f1009a020 con 0x559445efb790 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.152+0000 7f5f29d28b80 10 client.25694 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.152+0000 7f5f29d28b80 10 client.25694 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.152+0000 7f5f29d28b80 20 client.25694 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.152+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x559445f6a4e0 0x559445f6c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.152+0000 7f5f29d28b80 4 client.25694 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.152+0000 7f5f29d28b80 1 -- 172.21.15.79:0/810880725 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559445e36580 con 0x559445f6a4e0 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.152+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x559445f6a4e0 0x559445f6c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.153+0000 7f5efea00700 10 client.25694 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.154+0000 7f5f1ca00700 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x559445f6a4e0 0x559445f6c900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c692197a295f3bd server_cookie=d39d3e0a22367671 in_seq=0 out_seq=0 2024-06-03T02:23:05.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.154+0000 7f5efea00700 1 -- 172.21.15.79:0/810880725 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559445e36580 con 0x559445f6a4e0 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.154+0000 7f5efea00700 10 client.25694 handle_command_reply: tid=0 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.154+0000 7f5f29d28b80 1 client.25694 shutdown 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.154+0000 7f5f29d28b80 1 -- 172.21.15.79:0/810880725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445efb790 msgr2=0x559445f06db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.155+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445efb790 0x559445f06db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5f10020040 tx=0x7f5f10004be0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.155+0000 7f5f29d28b80 1 -- 172.21.15.79:0/810880725 shutdown_connections 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.155+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559445efb790 0x559445f06db0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.155+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559445eeb3a0 0x559445ef7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.155+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x559445f6a4e0 0x559445f6c900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.155+0000 7f5f29d28b80 1 --2- 172.21.15.79:0/810880725 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559445eebcc0 0x559445f06870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.155+0000 7f5f29d28b80 1 -- 172.21.15.79:0/810880725 >> 172.21.15.79:0/810880725 conn(0x559445e09170 msgr2=0x559445eedf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.155+0000 7f5efea00700 0 client.25694 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.156+0000 7f5f29d28b80 1 -- 172.21.15.79:0/810880725 shutdown_connections 2024-06-03T02:23:05.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.156+0000 7f5f29d28b80 1 -- 172.21.15.79:0/810880725 wait complete. 2024-06-03T02:23:05.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.157+0000 7f5f29d28b80 20 client.25694 trim_cache size 0 max 16384 2024-06-03T02:23:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:23:05.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 -- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f000a2e20 msgr2=0x7f5f000a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:05.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f000a2e20 0x7f5f000a52e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009680 tx=0x7f5f0c009260 comp rx=0 tx=0).stop 2024-06-03T02:23:05.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 -- 172.21.15.79:0/4248883026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f1810a6a0 msgr2=0x7f5f1819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:05.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f1810a6a0 0x7f5f1819f760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000b970 tx=0x7f5f1000bd30 comp rx=0 tx=0).stop 2024-06-03T02:23:05.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 -- 172.21.15.79:0/4248883026 shutdown_connections 2024-06-03T02:23:05.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f1810a6a0 0x7f5f1819f760 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f181046a0 0x7f5f1819aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f000a2e20 0x7f5f000a52e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 --2- 172.21.15.79:0/4248883026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f18105060 0x7f5f1819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:05.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 -- 172.21.15.79:0/4248883026 >> 172.21.15.79:0/4248883026 conn(0x7f5f18100170 msgr2=0x7f5f18100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:05.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 -- 172.21.15.79:0/4248883026 shutdown_connections 2024-06-03T02:23:05.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:05.159+0000 7f5f1f400700 1 -- 172.21.15.79:0/4248883026 wait complete. 2024-06-03T02:23:06.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:23:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:06 smithi123 ceph-mon[36640]: pgmap v607: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 525 op/s 2024-06-03T02:23:06.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:06 smithi079 ceph-mon[31819]: pgmap v607: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 525 op/s 2024-06-03T02:23:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:06 smithi135 ceph-mon[36653]: pgmap v607: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 525 op/s 2024-06-03T02:23:06.869 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 180.0 s, 1117.4 tps, lat 88.792 ms stddev 106.064 2024-06-03T02:23:08.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:08 smithi123 ceph-mon[36640]: pgmap v608: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 547 op/s 2024-06-03T02:23:08.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:08 smithi079 ceph-mon[31819]: pgmap v608: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 547 op/s 2024-06-03T02:23:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:08 smithi135 ceph-mon[36653]: pgmap v608: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 547 op/s 2024-06-03T02:23:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:10 smithi123 ceph-mon[36640]: pgmap v609: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.9 MiB/s wr, 660 op/s 2024-06-03T02:23:10.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:10 smithi079 ceph-mon[31819]: pgmap v609: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.9 MiB/s wr, 660 op/s 2024-06-03T02:23:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:10 smithi135 ceph-mon[36653]: pgmap v609: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.9 MiB/s wr, 660 op/s 2024-06-03T02:23:11.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.743+0000 7f2147800700 1 -- 172.21.15.79:0/2385109241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2140108950 msgr2=0x7f214010f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.743+0000 7f2147800700 1 --2- 172.21.15.79:0/2385109241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2140108950 0x7f214010f380 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2130009920 tx=0x7f2130009c30 comp rx=0 tx=0).stop 2024-06-03T02:23:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.744+0000 7f2147800700 1 -- 172.21.15.79:0/2385109241 shutdown_connections 2024-06-03T02:23:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.744+0000 7f2147800700 1 --2- 172.21.15.79:0/2385109241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2140108950 0x7f214010f380 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.744+0000 7f2147800700 1 --2- 172.21.15.79:0/2385109241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2140100870 0x7f2140100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.744+0000 7f2147800700 1 --2- 172.21.15.79:0/2385109241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2140101220 0x7f2140108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.744+0000 7f2147800700 1 -- 172.21.15.79:0/2385109241 >> 172.21.15.79:0/2385109241 conn(0x7f2140077590 msgr2=0x7f21400fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:11.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.745+0000 7f2147800700 1 -- 172.21.15.79:0/2385109241 shutdown_connections 2024-06-03T02:23:11.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.745+0000 7f2147800700 1 -- 172.21.15.79:0/2385109241 wait complete. 2024-06-03T02:23:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.748+0000 7f2147800700 1 Processor -- start 2024-06-03T02:23:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.748+0000 7f2147800700 1 -- start start 2024-06-03T02:23:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.749+0000 7f2147800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2140100870 0x7f2140073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.749+0000 7f2147800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2140101220 0x7f214006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.749+0000 7f2147800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2140108950 0x7f214006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.749+0000 7f2147800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2140118e80 con 0x7f2140108950 2024-06-03T02:23:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.749+0000 7f2147800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2140118d00 con 0x7f2140100870 2024-06-03T02:23:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.749+0000 7f2147800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2140119000 con 0x7f2140101220 2024-06-03T02:23:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.750+0000 7f2144e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2140100870 0x7f2140073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.750+0000 7f2144e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2140100870 0x7f2140073cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48530/0 (socket says 172.21.15.79:48530) 2024-06-03T02:23:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.750+0000 7f2144e00700 1 -- 172.21.15.79:0/3555167821 learned_addr learned my addr 172.21.15.79:0/3555167821 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.750+0000 7f2145800700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2140108950 0x7f214006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.750+0000 7f213fe00700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2140101220 0x7f214006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.750+0000 7f2144e00700 1 -- 172.21.15.79:0/3555167821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2140101220 msgr2=0x7f214006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.750+0000 7f2144e00700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2140101220 0x7f214006cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.751+0000 7f2144e00700 1 -- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2140108950 msgr2=0x7f214006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.751+0000 7f2144e00700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2140108950 0x7f214006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.751+0000 7f2144e00700 1 -- 172.21.15.79:0/3555167821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214006d910 con 0x7f2140100870 2024-06-03T02:23:11.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.752+0000 7f2144e00700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2140100870 0x7f2140073cd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2138007760 tx=0x7f2138007b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:11.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.752+0000 7f213d600700 1 -- 172.21.15.79:0/3555167821 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138014070 con 0x7f2140100870 2024-06-03T02:23:11.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.752+0000 7f2147800700 1 -- 172.21.15.79:0/3555167821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2140071800 con 0x7f2140100870 2024-06-03T02:23:11.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.752+0000 7f213d600700 1 -- 172.21.15.79:0/3555167821 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f213800b600 con 0x7f2140100870 2024-06-03T02:23:11.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.752+0000 7f213d600700 1 -- 172.21.15.79:0/3555167821 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380173b0 con 0x7f2140100870 2024-06-03T02:23:11.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.753+0000 7f2147800700 1 -- 172.21.15.79:0/3555167821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2140071d90 con 0x7f2140100870 2024-06-03T02:23:11.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.763+0000 7f213d600700 1 -- 172.21.15.79:0/3555167821 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2138003a00 con 0x7f2140100870 2024-06-03T02:23:11.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.768+0000 7f2152138b80 1 Processor -- start 2024-06-03T02:23:11.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.769+0000 7f2152138b80 1 -- start start 2024-06-03T02:23:11.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.769+0000 7f213d600700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f21240a2c70 0x7f21240a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.769+0000 7f213d600700 1 -- 172.21.15.79:0/3555167821 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f21380cee40 con 0x7f2140100870 2024-06-03T02:23:11.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.769+0000 7f2152138b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.769+0000 7f2152138b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f09cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.769+0000 7f2152138b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f09d240 0x55ca9f09d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.769+0000 7f2152138b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ca9efaba10 con 0x55ca9f09d240 2024-06-03T02:23:11.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.769+0000 7f2152138b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca9efeeb00 con 0x55ca9f0a34b0 2024-06-03T02:23:11.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.769+0000 7f2152138b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca9efa98f0 con 0x55ca9f0a2b90 2024-06-03T02:23:11.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.770+0000 7f213fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f09cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.770+0000 7f213fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f09cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48536/0 (socket says 172.21.15.79:48536) 2024-06-03T02:23:11.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.770+0000 7f213fe00700 1 -- 172.21.15.79:0/2555902052 learned_addr learned my addr 172.21.15.79:0/2555902052 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:11.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.770+0000 7f2144e00700 1 --2- 172.21.15.79:0/2555902052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.770+0000 7f213fe00700 1 -- 172.21.15.79:0/2555902052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 msgr2=0x55ca9f0a2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:11.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.770+0000 7f213fe00700 1 --2- 172.21.15.79:0/2555902052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0a2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.770+0000 7f213fe00700 1 -- 172.21.15.79:0/2555902052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f09d240 msgr2=0x55ca9f09d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:11.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.770+0000 7f213fe00700 1 --2- 172.21.15.79:0/2555902052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f09d240 0x55ca9f09d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.770+0000 7f213fe00700 1 -- 172.21.15.79:0/2555902052 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca9ef96f90 con 0x55ca9f0a34b0 2024-06-03T02:23:11.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.773+0000 7f213fe00700 1 --2- 172.21.15.79:0/2555902052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f09cd00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f212c009c20 tx=0x7f212c009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d971ba880d6f0dd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:11.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.773+0000 7f2136000700 1 -- 172.21.15.79:0/2555902052 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c010c30 con 0x55ca9f0a34b0 2024-06-03T02:23:11.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.773+0000 7f2136000700 1 -- 172.21.15.79:0/2555902052 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f212c004500 con 0x55ca9f0a34b0 2024-06-03T02:23:11.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.773+0000 7f2136000700 1 -- 172.21.15.79:0/2555902052 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c011440 con 0x55ca9f0a34b0 2024-06-03T02:23:11.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.773+0000 7f2145800700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f21240a2c70 0x7f21240a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.774+0000 7f2152138b80 1 -- 172.21.15.79:0/2555902052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 msgr2=0x55ca9f09cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.774+0000 7f2152138b80 1 --2- 172.21.15.79:0/2555902052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f09cd00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f212c009c20 tx=0x7f212c009f30 comp rx=0 tx=0).stop 2024-06-03T02:23:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.774+0000 7f2145800700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f21240a2c70 0x7f21240a5130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2130009680 tx=0x7f2130005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.776+0000 7f2152138b80 1 -- 172.21.15.79:0/2555902052 shutdown_connections 2024-06-03T02:23:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.776+0000 7f2152138b80 1 --2- 172.21.15.79:0/2555902052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f09d240 0x55ca9f09d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.776+0000 7f2152138b80 1 --2- 172.21.15.79:0/2555902052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.776+0000 7f2152138b80 1 --2- 172.21.15.79:0/2555902052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f09cd00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.776+0000 7f2152138b80 1 -- 172.21.15.79:0/2555902052 >> 172.21.15.79:0/2555902052 conn(0x55ca9efc9170 msgr2=0x55ca9f097ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.776+0000 7f2152138b80 1 -- 172.21.15.79:0/2555902052 shutdown_connections 2024-06-03T02:23:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.776+0000 7f2152138b80 1 -- 172.21.15.79:0/2555902052 wait complete. 2024-06-03T02:23:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.777+0000 7f2152138b80 1 -- start start 2024-06-03T02:23:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.778+0000 7f2152138b80 1 Processor -- start 2024-06-03T02:23:11.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.779+0000 7f2152138b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f09d240 0x55ca9f0a9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.782+0000 7f2152138b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0a6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.782+0000 7f2152138b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f0a6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.782+0000 7f2152138b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ca9ef8af60 con 0x55ca9f0a2b90 2024-06-03T02:23:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.782+0000 7f2152138b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca9efeaac0 con 0x55ca9f0a34b0 2024-06-03T02:23:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.782+0000 7f2152138b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca9f0a9ee0 con 0x55ca9f09d240 2024-06-03T02:23:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.782+0000 7f2152138b80 10 client.0 _ll_register_callbacks cb 0x55ca9ef84380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f2144e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f09d240 0x55ca9f0a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f2144e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f09d240 0x55ca9f0a9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33038/0 (socket says 172.21.15.79:33038) 2024-06-03T02:23:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f2144e00700 1 -- 172.21.15.79:0/3077650529 learned_addr learned my addr 172.21.15.79:0/3077650529 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f2145800700 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f0a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f213fe00700 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f2144e00700 1 -- 172.21.15.79:0/3077650529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 msgr2=0x55ca9f0a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:11.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f2144e00700 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f0a6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f2144e00700 1 -- 172.21.15.79:0/3077650529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a2b90 msgr2=0x55ca9f0a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:11.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f2144e00700 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.783+0000 7f2144e00700 1 -- 172.21.15.79:0/3077650529 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca9ef3d8d0 con 0x55ca9f09d240 2024-06-03T02:23:11.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.785+0000 7f2144e00700 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f09d240 0x55ca9f0a9150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2138097750 tx=0x7f213800f1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44469c76de7b2a62 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:11.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.785+0000 7f212aa00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:23:11.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.785+0000 7f212aa00700 1 -- 172.21.15.79:0/3077650529 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138095e60 con 0x55ca9f09d240 2024-06-03T02:23:11.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.785+0000 7f212aa00700 1 -- 172.21.15.79:0/3077650529 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2138003cf0 con 0x55ca9f09d240 2024-06-03T02:23:11.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.785+0000 7f2152138b80 1 -- 172.21.15.79:0/3077650529 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f212c004500 con 0x55ca9f09d240 2024-06-03T02:23:11.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.786+0000 7f212aa00700 1 -- 172.21.15.79:0/3077650529 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f213809b2f0 con 0x55ca9f09d240 2024-06-03T02:23:11.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.786+0000 7f2152138b80 10 client.25678 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:11.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.786+0000 7f2152138b80 1 -- 172.21.15.79:0/3077650529 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ca9f0b4690 con 0x55ca9f09d240 2024-06-03T02:23:11.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.786+0000 7f212aa00700 1 -- 172.21.15.79:0/3077650529 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f212c004500 con 0x55ca9f09d240 2024-06-03T02:23:11.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.786+0000 7f212aa00700 1 -- 172.21.15.79:0/3077650529 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2138097960 con 0x55ca9f09d240 2024-06-03T02:23:11.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.786+0000 7f2152138b80 10 client.25678 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:11.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.786+0000 7f2152138b80 10 client.25678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:23:11.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.786+0000 7f2152138b80 20 client.25678 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:11.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.787+0000 7f2152138b80 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55ca9f0b6c20 0x55ca9f0b9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.787+0000 7f2152138b80 4 client.25678 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:23:11.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.787+0000 7f2152138b80 1 -- 172.21.15.79:0/3077650529 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca9effc000 con 0x55ca9f0b6c20 2024-06-03T02:23:11.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.788+0000 7f213fe00700 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55ca9f0b6c20 0x55ca9f0b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.790+0000 7f213fe00700 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55ca9f0b6c20 0x55ca9f0b9010 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd9e2805be35736d server_cookie=3506b62d432aab58 in_seq=0 out_seq=0 2024-06-03T02:23:11.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.790+0000 7f212aa00700 10 client.25678 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:23:11.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.791+0000 7f212aa00700 1 -- 172.21.15.79:0/3077650529 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ca9effc000 con 0x55ca9f0b6c20 2024-06-03T02:23:11.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.792+0000 7f212aa00700 10 client.25678 handle_command_reply: tid=0 2024-06-03T02:23:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.795+0000 7f2152138b80 1 client.25678 shutdown 2024-06-03T02:23:11.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 -- 172.21.15.79:0/3077650529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f09d240 msgr2=0x55ca9f0a9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:11.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f09d240 0x55ca9f0a9150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2138097750 tx=0x7f213800f1d0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 -- 172.21.15.79:0/3077650529 shutdown_connections 2024-06-03T02:23:11.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55ca9f0b6c20 0x55ca9f0b9010 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a34b0 0x55ca9f0a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f09d240 0x55ca9f0a9150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 --2- 172.21.15.79:0/3077650529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 -- 172.21.15.79:0/3077650529 >> 172.21.15.79:0/3077650529 conn(0x55ca9efc9170 msgr2=0x55ca9f091790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:11.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f212aa00700 0 client.25678 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:23:11.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 -- 172.21.15.79:0/3077650529 shutdown_connections 2024-06-03T02:23:11.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 1 -- 172.21.15.79:0/3077650529 wait complete. 2024-06-03T02:23:11.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.797+0000 7f2152138b80 20 client.25678 trim_cache size 0 max 16384 2024-06-03T02:23:11.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.828+0000 7f2152138b80 1 Processor -- start 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.828+0000 7f2152138b80 1 -- start start 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.828+0000 7f2152138b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f09d240 0x55ca9f09cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.828+0000 7f2152138b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.828+0000 7f2152138b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a34b0 0x55ca9f092130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.828+0000 7f2152138b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f21401c5b50 con 0x55ca9f09d240 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.828+0000 7f2152138b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21401c7690 con 0x55ca9f0a2b90 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.828+0000 7f2152138b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f21401c6220 con 0x55ca9f0a34b0 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.829+0000 7f2145800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a34b0 0x55ca9f092130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.829+0000 7f2145800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a34b0 0x55ca9f092130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33046/0 (socket says 172.21.15.79:33046) 2024-06-03T02:23:11.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.829+0000 7f2145800700 1 -- 172.21.15.79:0/27286425 learned_addr learned my addr 172.21.15.79:0/27286425 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:11.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.832+0000 7f2145800700 1 -- 172.21.15.79:0/27286425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a2b90 msgr2=0x55ca9f0981a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:23:11.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.832+0000 7f2145800700 1 --2- 172.21.15.79:0/27286425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0981a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.833+0000 7f2145800700 1 -- 172.21.15.79:0/27286425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f09d240 msgr2=0x55ca9f09cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:11.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.833+0000 7f2145800700 1 --2- 172.21.15.79:0/27286425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f09d240 0x55ca9f09cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.833+0000 7f2145800700 1 -- 172.21.15.79:0/27286425 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca9f0a6220 con 0x55ca9f0a34b0 2024-06-03T02:23:11.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.833+0000 7f2145800700 1 --2- 172.21.15.79:0/27286425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a34b0 0x55ca9f092130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2130009920 tx=0x7f2130020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54dd50a9c87b1e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:11.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.834+0000 7f2135600700 1 -- 172.21.15.79:0/27286425 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130005490 con 0x55ca9f0a34b0 2024-06-03T02:23:11.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.834+0000 7f2135600700 1 -- 172.21.15.79:0/27286425 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2130005630 con 0x55ca9f0a34b0 2024-06-03T02:23:11.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.834+0000 7f2135600700 1 -- 172.21.15.79:0/27286425 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213000b920 con 0x55ca9f0a34b0 2024-06-03T02:23:11.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.836+0000 7f2152138b80 1 -- 172.21.15.79:0/27286425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a34b0 msgr2=0x55ca9f092130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:11.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.836+0000 7f2152138b80 1 --2- 172.21.15.79:0/27286425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a34b0 0x55ca9f092130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2130009920 tx=0x7f2130020000 comp rx=0 tx=0).stop 2024-06-03T02:23:11.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.837+0000 7f2152138b80 1 -- 172.21.15.79:0/27286425 shutdown_connections 2024-06-03T02:23:11.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.837+0000 7f2152138b80 1 --2- 172.21.15.79:0/27286425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a34b0 0x55ca9f092130 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.837+0000 7f2152138b80 1 --2- 172.21.15.79:0/27286425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f09d240 0x55ca9f09cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.837+0000 7f2152138b80 1 --2- 172.21.15.79:0/27286425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.837+0000 7f2152138b80 1 -- 172.21.15.79:0/27286425 >> 172.21.15.79:0/27286425 conn(0x55ca9efc9170 msgr2=0x55ca9f097a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:11.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.837+0000 7f2152138b80 1 -- 172.21.15.79:0/27286425 shutdown_connections 2024-06-03T02:23:11.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.838+0000 7f2152138b80 1 -- 172.21.15.79:0/27286425 wait complete. 2024-06-03T02:23:11.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.838+0000 7f2152138b80 1 -- start start 2024-06-03T02:23:11.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.839+0000 7f2152138b80 1 Processor -- start 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f2152138b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f09d240 0x55ca9f0bf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f2152138b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f2152138b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a34b0 0x55ca9f0af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f2152138b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f21401c6fc0 con 0x55ca9f0a34b0 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f2152138b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21401c68f0 con 0x55ca9f09d240 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f2152138b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f21401c3af0 con 0x55ca9f0a2b90 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f2152138b80 10 client.0 _ll_register_callbacks cb 0x55ca9ef84380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f213fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f213fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33060/0 (socket says 172.21.15.79:33060) 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f213fe00700 1 -- 172.21.15.79:0/3097669935 learned_addr learned my addr 172.21.15.79:0/3097669935 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.842+0000 7f2144e00700 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f09d240 0x55ca9f0bf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.844+0000 7f2145800700 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a34b0 0x55ca9f0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.844+0000 7f213fe00700 1 -- 172.21.15.79:0/3097669935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f09d240 msgr2=0x55ca9f0bf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:11.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.845+0000 7f213fe00700 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f09d240 0x55ca9f0bf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.845+0000 7f213fe00700 1 -- 172.21.15.79:0/3097669935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a34b0 msgr2=0x55ca9f0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:11.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.845+0000 7f213fe00700 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a34b0 0x55ca9f0af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.845+0000 7f213fe00700 1 -- 172.21.15.79:0/3097669935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca9f0bd860 con 0x55ca9f0a2b90 2024-06-03T02:23:11.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.846+0000 7f213fe00700 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0af1f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f212c003b90 tx=0x7f212c010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43688c137be7c3e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:11.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.846+0000 7f212aa00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:23:11.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.847+0000 7f212aa00700 1 -- 172.21.15.79:0/3097669935 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c004910 con 0x55ca9f0a2b90 2024-06-03T02:23:11.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.847+0000 7f2152138b80 1 -- 172.21.15.79:0/3097669935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2130005630 con 0x55ca9f0a2b90 2024-06-03T02:23:11.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.848+0000 7f212aa00700 1 -- 172.21.15.79:0/3097669935 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f212c004ab0 con 0x55ca9f0a2b90 2024-06-03T02:23:11.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.849+0000 7f212aa00700 1 -- 172.21.15.79:0/3097669935 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f212c035050 con 0x55ca9f0a2b90 2024-06-03T02:23:11.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.849+0000 7f2152138b80 10 client.25684 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:11.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.849+0000 7f2152138b80 1 -- 172.21.15.79:0/3097669935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ca9f120b10 con 0x55ca9f0a2b90 2024-06-03T02:23:11.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.849+0000 7f212aa00700 1 -- 172.21.15.79:0/3097669935 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2130005630 con 0x55ca9f0a2b90 2024-06-03T02:23:11.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.850+0000 7f212aa00700 1 -- 172.21.15.79:0/3097669935 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f212c003cc0 con 0x55ca9f0a2b90 2024-06-03T02:23:11.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.850+0000 7f2152138b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:11.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.850+0000 7f2152138b80 10 client.25684 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:23:11.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.851+0000 7f2152138b80 20 client.25684 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:11.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.851+0000 7f2152138b80 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55ca9f121090 0x55ca9f1234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:11.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.851+0000 7f2152138b80 4 client.25684 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:23:11.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.851+0000 7f2152138b80 1 -- 172.21.15.79:0/3097669935 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca9eff6580 con 0x55ca9f121090 2024-06-03T02:23:11.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.853+0000 7f2144e00700 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55ca9f121090 0x55ca9f1234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:11.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.854+0000 7f2144e00700 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55ca9f121090 0x55ca9f1234b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8eb606a0e604ab78 server_cookie=c8664b3ac6140c75 in_seq=0 out_seq=0 2024-06-03T02:23:11.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.854+0000 7f212aa00700 10 client.25684 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:23:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.854+0000 7f212aa00700 1 -- 172.21.15.79:0/3097669935 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca9eff6580 con 0x55ca9f121090 2024-06-03T02:23:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.854+0000 7f212aa00700 10 client.25684 handle_command_reply: tid=0 2024-06-03T02:23:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f2152138b80 1 client.25684 shutdown 2024-06-03T02:23:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f2152138b80 1 -- 172.21.15.79:0/3097669935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 msgr2=0x55ca9f0af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f2152138b80 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0af1f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f212c003b90 tx=0x7f212c010ab0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f2152138b80 1 -- 172.21.15.79:0/3097669935 shutdown_connections 2024-06-03T02:23:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f2152138b80 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ca9f0a34b0 0x55ca9f0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f2152138b80 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55ca9f121090 0x55ca9f1234b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f2152138b80 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca9f09d240 0x55ca9f0bf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f2152138b80 1 --2- 172.21.15.79:0/3097669935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca9f0a2b90 0x55ca9f0af1f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f2152138b80 1 -- 172.21.15.79:0/3097669935 >> 172.21.15.79:0/3097669935 conn(0x55ca9efc9170 msgr2=0x55ca9f0a3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.855+0000 7f212aa00700 0 client.25684 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:23:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.856+0000 7f2152138b80 1 -- 172.21.15.79:0/3097669935 shutdown_connections 2024-06-03T02:23:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.856+0000 7f2152138b80 1 -- 172.21.15.79:0/3097669935 wait complete. 2024-06-03T02:23:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.856+0000 7f2152138b80 20 client.25684 trim_cache size 0 max 16384 2024-06-03T02:23:11.858 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:23:11.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 -- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f21240a2c70 msgr2=0x7f21240a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:11.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f21240a2c70 0x7f21240a5130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2130009680 tx=0x7f2130005b40 comp rx=0 tx=0).stop 2024-06-03T02:23:11.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 -- 172.21.15.79:0/3555167821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2140100870 msgr2=0x7f2140073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:11.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2140100870 0x7f2140073cd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2138007760 tx=0x7f2138007b20 comp rx=0 tx=0).stop 2024-06-03T02:23:11.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 -- 172.21.15.79:0/3555167821 shutdown_connections 2024-06-03T02:23:11.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f21240a2c70 0x7f21240a5130 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2140108950 0x7f214006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2140100870 0x7f2140073cd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 --2- 172.21.15.79:0/3555167821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2140101220 0x7f214006cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:11.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.859+0000 7f2147800700 1 -- 172.21.15.79:0/3555167821 >> 172.21.15.79:0/3555167821 conn(0x7f2140077590 msgr2=0x7f21401065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:11.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.860+0000 7f2147800700 1 -- 172.21.15.79:0/3555167821 shutdown_connections 2024-06-03T02:23:11.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:11.860+0000 7f2147800700 1 -- 172.21.15.79:0/3555167821 wait complete. 2024-06-03T02:23:12.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:12 smithi123 ceph-mon[36640]: pgmap v610: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 561 op/s 2024-06-03T02:23:12.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:12 smithi079 ceph-mon[31819]: pgmap v610: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 561 op/s 2024-06-03T02:23:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:12 smithi135 ceph-mon[36653]: pgmap v610: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 561 op/s 2024-06-03T02:23:13.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:23:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 185.0 s, 911.2 tps, lat 109.607 ms stddev 153.012 2024-06-03T02:23:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:14.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:14 smithi079 ceph-mon[31819]: pgmap v611: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 841 op/s 2024-06-03T02:23:14.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:14 smithi123 ceph-mon[36640]: pgmap v611: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 841 op/s 2024-06-03T02:23:14.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:14 smithi135 ceph-mon[36653]: pgmap v611: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 841 op/s 2024-06-03T02:23:16.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:16 smithi079 ceph-mon[31819]: pgmap v612: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 MiB/s wr, 759 op/s 2024-06-03T02:23:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:16 smithi135 ceph-mon[36653]: pgmap v612: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 MiB/s wr, 759 op/s 2024-06-03T02:23:16.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:16 smithi123 ceph-mon[36640]: pgmap v612: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 MiB/s wr, 759 op/s 2024-06-03T02:23:17.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.293+0000 7fcbd5400700 1 -- 172.21.15.79:0/2970966781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbd0109530 msgr2=0x7fcbd0109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:17.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.293+0000 7fcbd5400700 1 --2- 172.21.15.79:0/2970966781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbd0109530 0x7fcbd0109910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009fc0 tx=0x7fcbc000b330 comp rx=0 tx=0).stop 2024-06-03T02:23:17.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.296+0000 7fcbd5400700 1 -- 172.21.15.79:0/2970966781 shutdown_connections 2024-06-03T02:23:17.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.296+0000 7fcbd5400700 1 --2- 172.21.15.79:0/2970966781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd010ddd0 0x7fcbd01158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.296+0000 7fcbd5400700 1 --2- 172.21.15.79:0/2970966781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbd0109530 0x7fcbd0109910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.296+0000 7fcbd5400700 1 --2- 172.21.15.79:0/2970966781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd0109ee0 0x7fcbd010d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.297+0000 7fcbd5400700 1 -- 172.21.15.79:0/2970966781 >> 172.21.15.79:0/2970966781 conn(0x7fcbd00781c0 msgr2=0x7fcbd007a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:17.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.297+0000 7fcbd5400700 1 -- 172.21.15.79:0/2970966781 shutdown_connections 2024-06-03T02:23:17.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.297+0000 7fcbd5400700 1 -- 172.21.15.79:0/2970966781 wait complete. 2024-06-03T02:23:17.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.298+0000 7fcbd5400700 1 Processor -- start 2024-06-03T02:23:17.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.299+0000 7fcbd5400700 1 -- start start 2024-06-03T02:23:17.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.299+0000 7fcbd5400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd0109530 0x7fcbd019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.299+0000 7fcbd5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd0109ee0 0x7fcbd019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.299+0000 7fcbd5400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbd010ddd0 0x7fcbd01a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.299+0000 7fcbd5400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0118d20 con 0x7fcbd010ddd0 2024-06-03T02:23:17.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.299+0000 7fcbd5400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0118ba0 con 0x7fcbd0109530 2024-06-03T02:23:17.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.299+0000 7fcbd5400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0118ea0 con 0x7fcbd0109ee0 2024-06-03T02:23:17.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.300+0000 7fcbcfe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd0109530 0x7fcbd019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.300+0000 7fcbcfe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd0109530 0x7fcbd019f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59134/0 (socket says 172.21.15.79:59134) 2024-06-03T02:23:17.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.300+0000 7fcbcfe00700 1 -- 172.21.15.79:0/3935090619 learned_addr learned my addr 172.21.15.79:0/3935090619 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:17.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.300+0000 7fcbd4a00700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbd010ddd0 0x7fcbd01a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.301+0000 7fcbd4a00700 1 -- 172.21.15.79:0/3935090619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd0109ee0 msgr2=0x7fcbd019f710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:23:17.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.301+0000 7fcbd4a00700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd0109ee0 0x7fcbd019f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.301+0000 7fcbd4a00700 1 -- 172.21.15.79:0/3935090619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd0109530 msgr2=0x7fcbd019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:17.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.301+0000 7fcbd4a00700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd0109530 0x7fcbd019f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.301+0000 7fcbd4a00700 1 -- 172.21.15.79:0/3935090619 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd01a41e0 con 0x7fcbd010ddd0 2024-06-03T02:23:17.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.301+0000 7fcbd4a00700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbd010ddd0 0x7fcbd01a3aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800d720 tx=0x7fcbc800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:17.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.302+0000 7fcbcd600700 1 -- 172.21.15.79:0/3935090619 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8017070 con 0x7fcbd010ddd0 2024-06-03T02:23:17.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.302+0000 7fcbcd600700 1 -- 172.21.15.79:0/3935090619 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc8005020 con 0x7fcbd010ddd0 2024-06-03T02:23:17.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.302+0000 7fcbcd600700 1 -- 172.21.15.79:0/3935090619 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc800fbf0 con 0x7fcbd010ddd0 2024-06-03T02:23:17.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.302+0000 7fcbd5400700 1 -- 172.21.15.79:0/3935090619 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd01a44d0 con 0x7fcbd010ddd0 2024-06-03T02:23:17.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.302+0000 7fcbd5400700 1 -- 172.21.15.79:0/3935090619 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd007cd00 con 0x7fcbd010ddd0 2024-06-03T02:23:17.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.313+0000 7fcbe13fcb80 1 Processor -- start 2024-06-03T02:23:17.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.315+0000 7fcbcd600700 1 -- 172.21.15.79:0/3935090619 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcbc80051d0 con 0x7fcbd010ddd0 2024-06-03T02:23:17.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.316+0000 7fcbe13fcb80 1 -- start start 2024-06-03T02:23:17.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.316+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709897140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.317+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709897680 0x55c7098a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.317+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7098a2530 0x55c7098a4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.317+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c7097a6a10 con 0x55c709897680 2024-06-03T02:23:17.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.317+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c7097e9b00 con 0x55c7098a2530 2024-06-03T02:23:17.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.317+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c7097a48f0 con 0x55c709896d60 2024-06-03T02:23:17.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.318+0000 7fcbcd600700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcbb40a2ca0 0x7fcbb40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.318+0000 7fcbcd600700 1 -- 172.21.15.79:0/3935090619 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fcbc80ceb00 con 0x7fcbd010ddd0 2024-06-03T02:23:17.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.318+0000 7fcbd4a00700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcbb40a2ca0 0x7fcbb40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.319+0000 7fcbd4a00700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcbb40a2ca0 0x7fcbb40a5160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80cc250 tx=0x7fcbc800df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:17.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.321+0000 7fcbc7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709897140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.321+0000 7fcbc7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7098a2530 0x55c7098a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.321+0000 7fcbc7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709897140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37298/0 (socket says 172.21.15.79:37298) 2024-06-03T02:23:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.321+0000 7fcbc7400700 1 -- 172.21.15.79:0/1602183972 learned_addr learned my addr 172.21.15.79:0/1602183972 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.321+0000 7fcbc7400700 1 -- 172.21.15.79:0/1602183972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 msgr2=0x55c709897140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.321+0000 7fcbc7400700 1 --2- 172.21.15.79:0/1602183972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709897140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.321+0000 7fcbc7400700 1 -- 172.21.15.79:0/1602183972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709897680 msgr2=0x55c7098a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:17.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.321+0000 7fcbc7400700 1 --2- 172.21.15.79:0/1602183972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709897680 0x55c7098a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.321+0000 7fcbc7400700 1 -- 172.21.15.79:0/1602183972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c709791f90 con 0x55c7098a2530 2024-06-03T02:23:17.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.323+0000 7fcbc7400700 1 --2- 172.21.15.79:0/1602183972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7098a2530 0x55c7098a4920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8013750 tx=0x7fcbb8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a91400e19c3e604e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:17.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.323+0000 7fcbc5600700 1 -- 172.21.15.79:0/1602183972 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8004a30 con 0x55c7098a2530 2024-06-03T02:23:17.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.323+0000 7fcbc5600700 1 -- 172.21.15.79:0/1602183972 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb8009e30 con 0x55c7098a2530 2024-06-03T02:23:17.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.323+0000 7fcbc5600700 1 -- 172.21.15.79:0/1602183972 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800f770 con 0x55c7098a2530 2024-06-03T02:23:17.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.324+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/1602183972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7098a2530 msgr2=0x55c7098a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:17.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.324+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/1602183972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7098a2530 0x55c7098a4920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8013750 tx=0x7fcbb8004750 comp rx=0 tx=0).stop 2024-06-03T02:23:17.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.324+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/1602183972 shutdown_connections 2024-06-03T02:23:17.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.324+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/1602183972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7098a2530 0x55c7098a4920 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8013750 tx=0x7fcbb8004750 comp rx=0 tx=0).stop 2024-06-03T02:23:17.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.324+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/1602183972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709897140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.324+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/1602183972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709897680 0x55c7098a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.325+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/1602183972 >> 172.21.15.79:0/1602183972 conn(0x55c7097c4170 msgr2=0x55c709893280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:17.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.325+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/1602183972 shutdown_connections 2024-06-03T02:23:17.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.325+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/1602183972 wait complete. 2024-06-03T02:23:17.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.326+0000 7fcbe13fcb80 1 -- start start 2024-06-03T02:23:17.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.326+0000 7fcbe13fcb80 1 Processor -- start 2024-06-03T02:23:17.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.328+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 0x55c7098a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.328+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709897680 0x55c70989f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.329+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989f720 0x55c70989fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.329+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c709785f60 con 0x55c709896d60 2024-06-03T02:23:17.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.329+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c7097e5ac0 con 0x55c70989f720 2024-06-03T02:23:17.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.329+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c7098a5020 con 0x55c709897680 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.329+0000 7fcbe13fcb80 10 client.0 _ll_register_callbacks cb 0x55c70977f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.329+0000 7fcbcfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709897680 0x55c70989f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.329+0000 7fcbcfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709897680 0x55c70989f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37308/0 (socket says 172.21.15.79:37308) 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.329+0000 7fcbcfe00700 1 -- 172.21.15.79:0/2339771137 learned_addr learned my addr 172.21.15.79:0/2339771137 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.329+0000 7fcbc7400700 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 0x55c7098a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.330+0000 7fcbcfe00700 1 -- 172.21.15.79:0/2339771137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989f720 msgr2=0x55c70989fb80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.330+0000 7fcbcfe00700 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989f720 0x55c70989fb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.330+0000 7fcbcfe00700 1 -- 172.21.15.79:0/2339771137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 msgr2=0x55c7098a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.330+0000 7fcbcfe00700 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 0x55c7098a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.330+0000 7fcbcfe00700 1 -- 172.21.15.79:0/2339771137 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7097388d0 con 0x55c709897680 2024-06-03T02:23:17.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.330+0000 7fcbc7400700 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 0x55c7098a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:23:17.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.330+0000 7fcbcfe00700 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709897680 0x55c70989f1e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0004db0 tx=0x7fcbc0004e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbe789134c166eaa server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:17.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.330+0000 7fcbbe000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:23:17.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.331+0000 7fcbbe000700 1 -- 172.21.15.79:0/2339771137 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0022070 con 0x55c709897680 2024-06-03T02:23:17.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.331+0000 7fcbbe000700 1 -- 172.21.15.79:0/2339771137 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fcbc001c440 con 0x55c709897680 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.331+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2339771137 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb8009e30 con 0x55c709897680 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.331+0000 7fcbbe000700 1 -- 172.21.15.79:0/2339771137 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbc002c050 con 0x55c709897680 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.331+0000 7fcbe13fcb80 10 client.25693 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.331+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2339771137 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c7098afdb0 con 0x55c709897680 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.332+0000 7fcbbe000700 1 -- 172.21.15.79:0/2339771137 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb8009e30 con 0x55c709897680 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.332+0000 7fcbbe000700 1 -- 172.21.15.79:0/2339771137 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcbc0011070 con 0x55c709897680 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.332+0000 7fcbe13fcb80 10 client.25693 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.332+0000 7fcbe13fcb80 10 client.25693 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.332+0000 7fcbe13fcb80 20 client.25693 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.332+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55c7098b2440 0x55c7098b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.332+0000 7fcbe13fcb80 4 client.25693 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:23:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.332+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2339771137 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7097f7000 con 0x55c7098b2440 2024-06-03T02:23:17.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.333+0000 7fcbc7400700 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55c7098b2440 0x55c7098b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.335+0000 7fcbc7400700 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55c7098b2440 0x55c7098b48a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b65f141bdec8ec5a server_cookie=f58499fbd32444d2 in_seq=0 out_seq=0 2024-06-03T02:23:17.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.335+0000 7fcbbe000700 10 client.25693 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:23:17.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.337+0000 7fcbbe000700 1 -- 172.21.15.79:0/2339771137 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c7097f7000 con 0x55c7098b2440 2024-06-03T02:23:17.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.337+0000 7fcbbe000700 10 client.25693 handle_command_reply: tid=0 2024-06-03T02:23:17.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.337+0000 7fcbe13fcb80 1 client.25693 shutdown 2024-06-03T02:23:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.338+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2339771137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709897680 msgr2=0x55c70989f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.338+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709897680 0x55c70989f1e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0004db0 tx=0x7fcbc0004e90 comp rx=0 tx=0).stop 2024-06-03T02:23:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.338+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2339771137 shutdown_connections 2024-06-03T02:23:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.338+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989f720 0x55c70989fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.338+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 0x55c7098a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.338+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55c7098b2440 0x55c7098b48a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.338+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2339771137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709897680 0x55c70989f1e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.338+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2339771137 >> 172.21.15.79:0/2339771137 conn(0x55c7097c4170 msgr2=0x55c709892e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:17.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.339+0000 7fcbbe000700 0 client.25693 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:23:17.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.339+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2339771137 shutdown_connections 2024-06-03T02:23:17.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.339+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2339771137 wait complete. 2024-06-03T02:23:17.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.339+0000 7fcbe13fcb80 20 client.25693 trim_cache size 0 max 16384 2024-06-03T02:23:17.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.375+0000 7fcbe13fcb80 1 Processor -- start 2024-06-03T02:23:17.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.376+0000 7fcbe13fcb80 1 -- start start 2024-06-03T02:23:17.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.376+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 0x55c70989dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.376+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70989f720 0x55c70989e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.376+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989ea90 0x55c70989eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.376+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01bd0f0 con 0x55c709896d60 2024-06-03T02:23:17.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.376+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01bec30 con 0x55c70989ea90 2024-06-03T02:23:17.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.376+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01bd7c0 con 0x55c70989f720 2024-06-03T02:23:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.377+0000 7fcbc7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989ea90 0x55c70989eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.377+0000 7fcbcfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70989f720 0x55c70989e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.377+0000 7fcbcfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70989f720 0x55c70989e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37318/0 (socket says 172.21.15.79:37318) 2024-06-03T02:23:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.377+0000 7fcbcfe00700 1 -- 172.21.15.79:0/3802518468 learned_addr learned my addr 172.21.15.79:0/3802518468 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.377+0000 7fcbc7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989ea90 0x55c70989eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59152/0 (socket says 172.21.15.79:59152) 2024-06-03T02:23:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.378+0000 7fcbcfe00700 1 -- 172.21.15.79:0/3802518468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989ea90 msgr2=0x55c70989eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.378+0000 7fcbcfe00700 1 --2- 172.21.15.79:0/3802518468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989ea90 0x55c70989eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.378+0000 7fcbcfe00700 1 -- 172.21.15.79:0/3802518468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 msgr2=0x55c70989dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:17.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.378+0000 7fcbcfe00700 1 --2- 172.21.15.79:0/3802518468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 0x55c70989dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.378+0000 7fcbcfe00700 1 -- 172.21.15.79:0/3802518468 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c709893d50 con 0x55c70989f720 2024-06-03T02:23:17.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.380+0000 7fcbcfe00700 1 --2- 172.21.15.79:0/3802518468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70989f720 0x55c70989e010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009960 tx=0x7fcbc0027d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616088ed0bfa87a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:17.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.380+0000 7fcbc4c00700 1 -- 172.21.15.79:0/3802518468 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc002c050 con 0x55c70989f720 2024-06-03T02:23:17.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.380+0000 7fcbc4c00700 1 -- 172.21.15.79:0/3802518468 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc000bd00 con 0x55c70989f720 2024-06-03T02:23:17.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.380+0000 7fcbc4c00700 1 -- 172.21.15.79:0/3802518468 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0031920 con 0x55c70989f720 2024-06-03T02:23:17.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.382+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/3802518468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70989f720 msgr2=0x55c70989e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:17.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.382+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/3802518468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70989f720 0x55c70989e010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009960 tx=0x7fcbc0027d90 comp rx=0 tx=0).stop 2024-06-03T02:23:17.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.383+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/3802518468 shutdown_connections 2024-06-03T02:23:17.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.383+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/3802518468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989ea90 0x55c70989eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.383+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/3802518468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c709896d60 0x55c70989dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.383+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/3802518468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c70989f720 0x55c70989e010 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.383+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/3802518468 >> 172.21.15.79:0/3802518468 conn(0x55c7097c4170 msgr2=0x55c7098b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:17.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.383+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/3802518468 shutdown_connections 2024-06-03T02:23:17.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.384+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/3802518468 wait complete. 2024-06-03T02:23:17.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.385+0000 7fcbe13fcb80 1 -- start start 2024-06-03T02:23:17.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.385+0000 7fcbe13fcb80 1 Processor -- start 2024-06-03T02:23:17.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.386+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709918130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.386+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c70989ea90 0x55c709918670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.387+0000 7fcbc7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709918130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.387+0000 7fcbe13fcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989f720 0x55c709918bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.387+0000 7fcbc7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709918130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37324/0 (socket says 172.21.15.79:37324) 2024-06-03T02:23:17.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.387+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01be560 con 0x55c70989ea90 2024-06-03T02:23:17.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.387+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01bde90 con 0x55c70989f720 2024-06-03T02:23:17.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.387+0000 7fcbe13fcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0083800 con 0x55c709896d60 2024-06-03T02:23:17.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.387+0000 7fcbe13fcb80 10 client.0 _ll_register_callbacks cb 0x55c70977f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:17.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.387+0000 7fcbcfe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c70989ea90 0x55c709918670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.387+0000 7fcbcfe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c70989ea90 0x55c709918670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41882/0 (socket says 172.21.15.79:41882) 2024-06-03T02:23:17.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.388+0000 7fcbcfe00700 1 -- 172.21.15.79:0/2173054039 learned_addr learned my addr 172.21.15.79:0/2173054039 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:17.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.388+0000 7fcbc7400700 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989f720 0x55c709918bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.388+0000 7fcbc7400700 1 -- 172.21.15.79:0/2173054039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989f720 msgr2=0x55c709918bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:17.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.388+0000 7fcbc7400700 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989f720 0x55c709918bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.388+0000 7fcbc7400700 1 -- 172.21.15.79:0/2173054039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c70989ea90 msgr2=0x55c709918670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:17.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.388+0000 7fcbc7400700 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c70989ea90 0x55c709918670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.388+0000 7fcbc7400700 1 -- 172.21.15.79:0/2173054039 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7098a8260 con 0x55c709896d60 2024-06-03T02:23:17.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.389+0000 7fcbc7400700 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709918130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800d000 tx=0x7fcbb8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b483370cd31c571c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:17.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.389+0000 7fcbbe000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:23:17.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.389+0000 7fcbbe000700 1 -- 172.21.15.79:0/2173054039 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8004050 con 0x55c709896d60 2024-06-03T02:23:17.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.389+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2173054039 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc000bd00 con 0x55c709896d60 2024-06-03T02:23:17.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.389+0000 7fcbbe000700 1 -- 172.21.15.79:0/2173054039 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fcbb80041f0 con 0x55c709896d60 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.389+0000 7fcbbe000700 1 -- 172.21.15.79:0/2173054039 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbb80043f0 con 0x55c709896d60 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbe13fcb80 10 client.25699 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2173054039 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c70992e560 con 0x55c709896d60 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbbe000700 1 -- 172.21.15.79:0/2173054039 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc000bd00 con 0x55c709896d60 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbbe000700 1 -- 172.21.15.79:0/2173054039 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcbb8003980 con 0x55c709896d60 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbe13fcb80 10 client.25699 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbe13fcb80 10 client.25699 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbe13fcb80 20 client.25699 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55c70992eaa0 0x55c709930e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbe13fcb80 4 client.25699 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:23:17.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.390+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2173054039 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7097f1580 con 0x55c70992eaa0 2024-06-03T02:23:17.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.391+0000 7fcbcfe00700 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55c70992eaa0 0x55c709930e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.392+0000 7fcbcfe00700 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55c70992eaa0 0x55c709930e90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37f0e36ff004cb0e server_cookie=9575cfe1fd71222e in_seq=0 out_seq=0 2024-06-03T02:23:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.392+0000 7fcbbe000700 10 client.25699 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:23:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbbe000700 1 -- 172.21.15.79:0/2173054039 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7097f1580 con 0x55c70992eaa0 2024-06-03T02:23:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbbe000700 10 client.25699 handle_command_reply: tid=0 2024-06-03T02:23:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbe13fcb80 1 client.25699 shutdown 2024-06-03T02:23:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2173054039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 msgr2=0x55c709918130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709918130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800d000 tx=0x7fcbb8009260 comp rx=0 tx=0).stop 2024-06-03T02:23:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2173054039 shutdown_connections 2024-06-03T02:23:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c70989f720 0x55c709918bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55c70992eaa0 0x55c709930e90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709896d60 0x55c709918130 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbe13fcb80 1 --2- 172.21.15.79:0/2173054039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c70989ea90 0x55c709918670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2173054039 >> 172.21.15.79:0/2173054039 conn(0x55c7097c4170 msgr2=0x55c70989da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.393+0000 7fcbbe000700 0 client.25699 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:23:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.394+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2173054039 shutdown_connections 2024-06-03T02:23:17.396 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:23:17.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.394+0000 7fcbe13fcb80 1 -- 172.21.15.79:0/2173054039 wait complete. 2024-06-03T02:23:17.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.394+0000 7fcbe13fcb80 20 client.25699 trim_cache size 0 max 16384 2024-06-03T02:23:17.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 -- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcbb40a2ca0 msgr2=0x7fcbb40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:17.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcbb40a2ca0 0x7fcbb40a5160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80cc250 tx=0x7fcbc800df90 comp rx=0 tx=0).stop 2024-06-03T02:23:17.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 -- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbd010ddd0 msgr2=0x7fcbd01a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:17.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbd010ddd0 0x7fcbd01a3aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800d720 tx=0x7fcbc800dae0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 -- 172.21.15.79:0/3935090619 shutdown_connections 2024-06-03T02:23:17.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcbb40a2ca0 0x7fcbb40a5160 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbd010ddd0 0x7fcbd01a3aa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd0109530 0x7fcbd019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 --2- 172.21.15.79:0/3935090619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd0109ee0 0x7fcbd019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:17.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.397+0000 7fcbd5400700 1 -- 172.21.15.79:0/3935090619 >> 172.21.15.79:0/3935090619 conn(0x7fcbd00781c0 msgr2=0x7fcbd0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:17.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.398+0000 7fcbd5400700 1 -- 172.21.15.79:0/3935090619 shutdown_connections 2024-06-03T02:23:17.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:17.398+0000 7fcbd5400700 1 -- 172.21.15.79:0/3935090619 wait complete. 2024-06-03T02:23:18.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 190.0 s, 792.0 tps, lat 126.136 ms stddev 141.445 2024-06-03T02:23:18.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:18 smithi079 ceph-mon[31819]: pgmap v613: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 736 op/s 2024-06-03T02:23:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:18 smithi135 ceph-mon[36653]: pgmap v613: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 736 op/s 2024-06-03T02:23:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:18 smithi123 ceph-mon[36640]: pgmap v613: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 736 op/s 2024-06-03T02:23:19.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:23:19.961 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:20.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:20 smithi079 ceph-mon[31819]: pgmap v614: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 873 op/s 2024-06-03T02:23:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:20 smithi135 ceph-mon[36653]: pgmap v614: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 873 op/s 2024-06-03T02:23:20.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:20 smithi123 ceph-mon[36640]: pgmap v614: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 873 op/s 2024-06-03T02:23:22.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:22 smithi079 ceph-mon[31819]: pgmap v615: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 746 op/s 2024-06-03T02:23:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:22 smithi135 ceph-mon[36653]: pgmap v615: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 746 op/s 2024-06-03T02:23:22.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:22 smithi123 ceph-mon[36640]: pgmap v615: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 746 op/s 2024-06-03T02:23:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 195.0 s, 937.0 tps, lat 108.764 ms stddev 139.738 2024-06-03T02:23:24.267 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:24 smithi079 ceph-mon[31819]: pgmap v616: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 949 op/s 2024-06-03T02:23:24.268 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:24 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:23:24.525 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:24 smithi123 ceph-mon[36640]: pgmap v616: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 949 op/s 2024-06-03T02:23:24.525 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:24 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:23:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:24 smithi135 ceph-mon[36653]: pgmap v616: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 949 op/s 2024-06-03T02:23:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:24 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:23:25.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.155+0000 7f2450a00700 1 -- 172.21.15.79:0/4057211953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244c1073c0 msgr2=0x7f244c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:25.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.155+0000 7f2450a00700 1 --2- 172.21.15.79:0/4057211953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244c1073c0 0x7f244c06a1a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2438009990 tx=0x7f2438009ca0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.156+0000 7f2450a00700 1 -- 172.21.15.79:0/4057211953 shutdown_connections 2024-06-03T02:23:25.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.156+0000 7f2450a00700 1 --2- 172.21.15.79:0/4057211953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244c104ed0 0x7f244c105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.156+0000 7f2450a00700 1 --2- 172.21.15.79:0/4057211953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244c1073c0 0x7f244c06a1a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.156+0000 7f2450a00700 1 --2- 172.21.15.79:0/4057211953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f244c06a6e0 0x7f244c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.158+0000 7f2450a00700 1 -- 172.21.15.79:0/4057211953 >> 172.21.15.79:0/4057211953 conn(0x7f244c078340 msgr2=0x7f244c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:25.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.158+0000 7f2450a00700 1 -- 172.21.15.79:0/4057211953 shutdown_connections 2024-06-03T02:23:25.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.159+0000 7f2450a00700 1 -- 172.21.15.79:0/4057211953 wait complete. 2024-06-03T02:23:25.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.164+0000 7f2450a00700 1 Processor -- start 2024-06-03T02:23:25.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.164+0000 7f2450a00700 1 -- start start 2024-06-03T02:23:25.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.164+0000 7f2450a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244c06a6e0 0x7f244c073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.165+0000 7f2449e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244c06a6e0 0x7f244c073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.165+0000 7f2449e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244c06a6e0 0x7f244c073c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:47532/0 (socket says 172.21.15.79:47532) 2024-06-03T02:23:25.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.165+0000 7f2449e00700 1 -- 172.21.15.79:0/423355015 learned_addr learned my addr 172.21.15.79:0/423355015 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:25.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.166+0000 7f2450a00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f244c104ed0 0x7f244c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.166+0000 7f2450a00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244c1073c0 0x7f244c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.167+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f244c118da0 con 0x7f244c104ed0 2024-06-03T02:23:25.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.167+0000 7f244a800700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244c1073c0 0x7f244c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.167+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f244c118c20 con 0x7f244c1073c0 2024-06-03T02:23:25.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.168+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f244c118f20 con 0x7f244c06a6e0 2024-06-03T02:23:25.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.169+0000 7f2449e00700 1 -- 172.21.15.79:0/423355015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244c1073c0 msgr2=0x7f244c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:25.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.169+0000 7f2449e00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244c1073c0 0x7f244c06d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.169+0000 7f2449e00700 1 -- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f244c104ed0 msgr2=0x7f244c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:25.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.169+0000 7f2449e00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f244c104ed0 0x7f244c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.169+0000 7f2449e00700 1 -- 172.21.15.79:0/423355015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c06d830 con 0x7f244c06a6e0 2024-06-03T02:23:25.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.170+0000 7f244a800700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244c1073c0 0x7f244c06d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:23:25.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.171+0000 7f2449e00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244c06a6e0 0x7f244c073c70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2438000c00 tx=0x7f243800bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:25.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.172+0000 7f2442a00700 1 -- 172.21.15.79:0/423355015 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438027070 con 0x7f244c06a6e0 2024-06-03T02:23:25.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.172+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244c06dac0 con 0x7f244c06a6e0 2024-06-03T02:23:25.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.173+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244c071a20 con 0x7f244c06a6e0 2024-06-03T02:23:25.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.173+0000 7f2442a00700 1 -- 172.21.15.79:0/423355015 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438003d70 con 0x7f244c06a6e0 2024-06-03T02:23:25.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.174+0000 7f2442a00700 1 -- 172.21.15.79:0/423355015 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438027070 con 0x7f244c06a6e0 2024-06-03T02:23:25.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.186+0000 7f2442a00700 1 -- 172.21.15.79:0/423355015 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2438009d10 con 0x7f244c06a6e0 2024-06-03T02:23:25.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.189+0000 7f245b3cdb80 1 Processor -- start 2024-06-03T02:23:25.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.189+0000 7f245b3cdb80 1 -- start start 2024-06-03T02:23:25.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.189+0000 7f2442a00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f24300a2e20 0x7f24300a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.190+0000 7f2442a00700 1 -- 172.21.15.79:0/423355015 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f24380d0360 con 0x7f244c06a6e0 2024-06-03T02:23:25.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.191+0000 7f244a800700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f24300a2e20 0x7f24300a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.191+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c574070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.192+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c5745b0 0x55855c574a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.192+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 0x55855c5816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.192+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55855c483a10 con 0x55855c5745b0 2024-06-03T02:23:25.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.192+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55855c4c6b00 con 0x55855c4f0e40 2024-06-03T02:23:25.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.192+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55855c4818f0 con 0x55855c57f300 2024-06-03T02:23:25.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.192+0000 7f2449e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c574070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.192+0000 7f2449e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c574070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36350/0 (socket says 172.21.15.79:36350) 2024-06-03T02:23:25.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.193+0000 7f2449e00700 1 -- 172.21.15.79:0/4265594930 learned_addr learned my addr 172.21.15.79:0/4265594930 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:25.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.193+0000 7f2449400700 1 --2- 172.21.15.79:0/4265594930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c5745b0 0x55855c574a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.193+0000 7f2449e00700 1 -- 172.21.15.79:0/4265594930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 msgr2=0x55855c5816f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:23:25.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.193+0000 7f2449e00700 1 --2- 172.21.15.79:0/4265594930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 0x55855c5816f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.193+0000 7f2449e00700 1 -- 172.21.15.79:0/4265594930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c5745b0 msgr2=0x55855c574a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:25.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.193+0000 7f2449e00700 1 --2- 172.21.15.79:0/4265594930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c5745b0 0x55855c574a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.193+0000 7f2449e00700 1 -- 172.21.15.79:0/4265594930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855c46ef90 con 0x55855c4f0e40 2024-06-03T02:23:25.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.194+0000 7f2449e00700 1 --2- 172.21.15.79:0/4265594930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c574070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2438099940 tx=0x7f2438009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcfa53e3f94d5f10 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:25.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.194+0000 7f2437400700 1 -- 172.21.15.79:0/4265594930 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243809d4e0 con 0x55855c4f0e40 2024-06-03T02:23:25.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.195+0000 7f2437400700 1 -- 172.21.15.79:0/4265594930 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24380a1890 con 0x55855c4f0e40 2024-06-03T02:23:25.198 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.195+0000 7f2437400700 1 -- 172.21.15.79:0/4265594930 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380a5c30 con 0x55855c4f0e40 2024-06-03T02:23:25.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.197+0000 7f245b3cdb80 1 -- 172.21.15.79:0/4265594930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 msgr2=0x55855c574070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:25.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.197+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/4265594930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c574070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2438099940 tx=0x7f2438009ca0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.198+0000 7f245b3cdb80 1 -- 172.21.15.79:0/4265594930 shutdown_connections 2024-06-03T02:23:25.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.198+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/4265594930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 0x55855c5816f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.198+0000 7f244a800700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f24300a2e20 0x7f24300a52e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f243c009bc0 tx=0x7f243c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:25.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.199+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/4265594930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c574070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.199+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/4265594930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c5745b0 0x55855c574a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.199+0000 7f245b3cdb80 1 -- 172.21.15.79:0/4265594930 >> 172.21.15.79:0/4265594930 conn(0x55855c4a1170 msgr2=0x55855c5704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:25.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.204+0000 7f245b3cdb80 1 -- 172.21.15.79:0/4265594930 shutdown_connections 2024-06-03T02:23:25.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.204+0000 7f245b3cdb80 1 -- 172.21.15.79:0/4265594930 wait complete. 2024-06-03T02:23:25.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.206+0000 7f245b3cdb80 1 -- start start 2024-06-03T02:23:25.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.207+0000 7f245b3cdb80 1 Processor -- start 2024-06-03T02:23:25.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.211+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c4f0e40 0x55855c57ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.211+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c5745b0 0x55855c57bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.212+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 0x55855c57c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.212+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55855c462f60 con 0x55855c4f0e40 2024-06-03T02:23:25.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.212+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55855c4c2ac0 con 0x55855c5745b0 2024-06-03T02:23:25.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.212+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55855c581d70 con 0x55855c57f300 2024-06-03T02:23:25.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.212+0000 7f245b3cdb80 10 client.0 _ll_register_callbacks cb 0x55855c460720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:25.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.212+0000 7f244a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 0x55855c57c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.213+0000 7f244a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 0x55855c57c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:47536/0 (socket says 172.21.15.79:47536) 2024-06-03T02:23:25.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.213+0000 7f244a800700 1 -- 172.21.15.79:0/267343812 learned_addr learned my addr 172.21.15.79:0/267343812 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.213+0000 7f244a800700 1 -- 172.21.15.79:0/267343812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c5745b0 msgr2=0x55855c57bd60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.213+0000 7f244a800700 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c5745b0 0x55855c57bd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.213+0000 7f244a800700 1 -- 172.21.15.79:0/267343812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c4f0e40 msgr2=0x55855c57ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.213+0000 7f244a800700 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c4f0e40 0x55855c57ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.213+0000 7f244a800700 1 -- 172.21.15.79:0/267343812 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855c4158d0 con 0x55855c57f300 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.214+0000 7f244a800700 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 0x55855c57c2a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f243c00fda0 tx=0x7f243c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e72bc59c38d435f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.214+0000 7f242be00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.214+0000 7f242be00700 1 -- 172.21.15.79:0/267343812 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c022070 con 0x55855c57f300 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.215+0000 7f242be00700 1 -- 172.21.15.79:0/267343812 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f243c004a30 con 0x55855c57f300 2024-06-03T02:23:25.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.215+0000 7f245b3cdb80 1 -- 172.21.15.79:0/267343812 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24380a1890 con 0x55855c57f300 2024-06-03T02:23:25.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.217+0000 7f242be00700 1 -- 172.21.15.79:0/267343812 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f243c02b050 con 0x55855c57f300 2024-06-03T02:23:25.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.217+0000 7f245b3cdb80 10 client.25708 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:25.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.217+0000 7f245b3cdb80 1 -- 172.21.15.79:0/267343812 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55855c58c510 con 0x55855c57f300 2024-06-03T02:23:25.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.218+0000 7f242be00700 1 -- 172.21.15.79:0/267343812 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24380a1890 con 0x55855c57f300 2024-06-03T02:23:25.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.219+0000 7f242be00700 1 -- 172.21.15.79:0/267343812 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f243c026410 con 0x55855c57f300 2024-06-03T02:23:25.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.219+0000 7f245b3cdb80 10 client.25708 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:25.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.219+0000 7f245b3cdb80 10 client.25708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:23:25.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.219+0000 7f245b3cdb80 20 client.25708 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:25.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.219+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55855c58eb40 0x55855c590fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.219+0000 7f245b3cdb80 4 client.25708 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:23:25.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.219+0000 7f245b3cdb80 1 -- 172.21.15.79:0/267343812 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55855c4d4000 con 0x55855c58eb40 2024-06-03T02:23:25.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.219+0000 7f2449e00700 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55855c58eb40 0x55855c590fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.221+0000 7f2449e00700 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55855c58eb40 0x55855c590fc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431983b973c94b8c server_cookie=51fe401d4e4efbbb in_seq=0 out_seq=0 2024-06-03T02:23:25.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.221+0000 7f242be00700 10 client.25708 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:25.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.226+0000 7f242be00700 1 -- 172.21.15.79:0/267343812 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55855c4d4000 con 0x55855c58eb40 2024-06-03T02:23:25.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.226+0000 7f242be00700 10 client.25708 handle_command_reply: tid=0 2024-06-03T02:23:25.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.226+0000 7f245b3cdb80 1 client.25708 shutdown 2024-06-03T02:23:25.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.227+0000 7f245b3cdb80 1 -- 172.21.15.79:0/267343812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 msgr2=0x55855c57c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:25.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.227+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 0x55855c57c2a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f243c00fda0 tx=0x7f243c004750 comp rx=0 tx=0).stop 2024-06-03T02:23:25.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.229+0000 7f245b3cdb80 1 -- 172.21.15.79:0/267343812 shutdown_connections 2024-06-03T02:23:25.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.229+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c57f300 0x55855c57c2a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.229+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55855c58eb40 0x55855c590fc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.229+0000 7f242be00700 0 client.25708 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:25.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.229+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c4f0e40 0x55855c57ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.229+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/267343812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c5745b0 0x55855c57bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.229+0000 7f245b3cdb80 1 -- 172.21.15.79:0/267343812 >> 172.21.15.79:0/267343812 conn(0x55855c4a1170 msgr2=0x55855c56f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:25.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.230+0000 7f245b3cdb80 1 -- 172.21.15.79:0/267343812 shutdown_connections 2024-06-03T02:23:25.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.230+0000 7f245b3cdb80 1 -- 172.21.15.79:0/267343812 wait complete. 2024-06-03T02:23:25.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.231+0000 7f245b3cdb80 20 client.25708 trim_cache size 0 max 16384 2024-06-03T02:23:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.268+0000 7f245b3cdb80 1 Processor -- start 2024-06-03T02:23:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.268+0000 7f245b3cdb80 1 -- start start 2024-06-03T02:23:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.268+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c4f0e40 0x55855c590f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c5745b0 0x55855c57e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c57f300 0x55855c5806f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f242c00ebd0 con 0x55855c57f300 2024-06-03T02:23:25.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f242c010710 con 0x55855c5745b0 2024-06-03T02:23:25.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f242c00f2a0 con 0x55855c4f0e40 2024-06-03T02:23:25.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f2449e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c4f0e40 0x55855c590f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f2449e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c4f0e40 0x55855c590f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:47538/0 (socket says 172.21.15.79:47538) 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f2449e00700 1 -- 172.21.15.79:0/1437689755 learned_addr learned my addr 172.21.15.79:0/1437689755 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f2449e00700 1 -- 172.21.15.79:0/1437689755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c5745b0 msgr2=0x55855c57e5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f2449e00700 1 --2- 172.21.15.79:0/1437689755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c5745b0 0x55855c57e5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f2449e00700 1 -- 172.21.15.79:0/1437689755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c57f300 msgr2=0x55855c5806f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f2449e00700 1 --2- 172.21.15.79:0/1437689755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c57f300 0x55855c5806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.269+0000 7f2449e00700 1 -- 172.21.15.79:0/1437689755 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855c57a2d0 con 0x55855c4f0e40 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.270+0000 7f2449e00700 1 --2- 172.21.15.79:0/1437689755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c4f0e40 0x55855c590f60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f24380d02d0 tx=0x7f243800b490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b04f7863a196bb0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.270+0000 7f2436a00700 1 -- 172.21.15.79:0/1437689755 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438066070 con 0x55855c4f0e40 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.270+0000 7f2436a00700 1 -- 172.21.15.79:0/1437689755 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24380a1890 con 0x55855c4f0e40 2024-06-03T02:23:25.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.270+0000 7f2436a00700 1 -- 172.21.15.79:0/1437689755 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380a5e90 con 0x55855c4f0e40 2024-06-03T02:23:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.273+0000 7f245b3cdb80 1 -- 172.21.15.79:0/1437689755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c4f0e40 msgr2=0x55855c590f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.273+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/1437689755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c4f0e40 0x55855c590f60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f24380d02d0 tx=0x7f243800b490 comp rx=0 tx=0).stop 2024-06-03T02:23:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.273+0000 7f245b3cdb80 1 -- 172.21.15.79:0/1437689755 shutdown_connections 2024-06-03T02:23:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.273+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/1437689755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c57f300 0x55855c5806f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.273+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/1437689755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c4f0e40 0x55855c590f60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.273+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/1437689755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c5745b0 0x55855c57e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.273+0000 7f245b3cdb80 1 -- 172.21.15.79:0/1437689755 >> 172.21.15.79:0/1437689755 conn(0x55855c4a1170 msgr2=0x55855c57bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:25.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.273+0000 7f245b3cdb80 1 -- 172.21.15.79:0/1437689755 shutdown_connections 2024-06-03T02:23:25.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.273+0000 7f245b3cdb80 1 -- 172.21.15.79:0/1437689755 wait complete. 2024-06-03T02:23:25.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.274+0000 7f245b3cdb80 1 -- start start 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.274+0000 7f245b3cdb80 1 Processor -- start 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.276+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c57f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.276+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c5745b0 0x55855c57ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.276+0000 7f245b3cdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c57f300 0x55855c580460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.276+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f242c010040 con 0x55855c57f300 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.276+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f242c00f970 con 0x55855c4f0e40 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.276+0000 7f245b3cdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f242c00cb70 con 0x55855c5745b0 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.277+0000 7f245b3cdb80 10 client.0 _ll_register_callbacks cb 0x55855c460720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.278+0000 7f2449e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c57f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.278+0000 7f2449400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c5745b0 0x55855c57ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.278+0000 7f2449400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c5745b0 0x55855c57ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:47554/0 (socket says 172.21.15.79:47554) 2024-06-03T02:23:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.278+0000 7f2449400700 1 -- 172.21.15.79:0/2644872735 learned_addr learned my addr 172.21.15.79:0/2644872735 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:25.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.278+0000 7f244a800700 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c57f300 0x55855c580460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.279+0000 7f2449e00700 1 -- 172.21.15.79:0/2644872735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c5745b0 msgr2=0x55855c57ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:25.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.279+0000 7f2449e00700 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c5745b0 0x55855c57ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.279+0000 7f2449e00700 1 -- 172.21.15.79:0/2644872735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c57f300 msgr2=0x55855c580460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:25.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.279+0000 7f2449e00700 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c57f300 0x55855c580460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.279+0000 7f2449e00700 1 -- 172.21.15.79:0/2644872735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855c586040 con 0x55855c4f0e40 2024-06-03T02:23:25.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.280+0000 7f2449e00700 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c57f9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f24380227b0 tx=0x7f243809d020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c3a9749a45af800 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:25.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.280+0000 7f242be00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:23:25.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.281+0000 7f242be00700 1 -- 172.21.15.79:0/2644872735 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438066070 con 0x55855c4f0e40 2024-06-03T02:23:25.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.281+0000 7f242be00700 1 -- 172.21.15.79:0/2644872735 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f243809d740 con 0x55855c4f0e40 2024-06-03T02:23:25.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.281+0000 7f245b3cdb80 1 -- 172.21.15.79:0/2644872735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24380a1890 con 0x55855c4f0e40 2024-06-03T02:23:25.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.288+0000 7f242be00700 1 -- 172.21.15.79:0/2644872735 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f243809d940 con 0x55855c4f0e40 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.288+0000 7f245b3cdb80 10 client.25733 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.288+0000 7f245b3cdb80 1 -- 172.21.15.79:0/2644872735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55855c587340 con 0x55855c4f0e40 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.288+0000 7f242be00700 1 -- 172.21.15.79:0/2644872735 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24380a1890 con 0x55855c4f0e40 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.288+0000 7f242be00700 1 -- 172.21.15.79:0/2644872735 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f243809c930 con 0x55855c4f0e40 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.289+0000 7f245b3cdb80 10 client.25733 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.289+0000 7f245b3cdb80 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.289+0000 7f245b3cdb80 20 client.25733 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.289+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55855c58eb40 0x55855c587570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.289+0000 7f245b3cdb80 4 client.25733 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:23:25.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.289+0000 7f245b3cdb80 1 -- 172.21.15.79:0/2644872735 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55855c4ce580 con 0x55855c58eb40 2024-06-03T02:23:25.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.289+0000 7f2449400700 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55855c58eb40 0x55855c587570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:25.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.291+0000 7f2449400700 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55855c58eb40 0x55855c587570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454b1fc678fb108e server_cookie=f8125cfc9ae7e715 in_seq=0 out_seq=0 2024-06-03T02:23:25.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f242be00700 10 client.25733 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f242be00700 1 -- 172.21.15.79:0/2644872735 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55855c4ce580 con 0x55855c58eb40 2024-06-03T02:23:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f242be00700 10 client.25733 handle_command_reply: tid=0 2024-06-03T02:23:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f245b3cdb80 1 client.25733 shutdown 2024-06-03T02:23:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f245b3cdb80 1 -- 172.21.15.79:0/2644872735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 msgr2=0x55855c57f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:25.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c57f9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f24380227b0 tx=0x7f243809d020 comp rx=0 tx=0).stop 2024-06-03T02:23:25.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f245b3cdb80 1 -- 172.21.15.79:0/2644872735 shutdown_connections 2024-06-03T02:23:25.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55855c58eb40 0x55855c587570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55855c57f300 0x55855c580460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55855c4f0e40 0x55855c57f9e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f245b3cdb80 1 --2- 172.21.15.79:0/2644872735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55855c5745b0 0x55855c57ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.292+0000 7f245b3cdb80 1 -- 172.21.15.79:0/2644872735 >> 172.21.15.79:0/2644872735 conn(0x55855c4a1170 msgr2=0x55855c57a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:25.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.293+0000 7f242be00700 0 client.25733 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:25.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.295+0000 7f245b3cdb80 1 -- 172.21.15.79:0/2644872735 shutdown_connections 2024-06-03T02:23:25.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.295+0000 7f245b3cdb80 1 -- 172.21.15.79:0/2644872735 wait complete. 2024-06-03T02:23:25.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.295+0000 7f245b3cdb80 20 client.25733 trim_cache size 0 max 16384 2024-06-03T02:23:25.297 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:23:25.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.298+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f24300a2e20 msgr2=0x7f24300a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:25.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.298+0000 7f2450a00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f24300a2e20 0x7f24300a52e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f243c009bc0 tx=0x7f243c009260 comp rx=0 tx=0).stop 2024-06-03T02:23:25.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.298+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244c06a6e0 msgr2=0x7f244c073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:25.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.298+0000 7f2450a00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244c06a6e0 0x7f244c073c70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2438000c00 tx=0x7f243800bb60 comp rx=0 tx=0).stop 2024-06-03T02:23:25.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.299+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 shutdown_connections 2024-06-03T02:23:25.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.299+0000 7f2450a00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244c1073c0 0x7f244c06d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.299+0000 7f2450a00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244c06a6e0 0x7f244c073c70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.299+0000 7f2450a00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f24300a2e20 0x7f24300a52e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.299+0000 7f2450a00700 1 --2- 172.21.15.79:0/423355015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f244c104ed0 0x7f244c06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:25.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.299+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 >> 172.21.15.79:0/423355015 conn(0x7f244c078340 msgr2=0x7f244c1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:25.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.299+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 shutdown_connections 2024-06-03T02:23:25.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:25.300+0000 7f2450a00700 1 -- 172.21.15.79:0/423355015 wait complete. 2024-06-03T02:23:25.422 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.422 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.422 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.423 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.423 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.423 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.423 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.423 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:25 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:25 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:25.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:25 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:23:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:26 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-06-03T02:23:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:26 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-03T02:23:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:26 smithi079 ceph-mon[31819]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-03T02:23:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:26 smithi079 ceph-mon[31819]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-03T02:23:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:26 smithi079 ceph-mon[31819]: pgmap v617: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 MiB/s wr, 648 op/s 2024-06-03T02:23:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:23:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:23:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:23:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:26 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-06-03T02:23:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:26 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-03T02:23:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:26 smithi135 ceph-mon[36653]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-03T02:23:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:26 smithi135 ceph-mon[36653]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-03T02:23:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:26 smithi135 ceph-mon[36653]: pgmap v617: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 MiB/s wr, 648 op/s 2024-06-03T02:23:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:23:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:23:26.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:23:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:26 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-06-03T02:23:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:26 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-03T02:23:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:26 smithi123 ceph-mon[36640]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-03T02:23:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:26 smithi123 ceph-mon[36640]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-03T02:23:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:26 smithi123 ceph-mon[36640]: pgmap v617: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 MiB/s wr, 648 op/s 2024-06-03T02:23:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:23:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:23:26.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:23:27.795 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:23:27.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:23:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 200.0 s, 1907.2 tps, lat 50.751 ms stddev 55.035 2024-06-03T02:23:28.475 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:28.591 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:28 smithi079 ceph-mon[31819]: pgmap v618: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.0 MiB/s wr, 583 op/s 2024-06-03T02:23:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:28 smithi135 ceph-mon[36653]: pgmap v618: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.0 MiB/s wr, 583 op/s 2024-06-03T02:23:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:28 smithi123 ceph-mon[36640]: pgmap v618: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.0 MiB/s wr, 583 op/s 2024-06-03T02:23:30.316 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:30 smithi079 ceph-mon[31819]: pgmap v619: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 816 op/s 2024-06-03T02:23:30.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:30 smithi135 ceph-mon[36653]: pgmap v619: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 816 op/s 2024-06-03T02:23:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:30 smithi123 ceph-mon[36640]: pgmap v619: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 816 op/s 2024-06-03T02:23:30.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.928+0000 7f00be800700 1 -- 172.21.15.79:0/1128163311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b8102510 msgr2=0x7f00b8102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:30.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.928+0000 7f00be800700 1 --2- 172.21.15.79:0/1128163311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b8102510 0x7f00b8102990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f00a40098f0 tx=0x7f00a4009c00 comp rx=0 tx=0).stop 2024-06-03T02:23:30.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.928+0000 7f00be800700 1 -- 172.21.15.79:0/1128163311 shutdown_connections 2024-06-03T02:23:30.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.928+0000 7f00be800700 1 --2- 172.21.15.79:0/1128163311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b8102ed0 0x7f00b810f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.928+0000 7f00be800700 1 --2- 172.21.15.79:0/1128163311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b8108520 0x7f00b8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.928+0000 7f00be800700 1 --2- 172.21.15.79:0/1128163311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b8102510 0x7f00b8102990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.928+0000 7f00be800700 1 -- 172.21.15.79:0/1128163311 >> 172.21.15.79:0/1128163311 conn(0x7f00b8079640 msgr2=0x7f00b8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:30.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.928+0000 7f00be800700 1 -- 172.21.15.79:0/1128163311 shutdown_connections 2024-06-03T02:23:30.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.929+0000 7f00be800700 1 -- 172.21.15.79:0/1128163311 wait complete. 2024-06-03T02:23:30.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.930+0000 7f00be800700 1 Processor -- start 2024-06-03T02:23:30.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.930+0000 7f00be800700 1 -- start start 2024-06-03T02:23:30.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.931+0000 7f00be800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b8102510 0x7f00b8116b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.931+0000 7f00be800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b8102ed0 0x7f00b8117060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.931+0000 7f00be800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b8108520 0x7f00b8111ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.931+0000 7f00be800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8075ea0 con 0x7f00b8102ed0 2024-06-03T02:23:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.931+0000 7f00be800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8075d20 con 0x7f00b8108520 2024-06-03T02:23:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.931+0000 7f00be800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8076020 con 0x7f00b8102510 2024-06-03T02:23:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.931+0000 7f00b7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b8102510 0x7f00b8116b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b8108520 0x7f00b8111ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b8102510 0x7f00b8116b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:47568/0 (socket says 172.21.15.79:47568) 2024-06-03T02:23:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b7400700 1 -- 172.21.15.79:0/1113032107 learned_addr learned my addr 172.21.15.79:0/1113032107 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b6a00700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b8102ed0 0x7f00b8117060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b7400700 1 -- 172.21.15.79:0/1113032107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b8108520 msgr2=0x7f00b8111ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b7400700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b8108520 0x7f00b8111ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b7400700 1 -- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b8102ed0 msgr2=0x7f00b8117060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b7400700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b8102ed0 0x7f00b8117060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b7400700 1 -- 172.21.15.79:0/1113032107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b81125e0 con 0x7f00b8102510 2024-06-03T02:23:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.932+0000 7f00b7e00700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b8108520 0x7f00b8111ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:23:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.933+0000 7f00b7400700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b8102510 0x7f00b8116b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f00a800c920 tx=0x7f00a800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:30.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.934+0000 7f00afe00700 1 -- 172.21.15.79:0/1113032107 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8007b00 con 0x7f00b8102510 2024-06-03T02:23:30.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.934+0000 7f00afe00700 1 -- 172.21.15.79:0/1113032107 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a8004d10 con 0x7f00b8102510 2024-06-03T02:23:30.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.934+0000 7f00be800700 1 -- 172.21.15.79:0/1113032107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b81128d0 con 0x7f00b8102510 2024-06-03T02:23:30.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.934+0000 7f00afe00700 1 -- 172.21.15.79:0/1113032107 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8005680 con 0x7f00b8102510 2024-06-03T02:23:30.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.934+0000 7f00be800700 1 -- 172.21.15.79:0/1113032107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b811ca10 con 0x7f00b8102510 2024-06-03T02:23:30.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.941+0000 7f00afe00700 1 -- 172.21.15.79:0/1113032107 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f00a8004750 con 0x7f00b8102510 2024-06-03T02:23:30.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.944+0000 7f00c923cb80 1 Processor -- start 2024-06-03T02:23:30.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.944+0000 7f00c923cb80 1 -- start start 2024-06-03T02:23:30.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.944+0000 7f00afe00700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f009c0a2e20 0x7f009c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.944+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceecd1ec0 0x55eceed550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.944+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed55ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.944+0000 7f00afe00700 1 -- 172.21.15.79:0/1113032107 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f00a80cf1e0 con 0x7f00b8102510 2024-06-03T02:23:30.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 0x55eceed62770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00c923cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55eceec64a90 con 0x55eceed60380 2024-06-03T02:23:30.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00c923cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55eceeca7b80 con 0x55eceecd1ec0 2024-06-03T02:23:30.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00c923cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55eceec62970 con 0x55eceed55630 2024-06-03T02:23:30.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00b7e00700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f009c0a2e20 0x7f009c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00b7e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 0x55eceed62770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00b7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceecd1ec0 0x55eceed550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00b7e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 0x55eceed62770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37390/0 (socket says 172.21.15.79:37390) 2024-06-03T02:23:30.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00b7e00700 1 -- 172.21.15.79:0/2538154931 learned_addr learned my addr 172.21.15.79:0/2538154931 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:30.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.945+0000 7f00b6a00700 1 --2- 172.21.15.79:0/2538154931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed55ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.946+0000 7f00b6a00700 1 -- 172.21.15.79:0/2538154931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceecd1ec0 msgr2=0x55eceed550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:30.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.946+0000 7f00b6a00700 1 --2- 172.21.15.79:0/2538154931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceecd1ec0 0x55eceed550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.946+0000 7f00b6a00700 1 -- 172.21.15.79:0/2538154931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 msgr2=0x55eceed62770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:30.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.946+0000 7f00b6a00700 1 --2- 172.21.15.79:0/2538154931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 0x55eceed62770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.946+0000 7f00b6a00700 1 -- 172.21.15.79:0/2538154931 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eceec50010 con 0x55eceed55630 2024-06-03T02:23:30.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.946+0000 7f00b6a00700 1 --2- 172.21.15.79:0/2538154931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed55ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f00a40098c0 tx=0x7f00a40051d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1c36f60c4804d1b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:30.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.946+0000 7f00b7e00700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f009c0a2e20 0x7f009c0a52e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00b0019430 tx=0x7f00b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:30.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.947+0000 7f00acc00700 1 -- 172.21.15.79:0/2538154931 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4005340 con 0x55eceed55630 2024-06-03T02:23:30.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.947+0000 7f00acc00700 1 -- 172.21.15.79:0/2538154931 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a40054e0 con 0x55eceed55630 2024-06-03T02:23:30.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.947+0000 7f00acc00700 1 -- 172.21.15.79:0/2538154931 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a400ba90 con 0x55eceed55630 2024-06-03T02:23:30.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.948+0000 7f00c923cb80 1 -- 172.21.15.79:0/2538154931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 msgr2=0x55eceed55ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:30.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.948+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2538154931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed55ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f00a40098c0 tx=0x7f00a40051d0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.948+0000 7f00c923cb80 1 -- 172.21.15.79:0/2538154931 shutdown_connections 2024-06-03T02:23:30.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.948+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2538154931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 0x55eceed62770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.948+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2538154931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceecd1ec0 0x55eceed550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.948+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2538154931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed55ab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.948+0000 7f00c923cb80 1 -- 172.21.15.79:0/2538154931 >> 172.21.15.79:0/2538154931 conn(0x55eceec821f0 msgr2=0x55eceed51530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:30.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.948+0000 7f00c923cb80 1 -- 172.21.15.79:0/2538154931 shutdown_connections 2024-06-03T02:23:30.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.948+0000 7f00c923cb80 1 -- 172.21.15.79:0/2538154931 wait complete. 2024-06-03T02:23:30.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.949+0000 7f00c923cb80 1 -- start start 2024-06-03T02:23:30.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.949+0000 7f00c923cb80 1 Processor -- start 2024-06-03T02:23:30.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.950+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.950+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed5cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.950+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed60380 0x55eceed5d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00c923cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55eceec43fe0 con 0x55eceecd1ec0 2024-06-03T02:23:30.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00c923cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55eceeca3b40 con 0x55eceed55630 2024-06-03T02:23:30.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00c923cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55eceed62fe0 con 0x55eceed60380 2024-06-03T02:23:30.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00c923cb80 10 client.0 _ll_register_callbacks cb 0x55eceec417a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:30.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b7400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b7400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37392/0 (socket says 172.21.15.79:37392) 2024-06-03T02:23:30.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b7400700 1 -- 172.21.15.79:0/755706946 learned_addr learned my addr 172.21.15.79:0/755706946 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:30.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b6a00700 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed5cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b7e00700 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed60380 0x55eceed5d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b6a00700 1 -- 172.21.15.79:0/755706946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed60380 msgr2=0x55eceed5d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:30.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b6a00700 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed60380 0x55eceed5d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b6a00700 1 -- 172.21.15.79:0/755706946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 msgr2=0x55eceed5fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:30.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b6a00700 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.951+0000 7f00b6a00700 1 -- 172.21.15.79:0/755706946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eceebf6950 con 0x55eceed55630 2024-06-03T02:23:30.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.952+0000 7f00b6a00700 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed5cde0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009810 tx=0x7f00a4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=225465519c3ea492 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:30.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.952+0000 7f00a1600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:23:30.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.952+0000 7f00a1600700 1 -- 172.21.15.79:0/755706946 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a402f060 con 0x55eceed55630 2024-06-03T02:23:30.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.952+0000 7f00a1600700 1 -- 172.21.15.79:0/755706946 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f00a400b8c0 con 0x55eceed55630 2024-06-03T02:23:30.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.953+0000 7f00c923cb80 1 -- 172.21.15.79:0/755706946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a40054e0 con 0x55eceed55630 2024-06-03T02:23:30.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.954+0000 7f00a1600700 1 -- 172.21.15.79:0/755706946 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00a4036050 con 0x55eceed55630 2024-06-03T02:23:30.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.954+0000 7f00c923cb80 10 client.25745 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:30.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.954+0000 7f00c923cb80 1 -- 172.21.15.79:0/755706946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eceed6d4e0 con 0x55eceed55630 2024-06-03T02:23:30.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.955+0000 7f00a1600700 1 -- 172.21.15.79:0/755706946 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a40054e0 con 0x55eceed55630 2024-06-03T02:23:30.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.956+0000 7f00a1600700 1 -- 172.21.15.79:0/755706946 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f00a400f990 con 0x55eceed55630 2024-06-03T02:23:30.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.956+0000 7f00c923cb80 10 client.25745 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:30.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.956+0000 7f00c923cb80 10 client.25745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:23:30.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.956+0000 7f00c923cb80 20 client.25745 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:30.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.956+0000 7f00c923cb80 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55eceed6f9b0 0x55eceed71da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.956+0000 7f00c923cb80 4 client.25745 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:23:30.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.956+0000 7f00c923cb80 1 -- 172.21.15.79:0/755706946 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eceecb5080 con 0x55eceed6f9b0 2024-06-03T02:23:30.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.957+0000 7f00b7400700 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55eceed6f9b0 0x55eceed71da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.957+0000 7f00b7400700 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55eceed6f9b0 0x55eceed71da0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8334123e1265520c server_cookie=d5ededac9769654a in_seq=0 out_seq=0 2024-06-03T02:23:30.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.957+0000 7f00a1600700 10 client.25745 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:30.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.958+0000 7f00a1600700 1 -- 172.21.15.79:0/755706946 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eceecb5080 con 0x55eceed6f9b0 2024-06-03T02:23:30.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.958+0000 7f00a1600700 10 client.25745 handle_command_reply: tid=0 2024-06-03T02:23:30.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.958+0000 7f00c923cb80 1 client.25745 shutdown 2024-06-03T02:23:30.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.958+0000 7f00c923cb80 1 -- 172.21.15.79:0/755706946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 msgr2=0x55eceed5cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:30.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.958+0000 7f00c923cb80 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed5cde0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009810 tx=0x7f00a4009260 comp rx=0 tx=0).stop 2024-06-03T02:23:30.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00c923cb80 1 -- 172.21.15.79:0/755706946 shutdown_connections 2024-06-03T02:23:30.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00c923cb80 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed60380 0x55eceed5d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00c923cb80 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55eceed6f9b0 0x55eceed71da0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00c923cb80 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00c923cb80 1 --2- 172.21.15.79:0/755706946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed5cde0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00c923cb80 1 -- 172.21.15.79:0/755706946 >> 172.21.15.79:0/755706946 conn(0x55eceec821f0 msgr2=0x55eceed55a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:30.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00a1600700 0 client.25745 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:30.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00c923cb80 1 -- 172.21.15.79:0/755706946 shutdown_connections 2024-06-03T02:23:30.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00c923cb80 1 -- 172.21.15.79:0/755706946 wait complete. 2024-06-03T02:23:30.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.959+0000 7f00c923cb80 20 client.25745 trim_cache size 0 max 16384 2024-06-03T02:23:30.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.990+0000 7f00c923cb80 1 Processor -- start 2024-06-03T02:23:30.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.990+0000 7f00c923cb80 1 -- start start 2024-06-03T02:23:30.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.990+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.990+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.990+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed60380 0x55eceed61cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00c923cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81bd200 con 0x55eceecd1ec0 2024-06-03T02:23:30.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00c923cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81bed40 con 0x55eceed60380 2024-06-03T02:23:30.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00c923cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81bd8d0 con 0x55eceed55630 2024-06-03T02:23:30.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b7400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed61770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:47586/0 (socket says 172.21.15.79:47586) 2024-06-03T02:23:30.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b7400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37398/0 (socket says 172.21.15.79:37398) 2024-06-03T02:23:30.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b6a00700 1 -- 172.21.15.79:0/3948964874 learned_addr learned my addr 172.21.15.79:0/3948964874 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:30.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b7e00700 1 --2- 172.21.15.79:0/3948964874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed60380 0x55eceed61cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b7400700 1 -- 172.21.15.79:0/3948964874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 msgr2=0x55eceed61770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b7400700 1 --2- 172.21.15.79:0/3948964874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed61770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b7400700 1 -- 172.21.15.79:0/3948964874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed60380 msgr2=0x55eceed61cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b7400700 1 --2- 172.21.15.79:0/3948964874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed60380 0x55eceed61cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.991+0000 7f00b7400700 1 -- 172.21.15.79:0/3948964874 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eceed5dcc0 con 0x55eceecd1ec0 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.992+0000 7f00b7400700 1 --2- 172.21.15.79:0/3948964874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5d410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f00a80cc8d0 tx=0x7f00a80cc7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f7ac26e4b06dcd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.992+0000 7f00a3e00700 1 -- 172.21.15.79:0/3948964874 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a801d040 con 0x55eceecd1ec0 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.992+0000 7f00a3e00700 1 -- 172.21.15.79:0/3948964874 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a80a42a0 con 0x55eceecd1ec0 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.992+0000 7f00a3e00700 1 -- 172.21.15.79:0/3948964874 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8004a40 con 0x55eceecd1ec0 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.993+0000 7f00c923cb80 1 -- 172.21.15.79:0/3948964874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 msgr2=0x55eceed5d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.993+0000 7f00c923cb80 1 --2- 172.21.15.79:0/3948964874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5d410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f00a80cc8d0 tx=0x7f00a80cc7a0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.993+0000 7f00c923cb80 1 -- 172.21.15.79:0/3948964874 shutdown_connections 2024-06-03T02:23:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.993+0000 7f00c923cb80 1 --2- 172.21.15.79:0/3948964874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed60380 0x55eceed61cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.993+0000 7f00c923cb80 1 --2- 172.21.15.79:0/3948964874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceecd1ec0 0x55eceed5d410 secure :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f00a80cc8d0 tx=0x7f00a80cc7a0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.993+0000 7f00c923cb80 1 --2- 172.21.15.79:0/3948964874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed55630 0x55eceed61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.994+0000 7f00c923cb80 1 -- 172.21.15.79:0/3948964874 >> 172.21.15.79:0/3948964874 conn(0x55eceec821f0 msgr2=0x55eceed5b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:30.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.996+0000 7f00c923cb80 1 -- 172.21.15.79:0/3948964874 shutdown_connections 2024-06-03T02:23:30.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.996+0000 7f00c923cb80 1 -- 172.21.15.79:0/3948964874 wait complete. 2024-06-03T02:23:30.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.997+0000 7f00c923cb80 1 -- start start 2024-06-03T02:23:30.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.997+0000 7f00c923cb80 1 Processor -- start 2024-06-03T02:23:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.997+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed75630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.997+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 0x55eceed71920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.997+0000 7f00c923cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed71eb0 0x55eceed72310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.997+0000 7f00c923cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81be670 con 0x55eceed60380 2024-06-03T02:23:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.997+0000 7f00c923cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81bdfa0 con 0x55eceed55630 2024-06-03T02:23:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00c923cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81bb160 con 0x55eceed71eb0 2024-06-03T02:23:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00c923cb80 10 client.0 _ll_register_callbacks cb 0x55eceec417a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed75630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed75630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36464/0 (socket says 172.21.15.79:36464) 2024-06-03T02:23:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b7400700 1 -- 172.21.15.79:0/2004069331 learned_addr learned my addr 172.21.15.79:0/2004069331 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b7e00700 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed71eb0 0x55eceed72310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b6a00700 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 0x55eceed71920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b7400700 1 -- 172.21.15.79:0/2004069331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed71eb0 msgr2=0x55eceed72310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b7400700 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed71eb0 0x55eceed72310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b7400700 1 -- 172.21.15.79:0/2004069331 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 msgr2=0x55eceed71920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b7400700 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 0x55eceed71920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.998+0000 7f00b7400700 1 -- 172.21.15.79:0/2004069331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eceedeb540 con 0x55eceed55630 2024-06-03T02:23:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.999+0000 7f00b7400700 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed75630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f00a809b5c0 tx=0x7f00a8005120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc15c340c14eb72 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.999+0000 7f00a1600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:23:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.999+0000 7f00a1600700 1 -- 172.21.15.79:0/2004069331 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a80b8020 con 0x55eceed55630 2024-06-03T02:23:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.999+0000 7f00a1600700 1 -- 172.21.15.79:0/2004069331 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f00a80cc9e0 con 0x55eceed55630 2024-06-03T02:23:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:30.999+0000 7f00c923cb80 1 -- 172.21.15.79:0/2004069331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a80a42a0 con 0x55eceed55630 2024-06-03T02:23:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.000+0000 7f00a1600700 1 -- 172.21.15.79:0/2004069331 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00a801d040 con 0x55eceed55630 2024-06-03T02:23:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.000+0000 7f00c923cb80 10 client.25751 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.000+0000 7f00c923cb80 1 -- 172.21.15.79:0/2004069331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eceedec860 con 0x55eceed55630 2024-06-03T02:23:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.000+0000 7f00a1600700 1 -- 172.21.15.79:0/2004069331 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a80a42a0 con 0x55eceed55630 2024-06-03T02:23:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.000+0000 7f00a1600700 1 -- 172.21.15.79:0/2004069331 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f00a80a3e10 con 0x55eceed55630 2024-06-03T02:23:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.001+0000 7f00c923cb80 10 client.25751 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.001+0000 7f00c923cb80 10 client.25751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:23:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.001+0000 7f00c923cb80 20 client.25751 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.001+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55eceedecb80 0x55eceedeef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.001+0000 7f00c923cb80 4 client.25751 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:23:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.001+0000 7f00c923cb80 1 -- 172.21.15.79:0/2004069331 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eceecaf600 con 0x55eceedecb80 2024-06-03T02:23:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.001+0000 7f00b6a00700 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55eceedecb80 0x55eceedeef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.001+0000 7f00a1600700 10 client.25751 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.002+0000 7f00b6a00700 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55eceedecb80 0x55eceedeef70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9daab63b6e9b8cf8 server_cookie=ceaf94ac17d86237 in_seq=0 out_seq=0 2024-06-03T02:23:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00a1600700 1 -- 172.21.15.79:0/2004069331 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eceecaf600 con 0x55eceedecb80 2024-06-03T02:23:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00a1600700 10 client.25751 handle_command_reply: tid=0 2024-06-03T02:23:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 client.25751 shutdown 2024-06-03T02:23:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 -- 172.21.15.79:0/2004069331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 msgr2=0x55eceed75630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed75630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f00a809b5c0 tx=0x7f00a8005120 comp rx=0 tx=0).stop 2024-06-03T02:23:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 -- 172.21.15.79:0/2004069331 shutdown_connections 2024-06-03T02:23:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eceed71eb0 0x55eceed72310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eceed55630 0x55eceed75630 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55eceedecb80 0x55eceedeef70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 --2- 172.21.15.79:0/2004069331 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eceed60380 0x55eceed71920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 -- 172.21.15.79:0/2004069331 >> 172.21.15.79:0/2004069331 conn(0x55eceec821f0 msgr2=0x55eceed4ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:31.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00a1600700 0 client.25751 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:31.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.003+0000 7f00c923cb80 1 -- 172.21.15.79:0/2004069331 shutdown_connections 2024-06-03T02:23:31.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.004+0000 7f00c923cb80 1 -- 172.21.15.79:0/2004069331 wait complete. 2024-06-03T02:23:31.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.004+0000 7f00c923cb80 20 client.25751 trim_cache size 0 max 16384 2024-06-03T02:23:31.005 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:23:31.005 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:23:31.006 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "7028133b-585d-478e-a1d0-865fd0bd58cf", 2024-06-03T02:23:31.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:23:31.006 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:23:31.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 -- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f009c0a2e20 msgr2=0x7f009c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:31.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f009c0a2e20 0x7f009c0a52e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00b0019430 tx=0x7f00b000b040 comp rx=0 tx=0).stop 2024-06-03T02:23:31.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 -- 172.21.15.79:0/1113032107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b8102510 msgr2=0x7f00b8116b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:31.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b8102510 0x7f00b8116b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f00a800c920 tx=0x7f00a800cc30 comp rx=0 tx=0).stop 2024-06-03T02:23:31.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 -- 172.21.15.79:0/1113032107 shutdown_connections 2024-06-03T02:23:31.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b8108520 0x7f00b8111ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b8102510 0x7f00b8116b20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f009c0a2e20 0x7f009c0a52e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 --2- 172.21.15.79:0/1113032107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00b8102ed0 0x7f00b8117060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:31.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 -- 172.21.15.79:0/1113032107 >> 172.21.15.79:0/1113032107 conn(0x7f00b8079640 msgr2=0x7f00b8100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:31.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 -- 172.21.15.79:0/1113032107 shutdown_connections 2024-06-03T02:23:31.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:31.005+0000 7f00be800700 1 -- 172.21.15.79:0/1113032107 wait complete. 2024-06-03T02:23:31.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:31 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:23:31.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:31 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:23:31.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:31 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:23:31.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:31 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:23:31.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:31 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:23:31.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:31 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:23:31.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:31 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:23:31.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:31 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:23:31.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:31 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:23:31.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:23:32.541 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:32.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:32 smithi079 ceph-mon[31819]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 644 op/s 2024-06-03T02:23:32.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:32 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:23:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:32 smithi135 ceph-mon[36653]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 644 op/s 2024-06-03T02:23:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:32 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:23:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:32 smithi123 ceph-mon[36640]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 644 op/s 2024-06-03T02:23:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:32 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:23:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 205.0 s, 1030.0 tps, lat 99.084 ms stddev 133.198 2024-06-03T02:23:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:34 smithi135 ceph-mon[36653]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 809 op/s 2024-06-03T02:23:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:34 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:23:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:34 smithi123 ceph-mon[36640]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 809 op/s 2024-06-03T02:23:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:34 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:23:34.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:34 smithi079 ceph-mon[31819]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 809 op/s 2024-06-03T02:23:34.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:34 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:23:35.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.886+0000 7f4e80200700 1 -- 172.21.15.79:0/3647237751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e78109db0 msgr2=0x7f4e781150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:35.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.886+0000 7f4e80200700 1 --2- 172.21.15.79:0/3647237751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e78109db0 0x7f4e781150a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e640098f0 tx=0x7f4e64009c00 comp rx=0 tx=0).stop 2024-06-03T02:23:35.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.887+0000 7f4e80200700 1 -- 172.21.15.79:0/3647237751 shutdown_connections 2024-06-03T02:23:35.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.887+0000 7f4e80200700 1 --2- 172.21.15.79:0/3647237751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e781155e0 0x7f4e781179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.887+0000 7f4e80200700 1 --2- 172.21.15.79:0/3647237751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e78109490 0x7f4e78109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.887+0000 7f4e80200700 1 --2- 172.21.15.79:0/3647237751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e78109db0 0x7f4e781150a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.887+0000 7f4e80200700 1 -- 172.21.15.79:0/3647237751 >> 172.21.15.79:0/3647237751 conn(0x7f4e7806d600 msgr2=0x7f4e7806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:35.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.889+0000 7f4e80200700 1 -- 172.21.15.79:0/3647237751 shutdown_connections 2024-06-03T02:23:35.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.889+0000 7f4e80200700 1 -- 172.21.15.79:0/3647237751 wait complete. 2024-06-03T02:23:35.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.891+0000 7f4e80200700 1 Processor -- start 2024-06-03T02:23:35.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.891+0000 7f4e80200700 1 -- start start 2024-06-03T02:23:35.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.893+0000 7f4e80200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e78109490 0x7f4e7810d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.893+0000 7f4e80200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e78109db0 0x7f4e7810d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.894+0000 7f4e80200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e781155e0 0x7f4e7810db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.894+0000 7f4e80200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7811b0c0 con 0x7f4e78109db0 2024-06-03T02:23:35.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.894+0000 7f4e80200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7811af40 con 0x7f4e78109490 2024-06-03T02:23:35.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.894+0000 7f4e80200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7811b240 con 0x7f4e781155e0 2024-06-03T02:23:35.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.894+0000 7f4e7ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e78109db0 0x7f4e7810d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.894+0000 7f4e7ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e78109db0 0x7f4e7810d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37688/0 (socket says 172.21.15.79:37688) 2024-06-03T02:23:35.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.894+0000 7f4e7ce00700 1 -- 172.21.15.79:0/929025993 learned_addr learned my addr 172.21.15.79:0/929025993 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:35.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.894+0000 7f4e7e200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e781155e0 0x7f4e7810db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.895+0000 7f4e7e200700 1 -- 172.21.15.79:0/929025993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e78109490 msgr2=0x7f4e7810d110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:23:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.895+0000 7f4e7e200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e78109490 0x7f4e7810d110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.895+0000 7f4e7e200700 1 -- 172.21.15.79:0/929025993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e78109db0 msgr2=0x7f4e7810d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.895+0000 7f4e7e200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e78109db0 0x7f4e7810d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.895+0000 7f4e7e200700 1 -- 172.21.15.79:0/929025993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e781b3ee0 con 0x7f4e781155e0 2024-06-03T02:23:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.896+0000 7f4e7e200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e781155e0 0x7f4e7810db90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000be50 tx=0x7f4e70009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.896+0000 7f4e76000700 1 -- 172.21.15.79:0/929025993 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70019070 con 0x7f4e781155e0 2024-06-03T02:23:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.896+0000 7f4e80200700 1 -- 172.21.15.79:0/929025993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e781b41d0 con 0x7f4e781155e0 2024-06-03T02:23:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.896+0000 7f4e80200700 1 -- 172.21.15.79:0/929025993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e781b4760 con 0x7f4e781155e0 2024-06-03T02:23:35.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.896+0000 7f4e76000700 1 -- 172.21.15.79:0/929025993 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e7000cc10 con 0x7f4e781155e0 2024-06-03T02:23:35.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.896+0000 7f4e76000700 1 -- 172.21.15.79:0/929025993 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70012670 con 0x7f4e781155e0 2024-06-03T02:23:35.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.904+0000 7f4e76000700 1 -- 172.21.15.79:0/929025993 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4e7001c070 con 0x7f4e781155e0 2024-06-03T02:23:35.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.907+0000 7f4e76000700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e600a2ca0 0x7f4e600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.907+0000 7f4e76000700 1 -- 172.21.15.79:0/929025993 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4e700ce8c0 con 0x7f4e781155e0 2024-06-03T02:23:35.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.907+0000 7f4e7d800700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e600a2ca0 0x7f4e600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.907+0000 7f4e8abffb80 1 Processor -- start 2024-06-03T02:23:35.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.908+0000 7f4e8abffb80 1 -- start start 2024-06-03T02:23:35.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.908+0000 7f4e7d800700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e600a2ca0 0x7f4e600a5160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c005e50 tx=0x7f4e6c005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:35.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.909+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559599144e40 0x5595991c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.909+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.909+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.909+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5595990d7a10 con 0x559599144e40 2024-06-03T02:23:35.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.909+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55959911ab00 con 0x5595991d3300 2024-06-03T02:23:35.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.909+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5595990d58f0 con 0x5595991c85b0 2024-06-03T02:23:35.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.909+0000 7f4e7d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559599144e40 0x5595991c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.909+0000 7f4e7d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559599144e40 0x5595991c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37698/0 (socket says 172.21.15.79:37698) 2024-06-03T02:23:35.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.910+0000 7f4e7d800700 1 -- 172.21.15.79:0/4779541 learned_addr learned my addr 172.21.15.79:0/4779541 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:35.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.910+0000 7f4e7e200700 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.910+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.911+0000 7f4e7d800700 1 -- 172.21.15.79:0/4779541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 msgr2=0x5595991c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:35.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.911+0000 7f4e7d800700 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.911+0000 7f4e7d800700 1 -- 172.21.15.79:0/4779541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 msgr2=0x5595991d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:35.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.911+0000 7f4e7d800700 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.911+0000 7f4e7d800700 1 -- 172.21.15.79:0/4779541 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595990c2f90 con 0x559599144e40 2024-06-03T02:23:35.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.911+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:23:35.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.912+0000 7f4e7d800700 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559599144e40 0x5595991c8070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00dbd0 tx=0x7f4e6c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737f60451116e1a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:35.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.912+0000 7f4e6aa00700 1 -- 172.21.15.79:0/4779541 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c010af0 con 0x559599144e40 2024-06-03T02:23:35.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.913+0000 7f4e6aa00700 1 -- 172.21.15.79:0/4779541 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c022070 con 0x559599144e40 2024-06-03T02:23:35.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.913+0000 7f4e6aa00700 1 -- 172.21.15.79:0/4779541 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00f620 con 0x559599144e40 2024-06-03T02:23:35.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.914+0000 7f4e8abffb80 1 -- 172.21.15.79:0/4779541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559599144e40 msgr2=0x5595991c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:35.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.914+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559599144e40 0x5595991c8070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00dbd0 tx=0x7f4e6c010000 comp rx=0 tx=0).stop 2024-06-03T02:23:35.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.914+0000 7f4e8abffb80 1 -- 172.21.15.79:0/4779541 shutdown_connections 2024-06-03T02:23:35.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.914+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.914+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559599144e40 0x5595991c8070 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00dbd0 tx=0x7f4e6c010000 comp rx=0 tx=0).stop 2024-06-03T02:23:35.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.914+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/4779541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.914+0000 7f4e8abffb80 1 -- 172.21.15.79:0/4779541 >> 172.21.15.79:0/4779541 conn(0x5595990f5170 msgr2=0x5595991c44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:35.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.919+0000 7f4e8abffb80 1 -- 172.21.15.79:0/4779541 shutdown_connections 2024-06-03T02:23:35.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.919+0000 7f4e8abffb80 1 -- 172.21.15.79:0/4779541 wait complete. 2024-06-03T02:23:35.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.920+0000 7f4e8abffb80 1 -- start start 2024-06-03T02:23:35.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.920+0000 7f4e8abffb80 1 Processor -- start 2024-06-03T02:23:35.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.921+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5595990b6f60 con 0x5595991c85b0 2024-06-03T02:23:35.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559599116ac0 con 0x5595991d3300 2024-06-03T02:23:35.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5595991d5f60 con 0x5595991d0400 2024-06-03T02:23:35.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e8abffb80 10 client.0 _ll_register_callbacks cb 0x5595990b4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:35.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e7e200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e7e200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:51730/0 (socket says 172.21.15.79:51730) 2024-06-03T02:23:35.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e7e200700 1 -- 172.21.15.79:0/83268393 learned_addr learned my addr 172.21.15.79:0/83268393 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:35.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.922+0000 7f4e7d800700 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.923+0000 7f4e7e200700 1 -- 172.21.15.79:0/83268393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 msgr2=0x5595991cfec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:23:35.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.923+0000 7f4e7e200700 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991cfec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.923+0000 7f4e7e200700 1 -- 172.21.15.79:0/83268393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 msgr2=0x5595991d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:35.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.923+0000 7f4e7e200700 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.923+0000 7f4e7e200700 1 -- 172.21.15.79:0/83268393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595990698d0 con 0x5595991d0400 2024-06-03T02:23:35.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.923+0000 7f4e7d800700 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:23:35.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.923+0000 7f4e7e200700 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d0880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4e700cc010 tx=0x7f4e7009f2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3fdb3b75a50a5f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:35.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.923+0000 7f4e5f400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:23:35.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.926+0000 7f4e5f400700 1 -- 172.21.15.79:0/83268393 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70014440 con 0x5595991d0400 2024-06-03T02:23:35.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.928+0000 7f4e5f400700 1 -- 172.21.15.79:0/83268393 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4e700145e0 con 0x5595991d0400 2024-06-03T02:23:35.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.928+0000 7f4e8abffb80 1 -- 172.21.15.79:0/83268393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e6c022070 con 0x5595991d0400 2024-06-03T02:23:35.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.929+0000 7f4e5f400700 1 -- 172.21.15.79:0/83268393 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e700a13b0 con 0x5595991d0400 2024-06-03T02:23:35.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.929+0000 7f4e8abffb80 10 client.25741 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:35.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.929+0000 7f4e8abffb80 1 -- 172.21.15.79:0/83268393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595991e0c70 con 0x5595991d0400 2024-06-03T02:23:35.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.929+0000 7f4e5f400700 1 -- 172.21.15.79:0/83268393 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c022070 con 0x5595991d0400 2024-06-03T02:23:35.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.929+0000 7f4e5f400700 1 -- 172.21.15.79:0/83268393 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4e700b9320 con 0x5595991d0400 2024-06-03T02:23:35.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.929+0000 7f4e8abffb80 10 client.25741 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:35.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.929+0000 7f4e8abffb80 10 client.25741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:23:35.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.929+0000 7f4e8abffb80 20 client.25741 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:35.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.929+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5595991e32a0 0x5595991e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.930+0000 7f4e8abffb80 4 client.25741 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:23:35.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.930+0000 7f4e8abffb80 1 -- 172.21.15.79:0/83268393 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559599128000 con 0x5595991e32a0 2024-06-03T02:23:35.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.931+0000 7f4e7d800700 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5595991e32a0 0x5595991e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.932+0000 7f4e7d800700 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5595991e32a0 0x5595991e56d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b60a6adef3b5c961 server_cookie=716f3bacb5b241f6 in_seq=0 out_seq=0 2024-06-03T02:23:35.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.932+0000 7f4e5f400700 10 client.25741 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.932+0000 7f4e5f400700 1 -- 172.21.15.79:0/83268393 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559599128000 con 0x5595991e32a0 2024-06-03T02:23:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.933+0000 7f4e5f400700 10 client.25741 handle_command_reply: tid=0 2024-06-03T02:23:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.933+0000 7f4e8abffb80 1 client.25741 shutdown 2024-06-03T02:23:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.934+0000 7f4e8abffb80 1 -- 172.21.15.79:0/83268393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 msgr2=0x5595991d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.934+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d0880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4e700cc010 tx=0x7f4e7009f2e0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.934+0000 7f4e8abffb80 1 -- 172.21.15.79:0/83268393 shutdown_connections 2024-06-03T02:23:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.934+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d0880 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.934+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5595991e32a0 0x5595991e56d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.934+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.934+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/83268393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.934+0000 7f4e8abffb80 1 -- 172.21.15.79:0/83268393 >> 172.21.15.79:0/83268393 conn(0x5595990f5170 msgr2=0x5595991c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:35.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.935+0000 7f4e5f400700 0 client.25741 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:35.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.935+0000 7f4e8abffb80 1 -- 172.21.15.79:0/83268393 shutdown_connections 2024-06-03T02:23:35.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.936+0000 7f4e8abffb80 1 -- 172.21.15.79:0/83268393 wait complete. 2024-06-03T02:23:35.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.936+0000 7f4e8abffb80 20 client.25741 trim_cache size 0 max 16384 2024-06-03T02:23:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.982+0000 7f4e8abffb80 1 Processor -- start 2024-06-03T02:23:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.982+0000 7f4e8abffb80 1 -- start start 2024-06-03T02:23:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.982+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5800eb40 con 0x5595991c85b0 2024-06-03T02:23:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58010680 con 0x5595991d3300 2024-06-03T02:23:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5800f210 con 0x5595991d0400 2024-06-03T02:23:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e7d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e7e200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e7d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37710/0 (socket says 172.21.15.79:37710) 2024-06-03T02:23:35.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e7e200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991d56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41342/0 (socket says 172.21.15.79:41342) 2024-06-03T02:23:35.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e7d800700 1 -- 172.21.15.79:0/2883969907 learned_addr learned my addr 172.21.15.79:0/2883969907 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:35.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.983+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/2883969907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.984+0000 7f4e7ce00700 1 -- 172.21.15.79:0/2883969907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 msgr2=0x5595991d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.984+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/2883969907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.984+0000 7f4e7ce00700 1 -- 172.21.15.79:0/2883969907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 msgr2=0x5595991d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.984+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/2883969907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.984+0000 7f4e7ce00700 1 -- 172.21.15.79:0/2883969907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595991ce430 con 0x5595991d0400 2024-06-03T02:23:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.985+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/2883969907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400b2e0 tx=0x7f4e64005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdea9559f03f0a03 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.986+0000 7f4e6a000700 1 -- 172.21.15.79:0/2883969907 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64028070 con 0x5595991d0400 2024-06-03T02:23:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.986+0000 7f4e6a000700 1 -- 172.21.15.79:0/2883969907 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e64003b60 con 0x5595991d0400 2024-06-03T02:23:35.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.986+0000 7f4e6a000700 1 -- 172.21.15.79:0/2883969907 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6400bd70 con 0x5595991d0400 2024-06-03T02:23:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.987+0000 7f4e8abffb80 1 -- 172.21.15.79:0/2883969907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 msgr2=0x5595991d51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.987+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/2883969907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400b2e0 tx=0x7f4e64005e00 comp rx=0 tx=0).stop 2024-06-03T02:23:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.987+0000 7f4e8abffb80 1 -- 172.21.15.79:0/2883969907 shutdown_connections 2024-06-03T02:23:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.987+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/2883969907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d3300 0x5595991d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.987+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/2883969907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991c85b0 0x5595991d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.987+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/2883969907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991d0400 0x5595991d51b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.987+0000 7f4e8abffb80 1 -- 172.21.15.79:0/2883969907 >> 172.21.15.79:0/2883969907 conn(0x5595990f5170 msgr2=0x5595991cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:35.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.988+0000 7f4e8abffb80 1 -- 172.21.15.79:0/2883969907 shutdown_connections 2024-06-03T02:23:35.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.988+0000 7f4e8abffb80 1 -- 172.21.15.79:0/2883969907 wait complete. 2024-06-03T02:23:35.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.988+0000 7f4e8abffb80 1 -- start start 2024-06-03T02:23:35.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.988+0000 7f4e8abffb80 1 Processor -- start 2024-06-03T02:23:35.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.989+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.989+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d0400 0x5595991d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.989+0000 7f4e8abffb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991d3300 0x5595991d4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.990+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5800ffb0 con 0x5595991d3300 2024-06-03T02:23:35.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.990+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5800f8e0 con 0x5595991d0400 2024-06-03T02:23:35.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.990+0000 7f4e8abffb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5800cae0 con 0x5595991c85b0 2024-06-03T02:23:35.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.990+0000 7f4e8abffb80 10 client.0 _ll_register_callbacks cb 0x5595990b4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:35.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.990+0000 7f4e7d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.990+0000 7f4e7d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991d3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:51750/0 (socket says 172.21.15.79:51750) 2024-06-03T02:23:35.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.990+0000 7f4e7d800700 1 -- 172.21.15.79:0/1146582065 learned_addr learned my addr 172.21.15.79:0/1146582065 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:35.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.991+0000 7f4e7d800700 1 -- 172.21.15.79:0/1146582065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d0400 msgr2=0x5595991d4240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:23:35.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.991+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d0400 0x5595991d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.992+0000 7f4e7e200700 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991d3300 0x5595991d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.992+0000 7f4e7d800700 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d0400 0x5595991d4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.992+0000 7f4e7d800700 1 -- 172.21.15.79:0/1146582065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991d3300 msgr2=0x5595991d4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:35.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.992+0000 7f4e7d800700 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991d3300 0x5595991d4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.992+0000 7f4e7d800700 1 -- 172.21.15.79:0/1146582065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595991da3c0 con 0x5595991c85b0 2024-06-03T02:23:35.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.992+0000 7f4e7d800700 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991d3d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00dba0 tx=0x7f4e6c01dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1456d65c50cba2c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:35.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.992+0000 7f4e5f400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:23:35.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.992+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d0400 0x5595991d4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:23:35.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.993+0000 7f4e5f400700 1 -- 172.21.15.79:0/1146582065 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c01d740 con 0x5595991c85b0 2024-06-03T02:23:35.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.993+0000 7f4e8abffb80 1 -- 172.21.15.79:0/1146582065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e64003b60 con 0x5595991c85b0 2024-06-03T02:23:35.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.993+0000 7f4e5f400700 1 -- 172.21.15.79:0/1146582065 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4e6c01d8e0 con 0x5595991c85b0 2024-06-03T02:23:35.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.993+0000 7f4e5f400700 1 -- 172.21.15.79:0/1146582065 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e6c041050 con 0x5595991c85b0 2024-06-03T02:23:35.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.993+0000 7f4e8abffb80 10 client.25747 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:35.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.993+0000 7f4e8abffb80 1 -- 172.21.15.79:0/1146582065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595991db710 con 0x5595991c85b0 2024-06-03T02:23:35.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.994+0000 7f4e5f400700 1 -- 172.21.15.79:0/1146582065 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e64003b60 con 0x5595991c85b0 2024-06-03T02:23:35.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.994+0000 7f4e5f400700 1 -- 172.21.15.79:0/1146582065 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4e6c0055b0 con 0x5595991c85b0 2024-06-03T02:23:35.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.994+0000 7f4e8abffb80 10 client.25747 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:35.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.994+0000 7f4e8abffb80 10 client.25747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:23:35.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.994+0000 7f4e8abffb80 20 client.25747 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:35.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.994+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5595991dbb40 0x55959924ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:35.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.994+0000 7f4e8abffb80 4 client.25747 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:23:35.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.994+0000 7f4e8abffb80 1 -- 172.21.15.79:0/1146582065 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559599122580 con 0x5595991dbb40 2024-06-03T02:23:35.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.995+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5595991dbb40 0x55959924ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:35.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.996+0000 7f4e7ce00700 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5595991dbb40 0x55959924ff80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de63a8b1482bf8f server_cookie=83325e4bb2d23671 in_seq=0 out_seq=0 2024-06-03T02:23:35.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.996+0000 7f4e5f400700 10 client.25747 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:35.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.996+0000 7f4e5f400700 1 -- 172.21.15.79:0/1146582065 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559599122580 con 0x5595991dbb40 2024-06-03T02:23:35.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.996+0000 7f4e5f400700 10 client.25747 handle_command_reply: tid=0 2024-06-03T02:23:35.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.996+0000 7f4e8abffb80 1 client.25747 shutdown 2024-06-03T02:23:35.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.997+0000 7f4e8abffb80 1 -- 172.21.15.79:0/1146582065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 msgr2=0x5595991d3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:35.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.997+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991d3d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00dba0 tx=0x7f4e6c01dbb0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.998+0000 7f4e8abffb80 1 -- 172.21.15.79:0/1146582065 shutdown_connections 2024-06-03T02:23:35.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.998+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5595991dbb40 0x55959924ff80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:35.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.998+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5595991d3300 0x5595991d4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.998+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595991c85b0 0x5595991d3d00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.998+0000 7f4e8abffb80 1 --2- 172.21.15.79:0/1146582065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595991d0400 0x5595991d4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.998+0000 7f4e5f400700 0 client.25747 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:23:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.998+0000 7f4e8abffb80 1 -- 172.21.15.79:0/1146582065 >> 172.21.15.79:0/1146582065 conn(0x5595990f5170 msgr2=0x5595991c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.999+0000 7f4e8abffb80 1 -- 172.21.15.79:0/1146582065 shutdown_connections 2024-06-03T02:23:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.999+0000 7f4e8abffb80 1 -- 172.21.15.79:0/1146582065 wait complete. 2024-06-03T02:23:36.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:35.999+0000 7f4e8abffb80 20 client.25747 trim_cache size 0 max 16384 2024-06-03T02:23:36.001 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:23:36.001 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:23:36.001 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:23:36.001 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:23:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.000+0000 7f4e80200700 1 -- 172.21.15.79:0/929025993 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e600a2ca0 msgr2=0x7f4e600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.000+0000 7f4e80200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e600a2ca0 0x7f4e600a5160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c005e50 tx=0x7f4e6c005de0 comp rx=0 tx=0).stop 2024-06-03T02:23:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.000+0000 7f4e80200700 1 -- 172.21.15.79:0/929025993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e781155e0 msgr2=0x7f4e7810db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.000+0000 7f4e80200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e781155e0 0x7f4e7810db90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000be50 tx=0x7f4e70009ed0 comp rx=0 tx=0).stop 2024-06-03T02:23:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.001+0000 7f4e80200700 1 -- 172.21.15.79:0/929025993 shutdown_connections 2024-06-03T02:23:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.001+0000 7f4e80200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e781155e0 0x7f4e7810db90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.001+0000 7f4e80200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e78109490 0x7f4e7810d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.001+0000 7f4e80200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4e600a2ca0 0x7f4e600a5160 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:36.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.001+0000 7f4e80200700 1 --2- 172.21.15.79:0/929025993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e78109db0 0x7f4e7810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:36.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.001+0000 7f4e80200700 1 -- 172.21.15.79:0/929025993 >> 172.21.15.79:0/929025993 conn(0x7f4e7806d600 msgr2=0x7f4e7810ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:36.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.001+0000 7f4e80200700 1 -- 172.21.15.79:0/929025993 shutdown_connections 2024-06-03T02:23:36.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:36.001+0000 7f4e80200700 1 -- 172.21.15.79:0/929025993 wait complete. 2024-06-03T02:23:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:36 smithi135 ceph-mon[36653]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 616 op/s 2024-06-03T02:23:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:36 smithi123 ceph-mon[36640]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 616 op/s 2024-06-03T02:23:36.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:36 smithi079 ceph-mon[31819]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 616 op/s 2024-06-03T02:23:37.637 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:23:37.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:23:38.304 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 210.0 s, 929.4 tps, lat 107.129 ms stddev 156.776 2024-06-03T02:23:38.677 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:38 smithi079 ceph-mon[31819]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.5 MiB/s wr, 562 op/s 2024-06-03T02:23:38.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:38 smithi123 ceph-mon[36640]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.5 MiB/s wr, 562 op/s 2024-06-03T02:23:38.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:38 smithi135 ceph-mon[36653]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.5 MiB/s wr, 562 op/s 2024-06-03T02:23:40.431 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:40 smithi079 ceph-mon[31819]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 741 op/s 2024-06-03T02:23:40.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:40 smithi123 ceph-mon[36640]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 741 op/s 2024-06-03T02:23:40.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.763+0000 7f95fc800700 1 -- 172.21.15.79:0/1792393326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f4111060 msgr2=0x7f95f41099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:40.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.763+0000 7f95fc800700 1 --2- 172.21.15.79:0/1792393326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f4111060 0x7f95f41099d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009990 tx=0x7f95e4009ca0 comp rx=0 tx=0).stop 2024-06-03T02:23:40.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.763+0000 7f95fc800700 1 -- 172.21.15.79:0/1792393326 shutdown_connections 2024-06-03T02:23:40.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.763+0000 7f95fc800700 1 --2- 172.21.15.79:0/1792393326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95f411bc70 0x7f95f411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:40.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.763+0000 7f95fc800700 1 --2- 172.21.15.79:0/1792393326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f4110bf0 0x7f95f4109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:40.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.763+0000 7f95fc800700 1 --2- 172.21.15.79:0/1792393326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f4111060 0x7f95f41099d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:40.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.763+0000 7f95fc800700 1 -- 172.21.15.79:0/1792393326 >> 172.21.15.79:0/1792393326 conn(0x7f95f406d600 msgr2=0x7f95f406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:40.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.763+0000 7f95fc800700 1 -- 172.21.15.79:0/1792393326 shutdown_connections 2024-06-03T02:23:40.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.763+0000 7f95fc800700 1 -- 172.21.15.79:0/1792393326 wait complete. 2024-06-03T02:23:40.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.765+0000 7f95fc800700 1 Processor -- start 2024-06-03T02:23:40.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.765+0000 7f95fc800700 1 -- start start 2024-06-03T02:23:40.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.765+0000 7f95fc800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f4110bf0 0x7f95f41ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:40.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.765+0000 7f95fc800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95f411bc70 0x7f95f41adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:40.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.765+0000 7f95fc800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41a78e0 0x7f95f41a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:40.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95fc800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4121640 con 0x7f95f411bc70 2024-06-03T02:23:40.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95fc800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41214c0 con 0x7f95f41a78e0 2024-06-03T02:23:40.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95fc800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41217c0 con 0x7f95f4110bf0 2024-06-03T02:23:40.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95f9400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95f411bc70 0x7f95f41adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:40.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95f9400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95f411bc70 0x7f95f41adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37736/0 (socket says 172.21.15.79:37736) 2024-06-03T02:23:40.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95f9400700 1 -- 172.21.15.79:0/699309287 learned_addr learned my addr 172.21.15.79:0/699309287 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:40.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95fa800700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41a78e0 0x7f95f41a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:40.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95f9400700 1 -- 172.21.15.79:0/699309287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f4110bf0 msgr2=0x7f95f41ad720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:23:40.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95f9400700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f4110bf0 0x7f95f41ad720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:40.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.766+0000 7f95f9400700 1 -- 172.21.15.79:0/699309287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41a78e0 msgr2=0x7f95f41a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:40.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.767+0000 7f95f9400700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41a78e0 0x7f95f41a7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:40.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.767+0000 7f95f9400700 1 -- 172.21.15.79:0/699309287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f41a8460 con 0x7f95f411bc70 2024-06-03T02:23:40.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.767+0000 7f95fa800700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41a78e0 0x7f95f41a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:23:40.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.767+0000 7f95f9400700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95f411bc70 0x7f95f41adc60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95e4000c00 tx=0x7f95e400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:40.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.768+0000 7f95eea00700 1 -- 172.21.15.79:0/699309287 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4027070 con 0x7f95f411bc70 2024-06-03T02:23:40.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.768+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f41a86f0 con 0x7f95f411bc70 2024-06-03T02:23:40.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.768+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f41b48d0 con 0x7f95f411bc70 2024-06-03T02:23:40.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.768+0000 7f95eea00700 1 -- 172.21.15.79:0/699309287 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e4003e30 con 0x7f95f411bc70 2024-06-03T02:23:40.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.768+0000 7f95eea00700 1 -- 172.21.15.79:0/699309287 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4022d70 con 0x7f95f411bc70 2024-06-03T02:23:40.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.770+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d8005330 con 0x7f95f411bc70 2024-06-03T02:23:40.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.773+0000 7f95eea00700 1 -- 172.21.15.79:0/699309287 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f95e4009d10 con 0x7f95f411bc70 2024-06-03T02:23:40.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.776+0000 7f95eea00700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f95dc0a2d20 0x7f95dc0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:40.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.776+0000 7f95eea00700 1 -- 172.21.15.79:0/699309287 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f95e40d05e0 con 0x7f95f411bc70 2024-06-03T02:23:40.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.777+0000 7f95f9e00700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f95dc0a2d20 0x7f95dc0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:40.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.779+0000 7f95f9e00700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f95dc0a2d20 0x7f95dc0a51e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f95f41a8a20 tx=0x7f95e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:40.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:40.780+0000 7f95eea00700 1 -- 172.21.15.79:0/699309287 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f95e4013070 con 0x7f95f411bc70 2024-06-03T02:23:40.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:40 smithi135 ceph-mon[36653]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 741 op/s 2024-06-03T02:23:41.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.260+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95d8005620 con 0x7f95f411bc70 2024-06-03T02:23:41.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.263+0000 7f95eea00700 1 -- 172.21.15.79:0/699309287 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f95e409d1c0 con 0x7f95f411bc70 2024-06-03T02:23:41.264 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:23:41.264 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:23:41.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.267+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f95dc0a2d20 msgr2=0x7f95dc0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:41.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.267+0000 7f95fc800700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f95dc0a2d20 0x7f95dc0a51e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f95f41a8a20 tx=0x7f95e8009260 comp rx=0 tx=0).stop 2024-06-03T02:23:41.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.267+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95f411bc70 msgr2=0x7f95f41adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:41.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.267+0000 7f95fc800700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95f411bc70 0x7f95f41adc60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95e4000c00 tx=0x7f95e400ba90 comp rx=0 tx=0).stop 2024-06-03T02:23:41.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.267+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 shutdown_connections 2024-06-03T02:23:41.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.267+0000 7f95fc800700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41a78e0 0x7f95f41a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:41.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.267+0000 7f95fc800700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f4110bf0 0x7f95f41ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:41.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.267+0000 7f95fc800700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f95dc0a2d20 0x7f95dc0a51e0 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f95f41a8a20 tx=0x7f95e8009260 comp rx=0 tx=0).stop 2024-06-03T02:23:41.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.267+0000 7f95fc800700 1 --2- 172.21.15.79:0/699309287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95f411bc70 0x7f95f41adc60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:41.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.268+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 >> 172.21.15.79:0/699309287 conn(0x7f95f406d600 msgr2=0x7f95f4119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:41.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.268+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 shutdown_connections 2024-06-03T02:23:41.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:41.268+0000 7f95fc800700 1 -- 172.21.15.79:0/699309287 wait complete. 2024-06-03T02:23:41.272 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:23:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:41 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/699309287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:23:41.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:41 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/699309287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:23:41.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:41 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/699309287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:23:42.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:23:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:42 smithi079 ceph-mon[31819]: pgmap v625: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-06-03T02:23:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:42 smithi123 ceph-mon[36640]: pgmap v625: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-06-03T02:23:42.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:42 smithi135 ceph-mon[36653]: pgmap v625: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-06-03T02:23:42.976 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 215.0 s, 1069.4 tps, lat 92.603 ms stddev 112.987 2024-06-03T02:23:43.490 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:43 smithi079 ceph-mon[31819]: pgmap v626: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 MiB/s wr, 646 op/s 2024-06-03T02:23:43.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:43 smithi123 ceph-mon[36640]: pgmap v626: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 MiB/s wr, 646 op/s 2024-06-03T02:23:43.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:43 smithi135 ceph-mon[36653]: pgmap v626: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 MiB/s wr, 646 op/s 2024-06-03T02:23:45.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:45 smithi079 ceph-mon[31819]: pgmap v627: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.5 MiB/s wr, 528 op/s 2024-06-03T02:23:45.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:45 smithi135 ceph-mon[36653]: pgmap v627: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.5 MiB/s wr, 528 op/s 2024-06-03T02:23:45.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:45 smithi123 ceph-mon[36640]: pgmap v627: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.5 MiB/s wr, 528 op/s 2024-06-03T02:23:47.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.397+0000 7f7ac3c00700 1 -- 172.21.15.79:0/3830082412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 msgr2=0x7f7abc109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:47.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.398+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/3830082412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 0x7f7abc109ad0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009990 tx=0x7f7ab0009ca0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.400+0000 7f7ac3c00700 1 -- 172.21.15.79:0/3830082412 shutdown_connections 2024-06-03T02:23:47.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.400+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/3830082412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7abc10e220 0x7f7abc115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.400+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/3830082412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 0x7f7abc109ad0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.400+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/3830082412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7abc10a0a0 0x7f7abc10db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.400+0000 7f7ac3c00700 1 -- 172.21.15.79:0/3830082412 >> 172.21.15.79:0/3830082412 conn(0x7f7abc06d600 msgr2=0x7f7abc06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:47.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.401+0000 7f7ac3c00700 1 -- 172.21.15.79:0/3830082412 shutdown_connections 2024-06-03T02:23:47.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.401+0000 7f7ac3c00700 1 -- 172.21.15.79:0/3830082412 wait complete. 2024-06-03T02:23:47.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.402+0000 7f7ac3c00700 1 Processor -- start 2024-06-03T02:23:47.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.403+0000 7f7ac3c00700 1 -- start start 2024-06-03T02:23:47.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.403+0000 7f7ac3c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 0x7f7abc10fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.404+0000 7f7ac1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 0x7f7abc10fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.404+0000 7f7ac3c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7abc10a0a0 0x7f7abc1103d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.404+0000 7f7ac1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 0x7f7abc10fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:43394/0 (socket says 172.21.15.79:43394) 2024-06-03T02:23:47.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.405+0000 7f7ac3c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7abc10e220 0x7f7abc110910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.405+0000 7f7ac3c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc116d00 con 0x7f7abc10e220 2024-06-03T02:23:47.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.405+0000 7f7ac3c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc116b80 con 0x7f7abc10a0a0 2024-06-03T02:23:47.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.405+0000 7f7ac3c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc116e80 con 0x7f7abc1096f0 2024-06-03T02:23:47.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.405+0000 7f7ac1200700 1 -- 172.21.15.79:0/1853503548 learned_addr learned my addr 172.21.15.79:0/1853503548 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:47.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.405+0000 7f7abbe00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7abc10a0a0 0x7f7abc1103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.406+0000 7f7abbe00700 1 -- 172.21.15.79:0/1853503548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 msgr2=0x7f7abc10fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:47.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.407+0000 7f7abbe00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 0x7f7abc10fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.407+0000 7f7abbe00700 1 -- 172.21.15.79:0/1853503548 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7abc10e220 msgr2=0x7f7abc110910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:23:47.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.407+0000 7f7abbe00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7abc10e220 0x7f7abc110910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.407+0000 7f7abbe00700 1 -- 172.21.15.79:0/1853503548 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc1af9e0 con 0x7f7abc10a0a0 2024-06-03T02:23:47.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.407+0000 7f7ac1200700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 0x7f7abc10fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:23:47.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.409+0000 7f7abbe00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7abc10a0a0 0x7f7abc1103d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7abc06bc00 tx=0x7f7aa800b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:47.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.409+0000 7f7ab9600700 1 -- 172.21.15.79:0/1853503548 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8015020 con 0x7f7abc10a0a0 2024-06-03T02:23:47.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.409+0000 7f7ac3c00700 1 -- 172.21.15.79:0/1853503548 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc1afcd0 con 0x7f7abc10a0a0 2024-06-03T02:23:47.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.409+0000 7f7ac3c00700 1 -- 172.21.15.79:0/1853503548 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc1b01c0 con 0x7f7abc10a0a0 2024-06-03T02:23:47.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.410+0000 7f7ab9600700 1 -- 172.21.15.79:0/1853503548 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa8011d10 con 0x7f7abc10a0a0 2024-06-03T02:23:47.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.410+0000 7f7ab9600700 1 -- 172.21.15.79:0/1853503548 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8010d20 con 0x7f7abc10a0a0 2024-06-03T02:23:47.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.419+0000 7f7ab9600700 1 -- 172.21.15.79:0/1853503548 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7aa8018020 con 0x7f7abc10a0a0 2024-06-03T02:23:47.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.420+0000 7f7ace538b80 1 Processor -- start 2024-06-03T02:23:47.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.422+0000 7f7ab9600700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7aa40a2c20 0x7f7aa40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.422+0000 7f7ab9600700 1 -- 172.21.15.79:0/1853503548 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7aa80ce9f0 con 0x7f7abc10a0a0 2024-06-03T02:23:47.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.423+0000 7f7ace538b80 1 -- start start 2024-06-03T02:23:47.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.423+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7aa40a2c20 0x7f7aa40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.424+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7aa40a2c20 0x7f7aa40a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400e350 tx=0x7f7ab400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c5b90 0x55e0452c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452bfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452c0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7ace538b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e0451cea10 con 0x55e0452c5b90 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7ace538b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e045211b00 con 0x55e0452c0240 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7ace538b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e0451cc8f0 con 0x55e0452c64b0 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7ac1c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7abbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452bfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7ac1c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452c0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60024/0 (socket says 172.21.15.79:60024) 2024-06-03T02:23:47.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.425+0000 7f7ac1c00700 1 -- 172.21.15.79:0/1916578384 learned_addr learned my addr 172.21.15.79:0/1916578384 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:47.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.426+0000 7f7ac1200700 1 --2- 172.21.15.79:0/1916578384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c5b90 0x55e0452c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.426+0000 7f7ac1c00700 1 -- 172.21.15.79:0/1916578384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 msgr2=0x55e0452bfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:47.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.426+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/1916578384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452bfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.426+0000 7f7ac1c00700 1 -- 172.21.15.79:0/1916578384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c5b90 msgr2=0x55e0452c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:47.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.426+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/1916578384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c5b90 0x55e0452c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.426+0000 7f7ac1c00700 1 -- 172.21.15.79:0/1916578384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0451b9f90 con 0x55e0452c0240 2024-06-03T02:23:47.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.426+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/1916578384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452c0680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4015470 tx=0x7f7ab4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2fd3b4092888b0e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:47.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.427+0000 7f7aae000700 1 -- 172.21.15.79:0/1916578384 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab401b040 con 0x55e0452c0240 2024-06-03T02:23:47.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.427+0000 7f7aae000700 1 -- 172.21.15.79:0/1916578384 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab4015cc0 con 0x55e0452c0240 2024-06-03T02:23:47.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.427+0000 7f7aae000700 1 -- 172.21.15.79:0/1916578384 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab401b040 con 0x55e0452c0240 2024-06-03T02:23:47.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.429+0000 7f7ace538b80 1 -- 172.21.15.79:0/1916578384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 msgr2=0x55e0452c0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:47.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.429+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1916578384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452c0680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4015470 tx=0x7f7ab4011000 comp rx=0 tx=0).stop 2024-06-03T02:23:47.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.430+0000 7f7ace538b80 1 -- 172.21.15.79:0/1916578384 shutdown_connections 2024-06-03T02:23:47.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.430+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1916578384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452c0680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.430+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1916578384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c5b90 0x55e0452c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.430+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1916578384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452bfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.430+0000 7f7ace538b80 1 -- 172.21.15.79:0/1916578384 >> 172.21.15.79:0/1916578384 conn(0x55e0451ec170 msgr2=0x55e0452baff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:47.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.430+0000 7f7ace538b80 1 -- 172.21.15.79:0/1916578384 shutdown_connections 2024-06-03T02:23:47.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.430+0000 7f7ace538b80 1 -- 172.21.15.79:0/1916578384 wait complete. 2024-06-03T02:23:47.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.431+0000 7f7ace538b80 1 -- start start 2024-06-03T02:23:47.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.431+0000 7f7ace538b80 1 Processor -- start 2024-06-03T02:23:47.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.434+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 0x55e0452cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.434+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c5b90 0x55e0452c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.434+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.434+0000 7f7abbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c5b90 0x55e0452c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.434+0000 7f7abbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c5b90 0x55e0452c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60040/0 (socket says 172.21.15.79:60040) 2024-06-03T02:23:47.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.434+0000 7f7abbe00700 1 -- 172.21.15.79:0/3288843145 learned_addr learned my addr 172.21.15.79:0/3288843145 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:47.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.435+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e0451adf60 con 0x55e0452c0240 2024-06-03T02:23:47.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.435+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e04520dac0 con 0x55e0452c5b90 2024-06-03T02:23:47.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.435+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e0452cce90 con 0x55e0452c64b0 2024-06-03T02:23:47.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.435+0000 7f7ac1200700 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 0x55e0452cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.435+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.435+0000 7f7ace538b80 10 client.0 _ll_register_callbacks cb 0x55e0451a7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.436+0000 7f7abbe00700 1 -- 172.21.15.79:0/3288843145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 msgr2=0x55e0452c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.436+0000 7f7abbe00700 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.436+0000 7f7abbe00700 1 -- 172.21.15.79:0/3288843145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 msgr2=0x55e0452cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.436+0000 7f7abbe00700 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 0x55e0452cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.436+0000 7f7abbe00700 1 -- 172.21.15.79:0/3288843145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0451608d0 con 0x55e0452c5b90 2024-06-03T02:23:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.436+0000 7f7abbe00700 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c5b90 0x55e0452c9540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7aa809aba0 tx=0x7f7aa809a800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41f221daf751e2e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.436+0000 7f7aa2a00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:23:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.436+0000 7f7aa2a00700 1 -- 172.21.15.79:0/3288843145 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa809f250 con 0x55e0452c5b90 2024-06-03T02:23:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.436+0000 7f7aa2a00700 1 -- 172.21.15.79:0/3288843145 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7aa809b100 con 0x55e0452c5b90 2024-06-03T02:23:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.439+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab4015cc0 con 0x55e0452c5b90 2024-06-03T02:23:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.440+0000 7f7aa2a00700 1 -- 172.21.15.79:0/3288843145 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7aa80a3b10 con 0x55e0452c5b90 2024-06-03T02:23:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.440+0000 7f7ace538b80 10 client.25775 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.440+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e0452d74f0 con 0x55e0452c5b90 2024-06-03T02:23:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.440+0000 7f7aa2a00700 1 -- 172.21.15.79:0/3288843145 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab4015cc0 con 0x55e0452c5b90 2024-06-03T02:23:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.440+0000 7f7aa2a00700 1 -- 172.21.15.79:0/3288843145 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7aa8010380 con 0x55e0452c5b90 2024-06-03T02:23:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.441+0000 7f7ace538b80 10 client.25775 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.441+0000 7f7ace538b80 10 client.25775 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:23:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.441+0000 7f7ace538b80 20 client.25775 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.441+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55e0452d9b30 0x55e0452dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.441+0000 7f7ace538b80 4 client.25775 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:23:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.441+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e04521f000 con 0x55e0452d9b30 2024-06-03T02:23:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.441+0000 7f7ac1200700 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55e0452d9b30 0x55e0452dbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.442+0000 7f7aa2a00700 10 client.25775 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:23:47.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.442+0000 7f7ac1200700 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55e0452d9b30 0x55e0452dbf20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d16d45e1745bfefe server_cookie=e0ace8eced301275 in_seq=0 out_seq=0 2024-06-03T02:23:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.443+0000 7f7aa2a00700 1 -- 172.21.15.79:0/3288843145 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e04521f000 con 0x55e0452d9b30 2024-06-03T02:23:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.443+0000 7f7aa2a00700 10 client.25775 handle_command_reply: tid=0 2024-06-03T02:23:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.443+0000 7f7ace538b80 1 client.25775 shutdown 2024-06-03T02:23:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c5b90 msgr2=0x55e0452c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c5b90 0x55e0452c9540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7aa809aba0 tx=0x7f7aa809a800 comp rx=0 tx=0).stop 2024-06-03T02:23:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 shutdown_connections 2024-06-03T02:23:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55e0452d9b30 0x55e0452dbf20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 0x55e0452cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3288843145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c5b90 0x55e0452c9540 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 >> 172.21.15.79:0/3288843145 conn(0x55e0451ec170 msgr2=0x55e0452b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7aa2a00700 0 client.25775 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:23:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.444+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 shutdown_connections 2024-06-03T02:23:47.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.445+0000 7f7ace538b80 1 -- 172.21.15.79:0/3288843145 wait complete. 2024-06-03T02:23:47.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.445+0000 7f7ace538b80 20 client.25775 trim_cache size 0 max 16384 2024-06-03T02:23:47.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.473+0000 7f7ace538b80 1 Processor -- start 2024-06-03T02:23:47.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.473+0000 7f7ace538b80 1 -- start start 2024-06-03T02:23:47.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.473+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 0x55e0452bfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.473+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.473+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c9ab0 0x55e0452c7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.473+0000 7f7ace538b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00eb20 con 0x55e0452c0240 2024-06-03T02:23:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.473+0000 7f7ace538b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c010660 con 0x55e0452c9ab0 2024-06-03T02:23:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.473+0000 7f7ace538b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00f1f0 con 0x55e0452c64b0 2024-06-03T02:23:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7ac1c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c9ab0 0x55e0452c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7ac1c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c9ab0 0x55e0452c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60056/0 (socket says 172.21.15.79:60056) 2024-06-03T02:23:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7ac1c00700 1 -- 172.21.15.79:0/3539432894 learned_addr learned my addr 172.21.15.79:0/3539432894 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7abbe00700 1 --2- 172.21.15.79:0/3539432894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7ac1200700 1 --2- 172.21.15.79:0/3539432894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 0x55e0452bfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7ac1c00700 1 -- 172.21.15.79:0/3539432894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 msgr2=0x55e0452b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/3539432894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7ac1c00700 1 -- 172.21.15.79:0/3539432894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 msgr2=0x55e0452bfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/3539432894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 0x55e0452bfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.474+0000 7f7ac1c00700 1 -- 172.21.15.79:0/3539432894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0452c75a0 con 0x55e0452c9ab0 2024-06-03T02:23:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.475+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/3539432894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c9ab0 0x55e0452c7060 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4022000 tx=0x7f7ab4009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcc22524692d7ad server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.476+0000 7f7aad600700 1 -- 172.21.15.79:0/3539432894 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4005610 con 0x55e0452c9ab0 2024-06-03T02:23:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.476+0000 7f7aad600700 1 -- 172.21.15.79:0/3539432894 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab40057b0 con 0x55e0452c9ab0 2024-06-03T02:23:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.476+0000 7f7aad600700 1 -- 172.21.15.79:0/3539432894 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4005610 con 0x55e0452c9ab0 2024-06-03T02:23:47.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.478+0000 7f7ace538b80 1 -- 172.21.15.79:0/3539432894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c9ab0 msgr2=0x55e0452c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:47.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.478+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3539432894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c9ab0 0x55e0452c7060 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4022000 tx=0x7f7ab4009bd0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.478+0000 7f7ace538b80 1 -- 172.21.15.79:0/3539432894 shutdown_connections 2024-06-03T02:23:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.478+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3539432894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c9ab0 0x55e0452c7060 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.478+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3539432894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c0240 0x55e0452bfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.478+0000 7f7ace538b80 1 --2- 172.21.15.79:0/3539432894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.478+0000 7f7ace538b80 1 -- 172.21.15.79:0/3539432894 >> 172.21.15.79:0/3539432894 conn(0x55e0451ec170 msgr2=0x55e0452c6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:47.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.482+0000 7f7ace538b80 1 -- 172.21.15.79:0/3539432894 shutdown_connections 2024-06-03T02:23:47.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.483+0000 7f7ace538b80 1 -- 172.21.15.79:0/3539432894 wait complete. 2024-06-03T02:23:47.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.483+0000 7f7ace538b80 1 -- start start 2024-06-03T02:23:47.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.483+0000 7f7ace538b80 1 Processor -- start 2024-06-03T02:23:47.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.486+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.487+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.487+0000 7f7ace538b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c9ab0 0x55e0452e0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.487+0000 7f7ace538b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00ff90 con 0x55e0452c9ab0 2024-06-03T02:23:47.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.487+0000 7f7ace538b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00f8c0 con 0x55e0452c0240 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.487+0000 7f7ace538b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c00cac0 con 0x55e0452c64b0 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.487+0000 7f7ace538b80 10 client.0 _ll_register_callbacks cb 0x55e0451a7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.487+0000 7f7abbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.487+0000 7f7ac1200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.488+0000 7f7ac1200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60070/0 (socket says 172.21.15.79:60070) 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.488+0000 7f7ac1200700 1 -- 172.21.15.79:0/1985325137 learned_addr learned my addr 172.21.15.79:0/1985325137 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.488+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c9ab0 0x55e0452e0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.488+0000 7f7ac1c00700 1 -- 172.21.15.79:0/1985325137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 msgr2=0x55e0452d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.488+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.488+0000 7f7ac1c00700 1 -- 172.21.15.79:0/1985325137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 msgr2=0x55e0452d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.488+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.488+0000 7f7ac1c00700 1 -- 172.21.15.79:0/1985325137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0452e0d80 con 0x55e0452c9ab0 2024-06-03T02:23:47.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.489+0000 7f7ac1200700 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:23:47.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.491+0000 7f7ac1c00700 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c9ab0 0x55e0452e0840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40061e0 tx=0x7f7ab4004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2722230147c84cba server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:47.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.491+0000 7f7aa2a00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:23:47.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.492+0000 7f7aa2a00700 1 -- 172.21.15.79:0/1985325137 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab403f060 con 0x55e0452c9ab0 2024-06-03T02:23:47.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.492+0000 7f7aa2a00700 1 -- 172.21.15.79:0/1985325137 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7ab4024040 con 0x55e0452c9ab0 2024-06-03T02:23:47.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.493+0000 7f7ace538b80 1 -- 172.21.15.79:0/1985325137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab40057b0 con 0x55e0452c9ab0 2024-06-03T02:23:47.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.495+0000 7f7aa2a00700 1 -- 172.21.15.79:0/1985325137 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ab403a460 con 0x55e0452c9ab0 2024-06-03T02:23:47.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.495+0000 7f7ace538b80 10 client.15531 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:47.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.495+0000 7f7ace538b80 1 -- 172.21.15.79:0/1985325137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e045343f40 con 0x55e0452c9ab0 2024-06-03T02:23:47.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.497+0000 7f7aa2a00700 1 -- 172.21.15.79:0/1985325137 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab40057b0 con 0x55e0452c9ab0 2024-06-03T02:23:47.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.498+0000 7f7aa2a00700 1 -- 172.21.15.79:0/1985325137 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7ab401e030 con 0x55e0452c9ab0 2024-06-03T02:23:47.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.498+0000 7f7ace538b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:47.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.498+0000 7f7ace538b80 10 client.15531 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:23:47.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.498+0000 7f7ace538b80 20 client.15531 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:47.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.498+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55e045344250 0x55e045346640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:47.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.498+0000 7f7ace538b80 4 client.15531 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:23:47.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.498+0000 7f7ace538b80 1 -- 172.21.15.79:0/1985325137 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e045219580 con 0x55e045344250 2024-06-03T02:23:47.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.499+0000 7f7ac1200700 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55e045344250 0x55e045346640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:47.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.500+0000 7f7ac1200700 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55e045344250 0x55e045346640 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32270371f60b0bb2 server_cookie=cdd6226c9386dc7d in_seq=0 out_seq=0 2024-06-03T02:23:47.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.500+0000 7f7aa2a00700 10 client.15531 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:23:47.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.500+0000 7f7aa2a00700 1 -- 172.21.15.79:0/1985325137 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e045219580 con 0x55e045344250 2024-06-03T02:23:47.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.500+0000 7f7aa2a00700 10 client.15531 handle_command_reply: tid=0 2024-06-03T02:23:47.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.501+0000 7f7ace538b80 1 client.15531 shutdown 2024-06-03T02:23:47.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.501+0000 7f7ace538b80 1 -- 172.21.15.79:0/1985325137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c9ab0 msgr2=0x55e0452e0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:47.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.501+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c9ab0 0x55e0452e0840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40061e0 tx=0x7f7ab4004d20 comp rx=0 tx=0).stop 2024-06-03T02:23:47.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.502+0000 7f7ace538b80 1 -- 172.21.15.79:0/1985325137 shutdown_connections 2024-06-03T02:23:47.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.502+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55e045344250 0x55e045346640 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.502+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0452c9ab0 0x55e0452e0840 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.502+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0452c0240 0x55e0452d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.502+0000 7f7ace538b80 1 --2- 172.21.15.79:0/1985325137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0452c64b0 0x55e0452d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.502+0000 7f7ace538b80 1 -- 172.21.15.79:0/1985325137 >> 172.21.15.79:0/1985325137 conn(0x55e0451ec170 msgr2=0x55e0452c9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:47.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.502+0000 7f7aa2a00700 0 client.15531 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:23:47.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.502+0000 7f7ace538b80 1 -- 172.21.15.79:0/1985325137 shutdown_connections 2024-06-03T02:23:47.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.503+0000 7f7ace538b80 1 -- 172.21.15.79:0/1985325137 wait complete. 2024-06-03T02:23:47.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.505+0000 7f7ace538b80 20 client.15531 trim_cache size 0 max 16384 2024-06-03T02:23:47.506 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:23:47.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 -- 172.21.15.79:0/1853503548 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7aa40a2c20 msgr2=0x7f7aa40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:47.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7aa40a2c20 0x7f7aa40a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400e350 tx=0x7f7ab400b040 comp rx=0 tx=0).stop 2024-06-03T02:23:47.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 -- 172.21.15.79:0/1853503548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7abc10a0a0 msgr2=0x7f7abc1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:47.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7abc10a0a0 0x7f7abc1103d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7abc06bc00 tx=0x7f7aa800b750 comp rx=0 tx=0).stop 2024-06-03T02:23:47.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 -- 172.21.15.79:0/1853503548 shutdown_connections 2024-06-03T02:23:47.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7aa40a2c20 0x7f7aa40a50e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7abc10e220 0x7f7abc110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7abc1096f0 0x7f7abc10fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 --2- 172.21.15.79:0/1853503548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7abc10a0a0 0x7f7abc1103d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:47.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.506+0000 7f7ac3c00700 1 -- 172.21.15.79:0/1853503548 >> 172.21.15.79:0/1853503548 conn(0x7f7abc06d600 msgr2=0x7f7abc10cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:47.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.507+0000 7f7ac3c00700 1 -- 172.21.15.79:0/1853503548 shutdown_connections 2024-06-03T02:23:47.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:47.507+0000 7f7ac3c00700 1 -- 172.21.15.79:0/1853503548 wait complete. 2024-06-03T02:23:47.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:47 smithi079 ceph-mon[31819]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.3 MiB/s wr, 492 op/s 2024-06-03T02:23:47.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:47 smithi135 ceph-mon[36653]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.3 MiB/s wr, 492 op/s 2024-06-03T02:23:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:47 smithi123 ceph-mon[36640]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.3 MiB/s wr, 492 op/s 2024-06-03T02:23:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 220.0 s, 1630.0 tps, lat 62.882 ms stddev 93.498 2024-06-03T02:23:48.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:23:49.401 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:49.841 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:49 smithi079 ceph-mon[31819]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.1 MiB/s wr, 697 op/s 2024-06-03T02:23:49.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:49 smithi135 ceph-mon[36653]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.1 MiB/s wr, 697 op/s 2024-06-03T02:23:49.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:49 smithi123 ceph-mon[36640]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.1 MiB/s wr, 697 op/s 2024-06-03T02:23:51.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:51 smithi079 ceph-mon[31819]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.8 MiB/s wr, 541 op/s 2024-06-03T02:23:51.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:51 smithi135 ceph-mon[36653]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.8 MiB/s wr, 541 op/s 2024-06-03T02:23:51.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:51 smithi123 ceph-mon[36640]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.8 MiB/s wr, 541 op/s 2024-06-03T02:23:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.110+0000 7f51baa00700 1 -- 172.21.15.79:0/1004187965 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00bc40 con 0x7f51bc108500 2024-06-03T02:23:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.110+0000 7f51c2e00700 1 -- 172.21.15.79:0/1004187965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51bc108500 msgr2=0x7f51bc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.111+0000 7f51c2e00700 1 --2- 172.21.15.79:0/1004187965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51bc108500 0x7f51bc1088e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009990 tx=0x7f51ac009ca0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.111+0000 7f51c2e00700 1 -- 172.21.15.79:0/1004187965 shutdown_connections 2024-06-03T02:23:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.111+0000 7f51c2e00700 1 --2- 172.21.15.79:0/1004187965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51bc06ab30 0x7f51bc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.111+0000 7f51c2e00700 1 --2- 172.21.15.79:0/1004187965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51bc108500 0x7f51bc1088e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.111+0000 7f51c2e00700 1 --2- 172.21.15.79:0/1004187965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc06a170 0x7f51bc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.111+0000 7f51c2e00700 1 -- 172.21.15.79:0/1004187965 >> 172.21.15.79:0/1004187965 conn(0x7f51bc078340 msgr2=0x7f51bc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.111+0000 7f51c2e00700 1 -- 172.21.15.79:0/1004187965 shutdown_connections 2024-06-03T02:23:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.111+0000 7f51c2e00700 1 -- 172.21.15.79:0/1004187965 wait complete. 2024-06-03T02:23:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.114+0000 7f51c2e00700 1 Processor -- start 2024-06-03T02:23:53.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.114+0000 7f51c2e00700 1 -- start start 2024-06-03T02:23:53.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.115+0000 7f51c2e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51bc06a170 0x7f51bc19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.115+0000 7f51c2e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc06ab30 0x7f51bc19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.116+0000 7f51c2e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51bc108500 0x7f51bc1a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.116+0000 7f51bbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51bc06a170 0x7f51bc19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.116+0000 7f51bbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51bc06a170 0x7f51bc19f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:43446/0 (socket says 172.21.15.79:43446) 2024-06-03T02:23:53.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.116+0000 7f51bbe00700 1 -- 172.21.15.79:0/2705239357 learned_addr learned my addr 172.21.15.79:0/2705239357 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:53.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.116+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc102a70 con 0x7f51bc108500 2024-06-03T02:23:53.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.116+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1028f0 con 0x7f51bc06ab30 2024-06-03T02:23:53.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.116+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc102bf0 con 0x7f51bc06a170 2024-06-03T02:23:53.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.116+0000 7f51bb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc06ab30 0x7f51bc19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.117+0000 7f51c0e00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51bc108500 0x7f51bc1a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.117+0000 7f51bb400700 1 -- 172.21.15.79:0/2705239357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51bc06a170 msgr2=0x7f51bc19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:53.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.117+0000 7f51bb400700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51bc06a170 0x7f51bc19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.117+0000 7f51bb400700 1 -- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51bc108500 msgr2=0x7f51bc1a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:53.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.118+0000 7f51bb400700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51bc108500 0x7f51bc1a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.118+0000 7f51bb400700 1 -- 172.21.15.79:0/2705239357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc1a41c0 con 0x7f51bc06ab30 2024-06-03T02:23:53.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.118+0000 7f51c0e00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51bc108500 0x7f51bc1a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:23:53.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.118+0000 7f51bb400700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc06ab30 0x7f51bc19f6f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f51a800e9f0 tx=0x7f51a800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:53.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.119+0000 7f51b8c00700 1 -- 172.21.15.79:0/2705239357 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a800c9c0 con 0x7f51bc06ab30 2024-06-03T02:23:53.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.119+0000 7f51b8c00700 1 -- 172.21.15.79:0/2705239357 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a800cb60 con 0x7f51bc06ab30 2024-06-03T02:23:53.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.119+0000 7f51b8c00700 1 -- 172.21.15.79:0/2705239357 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8018680 con 0x7f51bc06ab30 2024-06-03T02:23:53.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.119+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51bc1a44b0 con 0x7f51bc06ab30 2024-06-03T02:23:53.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.119+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51bc1abd30 con 0x7f51bc06ab30 2024-06-03T02:23:53.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.128+0000 7f51b8c00700 1 -- 172.21.15.79:0/2705239357 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f51a8010430 con 0x7f51bc06ab30 2024-06-03T02:23:53.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.129+0000 7f51b8c00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a00a2c90 0x7f51a00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.130+0000 7f51cd6e2b80 1 Processor -- start 2024-06-03T02:23:53.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.130+0000 7f51b8c00700 1 -- 172.21.15.79:0/2705239357 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51a80cf470 con 0x7f51bc06ab30 2024-06-03T02:23:53.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.130+0000 7f51cd6e2b80 1 -- start start 2024-06-03T02:23:53.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.130+0000 7f51bbe00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a00a2c90 0x7f51a00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.131+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47ddc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.131+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47ddca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.131+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47de96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.131+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555d47ceba10 con 0x555d47de7300 2024-06-03T02:23:53.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.131+0000 7f51bb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47ddca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.131+0000 7f51bb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47ddca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:43452/0 (socket says 172.21.15.79:43452) 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.131+0000 7f51bb400700 1 -- 172.21.15.79:0/1670506103 learned_addr learned my addr 172.21.15.79:0/1670506103 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.131+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1670506103 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555d47d2eb00 con 0x555d47d58e40 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.131+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1670506103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d47ce98f0 con 0x555d47ddc5b0 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51bbe00700 1 --2- 172.21.15.79:0/1670506103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47ddc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51c0e00700 1 --2- 172.21.15.79:0/1670506103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47de96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51bb400700 1 -- 172.21.15.79:0/1670506103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 msgr2=0x555d47ddc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51bb400700 1 --2- 172.21.15.79:0/1670506103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47ddc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51bb400700 1 -- 172.21.15.79:0/1670506103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 msgr2=0x555d47de96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51bb400700 1 --2- 172.21.15.79:0/1670506103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47de96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51bb400700 1 -- 172.21.15.79:0/1670506103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d47cd6f90 con 0x555d47ddc5b0 2024-06-03T02:23:53.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51bb400700 1 --2- 172.21.15.79:0/1670506103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47ddca30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f51a80a4060 tx=0x7f51a800cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19f72d67fdafb459 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:53.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51b1600700 1 -- 172.21.15.79:0/1670506103 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a809fcd0 con 0x555d47ddc5b0 2024-06-03T02:23:53.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51b1600700 1 -- 172.21.15.79:0/1670506103 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a80a4590 con 0x555d47ddc5b0 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.132+0000 7f51b1600700 1 -- 172.21.15.79:0/1670506103 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8010720 con 0x555d47ddc5b0 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.133+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1670506103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 msgr2=0x555d47ddca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.133+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/1670506103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47ddca30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f51a80a4060 tx=0x7f51a800cd10 comp rx=0 tx=0).stop 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.133+0000 7f51bbe00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a00a2c90 0x7f51a00a5150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009750 tx=0x7f51ac00b7c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.134+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1670506103 shutdown_connections 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.134+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/1670506103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47de96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.134+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/1670506103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47ddc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.134+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/1670506103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47ddca30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.134+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1670506103 >> 172.21.15.79:0/1670506103 conn(0x555d47d09170 msgr2=0x555d47dd84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:53.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.134+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1670506103 shutdown_connections 2024-06-03T02:23:53.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.134+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1670506103 wait complete. 2024-06-03T02:23:53.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.135+0000 7f51cd6e2b80 1 -- start start 2024-06-03T02:23:53.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.135+0000 7f51cd6e2b80 1 Processor -- start 2024-06-03T02:23:53.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.136+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47d58e40 0x555d47de6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.136+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47ddc5b0 0x555d47de3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.136+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47de4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.136+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555d47ccaf60 con 0x555d47de7300 2024-06-03T02:23:53.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.136+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555d47d2aac0 con 0x555d47ddc5b0 2024-06-03T02:23:53.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.136+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d47de9f60 con 0x555d47d58e40 2024-06-03T02:23:53.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.136+0000 7f51cd6e2b80 10 client.0 _ll_register_callbacks cb 0x555d47cc8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:53.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.136+0000 7f51bb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47ddc5b0 0x555d47de3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47ddc5b0 0x555d47de3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60114/0 (socket says 172.21.15.79:60114) 2024-06-03T02:23:53.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bb400700 1 -- 172.21.15.79:0/3596763927 learned_addr learned my addr 172.21.15.79:0/3596763927 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:53.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bbe00700 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47d58e40 0x555d47de6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51c0e00700 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47de4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bb400700 1 -- 172.21.15.79:0/3596763927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47d58e40 msgr2=0x555d47de6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:53.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bb400700 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47d58e40 0x555d47de6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bb400700 1 -- 172.21.15.79:0/3596763927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 msgr2=0x555d47de4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:53.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bb400700 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47de4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bb400700 1 -- 172.21.15.79:0/3596763927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d47c7d8d0 con 0x555d47ddc5b0 2024-06-03T02:23:53.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bbe00700 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47d58e40 0x555d47de6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:23:53.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51bb400700 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47ddc5b0 0x555d47de3e20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51a809bf80 tx=0x7f51a8097cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e8ec860e5f49bd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:53.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.137+0000 7f51a6000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:23:53.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.138+0000 7f51a6000700 1 -- 172.21.15.79:0/3596763927 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8010720 con 0x555d47ddc5b0 2024-06-03T02:23:53.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.138+0000 7f51a6000700 1 -- 172.21.15.79:0/3596763927 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51a80108c0 con 0x555d47ddc5b0 2024-06-03T02:23:53.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.138+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/3596763927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51a80a4590 con 0x555d47ddc5b0 2024-06-03T02:23:53.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.138+0000 7f51a6000700 1 -- 172.21.15.79:0/3596763927 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51a80ba150 con 0x555d47ddc5b0 2024-06-03T02:23:53.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.138+0000 7f51cd6e2b80 10 client.25793 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:53.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.138+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/3596763927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555d47df4780 con 0x555d47ddc5b0 2024-06-03T02:23:53.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.139+0000 7f51a6000700 1 -- 172.21.15.79:0/3596763927 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a80a4590 con 0x555d47ddc5b0 2024-06-03T02:23:53.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.142+0000 7f51a6000700 1 -- 172.21.15.79:0/3596763927 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f51a809ad70 con 0x555d47ddc5b0 2024-06-03T02:23:53.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.143+0000 7f51cd6e2b80 10 client.25793 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:53.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.143+0000 7f51cd6e2b80 10 client.25793 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:23:53.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.143+0000 7f51cd6e2b80 20 client.25793 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:53.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.143+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555d47df6c50 0x555d47df9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.143+0000 7f51cd6e2b80 4 client.25793 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:23:53.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.143+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/3596763927 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d47d3c000 con 0x555d47df6c50 2024-06-03T02:23:53.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.143+0000 7f51bbe00700 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555d47df6c50 0x555d47df9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.144+0000 7f51a6000700 10 client.25793 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:23:53.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.144+0000 7f51bbe00700 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555d47df6c50 0x555d47df9040 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=29d8c1b903ca58b3 server_cookie=229a45d37253cec8 in_seq=0 out_seq=0 2024-06-03T02:23:53.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51a6000700 1 -- 172.21.15.79:0/3596763927 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555d47d3c000 con 0x555d47df6c50 2024-06-03T02:23:53.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51a6000700 10 client.25793 handle_command_reply: tid=0 2024-06-03T02:23:53.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51cd6e2b80 1 client.25793 shutdown 2024-06-03T02:23:53.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/3596763927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47ddc5b0 msgr2=0x555d47de3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:53.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47ddc5b0 0x555d47de3e20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51a809bf80 tx=0x7f51a8097cf0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/3596763927 shutdown_connections 2024-06-03T02:23:53.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555d47df6c50 0x555d47df9040 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47de4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47d58e40 0x555d47de6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/3596763927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47ddc5b0 0x555d47de3e20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/3596763927 >> 172.21.15.79:0/3596763927 conn(0x555d47d09170 msgr2=0x555d47dd7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:53.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.145+0000 7f51a6000700 0 client.25793 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:23:53.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.146+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/3596763927 shutdown_connections 2024-06-03T02:23:53.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.146+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/3596763927 wait complete. 2024-06-03T02:23:53.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.146+0000 7f51cd6e2b80 20 client.25793 trim_cache size 0 max 16384 2024-06-03T02:23:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.172+0000 7f51cd6e2b80 1 Processor -- start 2024-06-03T02:23:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.172+0000 7f51cd6e2b80 1 -- start start 2024-06-03T02:23:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.172+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47d58e40 0x555d47de6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.172+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47de86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47de7300 0x555d47de8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1b4b30 con 0x555d47d58e40 2024-06-03T02:23:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1b6670 con 0x555d47de7300 2024-06-03T02:23:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1b5200 con 0x555d47ddc5b0 2024-06-03T02:23:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51c0e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47de7300 0x555d47de8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51c0e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47de7300 0x555d47de8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60130/0 (socket says 172.21.15.79:60130) 2024-06-03T02:23:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51c0e00700 1 -- 172.21.15.79:0/1355214045 learned_addr learned my addr 172.21.15.79:0/1355214045 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51bb400700 1 --2- 172.21.15.79:0/1355214045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47de86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51bb400700 1 -- 172.21.15.79:0/1355214045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47de7300 msgr2=0x555d47de8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51bb400700 1 --2- 172.21.15.79:0/1355214045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47de7300 0x555d47de8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51bb400700 1 -- 172.21.15.79:0/1355214045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47d58e40 msgr2=0x555d47de6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.173+0000 7f51bbe00700 1 --2- 172.21.15.79:0/1355214045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47d58e40 0x555d47de6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.174+0000 7f51bb400700 1 --2- 172.21.15.79:0/1355214045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47d58e40 0x555d47de6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.174+0000 7f51bb400700 1 -- 172.21.15.79:0/1355214045 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d47de3bf0 con 0x555d47ddc5b0 2024-06-03T02:23:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.174+0000 7f51bb400700 1 --2- 172.21.15.79:0/1355214045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47de86f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f51a809b760 tx=0x7f51a800cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb0a4a4e350327c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:53.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.176+0000 7f51b0c00700 1 -- 172.21.15.79:0/1355214045 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80ba2f0 con 0x555d47ddc5b0 2024-06-03T02:23:53.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.176+0000 7f51b0c00700 1 -- 172.21.15.79:0/1355214045 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a8099c20 con 0x555d47ddc5b0 2024-06-03T02:23:53.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.176+0000 7f51b0c00700 1 -- 172.21.15.79:0/1355214045 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8086070 con 0x555d47ddc5b0 2024-06-03T02:23:53.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.177+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1355214045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 msgr2=0x555d47de86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:53.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.177+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/1355214045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47de86f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f51a809b760 tx=0x7f51a800cd10 comp rx=0 tx=0).stop 2024-06-03T02:23:53.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.178+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1355214045 shutdown_connections 2024-06-03T02:23:53.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.178+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/1355214045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47de7300 0x555d47de8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.178+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/1355214045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47d58e40 0x555d47de6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.178+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/1355214045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47ddc5b0 0x555d47de86f0 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f51a809b760 tx=0x7f51a800cd10 comp rx=0 tx=0).stop 2024-06-03T02:23:53.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.178+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1355214045 >> 172.21.15.79:0/1355214045 conn(0x555d47d09170 msgr2=0x555d47de3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:53.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.178+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1355214045 shutdown_connections 2024-06-03T02:23:53.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.179+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/1355214045 wait complete. 2024-06-03T02:23:53.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.179+0000 7f51cd6e2b80 1 -- start start 2024-06-03T02:23:53.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.180+0000 7f51cd6e2b80 1 Processor -- start 2024-06-03T02:23:53.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.180+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47dfc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.180+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47df6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.180+0000 7f51cd6e2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47df72f0 0x555d47e5f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.180+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1b5fa0 con 0x555d47de7300 2024-06-03T02:23:53.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.180+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1b58d0 con 0x555d47d58e40 2024-06-03T02:23:53.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.180+0000 7f51cd6e2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc1b2ad0 con 0x555d47df72f0 2024-06-03T02:23:53.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.180+0000 7f51cd6e2b80 10 client.0 _ll_register_callbacks cb 0x555d47cc8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47dfc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47dfc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60134/0 (socket says 172.21.15.79:60134) 2024-06-03T02:23:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bbe00700 1 -- 172.21.15.79:0/2872114743 learned_addr learned my addr 172.21.15.79:0/2872114743 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51c0e00700 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47df72f0 0x555d47e5f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bb400700 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47df6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bbe00700 1 -- 172.21.15.79:0/2872114743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47df72f0 msgr2=0x555d47e5f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bbe00700 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47df72f0 0x555d47e5f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bbe00700 1 -- 172.21.15.79:0/2872114743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 msgr2=0x555d47df6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bbe00700 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47df6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bbe00700 1 -- 172.21.15.79:0/2872114743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d47df7a70 con 0x555d47d58e40 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51c0e00700 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47df72f0 0x555d47e5f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.181+0000 7f51bb400700 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47df6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.182+0000 7f51bbe00700 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47dfc530 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51ac005b40 tx=0x7f51ac009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d5655f1a8fa0e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.182+0000 7f51a6000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.182+0000 7f51a6000700 1 -- 172.21.15.79:0/2872114743 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac021040 con 0x555d47d58e40 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.182+0000 7f51a6000700 1 -- 172.21.15.79:0/2872114743 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51ac00bde0 con 0x555d47d58e40 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.182+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/2872114743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51a8099c20 con 0x555d47d58e40 2024-06-03T02:23:53.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.183+0000 7f51a6000700 1 -- 172.21.15.79:0/2872114743 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51ac00bc40 con 0x555d47d58e40 2024-06-03T02:23:53.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.183+0000 7f51cd6e2b80 10 client.25799 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:53.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.183+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/2872114743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555d47e609f0 con 0x555d47d58e40 2024-06-03T02:23:53.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.183+0000 7f51a6000700 1 -- 172.21.15.79:0/2872114743 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a8099c20 con 0x555d47d58e40 2024-06-03T02:23:53.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.184+0000 7f51a6000700 1 -- 172.21.15.79:0/2872114743 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f51ac00f620 con 0x555d47d58e40 2024-06-03T02:23:53.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.184+0000 7f51cd6e2b80 10 client.25799 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:53.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.184+0000 7f51cd6e2b80 10 client.25799 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:23:53.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.184+0000 7f51cd6e2b80 20 client.25799 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:53.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.184+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555d47e61080 0x555d47e63470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:53.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.185+0000 7f51cd6e2b80 4 client.25799 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:23:53.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.185+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/2872114743 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d47d36580 con 0x555d47e61080 2024-06-03T02:23:53.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.185+0000 7f51bb400700 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555d47e61080 0x555d47e63470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:53.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.186+0000 7f51bb400700 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555d47e61080 0x555d47e63470 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=48f2989fba3b0e07 server_cookie=c060fc56bd2c8a53 in_seq=0 out_seq=0 2024-06-03T02:23:53.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.186+0000 7f51a6000700 10 client.25799 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:23:53.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51a6000700 1 -- 172.21.15.79:0/2872114743 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d47d36580 con 0x555d47e61080 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51a6000700 10 client.25799 handle_command_reply: tid=0 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 client.25799 shutdown 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/2872114743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 msgr2=0x555d47dfc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47dfc530 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51ac005b40 tx=0x7f51ac009ca0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/2872114743 shutdown_connections 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d47df72f0 0x555d47e5f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d47d58e40 0x555d47dfc530 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555d47e61080 0x555d47e63470 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 --2- 172.21.15.79:0/2872114743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d47de7300 0x555d47df6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/2872114743 >> 172.21.15.79:0/2872114743 conn(0x555d47d09170 msgr2=0x555d47de2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:53.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51a6000700 0 client.25799 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:23:53.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/2872114743 shutdown_connections 2024-06-03T02:23:53.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.187+0000 7f51cd6e2b80 1 -- 172.21.15.79:0/2872114743 wait complete. 2024-06-03T02:23:53.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.188+0000 7f51cd6e2b80 20 client.25799 trim_cache size 0 max 16384 2024-06-03T02:23:53.189 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a00a2c90 msgr2=0x7f51a00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a00a2c90 0x7f51a00a5150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009750 tx=0x7f51ac00b7c0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc06ab30 msgr2=0x7f51bc19f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc06ab30 0x7f51bc19f6f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f51a800e9f0 tx=0x7f51a800ed00 comp rx=0 tx=0).stop 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 shutdown_connections 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51a00a2c90 0x7f51a00a5150 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51bc108500 0x7f51bc1a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51bc06a170 0x7f51bc19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 --2- 172.21.15.79:0/2705239357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51bc06ab30 0x7f51bc19f6f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:53.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.189+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 >> 172.21.15.79:0/2705239357 conn(0x7f51bc078340 msgr2=0x7f51bc1092d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:53.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.190+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 shutdown_connections 2024-06-03T02:23:53.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:53.190+0000 7f51c2e00700 1 -- 172.21.15.79:0/2705239357 wait complete. 2024-06-03T02:23:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 225.0 s, 1135.4 tps, lat 84.691 ms stddev 104.621 2024-06-03T02:23:53.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:53 smithi079 ceph-mon[31819]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 MiB/s wr, 681 op/s 2024-06-03T02:23:53.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:53 smithi135 ceph-mon[36653]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 MiB/s wr, 681 op/s 2024-06-03T02:23:53.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:53 smithi123 ceph-mon[36640]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 MiB/s wr, 681 op/s 2024-06-03T02:23:54.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:23:54.754 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:23:55.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:55 smithi079 ceph-mon[31819]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 MiB/s wr, 605 op/s 2024-06-03T02:23:55.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:55 smithi135 ceph-mon[36653]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 MiB/s wr, 605 op/s 2024-06-03T02:23:55.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:55 smithi123 ceph-mon[36640]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 MiB/s wr, 605 op/s 2024-06-03T02:23:57.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:57 smithi079 ceph-mon[31819]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.1 MiB/s wr, 538 op/s 2024-06-03T02:23:57.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:57 smithi135 ceph-mon[36653]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.1 MiB/s wr, 538 op/s 2024-06-03T02:23:57.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:57 smithi123 ceph-mon[36640]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.1 MiB/s wr, 538 op/s 2024-06-03T02:23:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 230.0 s, 1022.4 tps, lat 100.819 ms stddev 145.902 2024-06-03T02:23:58.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.545+0000 7f4fea400700 1 -- 172.21.15.79:0/618078702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fe41059c0 msgr2=0x7f4fe410d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:58.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.545+0000 7f4fea400700 1 --2- 172.21.15.79:0/618078702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fe41059c0 0x7f4fe410d540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00c340 tx=0x7f4fdc00c650 comp rx=0 tx=0).stop 2024-06-03T02:23:58.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.545+0000 7f4fea400700 1 -- 172.21.15.79:0/618078702 shutdown_connections 2024-06-03T02:23:58.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.545+0000 7f4fea400700 1 --2- 172.21.15.79:0/618078702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fe41059c0 0x7f4fe410d540 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.545+0000 7f4fea400700 1 --2- 172.21.15.79:0/618078702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fe4100a90 0x7f4fe4100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.545+0000 7f4fea400700 1 --2- 172.21.15.79:0/618078702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fe4101440 0x7f4fe4105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.545+0000 7f4fea400700 1 -- 172.21.15.79:0/618078702 >> 172.21.15.79:0/618078702 conn(0x7f4fe4076130 msgr2=0x7f4fe4076540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.545+0000 7f4fea400700 1 -- 172.21.15.79:0/618078702 shutdown_connections 2024-06-03T02:23:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.545+0000 7f4fea400700 1 -- 172.21.15.79:0/618078702 wait complete. 2024-06-03T02:23:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.547+0000 7f4fea400700 1 Processor -- start 2024-06-03T02:23:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.547+0000 7f4fea400700 1 -- start start 2024-06-03T02:23:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fea400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fe4100a90 0x7f4fe4073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fea400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fe4101440 0x7f4fe410f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fea400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fe41059c0 0x7f4fe410fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fea400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4112740 con 0x7f4fe4101440 2024-06-03T02:23:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fea400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41125c0 con 0x7f4fe4100a90 2024-06-03T02:23:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fea400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41128c0 con 0x7f4fe41059c0 2024-06-03T02:23:58.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fe9000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fe4100a90 0x7f4fe4073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fe9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fe41059c0 0x7f4fe410fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fe9000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fe4100a90 0x7f4fe4073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:46084/0 (socket says 172.21.15.79:46084) 2024-06-03T02:23:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fe9000700 1 -- 172.21.15.79:0/4222275049 learned_addr learned my addr 172.21.15.79:0/4222275049 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.548+0000 7f4fe9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fe41059c0 0x7f4fe410fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58050/0 (socket says 172.21.15.79:58050) 2024-06-03T02:23:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.549+0000 7f4fe3e00700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fe4101440 0x7f4fe410f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.549+0000 7f4fe9a00700 1 -- 172.21.15.79:0/4222275049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fe4100a90 msgr2=0x7f4fe4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.549+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fe4100a90 0x7f4fe4073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.549+0000 7f4fe9a00700 1 -- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fe4101440 msgr2=0x7f4fe410f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.550+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fe4101440 0x7f4fe410f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.550+0000 7f4fe9a00700 1 -- 172.21.15.79:0/4222275049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe4110540 con 0x7f4fe41059c0 2024-06-03T02:23:58.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.551+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fe41059c0 0x7f4fe410fea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00c9e0 tx=0x7f4fdc003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.552+0000 7f4fe1600700 1 -- 172.21.15.79:0/4222275049 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc021020 con 0x7f4fe41059c0 2024-06-03T02:23:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.552+0000 7f4fe1600700 1 -- 172.21.15.79:0/4222275049 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc0136b0 con 0x7f4fe41059c0 2024-06-03T02:23:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.552+0000 7f4fe1600700 1 -- 172.21.15.79:0/4222275049 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0055e0 con 0x7f4fe41059c0 2024-06-03T02:23:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.553+0000 7f4fea400700 1 -- 172.21.15.79:0/4222275049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe41afc00 con 0x7f4fe41059c0 2024-06-03T02:23:58.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.553+0000 7f4fea400700 1 -- 172.21.15.79:0/4222275049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe41affa0 con 0x7f4fe41059c0 2024-06-03T02:23:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.558+0000 7f4ff632bb80 1 Processor -- start 2024-06-03T02:23:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.558+0000 7f4fe1600700 1 -- 172.21.15.79:0/4222275049 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4fdc024020 con 0x7f4fe41059c0 2024-06-03T02:23:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4ff632bb80 1 -- start start 2024-06-03T02:23:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 0x55b890452420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890452de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b89045b570 0x55b89045d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b89035fa10 con 0x55b8903cce40 2024-06-03T02:23:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b8903a2b00 con 0x55b89045b570 2024-06-03T02:23:58.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b89035d8f0 con 0x55b890452960 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4fe9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890452de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4fe9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890452de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58064/0 (socket says 172.21.15.79:58064) 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.559+0000 7f4fe9000700 1 -- 172.21.15.79:0/611108774 learned_addr learned my addr 172.21.15.79:0/611108774 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.560+0000 7f4fe3e00700 1 --2- 172.21.15.79:0/611108774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 0x55b890452420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.560+0000 7f4fe9000700 1 -- 172.21.15.79:0/611108774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b89045b570 msgr2=0x55b89045d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.560+0000 7f4fe9000700 1 --2- 172.21.15.79:0/611108774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b89045b570 0x55b89045d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.560+0000 7f4fe9000700 1 -- 172.21.15.79:0/611108774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 msgr2=0x55b890452420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.560+0000 7f4fe9000700 1 --2- 172.21.15.79:0/611108774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 0x55b890452420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.560+0000 7f4fe9000700 1 -- 172.21.15.79:0/611108774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b89034af90 con 0x55b890452960 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.560+0000 7f4fe3e00700 1 --2- 172.21.15.79:0/611108774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 0x55b890452420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:23:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.560+0000 7f4fe9000700 1 --2- 172.21.15.79:0/611108774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890452de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009e50 tx=0x7f4fd8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b70e0c7f9a77a450 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:58.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.561+0000 7f4fd6000700 1 -- 172.21.15.79:0/611108774 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8004a30 con 0x55b890452960 2024-06-03T02:23:58.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.561+0000 7f4fd6000700 1 -- 172.21.15.79:0/611108774 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd8005500 con 0x55b890452960 2024-06-03T02:23:58.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.561+0000 7f4fd6000700 1 -- 172.21.15.79:0/611108774 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8019a70 con 0x55b890452960 2024-06-03T02:23:58.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.561+0000 7f4fe1600700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4fcc0a2e20 0x7f4fcc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.561+0000 7f4fe1600700 1 -- 172.21.15.79:0/4222275049 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4fdc010030 con 0x7f4fe41059c0 2024-06-03T02:23:58.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.561+0000 7f4fe3e00700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4fcc0a2e20 0x7f4fcc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.562+0000 7f4ff632bb80 1 -- 172.21.15.79:0/611108774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 msgr2=0x55b890452de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:58.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.562+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/611108774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890452de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009e50 tx=0x7f4fd8011720 comp rx=0 tx=0).stop 2024-06-03T02:23:58.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.562+0000 7f4ff632bb80 1 -- 172.21.15.79:0/611108774 shutdown_connections 2024-06-03T02:23:58.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.562+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/611108774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b89045b570 0x55b89045d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.562+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/611108774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 0x55b890452420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.563+0000 7f4fe3e00700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4fcc0a2e20 0x7f4fcc0a52e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000b6c0 tx=0x7f4fd000b930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:58.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.563+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/611108774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890452de0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.563+0000 7f4ff632bb80 1 -- 172.21.15.79:0/611108774 >> 172.21.15.79:0/611108774 conn(0x55b89037d170 msgr2=0x55b89044c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:58.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.563+0000 7f4ff632bb80 1 -- 172.21.15.79:0/611108774 shutdown_connections 2024-06-03T02:23:58.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.563+0000 7f4ff632bb80 1 -- 172.21.15.79:0/611108774 wait complete. 2024-06-03T02:23:58.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.564+0000 7f4ff632bb80 1 -- start start 2024-06-03T02:23:58.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.565+0000 7f4ff632bb80 1 Processor -- start 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b89045ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890458000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b89045b570 0x55b890458540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b89033ef60 con 0x55b89045b570 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b89039eac0 con 0x55b8903cce40 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b89045e120 con 0x55b890452960 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4ff632bb80 10 client.0 _ll_register_callbacks cb 0x55b89033c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4fe9000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b89045ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4fe9000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b89045ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:46102/0 (socket says 172.21.15.79:46102) 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.566+0000 7f4fe9000700 1 -- 172.21.15.79:0/3616396731 learned_addr learned my addr 172.21.15.79:0/3616396731 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.567+0000 7f4fe3e00700 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890458000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.567+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b89045b570 0x55b890458540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.567+0000 7f4fe9000700 1 -- 172.21.15.79:0/3616396731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 msgr2=0x55b890458000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:58.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.567+0000 7f4fe9000700 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890458000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.567+0000 7f4fe9000700 1 -- 172.21.15.79:0/3616396731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b89045b570 msgr2=0x55b890458540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:58.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.567+0000 7f4fe9000700 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b89045b570 0x55b890458540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.567+0000 7f4fe9000700 1 -- 172.21.15.79:0/3616396731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8902f18d0 con 0x55b8903cce40 2024-06-03T02:23:58.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.567+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b89045b570 0x55b890458540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:23:58.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.567+0000 7f4fe9000700 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b89045ad00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8004fa0 tx=0x7f4fd8004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b94787c9292c27fd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:58.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.568+0000 7f4fcaa00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:23:58.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.568+0000 7f4fcaa00700 1 -- 172.21.15.79:0/3616396731 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8027070 con 0x55b8903cce40 2024-06-03T02:23:58.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.568+0000 7f4fcaa00700 1 -- 172.21.15.79:0/3616396731 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4fd8004b60 con 0x55b8903cce40 2024-06-03T02:23:58.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.568+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3616396731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd8005500 con 0x55b8903cce40 2024-06-03T02:23:58.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.571+0000 7f4fcaa00700 1 -- 172.21.15.79:0/3616396731 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4fd8033050 con 0x55b8903cce40 2024-06-03T02:23:58.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.571+0000 7f4ff632bb80 10 client.25808 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:58.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.571+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3616396731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b89045f840 con 0x55b8903cce40 2024-06-03T02:23:58.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.571+0000 7f4fcaa00700 1 -- 172.21.15.79:0/3616396731 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd8005500 con 0x55b8903cce40 2024-06-03T02:23:58.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.571+0000 7f4fcaa00700 1 -- 172.21.15.79:0/3616396731 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4fd8003680 con 0x55b8903cce40 2024-06-03T02:23:58.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.572+0000 7f4ff632bb80 10 client.25808 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:58.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.572+0000 7f4ff632bb80 10 client.25808 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:23:58.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.572+0000 7f4ff632bb80 20 client.25808 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:58.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.572+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55b890461dd0 0x55b890464270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.572+0000 7f4ff632bb80 4 client.25808 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:23:58.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.572+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3616396731 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8903b0000 con 0x55b890461dd0 2024-06-03T02:23:58.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.572+0000 7f4fe3e00700 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55b890461dd0 0x55b890464270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.574+0000 7f4fe3e00700 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55b890461dd0 0x55b890464270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71990f96a93a41c5 server_cookie=4b153c1e91e84fee in_seq=0 out_seq=0 2024-06-03T02:23:58.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.574+0000 7f4fcaa00700 10 client.25808 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:23:58.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.575+0000 7f4fcaa00700 1 -- 172.21.15.79:0/3616396731 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b8903b0000 con 0x55b890461dd0 2024-06-03T02:23:58.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.575+0000 7f4fcaa00700 10 client.25808 handle_command_reply: tid=0 2024-06-03T02:23:58.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.575+0000 7f4ff632bb80 1 client.25808 shutdown 2024-06-03T02:23:58.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3616396731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 msgr2=0x55b89045ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:58.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b89045ad00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8004fa0 tx=0x7f4fd8004300 comp rx=0 tx=0).stop 2024-06-03T02:23:58.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3616396731 shutdown_connections 2024-06-03T02:23:58.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b89045b570 0x55b890458540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55b890461dd0 0x55b890464270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b89045ad00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3616396731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b890452960 0x55b890458000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3616396731 >> 172.21.15.79:0/3616396731 conn(0x55b89037d170 msgr2=0x55b89044c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:58.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4fcaa00700 0 client.25808 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:23:58.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3616396731 shutdown_connections 2024-06-03T02:23:58.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3616396731 wait complete. 2024-06-03T02:23:58.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.576+0000 7f4ff632bb80 20 client.25808 trim_cache size 0 max 16384 2024-06-03T02:23:58.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.608+0000 7f4ff632bb80 1 Processor -- start 2024-06-03T02:23:58.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.609+0000 7f4ff632bb80 1 -- start start 2024-06-03T02:23:58.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.610+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 0x55b89045a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.610+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b890452960 0x55b890485cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.610+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b890486210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.610+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b4720 con 0x55b8903cce40 2024-06-03T02:23:58.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.610+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b6260 con 0x55b890452960 2024-06-03T02:23:58.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.610+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b4df0 con 0x55b89045b570 2024-06-03T02:23:58.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b890486210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b890486210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58080/0 (socket says 172.21.15.79:58080) 2024-06-03T02:23:58.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe9a00700 1 -- 172.21.15.79:0/3804753578 learned_addr learned my addr 172.21.15.79:0/3804753578 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:58.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe9000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 0x55b89045a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe3e00700 1 --2- 172.21.15.79:0/3804753578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b890452960 0x55b890485cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe9a00700 1 -- 172.21.15.79:0/3804753578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b890452960 msgr2=0x55b890485cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:58.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/3804753578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b890452960 0x55b890485cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe9a00700 1 -- 172.21.15.79:0/3804753578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 msgr2=0x55b89045a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:58.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/3804753578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 0x55b89045a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.611+0000 7f4fe9a00700 1 -- 172.21.15.79:0/3804753578 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b890451570 con 0x55b89045b570 2024-06-03T02:23:58.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.612+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/3804753578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b890486210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0a4df0 tx=0x7f4fdc00c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c9d017bdd63b24 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:58.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.613+0000 7f4fd5600700 1 -- 172.21.15.79:0/3804753578 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc013860 con 0x55b89045b570 2024-06-03T02:23:58.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.613+0000 7f4fd5600700 1 -- 172.21.15.79:0/3804753578 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc01b3b0 con 0x55b89045b570 2024-06-03T02:23:58.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.613+0000 7f4fd5600700 1 -- 172.21.15.79:0/3804753578 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc098db0 con 0x55b89045b570 2024-06-03T02:23:58.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.614+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3804753578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 msgr2=0x55b890486210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:58.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.615+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3804753578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b890486210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0a4df0 tx=0x7f4fdc00c040 comp rx=0 tx=0).stop 2024-06-03T02:23:58.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.615+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3804753578 shutdown_connections 2024-06-03T02:23:58.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.615+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3804753578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b890486210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.615+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3804753578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8903cce40 0x55b89045a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.615+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/3804753578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b890452960 0x55b890485cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.615+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3804753578 >> 172.21.15.79:0/3804753578 conn(0x55b89037d170 msgr2=0x55b8904535f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:58.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.616+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3804753578 shutdown_connections 2024-06-03T02:23:58.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.616+0000 7f4ff632bb80 1 -- 172.21.15.79:0/3804753578 wait complete. 2024-06-03T02:23:58.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.616+0000 7f4ff632bb80 1 -- start start 2024-06-03T02:23:58.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.617+0000 7f4ff632bb80 1 Processor -- start 2024-06-03T02:23:58.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.617+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b890464d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.617+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b890452960 0x55b890465290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.617+0000 7f4ff632bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b8904e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.617+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b5b90 con 0x55b890452960 2024-06-03T02:23:58.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.617+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b54c0 con 0x55b8903cce40 2024-06-03T02:23:58.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.617+0000 7f4ff632bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b26c0 con 0x55b89045b570 2024-06-03T02:23:58.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.618+0000 7f4ff632bb80 10 client.0 _ll_register_callbacks cb 0x7f4fd000bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:23:58.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.618+0000 7f4fe9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b8904e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.618+0000 7f4fe9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b8904e4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58090/0 (socket says 172.21.15.79:58090) 2024-06-03T02:23:58.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.618+0000 7f4fe9a00700 1 -- 172.21.15.79:0/188356509 learned_addr learned my addr 172.21.15.79:0/188356509 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:23:58.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.619+0000 7f4fe9000700 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b890464d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.619+0000 7f4fe9a00700 1 -- 172.21.15.79:0/188356509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 msgr2=0x55b890464d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:58.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.619+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b890464d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.619+0000 7f4fe9a00700 1 -- 172.21.15.79:0/188356509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b890452960 msgr2=0x55b890465290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:23:58.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.619+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b890452960 0x55b890465290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.619+0000 7f4fe9a00700 1 -- 172.21.15.79:0/188356509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8904e51c0 con 0x55b89045b570 2024-06-03T02:23:58.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.620+0000 7f4fe9a00700 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b8904e4c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc09c6d0 tx=0x7f4fdc09ac50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f370d8a91d80e7b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:23:58.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.620+0000 7f4fcaa00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:23:58.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.620+0000 7f4fe9000700 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b890464d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:23:58.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.620+0000 7f4fcaa00700 1 -- 172.21.15.79:0/188356509 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc09cba0 con 0x55b89045b570 2024-06-03T02:23:58.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.621+0000 7f4ff632bb80 1 -- 172.21.15.79:0/188356509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fdc01b3b0 con 0x55b89045b570 2024-06-03T02:23:58.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.623+0000 7f4fcaa00700 1 -- 172.21.15.79:0/188356509 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4fdc0a4e90 con 0x55b89045b570 2024-06-03T02:23:58.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.623+0000 7f4fcaa00700 1 -- 172.21.15.79:0/188356509 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4fdc09c740 con 0x55b89045b570 2024-06-03T02:23:58.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.623+0000 7f4ff632bb80 10 client.25804 fetch_fsmap learned FSMap version 30 2024-06-03T02:23:58.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.623+0000 7f4ff632bb80 1 -- 172.21.15.79:0/188356509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b8904e6410 con 0x55b89045b570 2024-06-03T02:23:58.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.624+0000 7f4fcaa00700 1 -- 172.21.15.79:0/188356509 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc01b3b0 con 0x55b89045b570 2024-06-03T02:23:58.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.625+0000 7f4fcaa00700 1 -- 172.21.15.79:0/188356509 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4fdc0bc040 con 0x55b89045b570 2024-06-03T02:23:58.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.626+0000 7f4ff632bb80 10 client.25804 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:23:58.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.626+0000 7f4ff632bb80 10 client.25804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:23:58.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.626+0000 7f4ff632bb80 20 client.25804 populate_metadata read hostname 'smithi079' 2024-06-03T02:23:58.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.626+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55b8904e6a50 0x55b8904e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:23:58.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.626+0000 7f4ff632bb80 4 client.25804 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:23:58.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.626+0000 7f4ff632bb80 1 -- 172.21.15.79:0/188356509 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8903aa580 con 0x55b8904e6a50 2024-06-03T02:23:58.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.626+0000 7f4fe9000700 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55b8904e6a50 0x55b8904e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:23:58.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.627+0000 7f4fcaa00700 10 client.25804 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:23:58.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.627+0000 7f4fe9000700 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55b8904e6a50 0x55b8904e8e70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cc633809da6f9fe server_cookie=4d5cf8626ffb89ed in_seq=0 out_seq=0 2024-06-03T02:23:58.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.628+0000 7f4fcaa00700 1 -- 172.21.15.79:0/188356509 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8903aa580 con 0x55b8904e6a50 2024-06-03T02:23:58.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.628+0000 7f4fcaa00700 10 client.25804 handle_command_reply: tid=0 2024-06-03T02:23:58.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.628+0000 7f4ff632bb80 1 client.25804 shutdown 2024-06-03T02:23:58.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.630+0000 7f4ff632bb80 1 -- 172.21.15.79:0/188356509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 msgr2=0x55b8904e4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:58.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.630+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b8904e4c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc09c6d0 tx=0x7f4fdc09ac50 comp rx=0 tx=0).stop 2024-06-03T02:23:58.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.630+0000 7f4ff632bb80 1 -- 172.21.15.79:0/188356509 shutdown_connections 2024-06-03T02:23:58.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.630+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b89045b570 0x55b8904e4c80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.630+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55b8904e6a50 0x55b8904e8e70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.630+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8903cce40 0x55b890464d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.630+0000 7f4ff632bb80 1 --2- 172.21.15.79:0/188356509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b890452960 0x55b890465290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.630+0000 7f4fcaa00700 0 client.25804 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:23:58.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.630+0000 7f4ff632bb80 1 -- 172.21.15.79:0/188356509 >> 172.21.15.79:0/188356509 conn(0x55b89037d170 msgr2=0x55b8904567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:58.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.632+0000 7f4ff632bb80 1 -- 172.21.15.79:0/188356509 shutdown_connections 2024-06-03T02:23:58.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.632+0000 7f4ff632bb80 1 -- 172.21.15.79:0/188356509 wait complete. 2024-06-03T02:23:58.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.632+0000 7f4ff632bb80 20 client.25804 trim_cache size 0 max 16384 2024-06-03T02:23:58.633 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:23:58.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 -- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4fcc0a2e20 msgr2=0x7f4fcc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:58.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4fcc0a2e20 0x7f4fcc0a52e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000b6c0 tx=0x7f4fd000b930 comp rx=0 tx=0).stop 2024-06-03T02:23:58.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 -- 172.21.15.79:0/4222275049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fe41059c0 msgr2=0x7f4fe410fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:23:58.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fe41059c0 0x7f4fe410fea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00c9e0 tx=0x7f4fdc003c80 comp rx=0 tx=0).stop 2024-06-03T02:23:58.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 -- 172.21.15.79:0/4222275049 shutdown_connections 2024-06-03T02:23:58.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fe41059c0 0x7f4fe410fea0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fe4100a90 0x7f4fe4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f4fcc0a2e20 0x7f4fcc0a52e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 --2- 172.21.15.79:0/4222275049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fe4101440 0x7f4fe410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:23:58.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.634+0000 7f4fea400700 1 -- 172.21.15.79:0/4222275049 >> 172.21.15.79:0/4222275049 conn(0x7f4fe4076130 msgr2=0x7f4fe4104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:23:58.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.635+0000 7f4fea400700 1 -- 172.21.15.79:0/4222275049 shutdown_connections 2024-06-03T02:23:58.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:23:58.635+0000 7f4fea400700 1 -- 172.21.15.79:0/4222275049 wait complete. 2024-06-03T02:23:59.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:23:59 smithi079 ceph-mon[31819]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 MiB/s wr, 724 op/s 2024-06-03T02:23:59.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:23:59 smithi135 ceph-mon[36653]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 MiB/s wr, 724 op/s 2024-06-03T02:23:59.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:23:59 smithi123 ceph-mon[36640]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 MiB/s wr, 724 op/s 2024-06-03T02:24:01.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:24:01.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:01 smithi079 ceph-mon[31819]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 570 op/s 2024-06-03T02:24:01.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:01 smithi135 ceph-mon[36653]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 570 op/s 2024-06-03T02:24:01.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:01 smithi123 ceph-mon[36640]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 570 op/s 2024-06-03T02:24:02.019 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 235.0 s, 949.6 tps, lat 98.692 ms stddev 124.159 2024-06-03T02:24:03.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:03 smithi079 ceph-mon[31819]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 698 op/s 2024-06-03T02:24:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:03 smithi135 ceph-mon[36653]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 698 op/s 2024-06-03T02:24:03.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:03 smithi123 ceph-mon[36640]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 698 op/s 2024-06-03T02:24:05.749 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:05 smithi079 ceph-mon[31819]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-06-03T02:24:05.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:05 smithi135 ceph-mon[36653]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-06-03T02:24:05.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:05 smithi123 ceph-mon[36640]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-06-03T02:24:05.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.986+0000 7f0d57e00700 1 -- 172.21.15.79:0/1113871610 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c017400 con 0x7f0d58107450 2024-06-03T02:24:05.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.987+0000 7f0d60400700 1 -- 172.21.15.79:0/1113871610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d58107450 msgr2=0x7f0d58107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:05.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.987+0000 7f0d60400700 1 --2- 172.21.15.79:0/1113871610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d58107450 0x7f0d58107830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c0098f0 tx=0x7f0d4c009c00 comp rx=0 tx=0).stop 2024-06-03T02:24:05.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.988+0000 7f0d60400700 1 -- 172.21.15.79:0/1113871610 shutdown_connections 2024-06-03T02:24:05.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.988+0000 7f0d60400700 1 --2- 172.21.15.79:0/1113871610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d58102ee0 0x7f0d5810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:05.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.988+0000 7f0d60400700 1 --2- 172.21.15.79:0/1113871610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d58107450 0x7f0d58107830 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:05.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.988+0000 7f0d60400700 1 --2- 172.21.15.79:0/1113871610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d58102540 0x7f0d581029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:05.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.988+0000 7f0d60400700 1 -- 172.21.15.79:0/1113871610 >> 172.21.15.79:0/1113871610 conn(0x7f0d580795f0 msgr2=0x7f0d58079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:05.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.989+0000 7f0d60400700 1 -- 172.21.15.79:0/1113871610 shutdown_connections 2024-06-03T02:24:05.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.990+0000 7f0d60400700 1 -- 172.21.15.79:0/1113871610 wait complete. 2024-06-03T02:24:05.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.991+0000 7f0d60400700 1 Processor -- start 2024-06-03T02:24:05.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.991+0000 7f0d60400700 1 -- start start 2024-06-03T02:24:05.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.991+0000 7f0d60400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d58102540 0x7f0d580761f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.991+0000 7f0d60400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d58102ee0 0x7f0d580712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.991+0000 7f0d60400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d58107450 0x7f0d58071810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.991+0000 7f0d60400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d58112880 con 0x7f0d58102540 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.992+0000 7f0d60400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d58112700 con 0x7f0d58107450 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.992+0000 7f0d60400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d58112a00 con 0x7f0d58102ee0 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.992+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d58102ee0 0x7f0d580712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.992+0000 7f0d5e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d58107450 0x7f0d58071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.992+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d58102ee0 0x7f0d580712d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55454/0 (socket says 172.21.15.79:55454) 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.992+0000 7f0d5e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d58107450 0x7f0d58071810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:42678/0 (socket says 172.21.15.79:42678) 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.992+0000 7f0d5e400700 1 -- 172.21.15.79:0/385421532 learned_addr learned my addr 172.21.15.79:0/385421532 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:05.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.992+0000 7f0d5da00700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d58102540 0x7f0d580761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:05.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.994+0000 7f0d5e400700 1 -- 172.21.15.79:0/385421532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d58102ee0 msgr2=0x7f0d580712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:05.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.994+0000 7f0d5e400700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d58102ee0 0x7f0d580712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:05.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.994+0000 7f0d5e400700 1 -- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d58102540 msgr2=0x7f0d580761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:05.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.994+0000 7f0d5e400700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d58102540 0x7f0d580761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:05.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.994+0000 7f0d5e400700 1 -- 172.21.15.79:0/385421532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d580720e0 con 0x7f0d58107450 2024-06-03T02:24:05.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.994+0000 7f0d5da00700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d58102540 0x7f0d580761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:24:05.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.995+0000 7f0d5e400700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d58107450 0x7f0d58071810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000efc0 tx=0x7f0d5000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:05.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.996+0000 7f0d56000700 1 -- 172.21.15.79:0/385421532 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d500092e0 con 0x7f0d58107450 2024-06-03T02:24:05.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.996+0000 7f0d56000700 1 -- 172.21.15.79:0/385421532 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d50004bb0 con 0x7f0d58107450 2024-06-03T02:24:05.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.996+0000 7f0d56000700 1 -- 172.21.15.79:0/385421532 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50004040 con 0x7f0d58107450 2024-06-03T02:24:05.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.996+0000 7f0d60400700 1 -- 172.21.15.79:0/385421532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5819ec10 con 0x7f0d58107450 2024-06-03T02:24:05.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:05.996+0000 7f0d60400700 1 -- 172.21.15.79:0/385421532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5819f120 con 0x7f0d58107450 2024-06-03T02:24:06.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.002+0000 7f0d56000700 1 -- 172.21.15.79:0/385421532 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0d5001b030 con 0x7f0d58107450 2024-06-03T02:24:06.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.005+0000 7f0d56000700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0d400b3e00 0x7f0d400b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.006+0000 7f0d5da00700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0d400b3e00 0x7f0d400b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.007+0000 7f0d6ad1eb80 1 Processor -- start 2024-06-03T02:24:06.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.007+0000 7f0d56000700 1 -- 172.21.15.79:0/385421532 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0d500cec60 con 0x7f0d58107450 2024-06-03T02:24:06.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.008+0000 7f0d6ad1eb80 1 -- start start 2024-06-03T02:24:06.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.008+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.008+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdba7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.008+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdba7a50 0x55b5cdba7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.008+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b5cdaada10 con 0x55b5cdbad3a0 2024-06-03T02:24:06.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.008+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5cdaf0b00 con 0x55b5cdba7a50 2024-06-03T02:24:06.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.008+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b5cdaab8f0 con 0x55b5cdbadcc0 2024-06-03T02:24:06.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.009+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdba7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.009+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdba7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55462/0 (socket says 172.21.15.79:55462) 2024-06-03T02:24:06.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.009+0000 7f0d5d000700 1 -- 172.21.15.79:0/4190912958 learned_addr learned my addr 172.21.15.79:0/4190912958 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:06.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.009+0000 7f0d5da00700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0d400b3e00 0x7f0d400b62c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c005bf0 tx=0x7f0d4c005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:06.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.009+0000 7f0d5d000700 1 -- 172.21.15.79:0/4190912958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdba7a50 msgr2=0x55b5cdba7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:06.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.009+0000 7f0d5d000700 1 --2- 172.21.15.79:0/4190912958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdba7a50 0x55b5cdba7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.009+0000 7f0d5d000700 1 -- 172.21.15.79:0/4190912958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbad3a0 msgr2=0x55b5cdbad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:06.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.009+0000 7f0d5d000700 1 --2- 172.21.15.79:0/4190912958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.010+0000 7f0d5d000700 1 -- 172.21.15.79:0/4190912958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5cda98f90 con 0x55b5cdbadcc0 2024-06-03T02:24:06.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.010+0000 7f0d5d000700 1 --2- 172.21.15.79:0/4190912958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdba7510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009cf0 tx=0x7f0d44011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2b38b00e80b2a51 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:06.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.012+0000 7f0d4aa00700 1 -- 172.21.15.79:0/4190912958 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44011e30 con 0x55b5cdbadcc0 2024-06-03T02:24:06.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.012+0000 7f0d4aa00700 1 -- 172.21.15.79:0/4190912958 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d44004530 con 0x55b5cdbadcc0 2024-06-03T02:24:06.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.012+0000 7f0d4aa00700 1 -- 172.21.15.79:0/4190912958 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44005380 con 0x55b5cdbadcc0 2024-06-03T02:24:06.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.013+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/4190912958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 msgr2=0x55b5cdba7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:06.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.013+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/4190912958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdba7510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009cf0 tx=0x7f0d44011440 comp rx=0 tx=0).stop 2024-06-03T02:24:06.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.013+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/4190912958 shutdown_connections 2024-06-03T02:24:06.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.013+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/4190912958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdba7a50 0x55b5cdba7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.013+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/4190912958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.013+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/4190912958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdba7510 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.013+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/4190912958 >> 172.21.15.79:0/4190912958 conn(0x55b5cdacb170 msgr2=0x55b5cdb99ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:06.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.013+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/4190912958 shutdown_connections 2024-06-03T02:24:06.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.014+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/4190912958 wait complete. 2024-06-03T02:24:06.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.014+0000 7f0d6ad1eb80 1 -- start start 2024-06-03T02:24:06.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.015+0000 7f0d6ad1eb80 1 Processor -- start 2024-06-03T02:24:06.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.018+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdba7a50 0x55b5cdbb3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.018+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbb0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.018+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbb1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b5cda8cf60 con 0x55b5cdbad3a0 2024-06-03T02:24:06.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5cdaecac0 con 0x55b5cdbadcc0 2024-06-03T02:24:06.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b5cdbb46a0 con 0x55b5cdba7a50 2024-06-03T02:24:06.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdba7a50 0x55b5cdbb3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d5e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbb1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdba7a50 0x55b5cdbb3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55470/0 (socket says 172.21.15.79:55470) 2024-06-03T02:24:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d5da00700 1 -- 172.21.15.79:0/3168215966 learned_addr learned my addr 172.21.15.79:0/3168215966 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d5e400700 1 -- 172.21.15.79:0/3168215966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdba7a50 msgr2=0x55b5cdbb3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d5e400700 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdba7a50 0x55b5cdbb3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d5e400700 1 -- 172.21.15.79:0/3168215966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbad3a0 msgr2=0x55b5cdbb0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d5e400700 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbb0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.019+0000 7f0d5e400700 1 -- 172.21.15.79:0/3168215966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5cda3f8d0 con 0x55b5cdbadcc0 2024-06-03T02:24:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.020+0000 7f0d6ad1eb80 10 client.0 _ll_register_callbacks cb 0x55b5cda86380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.020+0000 7f0d5e400700 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbb1430 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c7a0 tx=0x7f0d5009b020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=370dd02e7b86b4b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.020+0000 7f0d3f400700 10 client.25817 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:06.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.020+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/3168215966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d44004530 con 0x55b5cdbadcc0 2024-06-03T02:24:06.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.021+0000 7f0d3f400700 1 -- 172.21.15.79:0/3168215966 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5009f4f0 con 0x55b5cdbadcc0 2024-06-03T02:24:06.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.021+0000 7f0d3f400700 1 -- 172.21.15.79:0/3168215966 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0d500977b0 con 0x55b5cdbadcc0 2024-06-03T02:24:06.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.021+0000 7f0d3f400700 1 -- 172.21.15.79:0/3168215966 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d500a3d20 con 0x55b5cdbadcc0 2024-06-03T02:24:06.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.023+0000 7f0d6ad1eb80 10 client.25817 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:06.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.023+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/3168215966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55b5cdbbef00 con 0x55b5cdbadcc0 2024-06-03T02:24:06.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.026+0000 7f0d3f400700 1 -- 172.21.15.79:0/3168215966 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0d5009f120 con 0x55b5cdbadcc0 2024-06-03T02:24:06.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.026+0000 7f0d6ad1eb80 10 client.25817 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:06.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.026+0000 7f0d6ad1eb80 10 client.25817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:24:06.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.026+0000 7f0d6ad1eb80 20 client.25817 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:06.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.026+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b5cdbc1430 0x55b5cdbc3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.026+0000 7f0d6ad1eb80 4 client.25817 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:24:06.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.026+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/3168215966 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5cdafe000 con 0x55b5cdbc1430 2024-06-03T02:24:06.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.027+0000 7f0d5da00700 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b5cdbc1430 0x55b5cdbc3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.029+0000 7f0d5da00700 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b5cdbc1430 0x55b5cdbc3890 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14dd034fa094fa88 server_cookie=b688bfb3bda34e75 in_seq=0 out_seq=0 2024-06-03T02:24:06.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.029+0000 7f0d3f400700 10 client.25817 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:24:06.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.032+0000 7f0d3f400700 1 -- 172.21.15.79:0/3168215966 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b5cdafe000 con 0x55b5cdbc1430 2024-06-03T02:24:06.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.032+0000 7f0d3f400700 10 client.25817 handle_command_reply: tid=0 2024-06-03T02:24:06.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.033+0000 7f0d6ad1eb80 1 client.25817 shutdown 2024-06-03T02:24:06.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.033+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/3168215966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbadcc0 msgr2=0x55b5cdbb1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:06.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.033+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbb1430 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c7a0 tx=0x7f0d5009b020 comp rx=0 tx=0).stop 2024-06-03T02:24:06.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.035+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/3168215966 shutdown_connections 2024-06-03T02:24:06.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.035+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbb1430 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.035+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b5cdbc1430 0x55b5cdbc3890 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.035+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdba7a50 0x55b5cdbb3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.035+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/3168215966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbb0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.035+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/3168215966 >> 172.21.15.79:0/3168215966 conn(0x55b5cdacb170 msgr2=0x55b5cdb93790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:06.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.036+0000 7f0d3f400700 0 client.25817 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:24:06.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.036+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/3168215966 shutdown_connections 2024-06-03T02:24:06.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.036+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/3168215966 wait complete. 2024-06-03T02:24:06.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.037+0000 7f0d6ad1eb80 20 client.25817 trim_cache size 0 max 16384 2024-06-03T02:24:06.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.077+0000 7f0d6ad1eb80 1 Processor -- start 2024-06-03T02:24:06.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.077+0000 7f0d6ad1eb80 1 -- start start 2024-06-03T02:24:06.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.077+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdba7a50 0x55b5cdba7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.078+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbad3a0 0x55b5cdb94130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.078+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbbd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.078+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581a3b40 con 0x55b5cdba7a50 2024-06-03T02:24:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.078+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581a5680 con 0x55b5cdbad3a0 2024-06-03T02:24:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.078+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581a4210 con 0x55b5cdbadcc0 2024-06-03T02:24:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.079+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdba7a50 0x55b5cdba7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.079+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbad3a0 0x55b5cdb94130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.079+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbad3a0 0x55b5cdb94130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:42708/0 (socket says 172.21.15.79:42708) 2024-06-03T02:24:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.079+0000 7f0d5d000700 1 -- 172.21.15.79:0/1086760885 learned_addr learned my addr 172.21.15.79:0/1086760885 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:06.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.079+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdba7a50 0x55b5cdba7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41682/0 (socket says 172.21.15.79:41682) 2024-06-03T02:24:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.079+0000 7f0d5e400700 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbbd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.080+0000 7f0d5da00700 1 -- 172.21.15.79:0/1086760885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 msgr2=0x55b5cdbbd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.080+0000 7f0d5da00700 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbbd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.080+0000 7f0d5da00700 1 -- 172.21.15.79:0/1086760885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbad3a0 msgr2=0x55b5cdb94130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.080+0000 7f0d5da00700 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbad3a0 0x55b5cdb94130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.080+0000 7f0d5da00700 1 -- 172.21.15.79:0/1086760885 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5cdbaf440 con 0x55b5cdba7a50 2024-06-03T02:24:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.080+0000 7f0d5e400700 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbbd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:24:06.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.080+0000 7f0d5d000700 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbad3a0 0x55b5cdb94130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:24:06.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.081+0000 7f0d5da00700 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdba7a50 0x55b5cdba7510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55b5cdbc4150 tx=0x7f0d4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d20ec9dc49bac64 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:06.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.081+0000 7f0d4a000700 1 -- 172.21.15.79:0/1086760885 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c035070 con 0x55b5cdba7a50 2024-06-03T02:24:06.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.081+0000 7f0d4a000700 1 -- 172.21.15.79:0/1086760885 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d4c00f9b0 con 0x55b5cdba7a50 2024-06-03T02:24:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.081+0000 7f0d4a000700 1 -- 172.21.15.79:0/1086760885 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00b560 con 0x55b5cdba7a50 2024-06-03T02:24:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.082+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1086760885 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdba7a50 msgr2=0x55b5cdba7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.082+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdba7a50 0x55b5cdba7510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55b5cdbc4150 tx=0x7f0d4c009c00 comp rx=0 tx=0).stop 2024-06-03T02:24:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.083+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1086760885 shutdown_connections 2024-06-03T02:24:06.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.083+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbbd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.083+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdba7a50 0x55b5cdba7510 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.083+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1086760885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdbad3a0 0x55b5cdb94130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.083+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1086760885 >> 172.21.15.79:0/1086760885 conn(0x55b5cdacb170 msgr2=0x55b5cdbaf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:06.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.083+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1086760885 shutdown_connections 2024-06-03T02:24:06.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.084+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1086760885 wait complete. 2024-06-03T02:24:06.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.084+0000 7f0d6ad1eb80 1 -- start start 2024-06-03T02:24:06.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.085+0000 7f0d6ad1eb80 1 Processor -- start 2024-06-03T02:24:06.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.086+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdba7a50 0x55b5cdbb9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.086+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbc89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.086+0000 7f0d6ad1eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbc8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.086+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581a4fb0 con 0x55b5cdbadcc0 2024-06-03T02:24:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.086+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581a48e0 con 0x55b5cdba7a50 2024-06-03T02:24:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.086+0000 7f0d6ad1eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581a1ae0 con 0x55b5cdbad3a0 2024-06-03T02:24:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.086+0000 7f0d6ad1eb80 10 client.0 _ll_register_callbacks cb 0x55b5cda86380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.088+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbc89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.088+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbc89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55476/0 (socket says 172.21.15.79:55476) 2024-06-03T02:24:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.088+0000 7f0d5d000700 1 -- 172.21.15.79:0/1502558816 learned_addr learned my addr 172.21.15.79:0/1502558816 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:06.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.088+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdba7a50 0x55b5cdbb9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.088+0000 7f0d5e400700 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbc8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.089+0000 7f0d5e400700 1 -- 172.21.15.79:0/1502558816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbad3a0 msgr2=0x55b5cdbc89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.089+0000 7f0d5e400700 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbc89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.089+0000 7f0d5e400700 1 -- 172.21.15.79:0/1502558816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdba7a50 msgr2=0x55b5cdbb9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.089+0000 7f0d5e400700 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdba7a50 0x55b5cdbb9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.089+0000 7f0d5e400700 1 -- 172.21.15.79:0/1502558816 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5cdbc9590 con 0x55b5cdbadcc0 2024-06-03T02:24:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.090+0000 7f0d5e400700 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbc8ef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55b5cdbbe3f0 tx=0x7f0d50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccbbf5b35d4a655 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:06.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.090+0000 7f0d3f400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:24:06.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.092+0000 7f0d3f400700 1 -- 172.21.15.79:0/1502558816 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5004c070 con 0x55b5cdbadcc0 2024-06-03T02:24:06.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.092+0000 7f0d3f400700 1 -- 172.21.15.79:0/1502558816 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0d500a3a90 con 0x55b5cdbadcc0 2024-06-03T02:24:06.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.093+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1502558816 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d4c00f9b0 con 0x55b5cdbadcc0 2024-06-03T02:24:06.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.093+0000 7f0d3f400700 1 -- 172.21.15.79:0/1502558816 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d5009f4f0 con 0x55b5cdbadcc0 2024-06-03T02:24:06.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.093+0000 7f0d6ad1eb80 10 client.15558 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:06.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.093+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1502558816 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b5cdc2bf60 con 0x55b5cdbadcc0 2024-06-03T02:24:06.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.094+0000 7f0d3f400700 1 -- 172.21.15.79:0/1502558816 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d4c00f9b0 con 0x55b5cdbadcc0 2024-06-03T02:24:06.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.095+0000 7f0d3f400700 1 -- 172.21.15.79:0/1502558816 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0d5009a000 con 0x55b5cdbadcc0 2024-06-03T02:24:06.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.095+0000 7f0d6ad1eb80 10 client.15558 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:06.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.095+0000 7f0d6ad1eb80 10 client.15558 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:24:06.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.095+0000 7f0d6ad1eb80 20 client.15558 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:06.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.095+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b5cdc2c660 0x55b5cdc2ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:06.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.095+0000 7f0d6ad1eb80 4 client.15558 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:24:06.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.095+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1502558816 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5cdaf8580 con 0x55b5cdc2c660 2024-06-03T02:24:06.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.096+0000 7f0d5da00700 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b5cdc2c660 0x55b5cdc2ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:06.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.097+0000 7f0d5da00700 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b5cdc2c660 0x55b5cdc2ea80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be11c19b073717f5 server_cookie=f12ea0bb13bdf095 in_seq=0 out_seq=0 2024-06-03T02:24:06.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.097+0000 7f0d3f400700 10 client.15558 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:24:06.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d3f400700 1 -- 172.21.15.79:0/1502558816 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5cdaf8580 con 0x55b5cdc2c660 2024-06-03T02:24:06.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d3f400700 10 client.15558 handle_command_reply: tid=0 2024-06-03T02:24:06.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d6ad1eb80 1 client.15558 shutdown 2024-06-03T02:24:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1502558816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbadcc0 msgr2=0x55b5cdbc8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbc8ef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55b5cdbbe3f0 tx=0x7f0d50009e30 comp rx=0 tx=0).stop 2024-06-03T02:24:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1502558816 shutdown_connections 2024-06-03T02:24:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b5cdbadcc0 0x55b5cdbc8ef0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5cdba7a50 0x55b5cdbb9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b5cdc2c660 0x55b5cdc2ea80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d6ad1eb80 1 --2- 172.21.15.79:0/1502558816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5cdbad3a0 0x55b5cdbc89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.098+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1502558816 >> 172.21.15.79:0/1502558816 conn(0x55b5cdacb170 msgr2=0x55b5cdbaff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.099+0000 7f0d3f400700 0 client.15558 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:24:06.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.099+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1502558816 shutdown_connections 2024-06-03T02:24:06.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.099+0000 7f0d6ad1eb80 1 -- 172.21.15.79:0/1502558816 wait complete. 2024-06-03T02:24:06.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.099+0000 7f0d6ad1eb80 20 client.15558 trim_cache size 0 max 16384 2024-06-03T02:24:06.100 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:24:06.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.100+0000 7f0d60400700 1 -- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0d400b3e00 msgr2=0x7f0d400b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:06.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.100+0000 7f0d60400700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0d400b3e00 0x7f0d400b62c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c005bf0 tx=0x7f0d4c005b40 comp rx=0 tx=0).stop 2024-06-03T02:24:06.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.100+0000 7f0d60400700 1 -- 172.21.15.79:0/385421532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d58107450 msgr2=0x7f0d58071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:06.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.100+0000 7f0d60400700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d58107450 0x7f0d58071810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000efc0 tx=0x7f0d5000c3a0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.101+0000 7f0d60400700 1 -- 172.21.15.79:0/385421532 shutdown_connections 2024-06-03T02:24:06.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.101+0000 7f0d60400700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d58107450 0x7f0d58071810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.101+0000 7f0d60400700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0d400b3e00 0x7f0d400b62c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.101+0000 7f0d60400700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d58102540 0x7f0d580761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.101+0000 7f0d60400700 1 --2- 172.21.15.79:0/385421532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d58102ee0 0x7f0d580712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:06.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.101+0000 7f0d60400700 1 -- 172.21.15.79:0/385421532 >> 172.21.15.79:0/385421532 conn(0x7f0d580795f0 msgr2=0x7f0d581003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:06.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.101+0000 7f0d60400700 1 -- 172.21.15.79:0/385421532 shutdown_connections 2024-06-03T02:24:06.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:06.102+0000 7f0d60400700 1 -- 172.21.15.79:0/385421532 wait complete. 2024-06-03T02:24:07.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:07 smithi079 ceph-mon[31819]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-06-03T02:24:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:07 smithi135 ceph-mon[36653]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-06-03T02:24:07.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:07 smithi123 ceph-mon[36640]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-06-03T02:24:08.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:24:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 240.0 s, 707.8 tps, lat 145.025 ms stddev 219.371 2024-06-03T02:24:08.731 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:09.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:09 smithi079 ceph-mon[31819]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.7 MiB/s wr, 787 op/s 2024-06-03T02:24:09.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:09 smithi135 ceph-mon[36653]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.7 MiB/s wr, 787 op/s 2024-06-03T02:24:09.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:09 smithi123 ceph-mon[36640]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.7 MiB/s wr, 787 op/s 2024-06-03T02:24:11.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.372+0000 7fc1bb600700 1 -- 172.21.15.79:0/909815630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406a170 msgr2=0x7fc1b406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:11.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.372+0000 7fc1bb600700 1 --2- 172.21.15.79:0/909815630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406a170 0x7fc1b406a5d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80099c0 tx=0x7fc1a8009cd0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.373+0000 7fc1bb600700 1 -- 172.21.15.79:0/909815630 shutdown_connections 2024-06-03T02:24:11.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.373+0000 7fc1bb600700 1 --2- 172.21.15.79:0/909815630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1b406ab10 0x7fc1b410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.373+0000 7fc1bb600700 1 --2- 172.21.15.79:0/909815630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b41084e0 0x7fc1b41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.373+0000 7fc1bb600700 1 --2- 172.21.15.79:0/909815630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406a170 0x7fc1b406a5d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.373+0000 7fc1bb600700 1 -- 172.21.15.79:0/909815630 >> 172.21.15.79:0/909815630 conn(0x7fc1b40782c0 msgr2=0x7fc1b40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:11.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.373+0000 7fc1bb600700 1 -- 172.21.15.79:0/909815630 shutdown_connections 2024-06-03T02:24:11.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.374+0000 7fc1bb600700 1 -- 172.21.15.79:0/909815630 wait complete. 2024-06-03T02:24:11.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.376+0000 7fc1bb600700 1 Processor -- start 2024-06-03T02:24:11.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.377+0000 7fc1bb600700 1 -- start start 2024-06-03T02:24:11.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.377+0000 7fc1bb600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b406a170 0x7fc1b419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.377+0000 7fc1bb600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406ab10 0x7fc1b419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.377+0000 7fc1bb600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1b41084e0 0x7fc1b419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.378+0000 7fc1b3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406ab10 0x7fc1b419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.378+0000 7fc1bb600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4114a80 con 0x7fc1b406a170 2024-06-03T02:24:11.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.378+0000 7fc1bb600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4114900 con 0x7fc1b41084e0 2024-06-03T02:24:11.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.378+0000 7fc1bb600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4114c00 con 0x7fc1b406ab10 2024-06-03T02:24:11.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.378+0000 7fc1b3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406ab10 0x7fc1b419b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55500/0 (socket says 172.21.15.79:55500) 2024-06-03T02:24:11.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.378+0000 7fc1b3e00700 1 -- 172.21.15.79:0/2002675820 learned_addr learned my addr 172.21.15.79:0/2002675820 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:11.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.379+0000 7fc1b9600700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1b41084e0 0x7fc1b419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.379+0000 7fc1b3e00700 1 -- 172.21.15.79:0/2002675820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1b41084e0 msgr2=0x7fc1b419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:11.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.379+0000 7fc1b3e00700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1b41084e0 0x7fc1b419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.379+0000 7fc1b3e00700 1 -- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b406a170 msgr2=0x7fc1b419af30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:24:11.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.380+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b406a170 0x7fc1b419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.380+0000 7fc1b3e00700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b406a170 0x7fc1b419af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.380+0000 7fc1b3e00700 1 -- 172.21.15.79:0/2002675820 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b419ff40 con 0x7fc1b406ab10 2024-06-03T02:24:11.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.380+0000 7fc1b9600700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1b41084e0 0x7fc1b419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:11.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.381+0000 7fc1b3e00700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406ab10 0x7fc1b419b470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800b580 tx=0x7fc1a8005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:11.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.382+0000 7fc1b1600700 1 -- 172.21.15.79:0/2002675820 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8026070 con 0x7fc1b406ab10 2024-06-03T02:24:11.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.382+0000 7fc1b1600700 1 -- 172.21.15.79:0/2002675820 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a8004050 con 0x7fc1b406ab10 2024-06-03T02:24:11.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.382+0000 7fc1bb600700 1 -- 172.21.15.79:0/2002675820 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b41a01d0 con 0x7fc1b406ab10 2024-06-03T02:24:11.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.382+0000 7fc1b1600700 1 -- 172.21.15.79:0/2002675820 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8021410 con 0x7fc1b406ab10 2024-06-03T02:24:11.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.382+0000 7fc1bb600700 1 -- 172.21.15.79:0/2002675820 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b41a7ab0 con 0x7fc1b406ab10 2024-06-03T02:24:11.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.388+0000 7fc1b1600700 1 -- 172.21.15.79:0/2002675820 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc1a8029070 con 0x7fc1b406ab10 2024-06-03T02:24:11.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.390+0000 7fc1b1600700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc19c0a2d70 0x7fc19c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.391+0000 7fc1b1600700 1 -- 172.21.15.79:0/2002675820 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc1a80d0220 con 0x7fc1b406ab10 2024-06-03T02:24:11.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.392+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc19c0a2d70 0x7fc19c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.392+0000 7fc1c5f70b80 1 Processor -- start 2024-06-03T02:24:11.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.392+0000 7fc1c5f70b80 1 -- start start 2024-06-03T02:24:11.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.392+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc19c0a2d70 0x7fc19c0a5230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4006fd0 tx=0x7fc1a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:11.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a4bce40 0x56069a540070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a5405b0 0x56069a540a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a54d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1c5f70b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56069a44fa10 con 0x56069a54b300 2024-06-03T02:24:11.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1c5f70b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56069a492b00 con 0x56069a5405b0 2024-06-03T02:24:11.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1c5f70b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56069a44d8f0 con 0x56069a4bce40 2024-06-03T02:24:11.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1b8c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a4bce40 0x56069a540070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1b8c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a4bce40 0x56069a540070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55514/0 (socket says 172.21.15.79:55514) 2024-06-03T02:24:11.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1b8c00700 1 -- 172.21.15.79:0/1212021148 learned_addr learned my addr 172.21.15.79:0/1212021148 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:11.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.393+0000 7fc1b3e00700 1 --2- 172.21.15.79:0/1212021148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a5405b0 0x56069a540a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.394+0000 7fc1b9600700 1 --2- 172.21.15.79:0/1212021148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a54d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.394+0000 7fc1b8c00700 1 -- 172.21.15.79:0/1212021148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a5405b0 msgr2=0x56069a540a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:11.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.394+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/1212021148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a5405b0 0x56069a540a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.394+0000 7fc1b8c00700 1 -- 172.21.15.79:0/1212021148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 msgr2=0x56069a54d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:11.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.394+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/1212021148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a54d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.394+0000 7fc1b8c00700 1 -- 172.21.15.79:0/1212021148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56069a43af90 con 0x56069a4bce40 2024-06-03T02:24:11.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.395+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/1212021148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a4bce40 0x56069a540070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4013970 tx=0x7fc1a400f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b96313cd49eddc5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:11.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.395+0000 7fc1a2000700 1 -- 172.21.15.79:0/1212021148 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4019040 con 0x56069a4bce40 2024-06-03T02:24:11.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.395+0000 7fc1a2000700 1 -- 172.21.15.79:0/1212021148 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a4022070 con 0x56069a4bce40 2024-06-03T02:24:11.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.396+0000 7fc1a2000700 1 -- 172.21.15.79:0/1212021148 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a401dbe0 con 0x56069a4bce40 2024-06-03T02:24:11.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.396+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/1212021148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a4bce40 msgr2=0x56069a540070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:11.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.396+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/1212021148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a4bce40 0x56069a540070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4013970 tx=0x7fc1a400f000 comp rx=0 tx=0).stop 2024-06-03T02:24:11.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.397+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/1212021148 shutdown_connections 2024-06-03T02:24:11.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.397+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/1212021148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a54d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.397+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/1212021148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a4bce40 0x56069a540070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.397+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/1212021148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a5405b0 0x56069a540a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.397+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/1212021148 >> 172.21.15.79:0/1212021148 conn(0x56069a46d170 msgr2=0x56069a53c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:11.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.397+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/1212021148 shutdown_connections 2024-06-03T02:24:11.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.397+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/1212021148 wait complete. 2024-06-03T02:24:11.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.398+0000 7fc1c5f70b80 1 -- start start 2024-06-03T02:24:11.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.398+0000 7fc1c5f70b80 1 Processor -- start 2024-06-03T02:24:11.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.399+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a54aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.400+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a547d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.400+0000 7fc1b8c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a54aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.400+0000 7fc1b8c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a54aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:42744/0 (socket says 172.21.15.79:42744) 2024-06-03T02:24:11.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.400+0000 7fc1b8c00700 1 -- 172.21.15.79:0/3238624653 learned_addr learned my addr 172.21.15.79:0/3238624653 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:11.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.400+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a5482a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.400+0000 7fc1b3e00700 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a547d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.400+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56069a42ef60 con 0x56069a54b300 2024-06-03T02:24:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.400+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56069a48eac0 con 0x56069a4bce40 2024-06-03T02:24:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.400+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56069a54dfc0 con 0x56069a5405b0 2024-06-03T02:24:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.401+0000 7fc1c5f70b80 10 client.0 _ll_register_callbacks cb 0x56069a42c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.401+0000 7fc1b9600700 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a5482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.401+0000 7fc1b8c00700 1 -- 172.21.15.79:0/3238624653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 msgr2=0x56069a547d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.401+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a547d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.401+0000 7fc1b8c00700 1 -- 172.21.15.79:0/3238624653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 msgr2=0x56069a5482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:11.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.401+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a5482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.401+0000 7fc1b8c00700 1 -- 172.21.15.79:0/3238624653 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56069a3e18d0 con 0x56069a4bce40 2024-06-03T02:24:11.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.401+0000 7fc1b9600700 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a5482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:24:11.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.402+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a54aa60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4013940 tx=0x7fc1a4013570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb972301d0e5e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:11.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.402+0000 7fc196a00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:11.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.402+0000 7fc196a00700 1 -- 172.21.15.79:0/3238624653 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a402b070 con 0x56069a4bce40 2024-06-03T02:24:11.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.402+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a4022070 con 0x56069a4bce40 2024-06-03T02:24:11.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.402+0000 7fc196a00700 1 -- 172.21.15.79:0/3238624653 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc1a401dc90 con 0x56069a4bce40 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.402+0000 7fc196a00700 1 -- 172.21.15.79:0/3238624653 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc1a4039050 con 0x56069a4bce40 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.402+0000 7fc1c5f70b80 10 client.25832 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.402+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56069a5586b0 con 0x56069a4bce40 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.403+0000 7fc196a00700 1 -- 172.21.15.79:0/3238624653 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a4022070 con 0x56069a4bce40 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.403+0000 7fc196a00700 1 -- 172.21.15.79:0/3238624653 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc1a401c030 con 0x56069a4bce40 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.404+0000 7fc1c5f70b80 10 client.25832 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.404+0000 7fc1c5f70b80 10 client.25832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.404+0000 7fc1c5f70b80 20 client.25832 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.404+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56069a55ac40 0x56069a55d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.404+0000 7fc1c5f70b80 4 client.25832 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:24:11.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.404+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56069a4a0000 con 0x56069a55ac40 2024-06-03T02:24:11.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.405+0000 7fc1b3e00700 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56069a55ac40 0x56069a55d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.405+0000 7fc1b3e00700 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56069a55ac40 0x56069a55d030 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83617de6ddc8bf6 server_cookie=3aaa0ede7ac4e74a in_seq=0 out_seq=0 2024-06-03T02:24:11.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.405+0000 7fc196a00700 10 client.25832 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc196a00700 1 -- 172.21.15.79:0/3238624653 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56069a4a0000 con 0x56069a55ac40 2024-06-03T02:24:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc196a00700 10 client.25832 handle_command_reply: tid=0 2024-06-03T02:24:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc1c5f70b80 1 client.25832 shutdown 2024-06-03T02:24:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 msgr2=0x56069a54aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a54aa60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4013940 tx=0x7fc1a4013570 comp rx=0 tx=0).stop 2024-06-03T02:24:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 shutdown_connections 2024-06-03T02:24:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56069a55ac40 0x56069a55d030 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a5482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a54aa60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc196a00700 0 client.25832 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:11.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3238624653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a547d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.406+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 >> 172.21.15.79:0/3238624653 conn(0x56069a46d170 msgr2=0x56069a53b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:11.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.407+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 shutdown_connections 2024-06-03T02:24:11.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.407+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3238624653 wait complete. 2024-06-03T02:24:11.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.407+0000 7fc1c5f70b80 20 client.25832 trim_cache size 0 max 16384 2024-06-03T02:24:11.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1c5f70b80 1 Processor -- start 2024-06-03T02:24:11.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1c5f70b80 1 -- start start 2024-06-03T02:24:11.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a53b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a54a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a54c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1c5f70b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41b0490 con 0x56069a54b300 2024-06-03T02:24:11.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1c5f70b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41b1fd0 con 0x56069a4bce40 2024-06-03T02:24:11.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1c5f70b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41b0b60 con 0x56069a5405b0 2024-06-03T02:24:11.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1b8c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a53b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1b8c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a53b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:42754/0 (socket says 172.21.15.79:42754) 2024-06-03T02:24:11.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1b8c00700 1 -- 172.21.15.79:0/3011301643 learned_addr learned my addr 172.21.15.79:0/3011301643 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:11.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.428+0000 7fc1b3e00700 1 --2- 172.21.15.79:0/3011301643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a54a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.429+0000 7fc1b8c00700 1 -- 172.21.15.79:0/3011301643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 msgr2=0x56069a54a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:11.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.429+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/3011301643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a54a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.429+0000 7fc1b8c00700 1 -- 172.21.15.79:0/3011301643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 msgr2=0x56069a54c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:11.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.429+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/3011301643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a54c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.429+0000 7fc1b8c00700 1 -- 172.21.15.79:0/3011301643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56069a542fd0 con 0x56069a4bce40 2024-06-03T02:24:11.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.430+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/3011301643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a53b5c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40048a0 tx=0x7fc1a401d650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a977eca05ac342eb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:11.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.431+0000 7fc1a1600700 1 -- 172.21.15.79:0/3011301643 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a402b040 con 0x56069a4bce40 2024-06-03T02:24:11.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.431+0000 7fc1a1600700 1 -- 172.21.15.79:0/3011301643 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a4022040 con 0x56069a4bce40 2024-06-03T02:24:11.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.431+0000 7fc1a1600700 1 -- 172.21.15.79:0/3011301643 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4030b40 con 0x56069a4bce40 2024-06-03T02:24:11.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.432+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3011301643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 msgr2=0x56069a53b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:11.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.432+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3011301643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a53b5c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40048a0 tx=0x7fc1a401d650 comp rx=0 tx=0).stop 2024-06-03T02:24:11.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.432+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3011301643 shutdown_connections 2024-06-03T02:24:11.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.432+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3011301643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a54b300 0x56069a54c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.432+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3011301643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a4bce40 0x56069a53b5c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.432+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/3011301643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a54a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.432+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3011301643 >> 172.21.15.79:0/3011301643 conn(0x56069a46d170 msgr2=0x56069a53aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:11.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.432+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3011301643 shutdown_connections 2024-06-03T02:24:11.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.432+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/3011301643 wait complete. 2024-06-03T02:24:11.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.433+0000 7fc1c5f70b80 1 -- start start 2024-06-03T02:24:11.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.433+0000 7fc1c5f70b80 1 Processor -- start 2024-06-03T02:24:11.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.434+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a4bce40 0x56069a553520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.434+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a553a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.434+0000 7fc1c5f70b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a54b300 0x56069a561ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.434+0000 7fc1c5f70b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41b1900 con 0x56069a4bce40 2024-06-03T02:24:11.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.434+0000 7fc1c5f70b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41b1230 con 0x56069a54b300 2024-06-03T02:24:11.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.434+0000 7fc1c5f70b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41ae430 con 0x56069a5405b0 2024-06-03T02:24:11.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.434+0000 7fc1c5f70b80 10 client.0 _ll_register_callbacks cb 0x56069a42c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:11.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.435+0000 7fc1b9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a54b300 0x56069a561ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.435+0000 7fc1b9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a54b300 0x56069a561ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:42758/0 (socket says 172.21.15.79:42758) 2024-06-03T02:24:11.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.435+0000 7fc1b9600700 1 -- 172.21.15.79:0/4120230768 learned_addr learned my addr 172.21.15.79:0/4120230768 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:11.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.435+0000 7fc1b9600700 1 -- 172.21.15.79:0/4120230768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 msgr2=0x56069a553a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:24:11.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.435+0000 7fc1b9600700 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a553a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.435+0000 7fc1b9600700 1 -- 172.21.15.79:0/4120230768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a4bce40 msgr2=0x56069a553520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.435+0000 7fc1b9600700 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a4bce40 0x56069a553520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.435+0000 7fc1b9600700 1 -- 172.21.15.79:0/4120230768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56069a562560 con 0x56069a54b300 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.436+0000 7fc1b9600700 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a54b300 0x56069a561ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac008ea0 tx=0x7fc1ac0124b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c6c190ac819dad server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.436+0000 7fc196a00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.436+0000 7fc196a00700 1 -- 172.21.15.79:0/4120230768 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac01b030 con 0x56069a54b300 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.436+0000 7fc196a00700 1 -- 172.21.15.79:0/4120230768 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc1ac013040 con 0x56069a54b300 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/4120230768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a4022040 con 0x56069a54b300 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc196a00700 1 -- 172.21.15.79:0/4120230768 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc1ac004bb0 con 0x56069a54b300 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc1c5f70b80 10 client.25838 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:11.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/4120230768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56069a552ef0 con 0x56069a54b300 2024-06-03T02:24:11.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc196a00700 1 -- 172.21.15.79:0/4120230768 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a4022040 con 0x56069a54b300 2024-06-03T02:24:11.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc196a00700 1 -- 172.21.15.79:0/4120230768 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc1ac004160 con 0x56069a54b300 2024-06-03T02:24:11.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc1c5f70b80 10 client.25838 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:11.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc1c5f70b80 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:24:11.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc1c5f70b80 20 client.25838 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:11.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56069a5ce210 0x56069a5d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:11.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc1c5f70b80 4 client.25838 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:24:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.437+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/4120230768 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56069a49a580 con 0x56069a5ce210 2024-06-03T02:24:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.438+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56069a5ce210 0x56069a5d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.438+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56069a5ce210 0x56069a5d0630 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=934abd34d26f32f3 server_cookie=da3cfeedcc18eb25 in_seq=0 out_seq=0 2024-06-03T02:24:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.438+0000 7fc196a00700 10 client.25838 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.438+0000 7fc196a00700 1 -- 172.21.15.79:0/4120230768 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56069a49a580 con 0x56069a5ce210 2024-06-03T02:24:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.438+0000 7fc196a00700 10 client.25838 handle_command_reply: tid=0 2024-06-03T02:24:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.438+0000 7fc1c5f70b80 1 client.25838 shutdown 2024-06-03T02:24:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.438+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/4120230768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a54b300 msgr2=0x56069a561ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.438+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a54b300 0x56069a561ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac008ea0 tx=0x7fc1ac0124b0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/4120230768 shutdown_connections 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56069a54b300 0x56069a561ca0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56069a5ce210 0x56069a5d0630 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56069a4bce40 0x56069a553520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc1c5f70b80 1 --2- 172.21.15.79:0/4120230768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56069a5405b0 0x56069a553a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc196a00700 0 client.25838 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/4120230768 >> 172.21.15.79:0/4120230768 conn(0x56069a46d170 msgr2=0x56069a542850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/4120230768 shutdown_connections 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc1c5f70b80 1 -- 172.21.15.79:0/4120230768 wait complete. 2024-06-03T02:24:11.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.439+0000 7fc1c5f70b80 20 client.25838 trim_cache size 0 max 16384 2024-06-03T02:24:11.444 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:24:11.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.440+0000 7fc1bb600700 1 -- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc19c0a2d70 msgr2=0x7fc19c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:11.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.440+0000 7fc1bb600700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc19c0a2d70 0x7fc19c0a5230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4006fd0 tx=0x7fc1a4008040 comp rx=0 tx=0).stop 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 -- 172.21.15.79:0/2002675820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406ab10 msgr2=0x7fc1b419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406ab10 0x7fc1b419b470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800b580 tx=0x7fc1a8005640 comp rx=0 tx=0).stop 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 -- 172.21.15.79:0/2002675820 shutdown_connections 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1b41084e0 0x7fc1b419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc19c0a2d70 0x7fc19c0a5230 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1b406a170 0x7fc1b419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 --2- 172.21.15.79:0/2002675820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1b406ab10 0x7fc1b419b470 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 -- 172.21.15.79:0/2002675820 >> 172.21.15.79:0/2002675820 conn(0x7fc1b40782c0 msgr2=0x7fc1b41051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 -- 172.21.15.79:0/2002675820 shutdown_connections 2024-06-03T02:24:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:11.441+0000 7fc1bb600700 1 -- 172.21.15.79:0/2002675820 wait complete. 2024-06-03T02:24:11.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:11 smithi079 ceph-mon[31819]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 643 op/s 2024-06-03T02:24:11.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:11 smithi135 ceph-mon[36653]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 643 op/s 2024-06-03T02:24:11.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:11 smithi123 ceph-mon[36640]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 643 op/s 2024-06-03T02:24:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 245.0 s, 1462.4 tps, lat 70.887 ms stddev 91.510 2024-06-03T02:24:13.400 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:24:13.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:24:13.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:13 smithi079 ceph-mon[31819]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 811 op/s 2024-06-03T02:24:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:13 smithi123 ceph-mon[36640]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 811 op/s 2024-06-03T02:24:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:14.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:13 smithi135 ceph-mon[36653]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 811 op/s 2024-06-03T02:24:16.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:16 smithi123 ceph-mon[36640]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.8 MiB/s wr, 733 op/s 2024-06-03T02:24:16.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:16 smithi079 ceph-mon[31819]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.8 MiB/s wr, 733 op/s 2024-06-03T02:24:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:16 smithi135 ceph-mon[36653]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.8 MiB/s wr, 733 op/s 2024-06-03T02:24:17.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.314+0000 7f8a64c00700 1 -- 172.21.15.79:0/135339948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a6006aa30 msgr2=0x7f8a6006ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:17.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.314+0000 7f8a64c00700 1 --2- 172.21.15.79:0/135339948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a6006aa30 0x7f8a6006ae10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009990 tx=0x7f8a48009ca0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.315+0000 7f8a64c00700 1 -- 172.21.15.79:0/135339948 shutdown_connections 2024-06-03T02:24:17.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.315+0000 7f8a64c00700 1 --2- 172.21.15.79:0/135339948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a6010ef30 0x7f8a60111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.315+0000 7f8a64c00700 1 --2- 172.21.15.79:0/135339948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a6006aa30 0x7f8a6006ae10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.315+0000 7f8a64c00700 1 --2- 172.21.15.79:0/135339948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a6006b350 0x7f8a6010e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.315+0000 7f8a64c00700 1 -- 172.21.15.79:0/135339948 >> 172.21.15.79:0/135339948 conn(0x7f8a60078300 msgr2=0x7f8a60078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:17.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.315+0000 7f8a64c00700 1 -- 172.21.15.79:0/135339948 shutdown_connections 2024-06-03T02:24:17.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.315+0000 7f8a64c00700 1 -- 172.21.15.79:0/135339948 wait complete. 2024-06-03T02:24:17.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.317+0000 7f8a64c00700 1 Processor -- start 2024-06-03T02:24:17.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.318+0000 7f8a64c00700 1 -- start start 2024-06-03T02:24:17.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.319+0000 7f8a64c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a6006aa30 0x7f8a6019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.319+0000 7f8a64c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a6006b350 0x7f8a6019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.319+0000 7f8a64c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a6010ef30 0x7f8a6019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.319+0000 7f8a64c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601149a0 con 0x7f8a6006aa30 2024-06-03T02:24:17.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.319+0000 7f8a64c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60114820 con 0x7f8a6010ef30 2024-06-03T02:24:17.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.319+0000 7f8a64c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60114b20 con 0x7f8a6006b350 2024-06-03T02:24:17.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.319+0000 7f8a5de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a6006aa30 0x7f8a6019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.320+0000 7f8a5de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a6006aa30 0x7f8a6019af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:32810/0 (socket says 172.21.15.79:32810) 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.320+0000 7f8a5de00700 1 -- 172.21.15.79:0/3083624467 learned_addr learned my addr 172.21.15.79:0/3083624467 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.319+0000 7f8a5e800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a6010ef30 0x7f8a6019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.320+0000 7f8a5e800700 1 -- 172.21.15.79:0/3083624467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a6006b350 msgr2=0x7f8a6019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.320+0000 7f8a5e800700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a6006b350 0x7f8a6019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.320+0000 7f8a5e800700 1 -- 172.21.15.79:0/3083624467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a6006aa30 msgr2=0x7f8a6019af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.320+0000 7f8a5e800700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a6006aa30 0x7f8a6019af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.320+0000 7f8a5e800700 1 -- 172.21.15.79:0/3083624467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6019ff40 con 0x7f8a6010ef30 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.321+0000 7f8a5e800700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a6010ef30 0x7f8a6019f800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800d720 tx=0x7f8a5800dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.321+0000 7f8a56a00700 1 -- 172.21.15.79:0/3083624467 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58016070 con 0x7f8a6010ef30 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.321+0000 7f8a56a00700 1 -- 172.21.15.79:0/3083624467 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a58004d10 con 0x7f8a6010ef30 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.321+0000 7f8a56a00700 1 -- 172.21.15.79:0/3083624467 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800f690 con 0x7f8a6010ef30 2024-06-03T02:24:17.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.321+0000 7f8a64c00700 1 -- 172.21.15.79:0/3083624467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a601a0230 con 0x7f8a6010ef30 2024-06-03T02:24:17.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.321+0000 7f8a64c00700 1 -- 172.21.15.79:0/3083624467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a601a7ab0 con 0x7f8a6010ef30 2024-06-03T02:24:17.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.326+0000 7f8a56a00700 1 -- 172.21.15.79:0/3083624467 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8a58004ec0 con 0x7f8a6010ef30 2024-06-03T02:24:17.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.328+0000 7f8a56a00700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a440a2ca0 0x7f8a440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.328+0000 7f8a56a00700 1 -- 172.21.15.79:0/3083624467 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8a580cefe0 con 0x7f8a6010ef30 2024-06-03T02:24:17.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.328+0000 7f8a5de00700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a440a2ca0 0x7f8a440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.331+0000 7f8a5de00700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a440a2ca0 0x7f8a440a5160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800b2e0 tx=0x7f8a4801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:17.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.334+0000 7f8a6f4e5b80 1 Processor -- start 2024-06-03T02:24:17.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.334+0000 7f8a6f4e5b80 1 -- start start 2024-06-03T02:24:17.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55de83ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 0x55a55de7dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de7e2c0 0x55a55de7e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a6f4e5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a55dd8ca90 con 0x55a55de84530 2024-06-03T02:24:17.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a6f4e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a55ddcfb80 con 0x55a55de83c10 2024-06-03T02:24:17.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a6f4e5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a55dd8a970 con 0x55a55de7e2c0 2024-06-03T02:24:17.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a5d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 0x55a55de7dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a5de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55de83ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a5d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 0x55a55de7dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:32824/0 (socket says 172.21.15.79:32824) 2024-06-03T02:24:17.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.335+0000 7f8a5d400700 1 -- 172.21.15.79:0/1200394551 learned_addr learned my addr 172.21.15.79:0/1200394551 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:17.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.336+0000 7f8a5de00700 1 -- 172.21.15.79:0/1200394551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de7e2c0 msgr2=0x55a55de7e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:17.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.336+0000 7f8a5de00700 1 --2- 172.21.15.79:0/1200394551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de7e2c0 0x55a55de7e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.336+0000 7f8a5de00700 1 -- 172.21.15.79:0/1200394551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 msgr2=0x55a55de7dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:17.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.336+0000 7f8a5de00700 1 --2- 172.21.15.79:0/1200394551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 0x55a55de7dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.336+0000 7f8a5de00700 1 -- 172.21.15.79:0/1200394551 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55dd78010 con 0x55a55de83c10 2024-06-03T02:24:17.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.336+0000 7f8a5de00700 1 --2- 172.21.15.79:0/1200394551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55de83ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006010 tx=0x7f8a4801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eca5fbe35838a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:17.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.336+0000 7f8a4f400700 1 -- 172.21.15.79:0/1200394551 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800fbd0 con 0x55a55de83c10 2024-06-03T02:24:17.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.336+0000 7f8a4f400700 1 -- 172.21.15.79:0/1200394551 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a48032070 con 0x55a55de83c10 2024-06-03T02:24:17.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.336+0000 7f8a4f400700 1 -- 172.21.15.79:0/1200394551 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4802d3f0 con 0x55a55de83c10 2024-06-03T02:24:17.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.339+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/1200394551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 msgr2=0x55a55de83ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:17.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.339+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/1200394551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55de83ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006010 tx=0x7f8a4801e000 comp rx=0 tx=0).stop 2024-06-03T02:24:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.339+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/1200394551 shutdown_connections 2024-06-03T02:24:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.339+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/1200394551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de7e2c0 0x55a55de7e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.339+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/1200394551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55de83ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.339+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/1200394551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 0x55a55de7dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.339+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/1200394551 >> 172.21.15.79:0/1200394551 conn(0x55a55ddaa1f0 msgr2=0x55a55de79070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:17.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.339+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/1200394551 shutdown_connections 2024-06-03T02:24:17.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.340+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/1200394551 wait complete. 2024-06-03T02:24:17.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.340+0000 7f8a6f4e5b80 1 -- start start 2024-06-03T02:24:17.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.340+0000 7f8a6f4e5b80 1 Processor -- start 2024-06-03T02:24:17.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.343+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de8a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.343+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de83c10 0x55a55de87670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.343+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 0x55a55de87bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.343+0000 7f8a6f4e5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a55dd6bfe0 con 0x55a55de84530 2024-06-03T02:24:17.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.343+0000 7f8a6f4e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a55ddcbb40 con 0x55a55de7e2c0 2024-06-03T02:24:17.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.343+0000 7f8a6f4e5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a55de8aeb0 con 0x55a55de83c10 2024-06-03T02:24:17.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.343+0000 7f8a6f4e5b80 10 client.0 _ll_register_callbacks cb 0x55a55dd65400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:17.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.344+0000 7f8a5d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de83c10 0x55a55de87670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.344+0000 7f8a5de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de8a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.344+0000 7f8a5de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de8a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38276/0 (socket says 172.21.15.79:38276) 2024-06-03T02:24:17.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.344+0000 7f8a5de00700 1 -- 172.21.15.79:0/4205431408 learned_addr learned my addr 172.21.15.79:0/4205431408 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:17.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.344+0000 7f8a5de00700 1 -- 172.21.15.79:0/4205431408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de83c10 msgr2=0x55a55de87670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:17.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.344+0000 7f8a5de00700 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de83c10 0x55a55de87670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.344+0000 7f8a5de00700 1 -- 172.21.15.79:0/4205431408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 msgr2=0x55a55de87bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:17.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.344+0000 7f8a5de00700 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 0x55a55de87bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.344+0000 7f8a5de00700 1 -- 172.21.15.79:0/4205431408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55dd1e950 con 0x55a55de7e2c0 2024-06-03T02:24:17.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.345+0000 7f8a5de00700 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de8a370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009720 tx=0x7f8a4800b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e1e334d2908afe server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:17.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.345+0000 7f8a43e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:17.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.346+0000 7f8a43e00700 1 -- 172.21.15.79:0/4205431408 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4803c070 con 0x55a55de7e2c0 2024-06-03T02:24:17.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.346+0000 7f8a43e00700 1 -- 172.21.15.79:0/4205431408 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8a48023610 con 0x55a55de7e2c0 2024-06-03T02:24:17.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.346+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/4205431408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a48032070 con 0x55a55de7e2c0 2024-06-03T02:24:17.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.346+0000 7f8a43e00700 1 -- 172.21.15.79:0/4205431408 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8a48045050 con 0x55a55de7e2c0 2024-06-03T02:24:17.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.346+0000 7f8a6f4e5b80 10 client.25850 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:17.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.346+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/4205431408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a55de9df20 con 0x55a55de7e2c0 2024-06-03T02:24:17.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.347+0000 7f8a43e00700 1 -- 172.21.15.79:0/4205431408 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a48032070 con 0x55a55de7e2c0 2024-06-03T02:24:17.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.347+0000 7f8a43e00700 1 -- 172.21.15.79:0/4205431408 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8a480364f0 con 0x55a55de7e2c0 2024-06-03T02:24:17.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.347+0000 7f8a6f4e5b80 10 client.25850 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:17.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.347+0000 7f8a6f4e5b80 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:24:17.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.347+0000 7f8a6f4e5b80 20 client.25850 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:17.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.347+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55a55dea03f0 0x55a55dea27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.347+0000 7f8a6f4e5b80 4 client.25850 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:24:17.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.347+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/4205431408 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a55dddd080 con 0x55a55dea03f0 2024-06-03T02:24:17.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.348+0000 7f8a5d400700 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55a55dea03f0 0x55a55dea27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.349+0000 7f8a5d400700 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55a55dea03f0 0x55a55dea27e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f4eb8e4b58cd9a server_cookie=fd9b4913454bf069 in_seq=0 out_seq=0 2024-06-03T02:24:17.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.349+0000 7f8a43e00700 10 client.25850 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:17.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.350+0000 7f8a43e00700 1 -- 172.21.15.79:0/4205431408 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a55dddd080 con 0x55a55dea03f0 2024-06-03T02:24:17.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.350+0000 7f8a43e00700 10 client.25850 handle_command_reply: tid=0 2024-06-03T02:24:17.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.350+0000 7f8a6f4e5b80 1 client.25850 shutdown 2024-06-03T02:24:17.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.351+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/4205431408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 msgr2=0x55a55de8a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:17.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.351+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de8a370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009720 tx=0x7f8a4800b740 comp rx=0 tx=0).stop 2024-06-03T02:24:17.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.352+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/4205431408 shutdown_connections 2024-06-03T02:24:17.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.352+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55a55dea03f0 0x55a55dea27e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.352+0000 7f8a43e00700 0 client.25850 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:17.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.352+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de84530 0x55a55de87bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.352+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de8a370 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.352+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/4205431408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de83c10 0x55a55de87670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.352+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/4205431408 >> 172.21.15.79:0/4205431408 conn(0x55a55ddaa1f0 msgr2=0x55a55de72810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:17.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.352+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/4205431408 shutdown_connections 2024-06-03T02:24:17.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.353+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/4205431408 wait complete. 2024-06-03T02:24:17.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.353+0000 7f8a6f4e5b80 20 client.25850 trim_cache size 0 max 16384 2024-06-03T02:24:17.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.393+0000 7f8a6f4e5b80 1 Processor -- start 2024-06-03T02:24:17.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a6f4e5b80 1 -- start start 2024-06-03T02:24:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de83c10 0x55a55de89eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de84530 0x55a55de87630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a6f4e5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601ac2a0 con 0x55a55de83c10 2024-06-03T02:24:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a6f4e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601adde0 con 0x55a55de7e2c0 2024-06-03T02:24:17.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a6f4e5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601ac970 con 0x55a55de84530 2024-06-03T02:24:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a5de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a5de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38280/0 (socket says 172.21.15.79:38280) 2024-06-03T02:24:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.394+0000 7f8a5de00700 1 -- 172.21.15.79:0/3515770944 learned_addr learned my addr 172.21.15.79:0/3515770944 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.395+0000 7f8a5de00700 1 -- 172.21.15.79:0/3515770944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de84530 msgr2=0x55a55de87630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:24:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.395+0000 7f8a5de00700 1 --2- 172.21.15.79:0/3515770944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de84530 0x55a55de87630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.395+0000 7f8a5de00700 1 -- 172.21.15.79:0/3515770944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de83c10 msgr2=0x55a55de89eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.395+0000 7f8a5de00700 1 --2- 172.21.15.79:0/3515770944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de83c10 0x55a55de89eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.395+0000 7f8a5de00700 1 -- 172.21.15.79:0/3515770944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55de836a0 con 0x55a55de7e2c0 2024-06-03T02:24:17.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.396+0000 7f8a5de00700 1 --2- 172.21.15.79:0/3515770944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de731b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005c20 tx=0x7f8a48036ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5be60166c33581a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:17.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.396+0000 7f8a4ea00700 1 -- 172.21.15.79:0/3515770944 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4803c040 con 0x55a55de7e2c0 2024-06-03T02:24:17.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.396+0000 7f8a4ea00700 1 -- 172.21.15.79:0/3515770944 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a48032050 con 0x55a55de7e2c0 2024-06-03T02:24:17.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.396+0000 7f8a4ea00700 1 -- 172.21.15.79:0/3515770944 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4802d610 con 0x55a55de7e2c0 2024-06-03T02:24:17.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.399+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/3515770944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 msgr2=0x55a55de731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:17.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.399+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/3515770944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de731b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005c20 tx=0x7f8a48036ce0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.399+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/3515770944 shutdown_connections 2024-06-03T02:24:17.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.399+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/3515770944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de84530 0x55a55de87630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.399+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/3515770944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de7e2c0 0x55a55de731b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.399+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/3515770944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de83c10 0x55a55de89eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.400+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/3515770944 >> 172.21.15.79:0/3515770944 conn(0x55a55ddaa1f0 msgr2=0x55a55de86f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:17.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.400+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/3515770944 shutdown_connections 2024-06-03T02:24:17.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.400+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/3515770944 wait complete. 2024-06-03T02:24:17.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.401+0000 7f8a6f4e5b80 1 -- start start 2024-06-03T02:24:17.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.401+0000 7f8a6f4e5b80 1 Processor -- start 2024-06-03T02:24:17.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.404+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de7e2c0 0x55a55dea5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.405+0000 7f8a6f4e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55dea4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.405+0000 7f8a5d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55dea4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.405+0000 7f8a5d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55dea4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38294/0 (socket says 172.21.15.79:38294) 2024-06-03T02:24:17.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.405+0000 7f8a5d400700 1 -- 172.21.15.79:0/749478021 learned_addr learned my addr 172.21.15.79:0/749478021 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:17.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.406+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de84530 0x55a55dea4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.406+0000 7f8a5de00700 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de7e2c0 0x55a55dea5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.408+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601ad710 con 0x55a55de7e2c0 2024-06-03T02:24:17.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.408+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601ad040 con 0x55a55de83c10 2024-06-03T02:24:17.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.408+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601aa240 con 0x55a55de84530 2024-06-03T02:24:17.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.408+0000 7f8a5e800700 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de84530 0x55a55dea4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.409+0000 7f8a6f4e5b80 10 client.0 _ll_register_callbacks cb 0x55a55dd65400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:17.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.409+0000 7f8a5d400700 1 -- 172.21.15.79:0/749478021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de84530 msgr2=0x55a55dea4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:17.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.409+0000 7f8a5d400700 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de84530 0x55a55dea4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.409+0000 7f8a5d400700 1 -- 172.21.15.79:0/749478021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de7e2c0 msgr2=0x55a55dea5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:17.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.409+0000 7f8a5d400700 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de7e2c0 0x55a55dea5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.409+0000 7f8a5d400700 1 -- 172.21.15.79:0/749478021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55dea4c60 con 0x55a55de83c10 2024-06-03T02:24:17.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.411+0000 7f8a5d400700 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55dea4110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a55de8af70 tx=0x7f8a50017640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e14695663ef8e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:17.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.411+0000 7f8a43e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:17.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.414+0000 7f8a43e00700 1 -- 172.21.15.79:0/749478021 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50017ce0 con 0x55a55de83c10 2024-06-03T02:24:17.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.414+0000 7f8a43e00700 1 -- 172.21.15.79:0/749478021 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8a50004bb0 con 0x55a55de83c10 2024-06-03T02:24:17.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.414+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a48032050 con 0x55a55de83c10 2024-06-03T02:24:17.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.423+0000 7f8a43e00700 1 -- 172.21.15.79:0/749478021 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8a50004db0 con 0x55a55de83c10 2024-06-03T02:24:17.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.423+0000 7f8a6f4e5b80 10 client.25856 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:17.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.423+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a55df0bc40 con 0x55a55de83c10 2024-06-03T02:24:17.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.423+0000 7f8a43e00700 1 -- 172.21.15.79:0/749478021 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a48032050 con 0x55a55de83c10 2024-06-03T02:24:17.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.423+0000 7f8a43e00700 1 -- 172.21.15.79:0/749478021 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8a50025580 con 0x55a55de83c10 2024-06-03T02:24:17.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.424+0000 7f8a6f4e5b80 10 client.25856 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:17.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.424+0000 7f8a6f4e5b80 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:24:17.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.424+0000 7f8a6f4e5b80 20 client.25856 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:17.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.424+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55a55df0c380 0x55a55df0e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:17.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.424+0000 7f8a6f4e5b80 4 client.25856 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:24:17.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.424+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a55ddd7600 con 0x55a55df0c380 2024-06-03T02:24:17.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.424+0000 7f8a5de00700 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55a55df0c380 0x55a55df0e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:17.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.426+0000 7f8a5de00700 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55a55df0c380 0x55a55df0e7a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c179c02c3a8b67aa server_cookie=f301fd884353ee5c in_seq=0 out_seq=0 2024-06-03T02:24:17.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.426+0000 7f8a43e00700 10 client.25856 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:17.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.429+0000 7f8a43e00700 1 -- 172.21.15.79:0/749478021 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a55ddd7600 con 0x55a55df0c380 2024-06-03T02:24:17.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.429+0000 7f8a43e00700 10 client.25856 handle_command_reply: tid=0 2024-06-03T02:24:17.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.429+0000 7f8a6f4e5b80 1 client.25856 shutdown 2024-06-03T02:24:17.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.429+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 msgr2=0x55a55dea4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:17.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.429+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55dea4110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a55de8af70 tx=0x7f8a50017640 comp rx=0 tx=0).stop 2024-06-03T02:24:17.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 shutdown_connections 2024-06-03T02:24:17.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55de84530 0x55a55dea4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55a55df0c380 0x55a55df0e7a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a55de7e2c0 0x55a55dea5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a6f4e5b80 1 --2- 172.21.15.79:0/749478021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a55de83c10 0x55a55dea4110 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 >> 172.21.15.79:0/749478021 conn(0x55a55ddaa1f0 msgr2=0x55a55de87720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:17.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a43e00700 0 client.25856 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:17.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 shutdown_connections 2024-06-03T02:24:17.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a6f4e5b80 1 -- 172.21.15.79:0/749478021 wait complete. 2024-06-03T02:24:17.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.432+0000 7f8a6f4e5b80 20 client.25856 trim_cache size 0 max 16384 2024-06-03T02:24:17.432 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:24:17.433 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:24:17.433 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "2c40ccfc-5d66-4c3f-b8eb-8e8ef4b60d82", 2024-06-03T02:24:17.433 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:24:17.433 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:24:17.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.433+0000 7f8a64c00700 1 -- 172.21.15.79:0/3083624467 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a440a2ca0 msgr2=0x7f8a440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.433+0000 7f8a64c00700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a440a2ca0 0x7f8a440a5160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800b2e0 tx=0x7f8a4801a040 comp rx=0 tx=0).stop 2024-06-03T02:24:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.433+0000 7f8a64c00700 1 -- 172.21.15.79:0/3083624467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a6010ef30 msgr2=0x7f8a6019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.433+0000 7f8a64c00700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a6010ef30 0x7f8a6019f800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800d720 tx=0x7f8a5800dae0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.434+0000 7f8a64c00700 1 -- 172.21.15.79:0/3083624467 shutdown_connections 2024-06-03T02:24:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.434+0000 7f8a64c00700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a6010ef30 0x7f8a6019f800 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.434+0000 7f8a64c00700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8a440a2ca0 0x7f8a440a5160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.434+0000 7f8a64c00700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a6006aa30 0x7f8a6019af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.434+0000 7f8a64c00700 1 --2- 172.21.15.79:0/3083624467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a6006b350 0x7f8a6019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.434+0000 7f8a64c00700 1 -- 172.21.15.79:0/3083624467 >> 172.21.15.79:0/3083624467 conn(0x7f8a60078300 msgr2=0x7f8a60110480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.434+0000 7f8a64c00700 1 -- 172.21.15.79:0/3083624467 shutdown_connections 2024-06-03T02:24:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:17.434+0000 7f8a64c00700 1 -- 172.21.15.79:0/3083624467 wait complete. 2024-06-03T02:24:18.314 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 250.0 s, 1256.5 tps, lat 78.062 ms stddev 91.053 2024-06-03T02:24:18.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:24:18.518 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:18 smithi079 ceph-mon[31819]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 659 op/s 2024-06-03T02:24:18.519 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:18 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:24:18.519 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:18 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:24:18.519 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:18 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:24:18.519 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:18 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:24:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:18 smithi135 ceph-mon[36653]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 659 op/s 2024-06-03T02:24:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:18 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:24:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:18 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:24:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:18 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:24:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:18 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:24:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:18 smithi123 ceph-mon[36640]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 659 op/s 2024-06-03T02:24:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:18 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:24:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:18 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:24:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:18 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:24:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:18 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:24:19.104 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:19.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:19 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:24:19.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:19 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:24:19.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:19 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:24:20.505 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:20 smithi079 ceph-mon[31819]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-06-03T02:24:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:20 smithi135 ceph-mon[36653]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-06-03T02:24:20.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:20 smithi123 ceph-mon[36640]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-06-03T02:24:22.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.166+0000 7f56d5600700 1 -- 172.21.15.79:0/2942326430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d01073c0 msgr2=0x7f56d006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:22.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.166+0000 7f56d5600700 1 --2- 172.21.15.79:0/2942326430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d01073c0 0x7f56d006a1a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56c00098f0 tx=0x7f56c0009c00 comp rx=0 tx=0).stop 2024-06-03T02:24:22.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.166+0000 7f56d5600700 1 -- 172.21.15.79:0/2942326430 shutdown_connections 2024-06-03T02:24:22.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.166+0000 7f56d5600700 1 --2- 172.21.15.79:0/2942326430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d0104ed0 0x7f56d0105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.166+0000 7f56d5600700 1 --2- 172.21.15.79:0/2942326430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d01073c0 0x7f56d006a1a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.166+0000 7f56d5600700 1 --2- 172.21.15.79:0/2942326430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56d006a6e0 0x7f56d006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.166+0000 7f56d5600700 1 -- 172.21.15.79:0/2942326430 >> 172.21.15.79:0/2942326430 conn(0x7f56d0078340 msgr2=0x7f56d0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:22.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.166+0000 7f56d5600700 1 -- 172.21.15.79:0/2942326430 shutdown_connections 2024-06-03T02:24:22.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.166+0000 7f56d5600700 1 -- 172.21.15.79:0/2942326430 wait complete. 2024-06-03T02:24:22.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.168+0000 7f56d5600700 1 Processor -- start 2024-06-03T02:24:22.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.168+0000 7f56d5600700 1 -- start start 2024-06-03T02:24:22.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.169+0000 7f56d5600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d006a6e0 0x7f56d0073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.169+0000 7f56d5600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56d0104ed0 0x7f56d006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.169+0000 7f56d5600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d01073c0 0x7f56d006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.169+0000 7f56cea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d006a6e0 0x7f56d0073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.169+0000 7f56cea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d006a6e0 0x7f56d0073ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36046/0 (socket says 172.21.15.79:36046) 2024-06-03T02:24:22.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.170+0000 7f56d5600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0118da0 con 0x7f56d01073c0 2024-06-03T02:24:22.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.170+0000 7f56d5600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0118c20 con 0x7f56d0104ed0 2024-06-03T02:24:22.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.170+0000 7f56d5600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0118f20 con 0x7f56d006a6e0 2024-06-03T02:24:22.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.170+0000 7f56ce000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56d0104ed0 0x7f56d006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.170+0000 7f56cf400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d01073c0 0x7f56d006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.170+0000 7f56cea00700 1 -- 172.21.15.79:0/1737383102 learned_addr learned my addr 172.21.15.79:0/1737383102 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56cea00700 1 -- 172.21.15.79:0/1737383102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56d0104ed0 msgr2=0x7f56d006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56cea00700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56d0104ed0 0x7f56d006cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56cea00700 1 -- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d01073c0 msgr2=0x7f56d006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56cea00700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d01073c0 0x7f56d006d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56cea00700 1 -- 172.21.15.79:0/1737383102 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d006d8b0 con 0x7f56d006a6e0 2024-06-03T02:24:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56cf400700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d01073c0 0x7f56d006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:24:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56cea00700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d006a6e0 0x7f56d0073ca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009230 tx=0x7f56c0005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:22.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56c7400700 1 -- 172.21.15.79:0/1737383102 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0027070 con 0x7f56d006a6e0 2024-06-03T02:24:22.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56d5600700 1 -- 172.21.15.79:0/1737383102 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d006db40 con 0x7f56d006a6e0 2024-06-03T02:24:22.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56c7400700 1 -- 172.21.15.79:0/1737383102 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c000b560 con 0x7f56d006a6e0 2024-06-03T02:24:22.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.171+0000 7f56c7400700 1 -- 172.21.15.79:0/1737383102 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00049c0 con 0x7f56d006a6e0 2024-06-03T02:24:22.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.172+0000 7f56d5600700 1 -- 172.21.15.79:0/1737383102 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d0071a50 con 0x7f56d006a6e0 2024-06-03T02:24:22.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.177+0000 7f56c7400700 1 -- 172.21.15.79:0/1737383102 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f56c002a070 con 0x7f56d006a6e0 2024-06-03T02:24:22.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.179+0000 7f56dfecbb80 1 Processor -- start 2024-06-03T02:24:22.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.179+0000 7f56dfecbb80 1 -- start start 2024-06-03T02:24:22.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.180+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae16f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.180+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae10d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.180+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 0x555a6ae11680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.180+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555a6ad1fa10 con 0x555a6ae174b0 2024-06-03T02:24:22.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.180+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555a6ad62b00 con 0x555a6ae16b90 2024-06-03T02:24:22.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.180+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555a6ad1d8f0 con 0x555a6ae11240 2024-06-03T02:24:22.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.180+0000 7f56c7400700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f56b40a2c20 0x7f56b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.181+0000 7f56c7400700 1 -- 172.21.15.79:0/1737383102 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f56c00cfdf0 con 0x7f56d006a6e0 2024-06-03T02:24:22.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.181+0000 7f56cea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae10d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.181+0000 7f56cea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae10d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:32878/0 (socket says 172.21.15.79:32878) 2024-06-03T02:24:22.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.181+0000 7f56cea00700 1 -- 172.21.15.79:0/1684236264 learned_addr learned my addr 172.21.15.79:0/1684236264 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:22.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.181+0000 7f56cea00700 1 -- 172.21.15.79:0/1684236264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 msgr2=0x555a6ae11680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:24:22.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.181+0000 7f56cea00700 1 --2- 172.21.15.79:0/1684236264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 0x555a6ae11680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.181+0000 7f56cf400700 1 --2- 172.21.15.79:0/1684236264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae16f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.182+0000 7f56cea00700 1 -- 172.21.15.79:0/1684236264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 msgr2=0x555a6ae16f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:22.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.182+0000 7f56cea00700 1 --2- 172.21.15.79:0/1684236264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae16f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.182+0000 7f56cea00700 1 -- 172.21.15.79:0/1684236264 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a6ad0af90 con 0x555a6ae174b0 2024-06-03T02:24:22.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.182+0000 7f56cf400700 1 --2- 172.21.15.79:0/1684236264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae16f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:24:22.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.182+0000 7f56cf400700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f56b40a2c20 0x7f56b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.182+0000 7f56cea00700 1 --2- 172.21.15.79:0/1684236264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae10d00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f56c0005e10 tx=0x7f56c000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51606f12deb82e90 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:22.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.183+0000 7f56bbe00700 1 -- 172.21.15.79:0/1684236264 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c009c330 con 0x555a6ae174b0 2024-06-03T02:24:22.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.183+0000 7f56bbe00700 1 -- 172.21.15.79:0/1684236264 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c00a06e0 con 0x555a6ae174b0 2024-06-03T02:24:22.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.183+0000 7f56bbe00700 1 -- 172.21.15.79:0/1684236264 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00a4a80 con 0x555a6ae174b0 2024-06-03T02:24:22.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.183+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1684236264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 msgr2=0x555a6ae10d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:22.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.183+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1684236264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae10d00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f56c0005e10 tx=0x7f56c000ba10 comp rx=0 tx=0).stop 2024-06-03T02:24:22.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.183+0000 7f56cf400700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f56b40a2c20 0x7f56b40a50e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009710 tx=0x7f56c8009100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:22.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.184+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1684236264 shutdown_connections 2024-06-03T02:24:22.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.184+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1684236264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 0x555a6ae11680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.184+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1684236264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae16f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.184+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1684236264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae10d00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.184+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1684236264 >> 172.21.15.79:0/1684236264 conn(0x555a6ad3d170 msgr2=0x555a6ae0bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:22.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.184+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1684236264 shutdown_connections 2024-06-03T02:24:22.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.184+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1684236264 wait complete. 2024-06-03T02:24:22.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.185+0000 7f56dfecbb80 1 -- start start 2024-06-03T02:24:22.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.185+0000 7f56dfecbb80 1 Processor -- start 2024-06-03T02:24:22.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.185+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae11240 0x555a6ae1d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.185+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae1a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae174b0 0x555a6ae1aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555a6acfef60 con 0x555a6ae11240 2024-06-03T02:24:22.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555a6ad5eac0 con 0x555a6ae16b90 2024-06-03T02:24:22.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555a6ae1dc40 con 0x555a6ae174b0 2024-06-03T02:24:22.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56dfecbb80 10 client.0 _ll_register_callbacks cb 0x555a6acf8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:22.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56cf400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae174b0 0x555a6ae1aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56cea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae11240 0x555a6ae1d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56cf400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae174b0 0x555a6ae1aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36058/0 (socket says 172.21.15.79:36058) 2024-06-03T02:24:22.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56cf400700 1 -- 172.21.15.79:0/1337972062 learned_addr learned my addr 172.21.15.79:0/1337972062 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:22.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.186+0000 7f56ce000700 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae1a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.187+0000 7f56cea00700 1 -- 172.21.15.79:0/1337972062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae174b0 msgr2=0x555a6ae1aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:22.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.187+0000 7f56cea00700 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae174b0 0x555a6ae1aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.187+0000 7f56cea00700 1 -- 172.21.15.79:0/1337972062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 msgr2=0x555a6ae1a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:22.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.187+0000 7f56cea00700 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae1a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.187+0000 7f56cea00700 1 -- 172.21.15.79:0/1337972062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a6acb18d0 con 0x555a6ae11240 2024-06-03T02:24:22.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.188+0000 7f56cea00700 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae11240 0x555a6ae1d210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f56c0005e10 tx=0x7f56c000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3e1e4fff7861c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:22.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.189+0000 7f56b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:24:22.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.189+0000 7f56b8c00700 1 -- 172.21.15.79:0/1337972062 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00ba060 con 0x555a6ae11240 2024-06-03T02:24:22.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.189+0000 7f56b8c00700 1 -- 172.21.15.79:0/1337972062 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f56c00b5170 con 0x555a6ae11240 2024-06-03T02:24:22.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.189+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1337972062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c00a06e0 con 0x555a6ae11240 2024-06-03T02:24:22.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.190+0000 7f56b8c00700 1 -- 172.21.15.79:0/1337972062 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56c00a4f10 con 0x555a6ae11240 2024-06-03T02:24:22.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.190+0000 7f56dfecbb80 10 client.15585 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:22.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.190+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1337972062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a6ae282f0 con 0x555a6ae11240 2024-06-03T02:24:22.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.191+0000 7f56b8c00700 1 -- 172.21.15.79:0/1337972062 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c00a06e0 con 0x555a6ae11240 2024-06-03T02:24:22.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.191+0000 7f56b8c00700 1 -- 172.21.15.79:0/1337972062 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f56c0004b60 con 0x555a6ae11240 2024-06-03T02:24:22.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.191+0000 7f56dfecbb80 10 client.15585 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:22.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.192+0000 7f56dfecbb80 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:24:22.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.192+0000 7f56dfecbb80 20 client.15585 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:22.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.192+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555a6ae2a5f0 0x555a6ae2c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.192+0000 7f56dfecbb80 4 client.15585 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:24:22.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.192+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1337972062 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a6ad70000 con 0x555a6ae2a5f0 2024-06-03T02:24:22.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.193+0000 7f56ce000700 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555a6ae2a5f0 0x555a6ae2c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.193+0000 7f56ce000700 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555a6ae2a5f0 0x555a6ae2c9e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=554e27d5a6fb2250 server_cookie=cb2b888281739e1d in_seq=0 out_seq=0 2024-06-03T02:24:22.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.193+0000 7f56b8c00700 10 client.15585 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:22.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.194+0000 7f56b8c00700 1 -- 172.21.15.79:0/1337972062 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a6ad70000 con 0x555a6ae2a5f0 2024-06-03T02:24:22.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.194+0000 7f56b8c00700 10 client.15585 handle_command_reply: tid=0 2024-06-03T02:24:22.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.194+0000 7f56dfecbb80 1 client.15585 shutdown 2024-06-03T02:24:22.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1337972062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae11240 msgr2=0x555a6ae1d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:22.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae11240 0x555a6ae1d210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f56c0005e10 tx=0x7f56c000b710 comp rx=0 tx=0).stop 2024-06-03T02:24:22.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1337972062 shutdown_connections 2024-06-03T02:24:22.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae174b0 0x555a6ae1aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555a6ae2a5f0 0x555a6ae2c9e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae11240 0x555a6ae1d210 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1337972062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae1a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56b8c00700 0 client.15585 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:22.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1337972062 >> 172.21.15.79:0/1337972062 conn(0x555a6ad3d170 msgr2=0x555a6ae05790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:22.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.195+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1337972062 shutdown_connections 2024-06-03T02:24:22.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.196+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1337972062 wait complete. 2024-06-03T02:24:22.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.197+0000 7f56dfecbb80 20 client.15585 trim_cache size 0 max 16384 2024-06-03T02:24:22.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.227+0000 7f56dfecbb80 1 Processor -- start 2024-06-03T02:24:22.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.227+0000 7f56dfecbb80 1 -- start start 2024-06-03T02:24:22.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.227+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae11240 0x555a6ae10d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.227+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae16b90 0x555a6ae14980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.227+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.227+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0110a90 con 0x555a6ae174b0 2024-06-03T02:24:22.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.227+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b6550 con 0x555a6ae11240 2024-06-03T02:24:22.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.227+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0111160 con 0x555a6ae16b90 2024-06-03T02:24:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.227+0000 7f56ce000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae16b90 0x555a6ae14980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.228+0000 7f56ce000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae16b90 0x555a6ae14980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36064/0 (socket says 172.21.15.79:36064) 2024-06-03T02:24:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.228+0000 7f56ce000700 1 -- 172.21.15.79:0/895475907 learned_addr learned my addr 172.21.15.79:0/895475907 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.228+0000 7f56cea00700 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae11240 0x555a6ae10d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.228+0000 7f56cf400700 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.228+0000 7f56cea00700 1 -- 172.21.15.79:0/895475907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae16b90 msgr2=0x555a6ae14980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.228+0000 7f56cea00700 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae16b90 0x555a6ae14980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.228+0000 7f56cea00700 1 -- 172.21.15.79:0/895475907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 msgr2=0x555a6ae181c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:22.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.228+0000 7f56cea00700 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae181c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.228+0000 7f56cea00700 1 -- 172.21.15.79:0/895475907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a6ae16690 con 0x555a6ae11240 2024-06-03T02:24:22.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.229+0000 7f56ce000700 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae16b90 0x555a6ae14980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:24:22.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.229+0000 7f56cf400700 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae181c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:24:22.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.230+0000 7f56cea00700 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae11240 0x555a6ae10d00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56c0098790 tx=0x7f56c000ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e8306ab9e7690 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:22.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.230+0000 7f56bb400700 1 -- 172.21.15.79:0/895475907 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00ba070 con 0x555a6ae11240 2024-06-03T02:24:22.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.230+0000 7f56bb400700 1 -- 172.21.15.79:0/895475907 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c00a06e0 con 0x555a6ae11240 2024-06-03T02:24:22.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.230+0000 7f56bb400700 1 -- 172.21.15.79:0/895475907 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c00a4a80 con 0x555a6ae11240 2024-06-03T02:24:22.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.231+0000 7f56dfecbb80 1 -- 172.21.15.79:0/895475907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae11240 msgr2=0x555a6ae10d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:22.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.231+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae11240 0x555a6ae10d00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56c0098790 tx=0x7f56c000ba10 comp rx=0 tx=0).stop 2024-06-03T02:24:22.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.231+0000 7f56dfecbb80 1 -- 172.21.15.79:0/895475907 shutdown_connections 2024-06-03T02:24:22.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.231+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae181c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.231+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae11240 0x555a6ae10d00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.231+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/895475907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae16b90 0x555a6ae14980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.231+0000 7f56dfecbb80 1 -- 172.21.15.79:0/895475907 >> 172.21.15.79:0/895475907 conn(0x555a6ad3d170 msgr2=0x555a6ae18ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:22.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.231+0000 7f56dfecbb80 1 -- 172.21.15.79:0/895475907 shutdown_connections 2024-06-03T02:24:22.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.231+0000 7f56dfecbb80 1 -- 172.21.15.79:0/895475907 wait complete. 2024-06-03T02:24:22.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.232+0000 7f56dfecbb80 1 -- start start 2024-06-03T02:24:22.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.232+0000 7f56dfecbb80 1 Processor -- start 2024-06-03T02:24:22.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 0x555a6ae235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae31970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56dfecbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b5e80 con 0x555a6ae174b0 2024-06-03T02:24:22.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b57b0 con 0x555a6ae16b90 2024-06-03T02:24:22.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56dfecbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01019b0 con 0x555a6ae11240 2024-06-03T02:24:22.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56dfecbb80 10 client.0 _ll_register_callbacks cb 0x555a6acf8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:22.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56cea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 0x555a6ae235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56ce000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae31970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56cf400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56cea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 0x555a6ae235d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36072/0 (socket says 172.21.15.79:36072) 2024-06-03T02:24:22.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.233+0000 7f56cea00700 1 -- 172.21.15.79:0/1166609446 learned_addr learned my addr 172.21.15.79:0/1166609446 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:22.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.234+0000 7f56ce000700 1 -- 172.21.15.79:0/1166609446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 msgr2=0x555a6ae235d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:22.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.234+0000 7f56ce000700 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 0x555a6ae235d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.234+0000 7f56ce000700 1 -- 172.21.15.79:0/1166609446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 msgr2=0x555a6ae31eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:22.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.234+0000 7f56ce000700 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae31eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.234+0000 7f56ce000700 1 -- 172.21.15.79:0/1166609446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a6ae326f0 con 0x555a6ae16b90 2024-06-03T02:24:22.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.234+0000 7f56ce000700 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae31970 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc021720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9acb9650ecc750bd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:22.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.234+0000 7f56b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:22.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.237+0000 7f56b8c00700 1 -- 172.21.15.79:0/1166609446 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004030 con 0x555a6ae16b90 2024-06-03T02:24:22.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.238+0000 7f56b8c00700 1 -- 172.21.15.79:0/1166609446 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f56bc0041d0 con 0x555a6ae16b90 2024-06-03T02:24:22.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.238+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1166609446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c00a06e0 con 0x555a6ae16b90 2024-06-03T02:24:22.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.239+0000 7f56b8c00700 1 -- 172.21.15.79:0/1166609446 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56bc0043d0 con 0x555a6ae16b90 2024-06-03T02:24:22.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.239+0000 7f56dfecbb80 10 client.25871 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:22.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.239+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1166609446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a6ae95080 con 0x555a6ae16b90 2024-06-03T02:24:22.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.239+0000 7f56b8c00700 1 -- 172.21.15.79:0/1166609446 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c00a06e0 con 0x555a6ae16b90 2024-06-03T02:24:22.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.240+0000 7f56b8c00700 1 -- 172.21.15.79:0/1166609446 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f56bc035030 con 0x555a6ae16b90 2024-06-03T02:24:22.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.240+0000 7f56dfecbb80 10 client.25871 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:22.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.240+0000 7f56dfecbb80 10 client.25871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:24:22.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.240+0000 7f56dfecbb80 20 client.25871 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:22.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.240+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555a6ae95740 0x555a6ae97b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:22.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.240+0000 7f56dfecbb80 4 client.25871 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:24:22.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.240+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1166609446 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a6ad6a580 con 0x555a6ae95740 2024-06-03T02:24:22.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.240+0000 7f56cea00700 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555a6ae95740 0x555a6ae97b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:22.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.241+0000 7f56cea00700 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555a6ae95740 0x555a6ae97b60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce81497dda28ae6 server_cookie=7a0275d70704da8b in_seq=0 out_seq=0 2024-06-03T02:24:22.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.241+0000 7f56b8c00700 10 client.25871 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:22.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.241+0000 7f56b8c00700 1 -- 172.21.15.79:0/1166609446 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a6ad6a580 con 0x555a6ae95740 2024-06-03T02:24:22.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.241+0000 7f56b8c00700 10 client.25871 handle_command_reply: tid=0 2024-06-03T02:24:22.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.241+0000 7f56dfecbb80 1 client.25871 shutdown 2024-06-03T02:24:22.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1166609446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 msgr2=0x555a6ae31970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:22.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae31970 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc021720 comp rx=0 tx=0).stop 2024-06-03T02:24:22.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1166609446 shutdown_connections 2024-06-03T02:24:22.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555a6ae95740 0x555a6ae97b60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555a6ae174b0 0x555a6ae31eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a6ae11240 0x555a6ae235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 --2- 172.21.15.79:0/1166609446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a6ae16b90 0x555a6ae31970 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1166609446 >> 172.21.15.79:0/1166609446 conn(0x555a6ad3d170 msgr2=0x555a6ae1bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:22.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56b8c00700 0 client.25871 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:22.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1166609446 shutdown_connections 2024-06-03T02:24:22.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.242+0000 7f56dfecbb80 1 -- 172.21.15.79:0/1166609446 wait complete. 2024-06-03T02:24:22.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.243+0000 7f56dfecbb80 20 client.25871 trim_cache size 0 max 16384 2024-06-03T02:24:22.247 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:24:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:24:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:24:22.248 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:24:22.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.248+0000 7f56d5600700 1 -- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f56b40a2c20 msgr2=0x7f56b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:22.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.248+0000 7f56d5600700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f56b40a2c20 0x7f56b40a50e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009710 tx=0x7f56c8009100 comp rx=0 tx=0).stop 2024-06-03T02:24:22.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.248+0000 7f56d5600700 1 -- 172.21.15.79:0/1737383102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d006a6e0 msgr2=0x7f56d0073ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:22.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.248+0000 7f56d5600700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d006a6e0 0x7f56d0073ca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009230 tx=0x7f56c0005e40 comp rx=0 tx=0).stop 2024-06-03T02:24:22.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.250+0000 7f56d5600700 1 -- 172.21.15.79:0/1737383102 shutdown_connections 2024-06-03T02:24:22.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.250+0000 7f56d5600700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f56b40a2c20 0x7f56b40a50e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.250+0000 7f56d5600700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d01073c0 0x7f56d006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.250+0000 7f56d5600700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d006a6e0 0x7f56d0073ca0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.250+0000 7f56d5600700 1 --2- 172.21.15.79:0/1737383102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56d0104ed0 0x7f56d006cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:22.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.250+0000 7f56d5600700 1 -- 172.21.15.79:0/1737383102 >> 172.21.15.79:0/1737383102 conn(0x7f56d0078340 msgr2=0x7f56d0100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:22.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.250+0000 7f56d5600700 1 -- 172.21.15.79:0/1737383102 shutdown_connections 2024-06-03T02:24:22.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:22.250+0000 7f56d5600700 1 -- 172.21.15.79:0/1737383102 wait complete. 2024-06-03T02:24:22.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:22 smithi079 ceph-mon[31819]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.0 MiB/s wr, 609 op/s 2024-06-03T02:24:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:22 smithi135 ceph-mon[36653]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.0 MiB/s wr, 609 op/s 2024-06-03T02:24:22.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:22 smithi123 ceph-mon[36640]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.0 MiB/s wr, 609 op/s 2024-06-03T02:24:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 255.0 s, 911.6 tps, lat 111.175 ms stddev 172.308 2024-06-03T02:24:23.498 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:24:23.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:24:24.128 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:24.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:24 smithi079 ceph-mon[31819]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 693 op/s 2024-06-03T02:24:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:24 smithi135 ceph-mon[36653]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 693 op/s 2024-06-03T02:24:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:24 smithi123 ceph-mon[36640]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 693 op/s 2024-06-03T02:24:26.525 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:26 smithi123 ceph-mon[36640]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-06-03T02:24:26.525 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:26 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:24:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:26 smithi079 ceph-mon[31819]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-06-03T02:24:26.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:26 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:24:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:26 smithi135 ceph-mon[36653]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-06-03T02:24:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:26 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:24:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 260.0 s, 812.6 tps, lat 123.776 ms stddev 174.752 2024-06-03T02:24:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:28 smithi079 ceph-mon[31819]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 480 op/s 2024-06-03T02:24:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:24:28.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:24:28.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:24:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:28 smithi135 ceph-mon[36653]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 480 op/s 2024-06-03T02:24:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:24:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:24:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:24:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:28 smithi123 ceph-mon[36640]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 480 op/s 2024-06-03T02:24:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:24:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:24:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:24:30.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.396+0000 7efe19600700 1 -- 172.21.15.79:0/753033508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe14075180 msgr2=0x7efe1410e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:30.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.397+0000 7efe19600700 1 --2- 172.21.15.79:0/753033508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe14075180 0x7efe1410e810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe04009990 tx=0x7efe04009ca0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.397+0000 7efe19600700 1 -- 172.21.15.79:0/753033508 shutdown_connections 2024-06-03T02:24:30.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.397+0000 7efe19600700 1 --2- 172.21.15.79:0/753033508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe1410ed50 0x7efe14111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.397+0000 7efe19600700 1 --2- 172.21.15.79:0/753033508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe14074860 0x7efe14074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.397+0000 7efe19600700 1 --2- 172.21.15.79:0/753033508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe14075180 0x7efe1410e810 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe04009990 tx=0x7efe04009ca0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.397+0000 7efe19600700 1 -- 172.21.15.79:0/753033508 >> 172.21.15.79:0/753033508 conn(0x7efe140fdf60 msgr2=0x7efe14100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:30.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.398+0000 7efe19600700 1 -- 172.21.15.79:0/753033508 shutdown_connections 2024-06-03T02:24:30.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.398+0000 7efe19600700 1 -- 172.21.15.79:0/753033508 wait complete. 2024-06-03T02:24:30.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.403+0000 7efe19600700 1 Processor -- start 2024-06-03T02:24:30.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.403+0000 7efe19600700 1 -- start start 2024-06-03T02:24:30.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.403+0000 7efe19600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe14074860 0x7efe141a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:30.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.403+0000 7efe19600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe1410ed50 0x7efe141a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:30.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.403+0000 7efe19600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe1410d430 0x7efe1410d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:30.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe19600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efe141036c0 con 0x7efe14074860 2024-06-03T02:24:30.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe19600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe14103540 con 0x7efe1410d430 2024-06-03T02:24:30.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe19600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efe14103840 con 0x7efe1410ed50 2024-06-03T02:24:30.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe12000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe1410ed50 0x7efe141a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:30.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe12000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe1410ed50 0x7efe141a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:43196/0 (socket says 172.21.15.79:43196) 2024-06-03T02:24:30.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe13400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe1410d430 0x7efe1410d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:30.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe12000700 1 -- 172.21.15.79:0/2462645935 learned_addr learned my addr 172.21.15.79:0/2462645935 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:30.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe12a00700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe14074860 0x7efe141a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:30.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe12000700 1 -- 172.21.15.79:0/2462645935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe1410d430 msgr2=0x7efe1410d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:30.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe12000700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe1410d430 0x7efe1410d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe12000700 1 -- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe14074860 msgr2=0x7efe141a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:30.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe12000700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe14074860 0x7efe141a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.404+0000 7efe12000700 1 -- 172.21.15.79:0/2462645935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1410dfb0 con 0x7efe1410ed50 2024-06-03T02:24:30.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.405+0000 7efe12a00700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe14074860 0x7efe141a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:30.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.405+0000 7efe13400700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe1410d430 0x7efe1410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:24:30.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.405+0000 7efe12000700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe1410ed50 0x7efe141a4690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe0400b5c0 tx=0x7efe040055b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:30.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.406+0000 7efe0b400700 1 -- 172.21.15.79:0/2462645935 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04003b50 con 0x7efe1410ed50 2024-06-03T02:24:30.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.406+0000 7efe0b400700 1 -- 172.21.15.79:0/2462645935 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe040041d0 con 0x7efe1410ed50 2024-06-03T02:24:30.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.406+0000 7efe0b400700 1 -- 172.21.15.79:0/2462645935 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040173f0 con 0x7efe1410ed50 2024-06-03T02:24:30.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.406+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1410e240 con 0x7efe1410ed50 2024-06-03T02:24:30.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.406+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe141ac310 con 0x7efe1410ed50 2024-06-03T02:24:30.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.409+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe14069180 con 0x7efe1410ed50 2024-06-03T02:24:30.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.413+0000 7efe0b400700 1 -- 172.21.15.79:0/2462645935 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efe0400f5f0 con 0x7efe1410ed50 2024-06-03T02:24:30.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.415+0000 7efe0b400700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efdf80a2cf0 0x7efdf80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:30.416 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:30 smithi079 ceph-mon[31819]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 666 op/s 2024-06-03T02:24:30.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.415+0000 7efe0b400700 1 -- 172.21.15.79:0/2462645935 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7efe040d04d0 con 0x7efe1410ed50 2024-06-03T02:24:30.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.416+0000 7efe12a00700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efdf80a2cf0 0x7efdf80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:30.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.417+0000 7efe12a00700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efdf80a2cf0 0x7efdf80a51b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00e300 tx=0x7efe0c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:30.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.421+0000 7efe0b400700 1 -- 172.21.15.79:0/2462645935 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe04013070 con 0x7efe1410ed50 2024-06-03T02:24:30.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:30 smithi135 ceph-mon[36653]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 666 op/s 2024-06-03T02:24:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:30 smithi123 ceph-mon[36640]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 666 op/s 2024-06-03T02:24:30.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.873+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe1402dca0 con 0x7efe1410ed50 2024-06-03T02:24:30.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.875+0000 7efe0b400700 1 -- 172.21.15.79:0/2462645935 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7efe040163a0 con 0x7efe1410ed50 2024-06-03T02:24:30.878 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:24:30.878 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:24:30.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efdf80a2cf0 msgr2=0x7efdf80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:30.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efdf80a2cf0 0x7efdf80a51b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00e300 tx=0x7efe0c009040 comp rx=0 tx=0).stop 2024-06-03T02:24:30.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe1410ed50 msgr2=0x7efe141a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:30.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe1410ed50 0x7efe141a4690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe0400b5c0 tx=0x7efe040055b0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 shutdown_connections 2024-06-03T02:24:30.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe1410d430 0x7efe1410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efdf80a2cf0 0x7efdf80a51b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe14074860 0x7efe141a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 --2- 172.21.15.79:0/2462645935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe1410ed50 0x7efe141a4690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:30.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.879+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 >> 172.21.15.79:0/2462645935 conn(0x7efe140fdf60 msgr2=0x7efe140fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:30.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.880+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 shutdown_connections 2024-06-03T02:24:30.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:30.880+0000 7efe19600700 1 -- 172.21.15.79:0/2462645935 wait complete. 2024-06-03T02:24:30.882 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:24:31.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:31 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2462645935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:24:31.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:31 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2462645935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:24:31.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:31 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2462645935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:24:32.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:32 smithi079 ceph-mon[31819]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.6 MiB/s wr, 546 op/s 2024-06-03T02:24:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:32 smithi135 ceph-mon[36653]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.6 MiB/s wr, 546 op/s 2024-06-03T02:24:32.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:24:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:32 smithi123 ceph-mon[36640]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.6 MiB/s wr, 546 op/s 2024-06-03T02:24:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 265.0 s, 652.4 tps, lat 148.886 ms stddev 218.121 2024-06-03T02:24:33.317 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:34.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:34 smithi079 ceph-mon[31819]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 691 op/s 2024-06-03T02:24:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:34 smithi135 ceph-mon[36653]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 691 op/s 2024-06-03T02:24:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:34 smithi123 ceph-mon[36640]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 691 op/s 2024-06-03T02:24:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.396+0000 7f66ac600700 1 -- 172.21.15.79:0/1727769818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a4107400 msgr2=0x7f66a41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:36.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.396+0000 7f66ac600700 1 --2- 172.21.15.79:0/1727769818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a4107400 0x7f66a41077e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f669800b0a0 tx=0x7f669800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.397 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:36 smithi079 ceph-mon[31819]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 745 op/s 2024-06-03T02:24:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.396+0000 7f66ac600700 1 -- 172.21.15.79:0/1727769818 shutdown_connections 2024-06-03T02:24:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.396+0000 7f66ac600700 1 --2- 172.21.15.79:0/1727769818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d50 0x7f66a410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.396+0000 7f66ac600700 1 --2- 172.21.15.79:0/1727769818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a4107400 0x7f66a41077e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.396+0000 7f66ac600700 1 --2- 172.21.15.79:0/1727769818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a4101390 0x7f66a4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.396+0000 7f66ac600700 1 -- 172.21.15.79:0/1727769818 >> 172.21.15.79:0/1727769818 conn(0x7f66a4078340 msgr2=0x7f66a4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.398+0000 7f66ac600700 1 -- 172.21.15.79:0/1727769818 shutdown_connections 2024-06-03T02:24:36.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.398+0000 7f66ac600700 1 -- 172.21.15.79:0/1727769818 wait complete. 2024-06-03T02:24:36.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.399+0000 7f66ac600700 1 Processor -- start 2024-06-03T02:24:36.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.399+0000 7f66ac600700 1 -- start start 2024-06-03T02:24:36.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.400+0000 7f66ac600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a4101390 0x7f66a4110c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.400+0000 7f66ac600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d50 0x7f66a41111c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.400+0000 7f66ac600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a4107400 0x7f66a4111700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.400+0000 7f66ac600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66a406ad00 con 0x7f66a4101390 2024-06-03T02:24:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.400+0000 7f66ac600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66a406ab80 con 0x7f66a4101d50 2024-06-03T02:24:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.400+0000 7f66ac600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a406ae80 con 0x7f66a4107400 2024-06-03T02:24:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.401+0000 7f66aa600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a4107400 0x7f66a4111700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.401+0000 7f66a9200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d50 0x7f66a41111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.401+0000 7f66a9200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d50 0x7f66a41111c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54360/0 (socket says 172.21.15.79:54360) 2024-06-03T02:24:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.401+0000 7f66a9200700 1 -- 172.21.15.79:0/2923251566 learned_addr learned my addr 172.21.15.79:0/2923251566 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.401+0000 7f66a9c00700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a4101390 0x7f66a4110c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.401+0000 7f66a9200700 1 -- 172.21.15.79:0/2923251566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a4107400 msgr2=0x7f66a4111700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.401+0000 7f66a9200700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a4107400 0x7f66a4111700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.401+0000 7f66a9200700 1 -- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a4101390 msgr2=0x7f66a4110c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:36.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.403+0000 7f66a9200700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a4101390 0x7f66a4110c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.403+0000 7f66a9200700 1 -- 172.21.15.79:0/2923251566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a41a7180 con 0x7f66a4101d50 2024-06-03T02:24:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.403+0000 7f66a9c00700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a4101390 0x7f66a4110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.403+0000 7f66aa600700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a4107400 0x7f66a4111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.403+0000 7f66a9200700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d50 0x7f66a41111c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f669000d980 tx=0x7f669000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.404+0000 7f66a2000700 1 -- 172.21.15.79:0/2923251566 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690016070 con 0x7f66a4101d50 2024-06-03T02:24:36.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.404+0000 7f66ac600700 1 -- 172.21.15.79:0/2923251566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a41a7470 con 0x7f66a4101d50 2024-06-03T02:24:36.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.404+0000 7f66a2000700 1 -- 172.21.15.79:0/2923251566 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6690004d10 con 0x7f66a4101d50 2024-06-03T02:24:36.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.404+0000 7f66a2000700 1 -- 172.21.15.79:0/2923251566 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690016070 con 0x7f66a4101d50 2024-06-03T02:24:36.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.405+0000 7f66ac600700 1 -- 172.21.15.79:0/2923251566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a41a7a00 con 0x7f66a4101d50 2024-06-03T02:24:36.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.412+0000 7f66b706cb80 1 Processor -- start 2024-06-03T02:24:36.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.417+0000 7f66b706cb80 1 -- start start 2024-06-03T02:24:36.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.417+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c8e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.417+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 0x557a6c8e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.417+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 0x557a6c8f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.417+0000 7f66b706cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557a6c7f1a10 con 0x557a6c85ee40 2024-06-03T02:24:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.417+0000 7f66b706cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557a6c834b00 con 0x557a6c8f5b10 2024-06-03T02:24:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.417+0000 7f66b706cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557a6c7ef8f0 con 0x557a6c8e25b0 2024-06-03T02:24:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.418+0000 7f66aa600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c8e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.418+0000 7f66aa600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c8e2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:55830/0 (socket says 172.21.15.79:55830) 2024-06-03T02:24:36.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.418+0000 7f66aa600700 1 -- 172.21.15.79:0/2422725071 learned_addr learned my addr 172.21.15.79:0/2422725071 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:36.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.419+0000 7f66aa600700 1 -- 172.21.15.79:0/2422725071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 msgr2=0x557a6c8e2a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:24:36.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.419+0000 7f66aa600700 1 --2- 172.21.15.79:0/2422725071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 0x557a6c8e2a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.419+0000 7f66aa600700 1 -- 172.21.15.79:0/2422725071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 msgr2=0x557a6c8f7f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:24:36.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.419+0000 7f66aa600700 1 --2- 172.21.15.79:0/2422725071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 0x557a6c8f7f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.419+0000 7f66aa600700 1 -- 172.21.15.79:0/2422725071 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6c7dcf90 con 0x557a6c85ee40 2024-06-03T02:24:36.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.419+0000 7f66aa600700 1 --2- 172.21.15.79:0/2422725071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c8e2070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f669c009d30 tx=0x7f669c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffb523999bb3e4a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:36.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.420+0000 7f6696a00700 1 -- 172.21.15.79:0/2422725071 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c011440 con 0x557a6c85ee40 2024-06-03T02:24:36.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.420+0000 7f6696a00700 1 -- 172.21.15.79:0/2422725071 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c011ac0 con 0x557a6c85ee40 2024-06-03T02:24:36.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.420+0000 7f6696a00700 1 -- 172.21.15.79:0/2422725071 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c02b950 con 0x557a6c85ee40 2024-06-03T02:24:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.423+0000 7f66a2000700 1 -- 172.21.15.79:0/2923251566 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f669003ebd0 con 0x7f66a4101d50 2024-06-03T02:24:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.425+0000 7f66b706cb80 1 -- 172.21.15.79:0/2422725071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 msgr2=0x557a6c8e2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.425+0000 7f66b706cb80 1 --2- 172.21.15.79:0/2422725071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c8e2070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f669c009d30 tx=0x7f669c0106b0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.425+0000 7f66b706cb80 1 -- 172.21.15.79:0/2422725071 shutdown_connections 2024-06-03T02:24:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.426+0000 7f66b706cb80 1 --2- 172.21.15.79:0/2422725071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 0x557a6c8f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.426+0000 7f66b706cb80 1 --2- 172.21.15.79:0/2422725071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c8e2070 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.426+0000 7f66b706cb80 1 --2- 172.21.15.79:0/2422725071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 0x557a6c8e2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.426+0000 7f66b706cb80 1 -- 172.21.15.79:0/2422725071 >> 172.21.15.79:0/2422725071 conn(0x557a6c80f170 msgr2=0x557a6c8de4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:36.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.426+0000 7f66b706cb80 1 -- 172.21.15.79:0/2422725071 shutdown_connections 2024-06-03T02:24:36.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.426+0000 7f66b706cb80 1 -- 172.21.15.79:0/2422725071 wait complete. 2024-06-03T02:24:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.427+0000 7f66b706cb80 1 -- start start 2024-06-03T02:24:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.428+0000 7f66a2000700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f668c0a2df0 0x7f668c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.428+0000 7f66a2000700 1 -- 172.21.15.79:0/2923251566 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f66900ce1e0 con 0x7f66a4101d50 2024-06-03T02:24:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.429+0000 7f66b706cb80 1 Processor -- start 2024-06-03T02:24:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.429+0000 7f66aa600700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f668c0a2df0 0x7f668c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c85ee40 0x557a6c8eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c8e25b0 0x557a6c8ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 0x557a6c8f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66b706cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557a6c7d0f60 con 0x557a6c8e25b0 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66b706cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557a6c830ac0 con 0x557a6c85ee40 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66b706cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557a6c8e9530 con 0x557a6c8f5b10 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66b706cb80 10 client.0 _ll_register_callbacks cb 0x557a6c7ce720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66a9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c85ee40 0x557a6c8eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66a9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c85ee40 0x557a6c8eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54376/0 (socket says 172.21.15.79:54376) 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.430+0000 7f66a9c00700 1 -- 172.21.15.79:0/1183867380 learned_addr learned my addr 172.21.15.79:0/1183867380 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.431+0000 7f66a9200700 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c8e25b0 0x557a6c8ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.431+0000 7f66aa600700 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 0x557a6c8f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.432+0000 7f66a9c00700 1 -- 172.21.15.79:0/1183867380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 msgr2=0x557a6c8f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.432+0000 7f66a9c00700 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 0x557a6c8f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.432+0000 7f66a9c00700 1 -- 172.21.15.79:0/1183867380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c8e25b0 msgr2=0x557a6c8ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.432+0000 7f66a9c00700 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c8e25b0 0x557a6c8ef480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.432+0000 7f66a9c00700 1 -- 172.21.15.79:0/1183867380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6c7838d0 con 0x557a6c85ee40 2024-06-03T02:24:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.433+0000 7f66aa600700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f668c0a2df0 0x7f668c0a52b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f669c009fa0 tx=0x7f669c009690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.434+0000 7f66a9c00700 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c85ee40 0x557a6c8eef40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f66980060b0 tx=0x7f669800be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f0835d58c1e647 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.434+0000 7f668b400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.435+0000 7f668b400700 1 -- 172.21.15.79:0/1183867380 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698022070 con 0x557a6c85ee40 2024-06-03T02:24:36.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.435+0000 7f668b400700 1 -- 172.21.15.79:0/1183867380 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f66980074e0 con 0x557a6c85ee40 2024-06-03T02:24:36.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.437+0000 7f66b706cb80 1 -- 172.21.15.79:0/1183867380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669c011ac0 con 0x557a6c85ee40 2024-06-03T02:24:36.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.438+0000 7f668b400700 1 -- 172.21.15.79:0/1183867380 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f669802b050 con 0x557a6c85ee40 2024-06-03T02:24:36.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.439+0000 7f66b706cb80 10 client.25886 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:36.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.439+0000 7f66b706cb80 1 -- 172.21.15.79:0/1183867380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a6c8eafd0 con 0x557a6c85ee40 2024-06-03T02:24:36.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.440+0000 7f668b400700 1 -- 172.21.15.79:0/1183867380 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c011ac0 con 0x557a6c85ee40 2024-06-03T02:24:36.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.440+0000 7f668b400700 1 -- 172.21.15.79:0/1183867380 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f669802e020 con 0x557a6c85ee40 2024-06-03T02:24:36.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.442+0000 7f66b706cb80 10 client.25886 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:36.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.442+0000 7f66b706cb80 10 client.25886 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:24:36.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.442+0000 7f66b706cb80 20 client.25886 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:36.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.442+0000 7f66b706cb80 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x557a6c8fcdd0 0x557a6c8ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.442+0000 7f66b706cb80 4 client.25886 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:24:36.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.442+0000 7f66b706cb80 1 -- 172.21.15.79:0/1183867380 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a6c842000 con 0x557a6c8fcdd0 2024-06-03T02:24:36.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.443+0000 7f66a9200700 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x557a6c8fcdd0 0x557a6c8ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.444+0000 7f66a9200700 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x557a6c8fcdd0 0x557a6c8ff1c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e77496c610dea6c9 server_cookie=2451ec24eb85989a in_seq=0 out_seq=0 2024-06-03T02:24:36.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.444+0000 7f668b400700 10 client.25886 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:24:36.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.445+0000 7f668b400700 1 -- 172.21.15.79:0/1183867380 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557a6c842000 con 0x557a6c8fcdd0 2024-06-03T02:24:36.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.445+0000 7f668b400700 10 client.25886 handle_command_reply: tid=0 2024-06-03T02:24:36.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.445+0000 7f66b706cb80 1 client.25886 shutdown 2024-06-03T02:24:36.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.445+0000 7f66b706cb80 1 -- 172.21.15.79:0/1183867380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c85ee40 msgr2=0x557a6c8eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:36.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.445+0000 7f66b706cb80 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c85ee40 0x557a6c8eef40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f66980060b0 tx=0x7f669800be00 comp rx=0 tx=0).stop 2024-06-03T02:24:36.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.445+0000 7f66b706cb80 1 -- 172.21.15.79:0/1183867380 shutdown_connections 2024-06-03T02:24:36.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.446+0000 7f66b706cb80 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x557a6c8fcdd0 0x557a6c8ff1c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.446+0000 7f668b400700 0 client.25886 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:24:36.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.446+0000 7f66b706cb80 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 0x557a6c8f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.446+0000 7f66b706cb80 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c85ee40 0x557a6c8eef40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.446+0000 7f66b706cb80 1 --2- 172.21.15.79:0/1183867380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c8e25b0 0x557a6c8ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.446+0000 7f66b706cb80 1 -- 172.21.15.79:0/1183867380 >> 172.21.15.79:0/1183867380 conn(0x557a6c80f170 msgr2=0x557a6c8dd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:36.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.446+0000 7f66b706cb80 1 -- 172.21.15.79:0/1183867380 shutdown_connections 2024-06-03T02:24:36.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.446+0000 7f66b706cb80 1 -- 172.21.15.79:0/1183867380 wait complete. 2024-06-03T02:24:36.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.447+0000 7f66b706cb80 20 client.25886 trim_cache size 0 max 16384 2024-06-03T02:24:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.475+0000 7f66b706cb80 1 Processor -- start 2024-06-03T02:24:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.476+0000 7f66b706cb80 1 -- start start 2024-06-03T02:24:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.476+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c8f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.476+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8e25b0 0x557a6c8e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.476+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 0x557a6c8e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.476+0000 7f66b706cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ac200 con 0x557a6c85ee40 2024-06-03T02:24:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.476+0000 7f66b706cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41add40 con 0x557a6c8e25b0 2024-06-03T02:24:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.476+0000 7f66b706cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ac8d0 con 0x557a6c8f5b10 2024-06-03T02:24:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66a9200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8e25b0 0x557a6c8e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66a9200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8e25b0 0x557a6c8e8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54382/0 (socket says 172.21.15.79:54382) 2024-06-03T02:24:36.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66a9200700 1 -- 172.21.15.79:0/4094566364 learned_addr learned my addr 172.21.15.79:0/4094566364 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:36.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66aa600700 1 --2- 172.21.15.79:0/4094566364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 0x557a6c8e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66a9200700 1 -- 172.21.15.79:0/4094566364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 msgr2=0x557a6c8e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:36.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66a9200700 1 --2- 172.21.15.79:0/4094566364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 0x557a6c8e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66a9200700 1 -- 172.21.15.79:0/4094566364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 msgr2=0x557a6c8f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:36.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66a9200700 1 --2- 172.21.15.79:0/4094566364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c8f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66a9200700 1 -- 172.21.15.79:0/4094566364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6c8f81b0 con 0x557a6c8e25b0 2024-06-03T02:24:36.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.477+0000 7f66a9200700 1 --2- 172.21.15.79:0/4094566364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8e25b0 0x557a6c8e8640 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f669000b540 tx=0x7f6690085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba1828ce27faad1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:36.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.478+0000 7f6696000700 1 -- 172.21.15.79:0/4094566364 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669009ad90 con 0x557a6c8e25b0 2024-06-03T02:24:36.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.478+0000 7f6696000700 1 -- 172.21.15.79:0/4094566364 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669009a690 con 0x557a6c8e25b0 2024-06-03T02:24:36.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.478+0000 7f6696000700 1 -- 172.21.15.79:0/4094566364 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669000bb60 con 0x557a6c8e25b0 2024-06-03T02:24:36.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.479+0000 7f66b706cb80 1 -- 172.21.15.79:0/4094566364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8e25b0 msgr2=0x557a6c8e8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:36.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.479+0000 7f66b706cb80 1 --2- 172.21.15.79:0/4094566364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8e25b0 0x557a6c8e8640 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f669000b540 tx=0x7f6690085040 comp rx=0 tx=0).stop 2024-06-03T02:24:36.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.480+0000 7f66b706cb80 1 -- 172.21.15.79:0/4094566364 shutdown_connections 2024-06-03T02:24:36.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.480+0000 7f66b706cb80 1 --2- 172.21.15.79:0/4094566364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8f5b10 0x557a6c8e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.480+0000 7f66b706cb80 1 --2- 172.21.15.79:0/4094566364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c8f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.480+0000 7f66b706cb80 1 --2- 172.21.15.79:0/4094566364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8e25b0 0x557a6c8e8640 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.480+0000 7f66b706cb80 1 -- 172.21.15.79:0/4094566364 >> 172.21.15.79:0/4094566364 conn(0x557a6c80f170 msgr2=0x557a6c8ddfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:36.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.480+0000 7f66b706cb80 1 -- 172.21.15.79:0/4094566364 shutdown_connections 2024-06-03T02:24:36.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.480+0000 7f66b706cb80 1 -- 172.21.15.79:0/4094566364 wait complete. 2024-06-03T02:24:36.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.481+0000 7f66b706cb80 1 -- start start 2024-06-03T02:24:36.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.481+0000 7f66b706cb80 1 Processor -- start 2024-06-03T02:24:36.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.481+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c910bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.481+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 0x557a6c8f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66b706cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 0x557a6c8f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66b706cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ad670 con 0x557a6c85ee40 2024-06-03T02:24:36.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66b706cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41acfa0 con 0x557a6c8f5b10 2024-06-03T02:24:36.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66b706cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41aa1d0 con 0x557a6c8e25b0 2024-06-03T02:24:36.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66b706cb80 10 client.0 _ll_register_callbacks cb 0x557a6c7ce720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:36.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c910bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c910bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:55864/0 (socket says 172.21.15.79:55864) 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9c00700 1 -- 172.21.15.79:0/3643730880 learned_addr learned my addr 172.21.15.79:0/3643730880 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9200700 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 0x557a6c8f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66aa600700 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 0x557a6c8f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9c00700 1 -- 172.21.15.79:0/3643730880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 msgr2=0x557a6c8f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9c00700 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 0x557a6c8f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9c00700 1 -- 172.21.15.79:0/3643730880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 msgr2=0x557a6c8f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9c00700 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 0x557a6c8f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9c00700 1 -- 172.21.15.79:0/3643730880 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6c8f7350 con 0x557a6c85ee40 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.482+0000 7f66a9200700 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 0x557a6c8f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.483+0000 7f66aa600700 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 0x557a6c8f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.483+0000 7f66a9c00700 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c910bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f669801cd00 tx=0x7f669800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e5a2d8b4b8861b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.483+0000 7f668b400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:24:36.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.483+0000 7f668b400700 1 -- 172.21.15.79:0/3643730880 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698022070 con 0x557a6c85ee40 2024-06-03T02:24:36.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.483+0000 7f668b400700 1 -- 172.21.15.79:0/3643730880 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f669801c440 con 0x557a6c85ee40 2024-06-03T02:24:36.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.483+0000 7f66b706cb80 1 -- 172.21.15.79:0/3643730880 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669009a690 con 0x557a6c85ee40 2024-06-03T02:24:36.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.484+0000 7f668b400700 1 -- 172.21.15.79:0/3643730880 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6698004ef0 con 0x557a6c85ee40 2024-06-03T02:24:36.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.484+0000 7f66b706cb80 10 client.15606 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:36.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.484+0000 7f66b706cb80 1 -- 172.21.15.79:0/3643730880 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a6c910400 con 0x557a6c85ee40 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.484+0000 7f668b400700 1 -- 172.21.15.79:0/3643730880 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669009a690 con 0x557a6c85ee40 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.484+0000 7f668b400700 1 -- 172.21.15.79:0/3643730880 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6698011070 con 0x557a6c85ee40 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.485+0000 7f66b706cb80 10 client.15606 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.485+0000 7f66b706cb80 10 client.15606 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.485+0000 7f66b706cb80 20 client.15606 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.485+0000 7f66b706cb80 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x557a6c8fcdd0 0x557a6c969c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.485+0000 7f66b706cb80 4 client.15606 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.485+0000 7f66b706cb80 1 -- 172.21.15.79:0/3643730880 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a6c83c580 con 0x557a6c8fcdd0 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.485+0000 7f66a9200700 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x557a6c8fcdd0 0x557a6c969c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.485+0000 7f66a9200700 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x557a6c8fcdd0 0x557a6c969c90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d19f1f648512e80f server_cookie=e5a6d106bf348a5 in_seq=0 out_seq=0 2024-06-03T02:24:36.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.485+0000 7f668b400700 10 client.15606 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:24:36.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.486+0000 7f668b400700 1 -- 172.21.15.79:0/3643730880 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a6c83c580 con 0x557a6c8fcdd0 2024-06-03T02:24:36.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.486+0000 7f668b400700 10 client.15606 handle_command_reply: tid=0 2024-06-03T02:24:36.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f66b706cb80 1 client.15606 shutdown 2024-06-03T02:24:36.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f66b706cb80 1 -- 172.21.15.79:0/3643730880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 msgr2=0x557a6c910bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f66b706cb80 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c910bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f669801cd00 tx=0x7f669800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f66b706cb80 1 -- 172.21.15.79:0/3643730880 shutdown_connections 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f66b706cb80 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x557a6c8fcdd0 0x557a6c969c90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f66b706cb80 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a6c8f5b10 0x557a6c8f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f66b706cb80 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557a6c85ee40 0x557a6c910bf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f66b706cb80 1 --2- 172.21.15.79:0/3643730880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a6c8e25b0 0x557a6c8f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f668b400700 0 client.15606 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.487+0000 7f66b706cb80 1 -- 172.21.15.79:0/3643730880 >> 172.21.15.79:0/3643730880 conn(0x557a6c80f170 msgr2=0x557a6c8e4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.488+0000 7f66b706cb80 1 -- 172.21.15.79:0/3643730880 shutdown_connections 2024-06-03T02:24:36.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.488+0000 7f66b706cb80 1 -- 172.21.15.79:0/3643730880 wait complete. 2024-06-03T02:24:36.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.488+0000 7f66b706cb80 20 client.15606 trim_cache size 0 max 16384 2024-06-03T02:24:36.490 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:24:36.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.489+0000 7f66ac600700 1 -- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f668c0a2df0 msgr2=0x7f668c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:36.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.489+0000 7f66ac600700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f668c0a2df0 0x7f668c0a52b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f669c009fa0 tx=0x7f669c009690 comp rx=0 tx=0).stop 2024-06-03T02:24:36.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.489+0000 7f66ac600700 1 -- 172.21.15.79:0/2923251566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d50 msgr2=0x7f66a41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:36.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.489+0000 7f66ac600700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d50 0x7f66a41111c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f669000d980 tx=0x7f669000dd40 comp rx=0 tx=0).stop 2024-06-03T02:24:36.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.490+0000 7f66ac600700 1 -- 172.21.15.79:0/2923251566 shutdown_connections 2024-06-03T02:24:36.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.490+0000 7f66ac600700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a4107400 0x7f66a4111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.490+0000 7f66ac600700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f668c0a2df0 0x7f668c0a52b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.490+0000 7f66ac600700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a4101390 0x7f66a4110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.490+0000 7f66ac600700 1 --2- 172.21.15.79:0/2923251566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d50 0x7f66a41111c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:36.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.490+0000 7f66ac600700 1 -- 172.21.15.79:0/2923251566 >> 172.21.15.79:0/2923251566 conn(0x7f66a4078340 msgr2=0x7f66a40ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:36.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.490+0000 7f66ac600700 1 -- 172.21.15.79:0/2923251566 shutdown_connections 2024-06-03T02:24:36.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:36.490+0000 7f66ac600700 1 -- 172.21.15.79:0/2923251566 wait complete. 2024-06-03T02:24:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:36 smithi135 ceph-mon[36653]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 745 op/s 2024-06-03T02:24:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:36 smithi123 ceph-mon[36640]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 745 op/s 2024-06-03T02:24:38.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:24:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 270.0 s, 910.2 tps, lat 111.184 ms stddev 157.564 2024-06-03T02:24:38.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:38 smithi079 ceph-mon[31819]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 683 op/s 2024-06-03T02:24:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:38 smithi135 ceph-mon[36653]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 683 op/s 2024-06-03T02:24:38.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:38 smithi123 ceph-mon[36640]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 683 op/s 2024-06-03T02:24:38.984 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:40.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:40 smithi079 ceph-mon[31819]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 824 op/s 2024-06-03T02:24:40.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:40 smithi135 ceph-mon[36653]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 824 op/s 2024-06-03T02:24:40.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:40 smithi123 ceph-mon[36640]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 824 op/s 2024-06-03T02:24:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.516+0000 7ff76ae00700 1 -- 172.21.15.79:0/279649722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76406a170 msgr2=0x7ff76406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.516+0000 7ff76ae00700 1 --2- 172.21.15.79:0/279649722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76406a170 0x7ff76406a5f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff750009990 tx=0x7ff750009ca0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.517+0000 7ff76ae00700 1 -- 172.21.15.79:0/279649722 shutdown_connections 2024-06-03T02:24:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.517+0000 7ff76ae00700 1 --2- 172.21.15.79:0/279649722 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff76406ab30 0x7ff76410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.517+0000 7ff76ae00700 1 --2- 172.21.15.79:0/279649722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff764108500 0x7ff7641088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.517+0000 7ff76ae00700 1 --2- 172.21.15.79:0/279649722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76406a170 0x7ff76406a5f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.517+0000 7ff76ae00700 1 -- 172.21.15.79:0/279649722 >> 172.21.15.79:0/279649722 conn(0x7ff764078340 msgr2=0x7ff764078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.518+0000 7ff76ae00700 1 -- 172.21.15.79:0/279649722 shutdown_connections 2024-06-03T02:24:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.518+0000 7ff76ae00700 1 -- 172.21.15.79:0/279649722 wait complete. 2024-06-03T02:24:42.520 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:42 smithi079 ceph-mon[31819]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.9 MiB/s wr, 703 op/s 2024-06-03T02:24:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.519+0000 7ff76ae00700 1 Processor -- start 2024-06-03T02:24:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.520+0000 7ff76ae00700 1 -- start start 2024-06-03T02:24:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.520+0000 7ff76ae00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff76406a170 0x7ff764073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.520+0000 7ff76ae00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76406ab30 0x7ff76406bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.520+0000 7ff76ae00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff764108500 0x7ff76406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.520+0000 7ff763400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76406ab30 0x7ff76406bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.520+0000 7ff763e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff76406a170 0x7ff764073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.520+0000 7ff763400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76406ab30 0x7ff76406bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54398/0 (socket says 172.21.15.79:54398) 2024-06-03T02:24:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.520+0000 7ff763e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff76406a170 0x7ff764073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58380/0 (socket says 172.21.15.79:58380) 2024-06-03T02:24:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff76ae00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff764110600 con 0x7ff764108500 2024-06-03T02:24:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff76ae00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff764110480 con 0x7ff76406ab30 2024-06-03T02:24:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff76ae00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff764110780 con 0x7ff76406a170 2024-06-03T02:24:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff763400700 1 -- 172.21.15.79:0/1010803982 learned_addr learned my addr 172.21.15.79:0/1010803982 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff768e00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff764108500 0x7ff76406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff763e00700 1 -- 172.21.15.79:0/1010803982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76406ab30 msgr2=0x7ff76406bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff763e00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76406ab30 0x7ff76406bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff763e00700 1 -- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff764108500 msgr2=0x7ff76406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff763e00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff764108500 0x7ff76406c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.521+0000 7ff763e00700 1 -- 172.21.15.79:0/1010803982 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76406cd20 con 0x7ff76406a170 2024-06-03T02:24:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.522+0000 7ff768e00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff764108500 0x7ff76406c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:24:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.522+0000 7ff763e00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff76406a170 0x7ff764073f80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff754011ab0 tx=0x7ff754011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.522+0000 7ff760c00700 1 -- 172.21.15.79:0/1010803982 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75401e070 con 0x7ff76406a170 2024-06-03T02:24:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.522+0000 7ff76ae00700 1 -- 172.21.15.79:0/1010803982 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7641ab8a0 con 0x7ff76406a170 2024-06-03T02:24:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.522+0000 7ff76ae00700 1 -- 172.21.15.79:0/1010803982 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7641abe30 con 0x7ff76406a170 2024-06-03T02:24:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.523+0000 7ff760c00700 1 -- 172.21.15.79:0/1010803982 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75400fdd0 con 0x7ff76406a170 2024-06-03T02:24:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.523+0000 7ff760c00700 1 -- 172.21.15.79:0/1010803982 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75401e070 con 0x7ff76406a170 2024-06-03T02:24:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.531+0000 7ff760c00700 1 -- 172.21.15.79:0/1010803982 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff754021070 con 0x7ff76406a170 2024-06-03T02:24:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.532+0000 7ff775717b80 1 Processor -- start 2024-06-03T02:24:42.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.533+0000 7ff775717b80 1 -- start start 2024-06-03T02:24:42.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.534+0000 7ff775717b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f2403a0 0x555b1f240780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.534+0000 7ff775717b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f23a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.534+0000 7ff768e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f2403a0 0x555b1f240780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.534+0000 7ff768e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f2403a0 0x555b1f240780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58382/0 (socket says 172.21.15.79:58382) 2024-06-03T02:24:42.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.534+0000 7ff760c00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7480a3010 0x7ff7480a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.534+0000 7ff760c00700 1 -- 172.21.15.79:0/1010803982 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff75400f6f0 con 0x7ff76406a170 2024-06-03T02:24:42.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.536+0000 7ff775717b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f23aa50 0x555b1f23ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.536+0000 7ff763e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f23a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.536+0000 7ff763e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f23a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54414/0 (socket says 172.21.15.79:54414) 2024-06-03T02:24:42.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.536+0000 7ff763400700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7480a3010 0x7ff7480a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.536+0000 7ff775717b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555b1f140a10 con 0x555b1f23aa50 2024-06-03T02:24:42.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.536+0000 7ff775717b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b1f183b00 con 0x555b1f240cc0 2024-06-03T02:24:42.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.536+0000 7ff775717b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555b1f13e8f0 con 0x555b1f2403a0 2024-06-03T02:24:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.537+0000 7ff768e00700 1 -- 172.21.15.79:0/4123032053 learned_addr learned my addr 172.21.15.79:0/4123032053 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.537+0000 7ff763e00700 1 -- 172.21.15.79:0/4123032053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f2403a0 msgr2=0x555b1f240780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.537+0000 7ff763e00700 1 --2- 172.21.15.79:0/4123032053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f2403a0 0x555b1f240780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.537+0000 7ff763e00700 1 -- 172.21.15.79:0/4123032053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f23aa50 msgr2=0x555b1f23ae90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:24:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.537+0000 7ff763e00700 1 --2- 172.21.15.79:0/4123032053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f23aa50 0x555b1f23ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.537+0000 7ff763e00700 1 -- 172.21.15.79:0/4123032053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1f12bf90 con 0x555b1f240cc0 2024-06-03T02:24:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.538+0000 7ff768e00700 1 --2- 172.21.15.79:0/4123032053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f2403a0 0x555b1f240780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:24:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.538+0000 7ff763400700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7480a3010 0x7ff7480a54d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff750009750 tx=0x7ff75000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.538+0000 7ff763e00700 1 --2- 172.21.15.79:0/4123032053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f23a510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff7540cfc30 tx=0x7ff75409e520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9854fb8388a380 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.538+0000 7ff759600700 1 -- 172.21.15.79:0/4123032053 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75401e070 con 0x555b1f240cc0 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.538+0000 7ff759600700 1 -- 172.21.15.79:0/4123032053 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7540a29d0 con 0x555b1f240cc0 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.538+0000 7ff759600700 1 -- 172.21.15.79:0/4123032053 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540a6e00 con 0x555b1f240cc0 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.540+0000 7ff775717b80 1 -- 172.21.15.79:0/4123032053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 msgr2=0x555b1f23a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.540+0000 7ff775717b80 1 --2- 172.21.15.79:0/4123032053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f23a510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff7540cfc30 tx=0x7ff75409e520 comp rx=0 tx=0).stop 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.540+0000 7ff775717b80 1 -- 172.21.15.79:0/4123032053 shutdown_connections 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.540+0000 7ff775717b80 1 --2- 172.21.15.79:0/4123032053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f23aa50 0x555b1f23ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.540+0000 7ff775717b80 1 --2- 172.21.15.79:0/4123032053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f2403a0 0x555b1f240780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.540+0000 7ff775717b80 1 --2- 172.21.15.79:0/4123032053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f23a510 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.540+0000 7ff775717b80 1 -- 172.21.15.79:0/4123032053 >> 172.21.15.79:0/4123032053 conn(0x555b1f15e170 msgr2=0x555b1f22cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:42.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.540+0000 7ff775717b80 1 -- 172.21.15.79:0/4123032053 shutdown_connections 2024-06-03T02:24:42.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.540+0000 7ff775717b80 1 -- 172.21.15.79:0/4123032053 wait complete. 2024-06-03T02:24:42.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.541+0000 7ff775717b80 1 -- start start 2024-06-03T02:24:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.541+0000 7ff775717b80 1 Processor -- start 2024-06-03T02:24:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.543+0000 7ff775717b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f23a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.543+0000 7ff775717b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f235560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.543+0000 7ff775717b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f240cc0 0x555b1f235aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.543+0000 7ff763e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f23a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.543+0000 7ff763e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f23a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58394/0 (socket says 172.21.15.79:58394) 2024-06-03T02:24:42.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.544+0000 7ff763400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f235560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.544+0000 7ff763400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f235560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54430/0 (socket says 172.21.15.79:54430) 2024-06-03T02:24:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555b1f11ff60 con 0x555b1f240cc0 2024-06-03T02:24:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b1f17fac0 con 0x555b1f2403a0 2024-06-03T02:24:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555b1f234150 con 0x555b1f23aa50 2024-06-03T02:24:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff775717b80 10 client.0 _ll_register_callbacks cb 0x555b1f119380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:42.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff763e00700 1 -- 172.21.15.79:0/3437029135 learned_addr learned my addr 172.21.15.79:0/3437029135 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff768e00700 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f240cc0 0x555b1f235aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff763e00700 1 -- 172.21.15.79:0/3437029135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 msgr2=0x555b1f235560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff763e00700 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f235560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff763e00700 1 -- 172.21.15.79:0/3437029135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f240cc0 msgr2=0x555b1f235aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:42.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff763e00700 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f240cc0 0x555b1f235aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff763400700 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f235560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-03T02:24:42.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.545+0000 7ff763e00700 1 -- 172.21.15.79:0/3437029135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1f0d28d0 con 0x555b1f23aa50 2024-06-03T02:24:42.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.546+0000 7ff763e00700 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f23a280 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff75400fbf0 tx=0x7ff754000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c40164eb60d94a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:42.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.546+0000 7ff74e000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:24:42.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.546+0000 7ff74e000700 1 -- 172.21.15.79:0/3437029135 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75401e070 con 0x555b1f23aa50 2024-06-03T02:24:42.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.547+0000 7ff74e000700 1 -- 172.21.15.79:0/3437029135 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff7540a7000 con 0x555b1f23aa50 2024-06-03T02:24:42.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.547+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7540a29d0 con 0x555b1f23aa50 2024-06-03T02:24:42.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.548+0000 7ff74e000700 1 -- 172.21.15.79:0/3437029135 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff754087bb0 con 0x555b1f23aa50 2024-06-03T02:24:42.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.548+0000 7ff775717b80 10 client.25882 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:42.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.548+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b1f249690 con 0x555b1f23aa50 2024-06-03T02:24:42.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.549+0000 7ff74e000700 1 -- 172.21.15.79:0/3437029135 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7540a29d0 con 0x555b1f23aa50 2024-06-03T02:24:42.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.549+0000 7ff74e000700 1 -- 172.21.15.79:0/3437029135 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff75409da70 con 0x555b1f23aa50 2024-06-03T02:24:42.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.549+0000 7ff775717b80 10 client.25882 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:42.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.549+0000 7ff775717b80 10 client.25882 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:24:42.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.549+0000 7ff775717b80 20 client.25882 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:42.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.549+0000 7ff775717b80 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555b1f24bb60 0x555b1f24df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.549+0000 7ff775717b80 4 client.25882 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:24:42.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.549+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b1f191000 con 0x555b1f24bb60 2024-06-03T02:24:42.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.549+0000 7ff763400700 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555b1f24bb60 0x555b1f24df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.550+0000 7ff74e000700 10 client.25882 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:24:42.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.551+0000 7ff763400700 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555b1f24bb60 0x555b1f24df50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38cb4ad7f210e7fe server_cookie=8288af422d33c084 in_seq=0 out_seq=0 2024-06-03T02:24:42.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.552+0000 7ff74e000700 1 -- 172.21.15.79:0/3437029135 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555b1f191000 con 0x555b1f24bb60 2024-06-03T02:24:42.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.552+0000 7ff74e000700 10 client.25882 handle_command_reply: tid=0 2024-06-03T02:24:42.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.552+0000 7ff775717b80 1 client.25882 shutdown 2024-06-03T02:24:42.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 msgr2=0x555b1f23a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:42.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f23a280 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff75400fbf0 tx=0x7ff754000bc0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 shutdown_connections 2024-06-03T02:24:42.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555b1f24bb60 0x555b1f24df50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f240cc0 0x555b1f235aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f23a280 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 --2- 172.21.15.79:0/3437029135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f235560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff74e000700 0 client.25882 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:24:42.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 >> 172.21.15.79:0/3437029135 conn(0x555b1f15e170 msgr2=0x555b1f226790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:42.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 shutdown_connections 2024-06-03T02:24:42.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 1 -- 172.21.15.79:0/3437029135 wait complete. 2024-06-03T02:24:42.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.553+0000 7ff775717b80 20 client.25882 trim_cache size 0 max 16384 2024-06-03T02:24:42.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.592+0000 7ff775717b80 1 Processor -- start 2024-06-03T02:24:42.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.592+0000 7ff775717b80 1 -- start start 2024-06-03T02:24:42.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff775717b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f239db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff775717b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f2403a0 0x555b1f22fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff775717b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f230230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff775717b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff74400eb90 con 0x555b1f2403a0 2024-06-03T02:24:42.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff775717b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7440106d0 con 0x555b1f240cc0 2024-06-03T02:24:42.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff775717b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff74400f260 con 0x555b1f23aa50 2024-06-03T02:24:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff768e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f230230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff768e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f230230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54434/0 (socket says 172.21.15.79:54434) 2024-06-03T02:24:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff768e00700 1 -- 172.21.15.79:0/1409080204 learned_addr learned my addr 172.21.15.79:0/1409080204 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.593+0000 7ff763e00700 1 --2- 172.21.15.79:0/1409080204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f239db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.594+0000 7ff763400700 1 --2- 172.21.15.79:0/1409080204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f2403a0 0x555b1f22fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.594+0000 7ff763e00700 1 -- 172.21.15.79:0/1409080204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 msgr2=0x555b1f230230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.594+0000 7ff763e00700 1 --2- 172.21.15.79:0/1409080204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f230230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.594+0000 7ff763e00700 1 -- 172.21.15.79:0/1409080204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f2403a0 msgr2=0x555b1f22fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.594+0000 7ff763e00700 1 --2- 172.21.15.79:0/1409080204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f2403a0 0x555b1f22fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.594+0000 7ff763e00700 1 -- 172.21.15.79:0/1409080204 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1f2409d0 con 0x555b1f23aa50 2024-06-03T02:24:42.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.594+0000 7ff763e00700 1 --2- 172.21.15.79:0/1409080204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f239db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff754019a60 tx=0x7ff754026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eef98608c3badf38 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:42.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:42 smithi135 ceph-mon[36653]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.9 MiB/s wr, 703 op/s 2024-06-03T02:24:42.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.595+0000 7ff758c00700 1 -- 172.21.15.79:0/1409080204 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75401e070 con 0x555b1f23aa50 2024-06-03T02:24:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.595+0000 7ff758c00700 1 -- 172.21.15.79:0/1409080204 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75409dc70 con 0x555b1f23aa50 2024-06-03T02:24:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.595+0000 7ff758c00700 1 -- 172.21.15.79:0/1409080204 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540b2b40 con 0x555b1f23aa50 2024-06-03T02:24:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.596+0000 7ff775717b80 1 -- 172.21.15.79:0/1409080204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 msgr2=0x555b1f239db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.596+0000 7ff775717b80 1 --2- 172.21.15.79:0/1409080204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f239db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff754019a60 tx=0x7ff754026000 comp rx=0 tx=0).stop 2024-06-03T02:24:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.597+0000 7ff775717b80 1 -- 172.21.15.79:0/1409080204 shutdown_connections 2024-06-03T02:24:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.597+0000 7ff775717b80 1 --2- 172.21.15.79:0/1409080204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f240cc0 0x555b1f230230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.597+0000 7ff775717b80 1 --2- 172.21.15.79:0/1409080204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f23aa50 0x555b1f239db0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.597+0000 7ff775717b80 1 --2- 172.21.15.79:0/1409080204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f2403a0 0x555b1f22fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.597+0000 7ff775717b80 1 -- 172.21.15.79:0/1409080204 >> 172.21.15.79:0/1409080204 conn(0x555b1f15e170 msgr2=0x555b1f226120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:42.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.597+0000 7ff775717b80 1 -- 172.21.15.79:0/1409080204 shutdown_connections 2024-06-03T02:24:42.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.597+0000 7ff775717b80 1 -- 172.21.15.79:0/1409080204 wait complete. 2024-06-03T02:24:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.597+0000 7ff775717b80 1 -- start start 2024-06-03T02:24:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.598+0000 7ff775717b80 1 Processor -- start 2024-06-03T02:24:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.598+0000 7ff775717b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f23aa50 0x555b1f244e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.598+0000 7ff775717b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f245360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.598+0000 7ff775717b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f240cc0 0x555b1f2ae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.598+0000 7ff775717b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff744010000 con 0x555b1f23aa50 2024-06-03T02:24:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.598+0000 7ff775717b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff74400f930 con 0x555b1f2403a0 2024-06-03T02:24:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.598+0000 7ff775717b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff74400cb30 con 0x555b1f240cc0 2024-06-03T02:24:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.599+0000 7ff775717b80 10 client.0 _ll_register_callbacks cb 0x555b1f119380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:42.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.599+0000 7ff763400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f245360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.599+0000 7ff763400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f245360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54446/0 (socket says 172.21.15.79:54446) 2024-06-03T02:24:42.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.599+0000 7ff763400700 1 -- 172.21.15.79:0/2690423119 learned_addr learned my addr 172.21.15.79:0/2690423119 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:42.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.599+0000 7ff763400700 1 -- 172.21.15.79:0/2690423119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f240cc0 msgr2=0x555b1f2ae460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:24:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.599+0000 7ff763400700 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f240cc0 0x555b1f2ae460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.599+0000 7ff763400700 1 -- 172.21.15.79:0/2690423119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f23aa50 msgr2=0x555b1f244e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.599+0000 7ff763400700 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f23aa50 0x555b1f244e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.599+0000 7ff763400700 1 -- 172.21.15.79:0/2690423119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1f2aea50 con 0x555b1f2403a0 2024-06-03T02:24:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.600+0000 7ff763400700 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f245360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff7500098e0 tx=0x7ff750009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96aad3b4f7b8f5d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.600+0000 7ff74e000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.600+0000 7ff74e000700 1 -- 172.21.15.79:0/2690423119 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750037060 con 0x555b1f2403a0 2024-06-03T02:24:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.600+0000 7ff74e000700 1 -- 172.21.15.79:0/2690423119 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff750003a90 con 0x555b1f2403a0 2024-06-03T02:24:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.600+0000 7ff775717b80 1 -- 172.21.15.79:0/2690423119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff75409dc70 con 0x555b1f2403a0 2024-06-03T02:24:42.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.601+0000 7ff74e000700 1 -- 172.21.15.79:0/2690423119 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff75000bea0 con 0x555b1f2403a0 2024-06-03T02:24:42.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.601+0000 7ff775717b80 10 client.25907 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:42.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.601+0000 7ff775717b80 1 -- 172.21.15.79:0/2690423119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b1f2b5e00 con 0x555b1f2403a0 2024-06-03T02:24:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.602+0000 7ff74e000700 1 -- 172.21.15.79:0/2690423119 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75409dc70 con 0x555b1f2403a0 2024-06-03T02:24:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.602+0000 7ff74e000700 1 -- 172.21.15.79:0/2690423119 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff75002f420 con 0x555b1f2403a0 2024-06-03T02:24:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.602+0000 7ff775717b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.602+0000 7ff775717b80 10 client.25907 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:24:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.602+0000 7ff775717b80 20 client.25907 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.602+0000 7ff775717b80 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555b1f2b6520 0x555b1f2b8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:42.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.603+0000 7ff775717b80 4 client.25907 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:24:42.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.603+0000 7ff775717b80 1 -- 172.21.15.79:0/2690423119 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b1f18b580 con 0x555b1f2b6520 2024-06-03T02:24:42.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.603+0000 7ff763e00700 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555b1f2b6520 0x555b1f2b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.607+0000 7ff763e00700 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555b1f2b6520 0x555b1f2b8940 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=402819d990d30dc7 server_cookie=38ef970c8f8dccd0 in_seq=0 out_seq=0 2024-06-03T02:24:42.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.607+0000 7ff74e000700 10 client.25907 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:24:42.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.607+0000 7ff74e000700 1 -- 172.21.15.79:0/2690423119 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b1f18b580 con 0x555b1f2b6520 2024-06-03T02:24:42.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.607+0000 7ff74e000700 10 client.25907 handle_command_reply: tid=0 2024-06-03T02:24:42.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.607+0000 7ff775717b80 1 client.25907 shutdown 2024-06-03T02:24:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.608+0000 7ff775717b80 1 -- 172.21.15.79:0/2690423119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 msgr2=0x555b1f245360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.608+0000 7ff775717b80 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f245360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff7500098e0 tx=0x7ff750009310 comp rx=0 tx=0).stop 2024-06-03T02:24:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.608+0000 7ff775717b80 1 -- 172.21.15.79:0/2690423119 shutdown_connections 2024-06-03T02:24:42.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.608+0000 7ff775717b80 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b1f240cc0 0x555b1f2ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.608+0000 7ff775717b80 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x555b1f2b6520 0x555b1f2b8940 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.608+0000 7ff775717b80 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555b1f23aa50 0x555b1f244e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.608+0000 7ff775717b80 1 --2- 172.21.15.79:0/2690423119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1f2403a0 0x555b1f245360 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.608+0000 7ff775717b80 1 -- 172.21.15.79:0/2690423119 >> 172.21.15.79:0/2690423119 conn(0x555b1f15e170 msgr2=0x555b1f2269b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.609+0000 7ff74e000700 0 client.25907 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:24:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.609+0000 7ff775717b80 1 -- 172.21.15.79:0/2690423119 shutdown_connections 2024-06-03T02:24:42.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.609+0000 7ff775717b80 1 -- 172.21.15.79:0/2690423119 wait complete. 2024-06-03T02:24:42.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.610+0000 7ff775717b80 20 client.25907 trim_cache size 0 max 16384 2024-06-03T02:24:42.612 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:24:42.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.612+0000 7ff76ae00700 1 -- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7480a3010 msgr2=0x7ff7480a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:42.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.612+0000 7ff76ae00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7480a3010 0x7ff7480a54d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff750009750 tx=0x7ff75000b280 comp rx=0 tx=0).stop 2024-06-03T02:24:42.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.613+0000 7ff76ae00700 1 -- 172.21.15.79:0/1010803982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff76406a170 msgr2=0x7ff764073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:42.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.613+0000 7ff76ae00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff76406a170 0x7ff764073f80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff754011ab0 tx=0x7ff754011dc0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.613+0000 7ff76ae00700 1 -- 172.21.15.79:0/1010803982 shutdown_connections 2024-06-03T02:24:42.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.613+0000 7ff76ae00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7480a3010 0x7ff7480a54d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.613+0000 7ff76ae00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff764108500 0x7ff76406c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.613+0000 7ff76ae00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff76406a170 0x7ff764073f80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.613+0000 7ff76ae00700 1 --2- 172.21.15.79:0/1010803982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76406ab30 0x7ff76406bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.613+0000 7ff76ae00700 1 -- 172.21.15.79:0/1010803982 >> 172.21.15.79:0/1010803982 conn(0x7ff764078340 msgr2=0x7ff764100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.614+0000 7ff76ae00700 1 -- 172.21.15.79:0/1010803982 shutdown_connections 2024-06-03T02:24:42.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:42.614+0000 7ff76ae00700 1 -- 172.21.15.79:0/1010803982 wait complete. 2024-06-03T02:24:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:42 smithi123 ceph-mon[36640]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.9 MiB/s wr, 703 op/s 2024-06-03T02:24:43.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 275.0 s, 1130.6 tps, lat 89.436 ms stddev 110.953 2024-06-03T02:24:43.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:24:44.198 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:44.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:44 smithi079 ceph-mon[31819]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 700 op/s 2024-06-03T02:24:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:44 smithi135 ceph-mon[36653]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 700 op/s 2024-06-03T02:24:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:44 smithi123 ceph-mon[36640]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 700 op/s 2024-06-03T02:24:46.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:46 smithi079 ceph-mon[31819]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 MiB/s wr, 690 op/s 2024-06-03T02:24:46.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:46 smithi135 ceph-mon[36653]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 MiB/s wr, 690 op/s 2024-06-03T02:24:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:46 smithi123 ceph-mon[36640]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 MiB/s wr, 690 op/s 2024-06-03T02:24:47.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.514+0000 7fb2fa800700 1 -- 172.21.15.79:0/684942671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2f4102ed0 msgr2=0x7fb2f410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:47.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.514+0000 7fb2fa800700 1 --2- 172.21.15.79:0/684942671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2f4102ed0 0x7fb2f410f330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00b0a0 tx=0x7fb2ec00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.514+0000 7fb2fa800700 1 -- 172.21.15.79:0/684942671 shutdown_connections 2024-06-03T02:24:47.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.514+0000 7fb2fa800700 1 --2- 172.21.15.79:0/684942671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2f4102ed0 0x7fb2f410f330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.514+0000 7fb2fa800700 1 --2- 172.21.15.79:0/684942671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f4108510 0x7fb2f41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.514+0000 7fb2fa800700 1 --2- 172.21.15.79:0/684942671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2f4102510 0x7fb2f4102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.514+0000 7fb2fa800700 1 -- 172.21.15.79:0/684942671 >> 172.21.15.79:0/684942671 conn(0x7fb2f4079560 msgr2=0x7fb2f4079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:47.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.514+0000 7fb2fa800700 1 -- 172.21.15.79:0/684942671 shutdown_connections 2024-06-03T02:24:47.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.514+0000 7fb2fa800700 1 -- 172.21.15.79:0/684942671 wait complete. 2024-06-03T02:24:47.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.516+0000 7fb2fa800700 1 Processor -- start 2024-06-03T02:24:47.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.516+0000 7fb2fa800700 1 -- start start 2024-06-03T02:24:47.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.517+0000 7fb2fa800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2f4102510 0x7fb2f4109320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.517+0000 7fb2fa800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2f4102ed0 0x7fb2f4109860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.517+0000 7fb2fa800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f4108510 0x7fb2f4109da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.517+0000 7fb2fa800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41127c0 con 0x7fb2f4102510 2024-06-03T02:24:47.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.517+0000 7fb2fa800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4112640 con 0x7fb2f4102ed0 2024-06-03T02:24:47.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.517+0000 7fb2fa800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4112940 con 0x7fb2f4108510 2024-06-03T02:24:47.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.517+0000 7fb2f3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2f4102510 0x7fb2f4109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.517+0000 7fb2f3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2f4102510 0x7fb2f4109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53028/0 (socket says 172.21.15.79:53028) 2024-06-03T02:24:47.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.517+0000 7fb2f3400700 1 -- 172.21.15.79:0/2351217573 learned_addr learned my addr 172.21.15.79:0/2351217573 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:47.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.518+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2f4102ed0 0x7fb2f4109860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.518+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f4108510 0x7fb2f4109da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.519+0000 7fb2f2a00700 1 -- 172.21.15.79:0/2351217573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f4108510 msgr2=0x7fb2f4109da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:47.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.519+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f4108510 0x7fb2f4109da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.519+0000 7fb2f2a00700 1 -- 172.21.15.79:0/2351217573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2f4102510 msgr2=0x7fb2f4109320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:47.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.519+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2f4102510 0x7fb2f4109320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.519+0000 7fb2f2a00700 1 -- 172.21.15.79:0/2351217573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f419abd0 con 0x7fb2f4102ed0 2024-06-03T02:24:47.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.521+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2f4102ed0 0x7fb2f4109860 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c9a0 tx=0x7fb2e000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:47.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.521+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2351217573 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0007ac0 con 0x7fb2f4102ed0 2024-06-03T02:24:47.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.521+0000 7fb2fa800700 1 -- 172.21.15.79:0/2351217573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f419aec0 con 0x7fb2f4102ed0 2024-06-03T02:24:47.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.521+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2351217573 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e0004d10 con 0x7fb2f4102ed0 2024-06-03T02:24:47.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.523+0000 7fb2fa800700 1 -- 172.21.15.79:0/2351217573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f419b450 con 0x7fb2f4102ed0 2024-06-03T02:24:47.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.523+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2351217573 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0007ac0 con 0x7fb2f4102ed0 2024-06-03T02:24:47.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.539+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2351217573 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb2e00053c0 con 0x7fb2f4102ed0 2024-06-03T02:24:47.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.539+0000 7fb305278b80 1 Processor -- start 2024-06-03T02:24:47.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.540+0000 7fb305278b80 1 -- start start 2024-06-03T02:24:47.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.540+0000 7fb305278b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e52f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.540+0000 7fb305278b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e534b0 0x55d9f1e4cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.540+0000 7fb305278b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e4d240 0x55d9f1e4d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.541+0000 7fb2f3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e534b0 0x55d9f1e4cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.541+0000 7fb2f3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e534b0 0x55d9f1e4cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53040/0 (socket says 172.21.15.79:53040) 2024-06-03T02:24:47.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.541+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e52f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.541+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e52f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54872/0 (socket says 172.21.15.79:54872) 2024-06-03T02:24:47.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.542+0000 7fb2f2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e4d240 0x55d9f1e4d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.542+0000 7fb2f2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e4d240 0x55d9f1e4d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:35466/0 (socket says 172.21.15.79:35466) 2024-06-03T02:24:47.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.542+0000 7fb305278b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d9f1d5ba10 con 0x55d9f1e534b0 2024-06-03T02:24:47.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.542+0000 7fb305278b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d9f1d9eb00 con 0x55d9f1e52b90 2024-06-03T02:24:47.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.542+0000 7fb305278b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9f1d598f0 con 0x55d9f1e4d240 2024-06-03T02:24:47.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.543+0000 7fb2f3400700 1 -- 172.21.15.79:0/4205623060 learned_addr learned my addr 172.21.15.79:0/4205623060 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:47.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.543+0000 7fb2f2a00700 1 -- 172.21.15.79:0/4205623060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 msgr2=0x55d9f1e52f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:47.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.543+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/4205623060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e52f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.543+0000 7fb2f2a00700 1 -- 172.21.15.79:0/4205623060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e534b0 msgr2=0x55d9f1e4cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:47.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.543+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/4205623060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e534b0 0x55d9f1e4cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.543+0000 7fb2f2a00700 1 -- 172.21.15.79:0/4205623060 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f1d46f90 con 0x55d9f1e4d240 2024-06-03T02:24:47.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.544+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/4205623060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e4d240 0x55d9f1e4d680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb2f419b670 tx=0x7fb2e00d4cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b64ea860f2fc114 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:47.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.544+0000 7fb2e8c00700 1 -- 172.21.15.79:0/4205623060 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00d2060 con 0x55d9f1e4d240 2024-06-03T02:24:47.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.544+0000 7fb2e8c00700 1 -- 172.21.15.79:0/4205623060 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e00d1e30 con 0x55d9f1e4d240 2024-06-03T02:24:47.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.544+0000 7fb2e8c00700 1 -- 172.21.15.79:0/4205623060 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00e03c0 con 0x55d9f1e4d240 2024-06-03T02:24:47.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.545+0000 7fb2ebe00700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d80a2ea0 0x7fb2d80a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.547+0000 7fb305278b80 1 -- 172.21.15.79:0/4205623060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e4d240 msgr2=0x55d9f1e4d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:47.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.547+0000 7fb305278b80 1 --2- 172.21.15.79:0/4205623060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e4d240 0x55d9f1e4d680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb2f419b670 tx=0x7fb2e00d4cc0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.547+0000 7fb2ebe00700 1 -- 172.21.15.79:0/2351217573 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb2e00d1170 con 0x7fb2f4102ed0 2024-06-03T02:24:47.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.548+0000 7fb305278b80 1 -- 172.21.15.79:0/4205623060 shutdown_connections 2024-06-03T02:24:47.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.548+0000 7fb305278b80 1 --2- 172.21.15.79:0/4205623060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e4d240 0x55d9f1e4d680 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.548+0000 7fb305278b80 1 --2- 172.21.15.79:0/4205623060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e52f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.548+0000 7fb305278b80 1 --2- 172.21.15.79:0/4205623060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e534b0 0x55d9f1e4cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.548+0000 7fb305278b80 1 -- 172.21.15.79:0/4205623060 >> 172.21.15.79:0/4205623060 conn(0x55d9f1d79170 msgr2=0x55d9f1e47ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:47.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.548+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d80a2ea0 0x7fb2d80a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.548+0000 7fb305278b80 1 -- 172.21.15.79:0/4205623060 shutdown_connections 2024-06-03T02:24:47.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.548+0000 7fb305278b80 1 -- 172.21.15.79:0/4205623060 wait complete. 2024-06-03T02:24:47.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.549+0000 7fb305278b80 1 -- start start 2024-06-03T02:24:47.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.549+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d80a2ea0 0x7fb2d80a5360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00b780 tx=0x7fb2ec009d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:47.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.549+0000 7fb305278b80 1 Processor -- start 2024-06-03T02:24:47.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.550+0000 7fb305278b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.550+0000 7fb305278b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.550+0000 7fb305278b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e534b0 0x55d9f1e56a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.550+0000 7fb305278b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d9f1d3af60 con 0x55d9f1e4d240 2024-06-03T02:24:47.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.550+0000 7fb305278b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d9f1d9aac0 con 0x55d9f1e52b90 2024-06-03T02:24:47.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.550+0000 7fb305278b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9f1e59e90 con 0x55d9f1e534b0 2024-06-03T02:24:47.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.550+0000 7fb305278b80 10 client.0 _ll_register_callbacks cb 0x55d9f1d34380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:47.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.551+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e534b0 0x55d9f1e56a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.551+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e534b0 0x55d9f1e56a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:35472/0 (socket says 172.21.15.79:35472) 2024-06-03T02:24:47.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.551+0000 7fb2f3e00700 1 -- 172.21.15.79:0/2705271638 learned_addr learned my addr 172.21.15.79:0/2705271638 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:47.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.551+0000 7fb2f3400700 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.552+0000 7fb2f3e00700 1 -- 172.21.15.79:0/2705271638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 msgr2=0x55d9f1e564d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:24:47.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.554+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.554+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.554+0000 7fb2f3e00700 1 -- 172.21.15.79:0/2705271638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 msgr2=0x55d9f1e591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:47.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.554+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.554+0000 7fb2f3e00700 1 -- 172.21.15.79:0/2705271638 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9f1ced8d0 con 0x55d9f1e534b0 2024-06-03T02:24:47.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.555+0000 7fb2f3400700 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:24:47.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.555+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:24:47.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.556+0000 7fb2dd600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:24:47.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.557+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e534b0 0x55d9f1e56a10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec004ce0 tx=0x7fb2ec00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d58ac8c5bedec6c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:47.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.557+0000 7fb2dd600700 1 -- 172.21.15.79:0/2705271638 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec02c030 con 0x55d9f1e534b0 2024-06-03T02:24:47.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.557+0000 7fb2dd600700 1 -- 172.21.15.79:0/2705271638 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb2ec02d070 con 0x55d9f1e534b0 2024-06-03T02:24:47.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.558+0000 7fb305278b80 1 -- 172.21.15.79:0/2705271638 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e00d1e30 con 0x55d9f1e534b0 2024-06-03T02:24:47.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.559+0000 7fb2dd600700 1 -- 172.21.15.79:0/2705271638 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2ec02c030 con 0x55d9f1e534b0 2024-06-03T02:24:47.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.559+0000 7fb305278b80 10 client.25897 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:47.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.559+0000 7fb305278b80 1 -- 172.21.15.79:0/2705271638 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9f1e64620 con 0x55d9f1e534b0 2024-06-03T02:24:47.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.560+0000 7fb2dd600700 1 -- 172.21.15.79:0/2705271638 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2ec009b80 con 0x55d9f1e534b0 2024-06-03T02:24:47.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.560+0000 7fb2dd600700 1 -- 172.21.15.79:0/2705271638 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb2ec03a020 con 0x55d9f1e534b0 2024-06-03T02:24:47.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.561+0000 7fb305278b80 10 client.25897 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:47.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.561+0000 7fb305278b80 10 client.25897 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:24:47.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.561+0000 7fb305278b80 20 client.25897 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:47.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.561+0000 7fb305278b80 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d9f1e66d00 0x55d9f1e690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.561+0000 7fb305278b80 4 client.25897 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:24:47.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.561+0000 7fb305278b80 1 -- 172.21.15.79:0/2705271638 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb2e00d1e30 con 0x55d9f1e66d00 2024-06-03T02:24:47.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.563+0000 7fb2f3400700 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d9f1e66d00 0x55d9f1e690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.563+0000 7fb2dd600700 10 client.25897 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.563+0000 7fb2f3400700 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d9f1e66d00 0x55d9f1e690f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffecfc763fb57668 server_cookie=84fe50db14e5c444 in_seq=0 out_seq=0 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.564+0000 7fb2dd600700 1 -- 172.21.15.79:0/2705271638 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fb2e00d1e30 con 0x55d9f1e66d00 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.564+0000 7fb2dd600700 10 client.25897 handle_command_reply: tid=0 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.565+0000 7fb305278b80 1 client.25897 shutdown 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.565+0000 7fb305278b80 1 -- 172.21.15.79:0/2705271638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e534b0 msgr2=0x55d9f1e56a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.565+0000 7fb305278b80 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e534b0 0x55d9f1e56a10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec004ce0 tx=0x7fb2ec00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.565+0000 7fb305278b80 1 -- 172.21.15.79:0/2705271638 shutdown_connections 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.565+0000 7fb305278b80 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e534b0 0x55d9f1e56a10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.565+0000 7fb305278b80 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.567+0000 7fb305278b80 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d9f1e66d00 0x55d9f1e690f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.567+0000 7fb305278b80 1 --2- 172.21.15.79:0/2705271638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.567+0000 7fb2dd600700 0 client.25897 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:24:47.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.567+0000 7fb305278b80 1 -- 172.21.15.79:0/2705271638 >> 172.21.15.79:0/2705271638 conn(0x55d9f1d79170 msgr2=0x55d9f1e41790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:47.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.567+0000 7fb305278b80 1 -- 172.21.15.79:0/2705271638 shutdown_connections 2024-06-03T02:24:47.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.567+0000 7fb305278b80 1 -- 172.21.15.79:0/2705271638 wait complete. 2024-06-03T02:24:47.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.567+0000 7fb305278b80 20 client.25897 trim_cache size 0 max 16384 2024-06-03T02:24:47.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.601+0000 7fb305278b80 1 Processor -- start 2024-06-03T02:24:47.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.601+0000 7fb305278b80 1 -- start start 2024-06-03T02:24:47.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.601+0000 7fb305278b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1d79170 0x55d9f1e4cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.601+0000 7fb305278b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e4d240 0x55d9f1e42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.601+0000 7fb305278b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e52b90 0x55d9f1e56a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.601+0000 7fb305278b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d400ebf0 con 0x55d9f1d79170 2024-06-03T02:24:47.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.601+0000 7fb305278b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d4010730 con 0x55d9f1e4d240 2024-06-03T02:24:47.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.601+0000 7fb305278b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d400f2c0 con 0x55d9f1e52b90 2024-06-03T02:24:47.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.602+0000 7fb2f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e4d240 0x55d9f1e42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.602+0000 7fb2f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e4d240 0x55d9f1e42130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54886/0 (socket says 172.21.15.79:54886) 2024-06-03T02:24:47.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.602+0000 7fb2f2a00700 1 -- 172.21.15.79:0/2504640891 learned_addr learned my addr 172.21.15.79:0/2504640891 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:47.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.604+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2504640891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e52b90 0x55d9f1e56a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.604+0000 7fb2f3400700 1 --2- 172.21.15.79:0/2504640891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1d79170 0x55d9f1e4cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.604+0000 7fb2f2a00700 1 -- 172.21.15.79:0/2504640891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e52b90 msgr2=0x55d9f1e56a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:47.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.604+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2504640891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e52b90 0x55d9f1e56a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.604+0000 7fb2f2a00700 1 -- 172.21.15.79:0/2504640891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1d79170 msgr2=0x55d9f1e4cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:47.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.604+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2504640891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1d79170 0x55d9f1e4cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.604+0000 7fb2f2a00700 1 -- 172.21.15.79:0/2504640891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f1e69770 con 0x55d9f1e4d240 2024-06-03T02:24:47.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.604+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2504640891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e4d240 0x55d9f1e42130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00a2a30 tx=0x7fb2e0005930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c2027f195c015f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:47.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.606+0000 7fb2dfe00700 1 -- 172.21.15.79:0/2504640891 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e009a280 con 0x55d9f1e4d240 2024-06-03T02:24:47.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.606+0000 7fb2dfe00700 1 -- 172.21.15.79:0/2504640891 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e00a2f60 con 0x55d9f1e4d240 2024-06-03T02:24:47.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.606+0000 7fb2dfe00700 1 -- 172.21.15.79:0/2504640891 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0099e10 con 0x55d9f1e4d240 2024-06-03T02:24:47.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.607+0000 7fb305278b80 1 -- 172.21.15.79:0/2504640891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e4d240 msgr2=0x55d9f1e42130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:47.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.607+0000 7fb305278b80 1 --2- 172.21.15.79:0/2504640891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e4d240 0x55d9f1e42130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00a2a30 tx=0x7fb2e0005930 comp rx=0 tx=0).stop 2024-06-03T02:24:47.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.607+0000 7fb305278b80 1 -- 172.21.15.79:0/2504640891 shutdown_connections 2024-06-03T02:24:47.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.607+0000 7fb305278b80 1 --2- 172.21.15.79:0/2504640891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1e52b90 0x55d9f1e56a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.607+0000 7fb305278b80 1 --2- 172.21.15.79:0/2504640891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1d79170 0x55d9f1e4cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.607+0000 7fb305278b80 1 --2- 172.21.15.79:0/2504640891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e4d240 0x55d9f1e42130 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.607+0000 7fb305278b80 1 -- 172.21.15.79:0/2504640891 >> 172.21.15.79:0/2504640891 conn(0x55d9f1e4dbd0 msgr2=0x55d9f1e53950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:47.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.608+0000 7fb305278b80 1 -- 172.21.15.79:0/2504640891 shutdown_connections 2024-06-03T02:24:47.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.608+0000 7fb305278b80 1 -- 172.21.15.79:0/2504640891 wait complete. 2024-06-03T02:24:47.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.609+0000 7fb305278b80 1 -- start start 2024-06-03T02:24:47.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.609+0000 7fb305278b80 1 Processor -- start 2024-06-03T02:24:47.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.610+0000 7fb305278b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1d79170 0x55d9f1e6e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.610+0000 7fb305278b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e6e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.610+0000 7fb305278b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e6eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.610+0000 7fb305278b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d4010060 con 0x55d9f1e4d240 2024-06-03T02:24:47.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.610+0000 7fb305278b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d400f990 con 0x55d9f1e52b90 2024-06-03T02:24:47.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.610+0000 7fb305278b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d400cb90 con 0x55d9f1d79170 2024-06-03T02:24:47.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.610+0000 7fb305278b80 10 client.0 _ll_register_callbacks cb 0x55d9f1d34380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:47.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1d79170 0x55d9f1e6e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1d79170 0x55d9f1e6e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:35492/0 (socket says 172.21.15.79:35492) 2024-06-03T02:24:47.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f3400700 1 -- 172.21.15.79:0/2394984477 learned_addr learned my addr 172.21.15.79:0/2394984477 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:47.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e6e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e6eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f3e00700 1 -- 172.21.15.79:0/2394984477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1d79170 msgr2=0x55d9f1e6e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:47.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1d79170 0x55d9f1e6e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f3e00700 1 -- 172.21.15.79:0/2394984477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 msgr2=0x55d9f1e6e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:47.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e6e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.611+0000 7fb2f3e00700 1 -- 172.21.15.79:0/2394984477 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9f1e49760 con 0x55d9f1e52b90 2024-06-03T02:24:47.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.612+0000 7fb2f2a00700 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e6e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:47.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.613+0000 7fb2f3e00700 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e6eea0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005090 tx=0x7fb2ec005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daed87fd554378d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:47.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.613+0000 7fb2dd600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:47.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.613+0000 7fb2dd600700 1 -- 172.21.15.79:0/2394984477 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0059c0 con 0x55d9f1e52b90 2024-06-03T02:24:47.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.613+0000 7fb2dd600700 1 -- 172.21.15.79:0/2394984477 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb2ec00f020 con 0x55d9f1e52b90 2024-06-03T02:24:47.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.613+0000 7fb305278b80 1 -- 172.21.15.79:0/2394984477 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e00a2f60 con 0x55d9f1e52b90 2024-06-03T02:24:47.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.614+0000 7fb2dd600700 1 -- 172.21.15.79:0/2394984477 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2ec007870 con 0x55d9f1e52b90 2024-06-03T02:24:47.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.614+0000 7fb305278b80 10 client.25922 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:47.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.614+0000 7fb305278b80 1 -- 172.21.15.79:0/2394984477 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9f1ed1910 con 0x55d9f1e52b90 2024-06-03T02:24:47.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.615+0000 7fb2dd600700 1 -- 172.21.15.79:0/2394984477 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e00a2f60 con 0x55d9f1e52b90 2024-06-03T02:24:47.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.615+0000 7fb2dd600700 1 -- 172.21.15.79:0/2394984477 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb2ec03a070 con 0x55d9f1e52b90 2024-06-03T02:24:47.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.615+0000 7fb305278b80 10 client.25922 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:47.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.615+0000 7fb305278b80 10 client.25922 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:24:47.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.615+0000 7fb305278b80 20 client.25922 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:47.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.615+0000 7fb305278b80 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d9f1e534b0 0x55d9f1ed3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:47.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.615+0000 7fb305278b80 4 client.25922 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:24:47.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.615+0000 7fb305278b80 1 -- 172.21.15.79:0/2394984477 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9f1dac000 con 0x55d9f1e534b0 2024-06-03T02:24:47.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.615+0000 7fb2f3400700 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d9f1e534b0 0x55d9f1ed3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:47.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.616+0000 7fb2f3400700 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d9f1e534b0 0x55d9f1ed3f20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f318649f408e8019 server_cookie=bd731e6b2bda9d18 in_seq=0 out_seq=0 2024-06-03T02:24:47.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.616+0000 7fb2dd600700 10 client.25922 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:24:47.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.616+0000 7fb2dd600700 1 -- 172.21.15.79:0/2394984477 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9f1dac000 con 0x55d9f1e534b0 2024-06-03T02:24:47.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.616+0000 7fb2dd600700 10 client.25922 handle_command_reply: tid=0 2024-06-03T02:24:47.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.616+0000 7fb305278b80 1 client.25922 shutdown 2024-06-03T02:24:47.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.617+0000 7fb305278b80 1 -- 172.21.15.79:0/2394984477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 msgr2=0x55d9f1e6eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:47.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.617+0000 7fb305278b80 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e6eea0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005090 tx=0x7fb2ec005f10 comp rx=0 tx=0).stop 2024-06-03T02:24:47.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.617+0000 7fb305278b80 1 -- 172.21.15.79:0/2394984477 shutdown_connections 2024-06-03T02:24:47.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.617+0000 7fb305278b80 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d9f1e534b0 0x55d9f1ed3f20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.617+0000 7fb305278b80 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9f1e52b90 0x55d9f1e6eea0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.617+0000 7fb305278b80 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9f1d79170 0x55d9f1e6e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.617+0000 7fb305278b80 1 --2- 172.21.15.79:0/2394984477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d9f1e4d240 0x55d9f1e6e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.617+0000 7fb305278b80 1 -- 172.21.15.79:0/2394984477 >> 172.21.15.79:0/2394984477 conn(0x55d9f1e4dbd0 msgr2=0x55d9f1e57d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:47.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.618+0000 7fb2dd600700 0 client.25922 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:24:47.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.618+0000 7fb305278b80 1 -- 172.21.15.79:0/2394984477 shutdown_connections 2024-06-03T02:24:47.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.618+0000 7fb305278b80 1 -- 172.21.15.79:0/2394984477 wait complete. 2024-06-03T02:24:47.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.618+0000 7fb305278b80 20 client.25922 trim_cache size 0 max 16384 2024-06-03T02:24:47.619 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:24:47.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.619+0000 7fb2fa800700 1 -- 172.21.15.79:0/2351217573 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d80a2ea0 msgr2=0x7fb2d80a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:47.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.619+0000 7fb2fa800700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d80a2ea0 0x7fb2d80a5360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00b780 tx=0x7fb2ec009d60 comp rx=0 tx=0).stop 2024-06-03T02:24:47.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.619+0000 7fb2fa800700 1 -- 172.21.15.79:0/2351217573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2f4102ed0 msgr2=0x7fb2f4109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:47.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.619+0000 7fb2fa800700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2f4102ed0 0x7fb2f4109860 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c9a0 tx=0x7fb2e000ccb0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.620+0000 7fb2fa800700 1 -- 172.21.15.79:0/2351217573 shutdown_connections 2024-06-03T02:24:47.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.620+0000 7fb2fa800700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f4108510 0x7fb2f4109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.620+0000 7fb2fa800700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2d80a2ea0 0x7fb2d80a5360 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.620+0000 7fb2fa800700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2f4102510 0x7fb2f4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.620+0000 7fb2fa800700 1 --2- 172.21.15.79:0/2351217573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2f4102ed0 0x7fb2f4109860 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:47.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.620+0000 7fb2fa800700 1 -- 172.21.15.79:0/2351217573 >> 172.21.15.79:0/2351217573 conn(0x7fb2f4079560 msgr2=0x7fb2f40748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:47.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.620+0000 7fb2fa800700 1 -- 172.21.15.79:0/2351217573 shutdown_connections 2024-06-03T02:24:47.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:47.621+0000 7fb2fa800700 1 -- 172.21.15.79:0/2351217573 wait complete. 2024-06-03T02:24:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 280.0 s, 1008.0 tps, lat 99.195 ms stddev 122.845 2024-06-03T02:24:48.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:24:48.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:48 smithi135 ceph-mon[36653]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2024-06-03T02:24:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:48 smithi123 ceph-mon[36640]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2024-06-03T02:24:48.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:48 smithi079 ceph-mon[31819]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2024-06-03T02:24:49.053 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:50.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:50 smithi079 ceph-mon[31819]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 MiB/s wr, 647 op/s 2024-06-03T02:24:50.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:50 smithi135 ceph-mon[36653]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 MiB/s wr, 647 op/s 2024-06-03T02:24:50.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:50 smithi123 ceph-mon[36640]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 MiB/s wr, 647 op/s 2024-06-03T02:24:52.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.451+0000 7f37b0000700 1 -- 172.21.15.79:0/3882688337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8102740 msgr2=0x7f37a8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:52.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.451+0000 7f37b0000700 1 --2- 172.21.15.79:0/3882688337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8102740 0x7f37a8102b20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f379c0099c0 tx=0x7f379c009cd0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.452+0000 7f37b0000700 1 -- 172.21.15.79:0/3882688337 shutdown_connections 2024-06-03T02:24:52.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.452+0000 7f37b0000700 1 --2- 172.21.15.79:0/3882688337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a8107840 0x7f37a810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.452+0000 7f37b0000700 1 --2- 172.21.15.79:0/3882688337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8102740 0x7f37a8102b20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.452+0000 7f37b0000700 1 --2- 172.21.15.79:0/3882688337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a81030f0 0x7f37a8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.452+0000 7f37b0000700 1 -- 172.21.15.79:0/3882688337 >> 172.21.15.79:0/3882688337 conn(0x7f37a80795f0 msgr2=0x7f37a8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:52.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.452+0000 7f37b0000700 1 -- 172.21.15.79:0/3882688337 shutdown_connections 2024-06-03T02:24:52.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.453+0000 7f37b0000700 1 -- 172.21.15.79:0/3882688337 wait complete. 2024-06-03T02:24:52.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.455+0000 7f37b0000700 1 Processor -- start 2024-06-03T02:24:52.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.457+0000 7f37b0000700 1 -- start start 2024-06-03T02:24:52.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.458+0000 7f37b0000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8102740 0x7f37a8073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.458+0000 7f37b0000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a81030f0 0x7f37a8074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.458+0000 7f37b0000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a8107840 0x7f37a806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.458+0000 7f37b0000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f37a8110660 con 0x7f37a8107840 2024-06-03T02:24:52.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.458+0000 7f37b0000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37a81104e0 con 0x7f37a8102740 2024-06-03T02:24:52.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.458+0000 7f37b0000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f37a81107e0 con 0x7f37a81030f0 2024-06-03T02:24:52.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37acc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a81030f0 0x7f37a8074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ae000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a8107840 0x7f37a806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ad600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8102740 0x7f37a8073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ae000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a8107840 0x7f37a806ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53086/0 (socket says 172.21.15.79:53086) 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ae000700 1 -- 172.21.15.79:0/95414704 learned_addr learned my addr 172.21.15.79:0/95414704 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ae000700 1 -- 172.21.15.79:0/95414704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a81030f0 msgr2=0x7f37a8074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ae000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a81030f0 0x7f37a8074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ae000700 1 -- 172.21.15.79:0/95414704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8102740 msgr2=0x7f37a8073b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ae000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8102740 0x7f37a8073b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ae000700 1 -- 172.21.15.79:0/95414704 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a806d5f0 con 0x7f37a8107840 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.459+0000 7f37ad600700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8102740 0x7f37a8073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:52.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.460+0000 7f37ae000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a8107840 0x7f37a806ccf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f37a0007ee0 tx=0x7f37a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:52.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.460+0000 7f37a6000700 1 -- 172.21.15.79:0/95414704 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a000dbd0 con 0x7f37a8107840 2024-06-03T02:24:52.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.461+0000 7f37b0000700 1 -- 172.21.15.79:0/95414704 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a806d8e0 con 0x7f37a8107840 2024-06-03T02:24:52.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.461+0000 7f37b0000700 1 -- 172.21.15.79:0/95414704 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a8071870 con 0x7f37a8107840 2024-06-03T02:24:52.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.461+0000 7f37a6000700 1 -- 172.21.15.79:0/95414704 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a000dd70 con 0x7f37a8107840 2024-06-03T02:24:52.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.461+0000 7f37a6000700 1 -- 172.21.15.79:0/95414704 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a00059f0 con 0x7f37a8107840 2024-06-03T02:24:52.464 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:52 smithi079 ceph-mon[31819]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 MiB/s wr, 547 op/s 2024-06-03T02:24:52.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.467+0000 7f37a6000700 1 -- 172.21.15.79:0/95414704 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f37a001a020 con 0x7f37a8107840 2024-06-03T02:24:52.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.469+0000 7f37a6000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37900a7080 0x7f37900a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.469+0000 7f37a6000700 1 -- 172.21.15.79:0/95414704 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f37a00990d0 con 0x7f37a8107840 2024-06-03T02:24:52.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.470+0000 7f37ad600700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37900a7080 0x7f37900a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.471+0000 7f37ad600700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37900a7080 0x7f37900a9540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f379c009f50 tx=0x7f379c005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:52.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.471+0000 7f37ba8d6b80 1 Processor -- start 2024-06-03T02:24:52.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.472+0000 7f37ba8d6b80 1 -- start start 2024-06-03T02:24:52.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.472+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee873a0 0x55bd9ee87780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.472+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.472+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee81a50 0x55bd9ee81e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.472+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bd9ed87a10 con 0x55bd9ee87cc0 2024-06-03T02:24:52.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.472+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bd9edcab00 con 0x55bd9ee873a0 2024-06-03T02:24:52.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.472+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bd9ed858f0 con 0x55bd9ee81a50 2024-06-03T02:24:52.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.473+0000 7f37ad600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee873a0 0x55bd9ee87780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.473+0000 7f37ad600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee873a0 0x55bd9ee87780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54924/0 (socket says 172.21.15.79:54924) 2024-06-03T02:24:52.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.473+0000 7f37ae000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee81a50 0x55bd9ee81e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.473+0000 7f37ad600700 1 -- 172.21.15.79:0/2974765602 learned_addr learned my addr 172.21.15.79:0/2974765602 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:52.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.473+0000 7f37acc00700 1 --2- 172.21.15.79:0/2974765602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee81510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.473+0000 7f37ae000700 1 -- 172.21.15.79:0/2974765602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee873a0 msgr2=0x55bd9ee87780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:52.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.473+0000 7f37ae000700 1 --2- 172.21.15.79:0/2974765602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee873a0 0x55bd9ee87780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.473+0000 7f37ae000700 1 -- 172.21.15.79:0/2974765602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 msgr2=0x55bd9ee81510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:52.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.473+0000 7f37ae000700 1 --2- 172.21.15.79:0/2974765602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee81510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.474+0000 7f37ae000700 1 -- 172.21.15.79:0/2974765602 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd9ed72f90 con 0x55bd9ee81a50 2024-06-03T02:24:52.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.474+0000 7f37ae000700 1 --2- 172.21.15.79:0/2974765602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee81a50 0x55bd9ee81e90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f37a0095b00 tx=0x7f37a00a26f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1234526d894e307a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:52.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.476+0000 7f3796a00700 1 -- 172.21.15.79:0/2974765602 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0097a60 con 0x55bd9ee81a50 2024-06-03T02:24:52.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.476+0000 7f3796a00700 1 -- 172.21.15.79:0/2974765602 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a009b730 con 0x55bd9ee81a50 2024-06-03T02:24:52.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.477+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2974765602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee81a50 msgr2=0x55bd9ee81e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:52.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.477+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2974765602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee81a50 0x55bd9ee81e90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f37a0095b00 tx=0x7f37a00a26f0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.478+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2974765602 shutdown_connections 2024-06-03T02:24:52.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.478+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2974765602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee81a50 0x55bd9ee81e90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.478+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2974765602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee873a0 0x55bd9ee87780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.479+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2974765602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee81510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.479+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2974765602 >> 172.21.15.79:0/2974765602 conn(0x55bd9eda5170 msgr2=0x55bd9ee73ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:52.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.480+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2974765602 shutdown_connections 2024-06-03T02:24:52.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.480+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2974765602 wait complete. 2024-06-03T02:24:52.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.481+0000 7f37ba8d6b80 1 -- start start 2024-06-03T02:24:52.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.482+0000 7f37ba8d6b80 1 Processor -- start 2024-06-03T02:24:52.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.483+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee81a50 0x55bd9ee8d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.483+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee873a0 0x55bd9ee8acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.483+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee8b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.483+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bd9ed66f60 con 0x55bd9ee87cc0 2024-06-03T02:24:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.483+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bd9edc6ac0 con 0x55bd9ee81a50 2024-06-03T02:24:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.483+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bd9ee8e660 con 0x55bd9ee873a0 2024-06-03T02:24:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.483+0000 7f37ba8d6b80 10 client.0 _ll_register_callbacks cb 0x55bd9ed60380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37acc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee873a0 0x55bd9ee8acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37ad600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee81a50 0x55bd9ee8d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37ae000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee8b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37acc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee873a0 0x55bd9ee8acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:35510/0 (socket says 172.21.15.79:35510) 2024-06-03T02:24:52.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37acc00700 1 -- 172.21.15.79:0/3956209129 learned_addr learned my addr 172.21.15.79:0/3956209129 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:52.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37acc00700 1 -- 172.21.15.79:0/3956209129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee81a50 msgr2=0x55bd9ee8d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:52.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37acc00700 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee81a50 0x55bd9ee8d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37acc00700 1 -- 172.21.15.79:0/3956209129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 msgr2=0x55bd9ee8b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37acc00700 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee8b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.484+0000 7f37acc00700 1 -- 172.21.15.79:0/3956209129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd9ed198d0 con 0x55bd9ee873a0 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.485+0000 7f37acc00700 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee873a0 0x55bd9ee8acf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f37980106a0 tx=0x7f37980109b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=161b414f564ce7cf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.485+0000 7f378b400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.485+0000 7f378b400700 1 -- 172.21.15.79:0/3956209129 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798004a30 con 0x55bd9ee873a0 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.485+0000 7f378b400700 1 -- 172.21.15.79:0/3956209129 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3798004bd0 con 0x55bd9ee873a0 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.485+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/3956209129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a009b730 con 0x55bd9ee873a0 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f378b400700 1 -- 172.21.15.79:0/3956209129 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3798021050 con 0x55bd9ee873a0 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f37ba8d6b80 10 client.25912 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:52.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/3956209129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd9ee98eb0 con 0x55bd9ee873a0 2024-06-03T02:24:52.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f378b400700 1 -- 172.21.15.79:0/3956209129 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a009b730 con 0x55bd9ee873a0 2024-06-03T02:24:52.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f378b400700 1 -- 172.21.15.79:0/3956209129 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f379800e030 con 0x55bd9ee873a0 2024-06-03T02:24:52.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f37ba8d6b80 10 client.25912 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:52.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f37ba8d6b80 10 client.25912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:24:52.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f37ba8d6b80 20 client.25912 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:52.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55bd9ee9b4f0 0x55bd9ee9d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f37ba8d6b80 4 client.25912 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:24:52.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.486+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/3956209129 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd9edd8000 con 0x55bd9ee9b4f0 2024-06-03T02:24:52.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.487+0000 7f37ad600700 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55bd9ee9b4f0 0x55bd9ee9d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.488+0000 7f378b400700 10 client.25912 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:24:52.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.488+0000 7f37ad600700 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55bd9ee9b4f0 0x55bd9ee9d8e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdfc3aa3fcedaf79 server_cookie=94602813184fb23b in_seq=0 out_seq=0 2024-06-03T02:24:52.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.490+0000 7f378b400700 1 -- 172.21.15.79:0/3956209129 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bd9edd8000 con 0x55bd9ee9b4f0 2024-06-03T02:24:52.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.490+0000 7f378b400700 10 client.25912 handle_command_reply: tid=0 2024-06-03T02:24:52.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f37ba8d6b80 1 client.25912 shutdown 2024-06-03T02:24:52.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/3956209129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee873a0 msgr2=0x55bd9ee8acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:52.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee873a0 0x55bd9ee8acf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f37980106a0 tx=0x7f37980109b0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/3956209129 shutdown_connections 2024-06-03T02:24:52.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee8b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee81a50 0x55bd9ee8d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55bd9ee9b4f0 0x55bd9ee9d8e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/3956209129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee873a0 0x55bd9ee8acf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/3956209129 >> 172.21.15.79:0/3956209129 conn(0x55bd9eda5170 msgr2=0x55bd9ee6d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:52.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.491+0000 7f378b400700 0 client.25912 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:24:52.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.492+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/3956209129 shutdown_connections 2024-06-03T02:24:52.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.492+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/3956209129 wait complete. 2024-06-03T02:24:52.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.492+0000 7f37ba8d6b80 20 client.25912 trim_cache size 0 max 16384 2024-06-03T02:24:52.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.525+0000 7f37ba8d6b80 1 Processor -- start 2024-06-03T02:24:52.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.525+0000 7f37ba8d6b80 1 -- start start 2024-06-03T02:24:52.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.525+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee81a50 0x55bd9ee81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.525+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.525+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9ee6e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.525+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f378c00eba0 con 0x55bd9ee87cc0 2024-06-03T02:24:52.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.525+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0106e0 con 0x55bd9ee81a50 2024-06-03T02:24:52.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.525+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f378c00f270 con 0x55bd9ee85540 2024-06-03T02:24:52.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.526+0000 7f37ae000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9ee6e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.526+0000 7f37ae000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9ee6e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:35516/0 (socket says 172.21.15.79:35516) 2024-06-03T02:24:52.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.526+0000 7f37ae000700 1 -- 172.21.15.79:0/2839318714 learned_addr learned my addr 172.21.15.79:0/2839318714 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:52.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.527+0000 7f37ae000700 1 -- 172.21.15.79:0/2839318714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee81a50 msgr2=0x55bd9ee81510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:24:52.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.527+0000 7f37ae000700 1 --2- 172.21.15.79:0/2839318714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee81a50 0x55bd9ee81510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.527+0000 7f37acc00700 1 --2- 172.21.15.79:0/2839318714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.528+0000 7f37ae000700 1 -- 172.21.15.79:0/2839318714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 msgr2=0x55bd9ee741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:52.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.528+0000 7f37ae000700 1 --2- 172.21.15.79:0/2839318714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee741a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.528+0000 7f37ae000700 1 -- 172.21.15.79:0/2839318714 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd9ee89260 con 0x55bd9ee85540 2024-06-03T02:24:52.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.528+0000 7f37acc00700 1 --2- 172.21.15.79:0/2839318714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:24:52.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.529+0000 7f37ae000700 1 --2- 172.21.15.79:0/2839318714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9ee6e130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f37a00061e0 tx=0x7f37a000df20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d9b82342296ede3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:52.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.529+0000 7f3796000700 1 -- 172.21.15.79:0/2839318714 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0097da0 con 0x55bd9ee85540 2024-06-03T02:24:52.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.530+0000 7f3796000700 1 -- 172.21.15.79:0/2839318714 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a0099330 con 0x55bd9ee85540 2024-06-03T02:24:52.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.530+0000 7f3796000700 1 -- 172.21.15.79:0/2839318714 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0097f70 con 0x55bd9ee85540 2024-06-03T02:24:52.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.532+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2839318714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 msgr2=0x55bd9ee6e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:52.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.533+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2839318714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9ee6e130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f37a00061e0 tx=0x7f37a000df20 comp rx=0 tx=0).stop 2024-06-03T02:24:52.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.535+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2839318714 shutdown_connections 2024-06-03T02:24:52.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.535+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2839318714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9ee6e130 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.535+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2839318714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee81a50 0x55bd9ee81510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.536+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2839318714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee87cc0 0x55bd9ee741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.536+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2839318714 >> 172.21.15.79:0/2839318714 conn(0x55bd9eda5170 msgr2=0x55bd9ee73a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:52.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.538+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2839318714 shutdown_connections 2024-06-03T02:24:52.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.538+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2839318714 wait complete. 2024-06-03T02:24:52.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.539+0000 7f37ba8d6b80 1 -- start start 2024-06-03T02:24:52.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.539+0000 7f37ba8d6b80 1 Processor -- start 2024-06-03T02:24:52.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.540+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee81a50 0x55bd9ee93e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.540+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9eea2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.541+0000 7f37ba8d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee87cc0 0x55bd9eea2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.541+0000 7f37ba8d6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f378c010010 con 0x55bd9ee81a50 2024-06-03T02:24:52.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.541+0000 7f37ae000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee87cc0 0x55bd9eea2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.541+0000 7f37ae000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee87cc0 0x55bd9eea2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54958/0 (socket says 172.21.15.79:54958) 2024-06-03T02:24:52.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.541+0000 7f37ae000700 1 -- 172.21.15.79:0/2518408773 learned_addr learned my addr 172.21.15.79:0/2518408773 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:52.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.542+0000 7f37acc00700 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9eea2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.542+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f378c00f940 con 0x55bd9ee87cc0 2024-06-03T02:24:52.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.542+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f378c00cb40 con 0x55bd9ee85540 2024-06-03T02:24:52.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.542+0000 7f37ba8d6b80 10 client.0 _ll_register_callbacks cb 0x55bd9ed60380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:52.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.542+0000 7f37ae000700 1 -- 172.21.15.79:0/2518408773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 msgr2=0x55bd9eea2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:52.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.543+0000 7f37ae000700 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9eea2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.543+0000 7f37ae000700 1 -- 172.21.15.79:0/2518408773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee81a50 msgr2=0x55bd9ee93e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:52.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.543+0000 7f37ae000700 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee81a50 0x55bd9ee93e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.543+0000 7f37ae000700 1 -- 172.21.15.79:0/2518408773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd9eea33b0 con 0x55bd9ee87cc0 2024-06-03T02:24:52.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.543+0000 7f378b400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:52.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.543+0000 7f37ae000700 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee87cc0 0x55bd9eea2e40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f37a0097080 tx=0x7f37a00a3de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e33e0e6c7169f00 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:52.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.544+0000 7f378b400700 1 -- 172.21.15.79:0/2518408773 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0098110 con 0x55bd9ee87cc0 2024-06-03T02:24:52.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.544+0000 7f378b400700 1 -- 172.21.15.79:0/2518408773 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f37a00a0d80 con 0x55bd9ee87cc0 2024-06-03T02:24:52.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.544+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a0099330 con 0x55bd9ee87cc0 2024-06-03T02:24:52.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.544+0000 7f378b400700 1 -- 172.21.15.79:0/2518408773 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37a007e050 con 0x55bd9ee87cc0 2024-06-03T02:24:52.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.546+0000 7f37ba8d6b80 10 client.25937 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:52.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.546+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd9ef05e70 con 0x55bd9ee87cc0 2024-06-03T02:24:52.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.546+0000 7f378b400700 1 -- 172.21.15.79:0/2518408773 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a0099330 con 0x55bd9ee87cc0 2024-06-03T02:24:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.546+0000 7f378b400700 1 -- 172.21.15.79:0/2518408773 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f37a00a0470 con 0x55bd9ee87cc0 2024-06-03T02:24:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.546+0000 7f37ba8d6b80 10 client.25937 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.547+0000 7f37ba8d6b80 10 client.25937 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:24:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.547+0000 7f37ba8d6b80 20 client.25937 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.547+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55bd9ef060a0 0x55bd9ef08490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.547+0000 7f37ba8d6b80 4 client.25937 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:24:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.547+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd9edd2580 con 0x55bd9ef060a0 2024-06-03T02:24:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.548+0000 7f37ad600700 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55bd9ef060a0 0x55bd9ef08490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:52.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.550+0000 7f37ad600700 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55bd9ef060a0 0x55bd9ef08490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfdc638a3ca77787 server_cookie=4ad7a7be7d2e4bef in_seq=0 out_seq=0 2024-06-03T02:24:52.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.550+0000 7f378b400700 10 client.25937 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:24:52.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.552+0000 7f378b400700 1 -- 172.21.15.79:0/2518408773 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd9edd2580 con 0x55bd9ef060a0 2024-06-03T02:24:52.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.552+0000 7f378b400700 10 client.25937 handle_command_reply: tid=0 2024-06-03T02:24:52.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.552+0000 7f37ba8d6b80 1 client.25937 shutdown 2024-06-03T02:24:52.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.555+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee87cc0 msgr2=0x55bd9eea2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:52.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.555+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee87cc0 0x55bd9eea2e40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f37a0097080 tx=0x7f37a00a3de0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.557+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 shutdown_connections 2024-06-03T02:24:52.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.557+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd9ee87cc0 0x55bd9eea2e40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.557+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bd9ee81a50 0x55bd9ee93e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.557+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55bd9ef060a0 0x55bd9ef08490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.557+0000 7f37ba8d6b80 1 --2- 172.21.15.79:0/2518408773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd9ee85540 0x55bd9eea2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.557+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 >> 172.21.15.79:0/2518408773 conn(0x55bd9eda5170 msgr2=0x55bd9ee880a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:52.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.557+0000 7f378b400700 0 client.25937 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:24:52.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.557+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 shutdown_connections 2024-06-03T02:24:52.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.557+0000 7f37ba8d6b80 1 -- 172.21.15.79:0/2518408773 wait complete. 2024-06-03T02:24:52.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.558+0000 7f37ba8d6b80 20 client.25937 trim_cache size 0 max 16384 2024-06-03T02:24:52.559 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:24:52.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 -- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37900a7080 msgr2=0x7f37900a9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:52.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37900a7080 0x7f37900a9540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f379c009f50 tx=0x7f379c005c00 comp rx=0 tx=0).stop 2024-06-03T02:24:52.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 -- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a8107840 msgr2=0x7f37a806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:52.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a8107840 0x7f37a806ccf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f37a0007ee0 tx=0x7f37a000d3b0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 -- 172.21.15.79:0/95414704 shutdown_connections 2024-06-03T02:24:52.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f37900a7080 0x7f37900a9540 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a8107840 0x7f37a806ccf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8102740 0x7f37a8073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 --2- 172.21.15.79:0/95414704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37a81030f0 0x7f37a8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:52.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 -- 172.21.15.79:0/95414704 >> 172.21.15.79:0/95414704 conn(0x7f37a80795f0 msgr2=0x7f37a8101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:52.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.559+0000 7f37b0000700 1 -- 172.21.15.79:0/95414704 shutdown_connections 2024-06-03T02:24:52.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:52.560+0000 7f37b0000700 1 -- 172.21.15.79:0/95414704 wait complete. 2024-06-03T02:24:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:52 smithi123 ceph-mon[36640]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 MiB/s wr, 547 op/s 2024-06-03T02:24:52.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:52 smithi135 ceph-mon[36653]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 MiB/s wr, 547 op/s 2024-06-03T02:24:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 285.0 s, 918.0 tps, lat 109.225 ms stddev 134.510 2024-06-03T02:24:54.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:24:54.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:54 smithi079 ceph-mon[31819]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.9 MiB/s wr, 558 op/s 2024-06-03T02:24:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:54 smithi123 ceph-mon[36640]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.9 MiB/s wr, 558 op/s 2024-06-03T02:24:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:54 smithi135 ceph-mon[36653]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.9 MiB/s wr, 558 op/s 2024-06-03T02:24:54.866 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:24:56.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:56 smithi123 ceph-mon[36640]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.4 MiB/s wr, 619 op/s 2024-06-03T02:24:56.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:56 smithi079 ceph-mon[31819]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.4 MiB/s wr, 619 op/s 2024-06-03T02:24:56.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:56 smithi135 ceph-mon[36653]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.4 MiB/s wr, 619 op/s 2024-06-03T02:24:58.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 290.0 s, 509.8 tps, lat 189.873 ms stddev 269.389 2024-06-03T02:24:58.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:24:58 smithi123 ceph-mon[36640]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-06-03T02:24:58.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:24:58 smithi079 ceph-mon[31819]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-06-03T02:24:58.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:24:58 smithi135 ceph-mon[36653]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-06-03T02:24:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.318+0000 7fc1c2e00700 1 -- 172.21.15.79:0/3461492883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc108500 msgr2=0x7fc1bc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.318+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/3461492883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc108500 0x7fc1bc1088e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0098f0 tx=0x7fc1ac009c00 comp rx=0 tx=0).stop 2024-06-03T02:24:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.318+0000 7fc1c2e00700 1 -- 172.21.15.79:0/3461492883 shutdown_connections 2024-06-03T02:24:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.318+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/3461492883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1bc06ab30 0x7fc1bc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.318+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/3461492883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc108500 0x7fc1bc1088e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.318+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/3461492883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1bc06a170 0x7fc1bc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.318+0000 7fc1c2e00700 1 -- 172.21.15.79:0/3461492883 >> 172.21.15.79:0/3461492883 conn(0x7fc1bc078340 msgr2=0x7fc1bc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:59.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.319+0000 7fc1c2e00700 1 -- 172.21.15.79:0/3461492883 shutdown_connections 2024-06-03T02:24:59.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.319+0000 7fc1c2e00700 1 -- 172.21.15.79:0/3461492883 wait complete. 2024-06-03T02:24:59.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.321+0000 7fc1c2e00700 1 Processor -- start 2024-06-03T02:24:59.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.321+0000 7fc1c2e00700 1 -- start start 2024-06-03T02:24:59.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.321+0000 7fc1c2e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1bc06a170 0x7fc1bc19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.321+0000 7fc1c2e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1bc06ab30 0x7fc1bc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.321+0000 7fc1c2e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc108500 0x7fc1bc196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.321+0000 7fc1c2e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc110600 con 0x7fc1bc06ab30 2024-06-03T02:24:59.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.321+0000 7fc1c2e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc110480 con 0x7fc1bc108500 2024-06-03T02:24:59.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.321+0000 7fc1c2e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc110780 con 0x7fc1bc06a170 2024-06-03T02:24:59.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.322+0000 7fc1bbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1bc06a170 0x7fc1bc19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.322+0000 7fc1c0e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc108500 0x7fc1bc196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.322+0000 7fc1bb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1bc06ab30 0x7fc1bc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.322+0000 7fc1c0e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc108500 0x7fc1bc196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59388/0 (socket says 172.21.15.79:59388) 2024-06-03T02:24:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.322+0000 7fc1c0e00700 1 -- 172.21.15.79:0/1291491354 learned_addr learned my addr 172.21.15.79:0/1291491354 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.322+0000 7fc1bbe00700 1 -- 172.21.15.79:0/1291491354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc108500 msgr2=0x7fc1bc196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.322+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc108500 0x7fc1bc196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.322+0000 7fc1bbe00700 1 -- 172.21.15.79:0/1291491354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1bc06ab30 msgr2=0x7fc1bc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.322+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1bc06ab30 0x7fc1bc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.323+0000 7fc1bbe00700 1 -- 172.21.15.79:0/1291491354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc1970a0 con 0x7fc1bc06a170 2024-06-03T02:24:59.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.323+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1bc06a170 0x7fc1bc19c780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000c00 tx=0x7fc1ac00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.324+0000 7fc1b8c00700 1 -- 172.21.15.79:0/1291491354 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac005200 con 0x7fc1bc06a170 2024-06-03T02:24:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.324+0000 7fc1b8c00700 1 -- 172.21.15.79:0/1291491354 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1ac00b560 con 0x7fc1bc06a170 2024-06-03T02:24:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.324+0000 7fc1b8c00700 1 -- 172.21.15.79:0/1291491354 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac003aa0 con 0x7fc1bc06a170 2024-06-03T02:24:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.324+0000 7fc1c2e00700 1 -- 172.21.15.79:0/1291491354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1bc197330 con 0x7fc1bc06a170 2024-06-03T02:24:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.325+0000 7fc1c2e00700 1 -- 172.21.15.79:0/1291491354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1bc197730 con 0x7fc1bc06a170 2024-06-03T02:24:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.331+0000 7fc1b8c00700 1 -- 172.21.15.79:0/1291491354 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc1ac00ba10 con 0x7fc1bc06a170 2024-06-03T02:24:59.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.336+0000 7fc1b8c00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc1a00a2d10 0x7fc1a00a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.336+0000 7fc1b8c00700 1 -- 172.21.15.79:0/1291491354 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc1ac0e1830 con 0x7fc1bc06a170 2024-06-03T02:24:59.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.337+0000 7fc1bb400700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc1a00a2d10 0x7fc1a00a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.338+0000 7fc1bb400700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc1a00a2d10 0x7fc1a00a51d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc078a40 tx=0x7fc1a8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:59.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.339+0000 7fc1cd874b80 1 Processor -- start 2024-06-03T02:24:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.340+0000 7fc1cd874b80 1 -- start start 2024-06-03T02:24:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.340+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x56037346a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.340+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.340+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 0x5603734776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.340+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560373379a10 con 0x560373475300 2024-06-03T02:24:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.340+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5603733bcb00 con 0x5603733e6e40 2024-06-03T02:24:59.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.340+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5603733778f0 con 0x56037346a5b0 2024-06-03T02:24:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.341+0000 7fc1bb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.341+0000 7fc1bb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41522/0 (socket says 172.21.15.79:41522) 2024-06-03T02:24:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.341+0000 7fc1bb400700 1 -- 172.21.15.79:0/3222315663 learned_addr learned my addr 172.21.15.79:0/3222315663 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.341+0000 7fc1c0e00700 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 0x5603734776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.341+0000 7fc1bb400700 1 -- 172.21.15.79:0/3222315663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 msgr2=0x56037346a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:59.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.341+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x56037346a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.342+0000 7fc1bb400700 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x56037346a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.342+0000 7fc1bb400700 1 -- 172.21.15.79:0/3222315663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 msgr2=0x5603734776f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.342+0000 7fc1bb400700 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 0x5603734776f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.342+0000 7fc1bb400700 1 -- 172.21.15.79:0/3222315663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560373364f90 con 0x56037346a5b0 2024-06-03T02:24:59.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.342+0000 7fc1c0e00700 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 0x5603734776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:59.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.342+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x56037346a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:24:59.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.343+0000 7fc1bb400700 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346aa30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8005b40 tx=0x7fc1a8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10b191c5b0ec865d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:59.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.343+0000 7fc1b1600700 1 -- 172.21.15.79:0/3222315663 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a801f420 con 0x56037346a5b0 2024-06-03T02:24:59.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.344+0000 7fc1b1600700 1 -- 172.21.15.79:0/3222315663 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a801faa0 con 0x56037346a5b0 2024-06-03T02:24:59.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.344+0000 7fc1b1600700 1 -- 172.21.15.79:0/3222315663 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a801dc10 con 0x56037346a5b0 2024-06-03T02:24:59.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.345+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3222315663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 msgr2=0x56037346aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:59.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.345+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346aa30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8005b40 tx=0x7fc1a8014000 comp rx=0 tx=0).stop 2024-06-03T02:24:59.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.346+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3222315663 shutdown_connections 2024-06-03T02:24:59.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.346+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 0x5603734776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.346+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x56037346a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.346+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3222315663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346aa30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.346+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3222315663 >> 172.21.15.79:0/3222315663 conn(0x560373397170 msgr2=0x5603734664b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:59.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.346+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3222315663 shutdown_connections 2024-06-03T02:24:59.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.346+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3222315663 wait complete. 2024-06-03T02:24:59.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.347+0000 7fc1cd874b80 1 -- start start 2024-06-03T02:24:59.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.347+0000 7fc1cd874b80 1 Processor -- start 2024-06-03T02:24:59.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.348+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603733e6e40 0x560373474ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56037346a5b0 0x560373471e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373472360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560373358f60 con 0x5603733e6e40 2024-06-03T02:24:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5603733b8ac0 con 0x56037346a5b0 2024-06-03T02:24:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560373477fc0 con 0x560373475300 2024-06-03T02:24:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1cd874b80 10 client.0 _ll_register_callbacks cb 0x560373356720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1c0e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373472360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1bb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56037346a5b0 0x560373471e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1c0e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373472360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41524/0 (socket says 172.21.15.79:41524) 2024-06-03T02:24:59.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1bb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56037346a5b0 0x560373471e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59404/0 (socket says 172.21.15.79:59404) 2024-06-03T02:24:59.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.349+0000 7fc1bb400700 1 -- 172.21.15.79:0/3100018417 learned_addr learned my addr 172.21.15.79:0/3100018417 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:59.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.350+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603733e6e40 0x560373474ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.350+0000 7fc1bb400700 1 -- 172.21.15.79:0/3100018417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 msgr2=0x560373472360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:59.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.350+0000 7fc1bb400700 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373472360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.350+0000 7fc1bb400700 1 -- 172.21.15.79:0/3100018417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603733e6e40 msgr2=0x560373474ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.350+0000 7fc1bb400700 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603733e6e40 0x560373474ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.350+0000 7fc1bb400700 1 -- 172.21.15.79:0/3100018417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037330b8d0 con 0x56037346a5b0 2024-06-03T02:24:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.350+0000 7fc1bb400700 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56037346a5b0 0x560373471e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc1a801d830 tx=0x7fc1a801f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7283b67c80ad140 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.351+0000 7fc1a6000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:24:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.351+0000 7fc1a6000700 1 -- 172.21.15.79:0/3100018417 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8036060 con 0x56037346a5b0 2024-06-03T02:24:59.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.351+0000 7fc1a6000700 1 -- 172.21.15.79:0/3100018417 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc1a801daa0 con 0x56037346a5b0 2024-06-03T02:24:59.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.351+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3100018417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a801faa0 con 0x56037346a5b0 2024-06-03T02:24:59.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.351+0000 7fc1a6000700 1 -- 172.21.15.79:0/3100018417 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc1a803d050 con 0x56037346a5b0 2024-06-03T02:24:59.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.352+0000 7fc1cd874b80 10 client.25946 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:59.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.352+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3100018417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560373482720 con 0x56037346a5b0 2024-06-03T02:24:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.352+0000 7fc1a6000700 1 -- 172.21.15.79:0/3100018417 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a801faa0 con 0x56037346a5b0 2024-06-03T02:24:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.352+0000 7fc1a6000700 1 -- 172.21.15.79:0/3100018417 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc1a8027c40 con 0x56037346a5b0 2024-06-03T02:24:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.353+0000 7fc1cd874b80 10 client.25946 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.353+0000 7fc1cd874b80 10 client.25946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:24:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.353+0000 7fc1cd874b80 20 client.25946 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.353+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560373484d00 0x5603734870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.353+0000 7fc1cd874b80 4 client.25946 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:24:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.353+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3100018417 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603733ca000 con 0x560373484d00 2024-06-03T02:24:59.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.353+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560373484d00 0x5603734870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.354+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560373484d00 0x5603734870f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268c5ca6f7fce81d server_cookie=7def74046f97a039 in_seq=0 out_seq=0 2024-06-03T02:24:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.354+0000 7fc1a6000700 10 client.25946 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.355+0000 7fc1a6000700 1 -- 172.21.15.79:0/3100018417 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5603733ca000 con 0x560373484d00 2024-06-03T02:24:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.355+0000 7fc1a6000700 10 client.25946 handle_command_reply: tid=0 2024-06-03T02:24:59.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.355+0000 7fc1cd874b80 1 client.25946 shutdown 2024-06-03T02:24:59.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.356+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3100018417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56037346a5b0 msgr2=0x560373471e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:59.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.356+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56037346a5b0 0x560373471e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc1a801d830 tx=0x7fc1a801f920 comp rx=0 tx=0).stop 2024-06-03T02:24:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.356+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3100018417 shutdown_connections 2024-06-03T02:24:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.356+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373472360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.356+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560373484d00 0x5603734870f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.356+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5603733e6e40 0x560373474ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.356+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/3100018417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56037346a5b0 0x560373471e20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.356+0000 7fc1a6000700 0 client.25946 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:59.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.356+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3100018417 >> 172.21.15.79:0/3100018417 conn(0x560373397170 msgr2=0x560373465740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:59.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.357+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3100018417 shutdown_connections 2024-06-03T02:24:59.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.357+0000 7fc1cd874b80 1 -- 172.21.15.79:0/3100018417 wait complete. 2024-06-03T02:24:59.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.357+0000 7fc1cd874b80 20 client.25946 trim_cache size 0 max 16384 2024-06-03T02:24:59.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.393+0000 7fc1cd874b80 1 Processor -- start 2024-06-03T02:24:59.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.393+0000 7fc1cd874b80 1 -- start start 2024-06-03T02:24:59.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.393+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373474660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.393+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.393+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 0x56037346d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.394+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1ac1d0 con 0x560373475300 2024-06-03T02:24:59.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.394+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1add10 con 0x5603733e6e40 2024-06-03T02:24:59.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.394+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1ac8a0 con 0x56037346a5b0 2024-06-03T02:24:59.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.394+0000 7fc1bbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373474660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.394+0000 7fc1bb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.394+0000 7fc1bbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373474660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59414/0 (socket says 172.21.15.79:59414) 2024-06-03T02:24:59.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.394+0000 7fc1bbe00700 1 -- 172.21.15.79:0/545868746 learned_addr learned my addr 172.21.15.79:0/545868746 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:59.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.394+0000 7fc1bb400700 1 -- 172.21.15.79:0/545868746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 msgr2=0x560373474660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:59.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.395+0000 7fc1bb400700 1 --2- 172.21.15.79:0/545868746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373474660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.395+0000 7fc1bb400700 1 -- 172.21.15.79:0/545868746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 msgr2=0x56037346d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:59.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.395+0000 7fc1bb400700 1 --2- 172.21.15.79:0/545868746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 0x56037346d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.396+0000 7fc1bb400700 1 -- 172.21.15.79:0/545868746 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037345de10 con 0x56037346a5b0 2024-06-03T02:24:59.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.396+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/545868746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373474660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:24:59.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.396+0000 7fc1bb400700 1 --2- 172.21.15.79:0/545868746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346cfd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1a801d860 tx=0x7fc1a801b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19b391cad18361b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:59.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.398+0000 7fc1b0c00700 1 -- 172.21.15.79:0/545868746 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8036070 con 0x56037346a5b0 2024-06-03T02:24:59.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.398+0000 7fc1b0c00700 1 -- 172.21.15.79:0/545868746 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a8009950 con 0x56037346a5b0 2024-06-03T02:24:59.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.398+0000 7fc1b0c00700 1 -- 172.21.15.79:0/545868746 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a801bd80 con 0x56037346a5b0 2024-06-03T02:24:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.399+0000 7fc1cd874b80 1 -- 172.21.15.79:0/545868746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 msgr2=0x56037346cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.399+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/545868746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346cfd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1a801d860 tx=0x7fc1a801b6e0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.400+0000 7fc1cd874b80 1 -- 172.21.15.79:0/545868746 shutdown_connections 2024-06-03T02:24:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.400+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/545868746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560373475300 0x56037346d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.400+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/545868746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373474660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.400+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/545868746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56037346a5b0 0x56037346cfd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.400+0000 7fc1cd874b80 1 -- 172.21.15.79:0/545868746 >> 172.21.15.79:0/545868746 conn(0x560373397170 msgr2=0x560373480250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:59.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.401+0000 7fc1cd874b80 1 -- 172.21.15.79:0/545868746 shutdown_connections 2024-06-03T02:24:59.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.401+0000 7fc1cd874b80 1 -- 172.21.15.79:0/545868746 wait complete. 2024-06-03T02:24:59.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.401+0000 7fc1cd874b80 1 -- start start 2024-06-03T02:24:59.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.402+0000 7fc1cd874b80 1 Processor -- start 2024-06-03T02:24:59.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.404+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373487100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.404+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56037346a5b0 0x560373487640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.404+0000 7fc1cd874b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373487b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.404+0000 7fc1bbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373487100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.405+0000 7fc1bbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373487100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59424/0 (socket says 172.21.15.79:59424) 2024-06-03T02:24:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.405+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1ad640 con 0x56037346a5b0 2024-06-03T02:24:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.405+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1acf70 con 0x5603733e6e40 2024-06-03T02:24:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.405+0000 7fc1cd874b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1aa170 con 0x560373475300 2024-06-03T02:24:59.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.405+0000 7fc1c0e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373487b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.405+0000 7fc1c0e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373487b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41548/0 (socket says 172.21.15.79:41548) 2024-06-03T02:24:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.405+0000 7fc1c0e00700 1 -- 172.21.15.79:0/1868506233 learned_addr learned my addr 172.21.15.79:0/1868506233 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:24:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.406+0000 7fc1c0e00700 1 -- 172.21.15.79:0/1868506233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 msgr2=0x560373487100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.406+0000 7fc1c0e00700 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373487100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.406+0000 7fc1c0e00700 1 -- 172.21.15.79:0/1868506233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56037346a5b0 msgr2=0x560373487640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:24:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.406+0000 7fc1c0e00700 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56037346a5b0 0x560373487640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.406+0000 7fc1c0e00700 1 -- 172.21.15.79:0/1868506233 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603734ed840 con 0x560373475300 2024-06-03T02:24:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.406+0000 7fc1cd874b80 10 client.0 _ll_register_callbacks cb 0x560373356720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:24:59.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.406+0000 7fc1cd874b80 1 -- 172.21.15.79:0/1868506233 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603733c4580 con 0x560373475300 2024-06-03T02:24:59.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.407+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373487100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-03T02:24:59.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.408+0000 7fc1c0e00700 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373487b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4007da0 tx=0x7fc1b4007e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d86f1b63eb258567 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:24:59.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.408+0000 7fc1a6000700 10 client.25936 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:24:59.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.409+0000 7fc1a6000700 1 -- 172.21.15.79:0/1868506233 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4016da0 con 0x560373475300 2024-06-03T02:24:59.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.409+0000 7fc1a6000700 1 -- 172.21.15.79:0/1868506233 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fc1b4003a00 con 0x560373475300 2024-06-03T02:24:59.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.409+0000 7fc1a6000700 1 -- 172.21.15.79:0/1868506233 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc1b4003c40 con 0x560373475300 2024-06-03T02:24:59.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.409+0000 7fc1cd874b80 10 client.25936 fetch_fsmap learned FSMap version 30 2024-06-03T02:24:59.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.410+0000 7fc1cd874b80 1 -- 172.21.15.79:0/1868506233 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5603734eea90 con 0x560373475300 2024-06-03T02:24:59.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.410+0000 7fc1a6000700 1 -- 172.21.15.79:0/1868506233 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc1b4021430 con 0x560373475300 2024-06-03T02:24:59.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.410+0000 7fc1cd874b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:24:59.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.410+0000 7fc1cd874b80 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:24:59.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.410+0000 7fc1cd874b80 20 client.25936 populate_metadata read hostname 'smithi079' 2024-06-03T02:24:59.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.410+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5603734ef090 0x5603734f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:24:59.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.410+0000 7fc1cd874b80 4 client.25936 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:24:59.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.410+0000 7fc1cd874b80 1 -- 172.21.15.79:0/1868506233 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56037323e250 con 0x5603734ef090 2024-06-03T02:24:59.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.411+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5603734ef090 0x5603734f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:24:59.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.411+0000 7fc1bbe00700 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5603734ef090 0x5603734f14f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5474009dc5da64e6 server_cookie=52175b59ae808f2c in_seq=0 out_seq=0 2024-06-03T02:24:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.411+0000 7fc1a6000700 10 client.25936 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1a6000700 1 -- 172.21.15.79:0/1868506233 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56037323e250 con 0x5603734ef090 2024-06-03T02:24:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1a6000700 10 client.25936 handle_command_reply: tid=0 2024-06-03T02:24:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1cd874b80 1 client.25936 shutdown 2024-06-03T02:24:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1cd874b80 1 -- 172.21.15.79:0/1868506233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 msgr2=0x560373487b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373487b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4007da0 tx=0x7fc1b4007e80 comp rx=0 tx=0).stop 2024-06-03T02:24:59.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1cd874b80 1 -- 172.21.15.79:0/1868506233 shutdown_connections 2024-06-03T02:24:59.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560373475300 0x560373487b80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603733e6e40 0x560373487100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5603734ef090 0x5603734f14f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1cd874b80 1 --2- 172.21.15.79:0/1868506233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56037346a5b0 0x560373487640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.412+0000 7fc1cd874b80 1 -- 172.21.15.79:0/1868506233 >> 172.21.15.79:0/1868506233 conn(0x560373397170 msgr2=0x560373480ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.413+0000 7fc1a6000700 0 client.25936 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:24:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.413+0000 7fc1cd874b80 1 -- 172.21.15.79:0/1868506233 shutdown_connections 2024-06-03T02:24:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.413+0000 7fc1cd874b80 1 -- 172.21.15.79:0/1868506233 wait complete. 2024-06-03T02:24:59.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.413+0000 7fc1cd874b80 20 client.25936 trim_cache size 0 max 16384 2024-06-03T02:24:59.417 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:24:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 -- 172.21.15.79:0/1291491354 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc1a00a2d10 msgr2=0x7fc1a00a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc1a00a2d10 0x7fc1a00a51d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc078a40 tx=0x7fc1a8009310 comp rx=0 tx=0).stop 2024-06-03T02:24:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 -- 172.21.15.79:0/1291491354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1bc06a170 msgr2=0x7fc1bc19c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:24:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1bc06a170 0x7fc1bc19c780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000c00 tx=0x7fc1ac00fa00 comp rx=0 tx=0).stop 2024-06-03T02:24:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 -- 172.21.15.79:0/1291491354 shutdown_connections 2024-06-03T02:24:59.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc108500 0x7fc1bc196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1bc06a170 0x7fc1bc19c780 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fc1a00a2d10 0x7fc1a00a51d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 --2- 172.21.15.79:0/1291491354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1bc06ab30 0x7fc1bc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:24:59.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 -- 172.21.15.79:0/1291491354 >> 172.21.15.79:0/1291491354 conn(0x7fc1bc078340 msgr2=0x7fc1bc100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:24:59.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 -- 172.21.15.79:0/1291491354 shutdown_connections 2024-06-03T02:24:59.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:24:59.414+0000 7fc1c2e00700 1 -- 172.21.15.79:0/1291491354 wait complete. 2024-06-03T02:25:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:00 smithi079 ceph-mon[31819]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 MiB/s wr, 597 op/s 2024-06-03T02:25:00.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:00 smithi123 ceph-mon[36640]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 MiB/s wr, 597 op/s 2024-06-03T02:25:00.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:00 smithi135 ceph-mon[36653]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 MiB/s wr, 597 op/s 2024-06-03T02:25:01.344 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:25:01.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:25:02.079 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:02.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:02 smithi079 ceph-mon[31819]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 498 op/s 2024-06-03T02:25:02.727 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:02 smithi123 ceph-mon[36640]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 498 op/s 2024-06-03T02:25:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:02 smithi135 ceph-mon[36653]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 498 op/s 2024-06-03T02:25:03.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 295.0 s, 564.4 tps, lat 173.962 ms stddev 252.182 2024-06-03T02:25:04.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:04 smithi079 ceph-mon[31819]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.7 MiB/s wr, 576 op/s 2024-06-03T02:25:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:04 smithi123 ceph-mon[36640]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.7 MiB/s wr, 576 op/s 2024-06-03T02:25:04.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:04 smithi135 ceph-mon[36653]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.7 MiB/s wr, 576 op/s 2024-06-03T02:25:05.436 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:05 smithi079 ceph-mon[31819]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 658 op/s 2024-06-03T02:25:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:05 smithi123 ceph-mon[36640]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 658 op/s 2024-06-03T02:25:05.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:05 smithi135 ceph-mon[36653]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 658 op/s 2024-06-03T02:25:06.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.144+0000 7f57a0200700 1 -- 172.21.15.79:0/4196645104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57981024e0 msgr2=0x7f5798102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:06.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.144+0000 7f57a0200700 1 --2- 172.21.15.79:0/4196645104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57981024e0 0x7f5798102960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5784009990 tx=0x7f5784009ca0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.145+0000 7f57a0200700 1 -- 172.21.15.79:0/4196645104 shutdown_connections 2024-06-03T02:25:06.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.145+0000 7f57a0200700 1 --2- 172.21.15.79:0/4196645104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5798102ea0 0x7f579810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.145+0000 7f57a0200700 1 --2- 172.21.15.79:0/4196645104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57981085f0 0x7f57981089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.145+0000 7f57a0200700 1 --2- 172.21.15.79:0/4196645104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57981024e0 0x7f5798102960 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.145+0000 7f57a0200700 1 -- 172.21.15.79:0/4196645104 >> 172.21.15.79:0/4196645104 conn(0x7f5798079640 msgr2=0x7f5798079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:06.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.145+0000 7f57a0200700 1 -- 172.21.15.79:0/4196645104 shutdown_connections 2024-06-03T02:25:06.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.145+0000 7f57a0200700 1 -- 172.21.15.79:0/4196645104 wait complete. 2024-06-03T02:25:06.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.146+0000 7f57a0200700 1 Processor -- start 2024-06-03T02:25:06.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.147+0000 7f57a0200700 1 -- start start 2024-06-03T02:25:06.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.147+0000 7f57a0200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57981024e0 0x7f5798074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.147+0000 7f57a0200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102ea0 0x7f5798074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.147+0000 7f57a0200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57981085f0 0x7f5798075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.147+0000 7f57a0200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f57981128b0 con 0x7f57981024e0 2024-06-03T02:25:06.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.147+0000 7f57a0200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5798112730 con 0x7f5798102ea0 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.147+0000 7f57a0200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5798112a30 con 0x7f57981085f0 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579e200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57981085f0 0x7f5798075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579e200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57981085f0 0x7f5798075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41296/0 (socket says 172.21.15.79:41296) 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579e200700 1 -- 172.21.15.79:0/3606296692 learned_addr learned my addr 172.21.15.79:0/3606296692 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579ce00700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102ea0 0x7f5798074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579e200700 1 -- 172.21.15.79:0/3606296692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102ea0 msgr2=0x7f5798074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579e200700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102ea0 0x7f5798074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579e200700 1 -- 172.21.15.79:0/3606296692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57981024e0 msgr2=0x7f5798074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579e200700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57981024e0 0x7f5798074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579e200700 1 -- 172.21.15.79:0/3606296692 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57981a72f0 con 0x7f57981085f0 2024-06-03T02:25:06.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.148+0000 7f579e200700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57981085f0 0x7f5798075210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f579000e970 tx=0x7f579000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:06.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.149+0000 7f5796000700 1 -- 172.21.15.79:0/3606296692 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579000c9d0 con 0x7f57981085f0 2024-06-03T02:25:06.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.149+0000 7f5796000700 1 -- 172.21.15.79:0/3606296692 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5790004d10 con 0x7f57981085f0 2024-06-03T02:25:06.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.149+0000 7f5796000700 1 -- 172.21.15.79:0/3606296692 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790005690 con 0x7f57981085f0 2024-06-03T02:25:06.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.149+0000 7f57a0200700 1 -- 172.21.15.79:0/3606296692 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57981a75e0 con 0x7f57981085f0 2024-06-03T02:25:06.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.149+0000 7f57a0200700 1 -- 172.21.15.79:0/3606296692 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57981a7ad0 con 0x7f57981085f0 2024-06-03T02:25:06.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.155+0000 7f5796000700 1 -- 172.21.15.79:0/3606296692 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5790021440 con 0x7f57981085f0 2024-06-03T02:25:06.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.157+0000 7f5796000700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57800a2e60 0x7f57800a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.157+0000 7f5796000700 1 -- 172.21.15.79:0/3606296692 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5790005430 con 0x7f57981085f0 2024-06-03T02:25:06.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.158+0000 7f579d800700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57800a2e60 0x7f57800a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.159+0000 7f579d800700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57800a2e60 0x7f57800a5320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f578c00ab90 tx=0x7f578c009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:06.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.160+0000 7f57aabe0b80 1 Processor -- start 2024-06-03T02:25:06.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.160+0000 7f57aabe0b80 1 -- start start 2024-06-03T02:25:06.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.160+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071667eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.160+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 0x560716678d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.160+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716679700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.160+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560716587a90 con 0x56071667f530 2024-06-03T02:25:06.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.160+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607165cab80 con 0x56071667ec10 2024-06-03T02:25:06.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.160+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560716585970 con 0x5607166792c0 2024-06-03T02:25:06.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.161+0000 7f579d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071667eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.161+0000 7f579d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071667eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33446/0 (socket says 172.21.15.79:33446) 2024-06-03T02:25:06.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.161+0000 7f579d800700 1 -- 172.21.15.79:0/399804626 learned_addr learned my addr 172.21.15.79:0/399804626 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:06.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.161+0000 7f579e200700 1 --2- 172.21.15.79:0/399804626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716679700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.161+0000 7f579ce00700 1 --2- 172.21.15.79:0/399804626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 0x560716678d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.162+0000 7f579d800700 1 -- 172.21.15.79:0/399804626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 msgr2=0x560716679700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:06.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.162+0000 7f579d800700 1 --2- 172.21.15.79:0/399804626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716679700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.162+0000 7f579d800700 1 -- 172.21.15.79:0/399804626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 msgr2=0x560716678d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:06.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.162+0000 7f579d800700 1 --2- 172.21.15.79:0/399804626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 0x560716678d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.162+0000 7f579d800700 1 -- 172.21.15.79:0/399804626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560716573010 con 0x56071667ec10 2024-06-03T02:25:06.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.165+0000 7f579d800700 1 --2- 172.21.15.79:0/399804626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071667eff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f578c00ce00 tx=0x7f578c00f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e56e065f9f1d2ba server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:06.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.165+0000 7f578aa00700 1 -- 172.21.15.79:0/399804626 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c019420 con 0x56071667ec10 2024-06-03T02:25:06.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.165+0000 7f578aa00700 1 -- 172.21.15.79:0/399804626 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f578c019aa0 con 0x56071667ec10 2024-06-03T02:25:06.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.165+0000 7f578aa00700 1 -- 172.21.15.79:0/399804626 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c021b00 con 0x56071667ec10 2024-06-03T02:25:06.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.166+0000 7f57aabe0b80 1 -- 172.21.15.79:0/399804626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 msgr2=0x56071667eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:06.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.166+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/399804626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071667eff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f578c00ce00 tx=0x7f578c00f000 comp rx=0 tx=0).stop 2024-06-03T02:25:06.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.167+0000 7f57aabe0b80 1 -- 172.21.15.79:0/399804626 shutdown_connections 2024-06-03T02:25:06.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.167+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/399804626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716679700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.167+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/399804626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071667eff0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.167+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/399804626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 0x560716678d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.167+0000 7f57aabe0b80 1 -- 172.21.15.79:0/399804626 >> 172.21.15.79:0/399804626 conn(0x5607165a51f0 msgr2=0x560716674070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:06.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.167+0000 7f57aabe0b80 1 -- 172.21.15.79:0/399804626 shutdown_connections 2024-06-03T02:25:06.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.167+0000 7f57aabe0b80 1 -- 172.21.15.79:0/399804626 wait complete. 2024-06-03T02:25:06.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.168+0000 7f57aabe0b80 1 -- start start 2024-06-03T02:25:06.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.169+0000 7f57aabe0b80 1 Processor -- start 2024-06-03T02:25:06.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.170+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716685240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.170+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667ec10 0x5607166825b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667f530 0x560716682af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f579d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716685240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560716566fe0 con 0x56071667ec10 2024-06-03T02:25:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607165c6b40 con 0x56071667f530 2024-06-03T02:25:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560716685f10 con 0x5607166792c0 2024-06-03T02:25:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f579d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716685240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41304/0 (socket says 172.21.15.79:41304) 2024-06-03T02:25:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f579d800700 1 -- 172.21.15.79:0/2218546135 learned_addr learned my addr 172.21.15.79:0/2218546135 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f57aabe0b80 10 client.0 _ll_register_callbacks cb 0x560716560400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f579ce00700 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667ec10 0x5607166825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.171+0000 7f579e200700 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667f530 0x560716682af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.172+0000 7f579d800700 1 -- 172.21.15.79:0/2218546135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667f530 msgr2=0x560716682af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.172+0000 7f579d800700 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667f530 0x560716682af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.172+0000 7f579d800700 1 -- 172.21.15.79:0/2218546135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667ec10 msgr2=0x5607166825b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.172+0000 7f579d800700 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667ec10 0x5607166825b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.172+0000 7f579d800700 1 -- 172.21.15.79:0/2218546135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560716519950 con 0x5607166792c0 2024-06-03T02:25:06.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.172+0000 7f579d800700 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716685240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f578c004c90 tx=0x7f578c019700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=688b303f71ee2456 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:06.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.172+0000 7f577f400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:25:06.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.172+0000 7f577f400700 1 -- 172.21.15.79:0/2218546135 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c02f060 con 0x5607166792c0 2024-06-03T02:25:06.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.173+0000 7f577f400700 1 -- 172.21.15.79:0/2218546135 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f578c021af0 con 0x5607166792c0 2024-06-03T02:25:06.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.173+0000 7f57aabe0b80 1 -- 172.21.15.79:0/2218546135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f578c019aa0 con 0x5607166792c0 2024-06-03T02:25:06.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.173+0000 7f577f400700 1 -- 172.21.15.79:0/2218546135 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f578c021cf0 con 0x5607166792c0 2024-06-03T02:25:06.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.173+0000 7f57aabe0b80 10 client.25948 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:06.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.173+0000 7f57aabe0b80 1 -- 172.21.15.79:0/2218546135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560716698f00 con 0x5607166792c0 2024-06-03T02:25:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.173+0000 7f577f400700 1 -- 172.21.15.79:0/2218546135 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f578c019aa0 con 0x5607166792c0 2024-06-03T02:25:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.174+0000 7f577f400700 1 -- 172.21.15.79:0/2218546135 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f578c028380 con 0x5607166792c0 2024-06-03T02:25:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.174+0000 7f57aabe0b80 10 client.25948 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.174+0000 7f57aabe0b80 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.174+0000 7f57aabe0b80 20 client.25948 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.174+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56071669b3d0 0x56071669d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.174+0000 7f57aabe0b80 4 client.25948 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:25:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.174+0000 7f57aabe0b80 1 -- 172.21.15.79:0/2218546135 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607165d8080 con 0x56071669b3d0 2024-06-03T02:25:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.174+0000 7f579ce00700 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56071669b3d0 0x56071669d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.176+0000 7f579ce00700 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56071669b3d0 0x56071669d7c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc1c072cb7c9eebf server_cookie=186a98211aa9373c in_seq=0 out_seq=0 2024-06-03T02:25:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.176+0000 7f577f400700 10 client.25948 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.177+0000 7f577f400700 1 -- 172.21.15.79:0/2218546135 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607165d8080 con 0x56071669b3d0 2024-06-03T02:25:06.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.177+0000 7f577f400700 10 client.25948 handle_command_reply: tid=0 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.177+0000 7f57aabe0b80 1 client.25948 shutdown 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f57aabe0b80 1 -- 172.21.15.79:0/2218546135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 msgr2=0x560716685240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716685240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f578c004c90 tx=0x7f578c019700 comp rx=0 tx=0).stop 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f57aabe0b80 1 -- 172.21.15.79:0/2218546135 shutdown_connections 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667f530 0x560716682af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x560716685240 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56071669b3d0 0x56071669d7c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/2218546135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667ec10 0x5607166825b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f57aabe0b80 1 -- 172.21.15.79:0/2218546135 >> 172.21.15.79:0/2218546135 conn(0x5607165a51f0 msgr2=0x56071666d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f577f400700 0 client.25948 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:06.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.178+0000 7f57aabe0b80 1 -- 172.21.15.79:0/2218546135 shutdown_connections 2024-06-03T02:25:06.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.179+0000 7f57aabe0b80 1 -- 172.21.15.79:0/2218546135 wait complete. 2024-06-03T02:25:06.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.180+0000 7f57aabe0b80 20 client.25948 trim_cache size 0 max 16384 2024-06-03T02:25:06.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.204+0000 7f57aabe0b80 1 Processor -- start 2024-06-03T02:25:06.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.204+0000 7f57aabe0b80 1 -- start start 2024-06-03T02:25:06.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607166792c0 0x5607166801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071666e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071667f530 0x560716682b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f577800ec20 con 0x5607166792c0 2024-06-03T02:25:06.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5778010760 con 0x56071667ec10 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f577800f2f0 con 0x56071667f530 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f579e200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071667f530 0x560716682b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f579e200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071667f530 0x560716682b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41318/0 (socket says 172.21.15.79:41318) 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f579d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607166792c0 0x5607166801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f579e200700 1 -- 172.21.15.79:0/4025253482 learned_addr learned my addr 172.21.15.79:0/4025253482 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.205+0000 7f579ce00700 1 --2- 172.21.15.79:0/4025253482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071666e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.206+0000 7f579e200700 1 -- 172.21.15.79:0/4025253482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 msgr2=0x56071666e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.206+0000 7f579e200700 1 --2- 172.21.15.79:0/4025253482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071666e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.206+0000 7f579e200700 1 -- 172.21.15.79:0/4025253482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607166792c0 msgr2=0x5607166801d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.206+0000 7f579e200700 1 --2- 172.21.15.79:0/4025253482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607166792c0 0x5607166801d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.206+0000 7f579e200700 1 -- 172.21.15.79:0/4025253482 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607166a3980 con 0x56071667f530 2024-06-03T02:25:06.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.206+0000 7f579e200700 1 --2- 172.21.15.79:0/4025253482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071667f530 0x560716682b20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5790098d30 tx=0x7f57900cdf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=334a482dd6691f35 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:06.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.206+0000 7f578a000700 1 -- 172.21.15.79:0/4025253482 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790085e60 con 0x56071667f530 2024-06-03T02:25:06.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.206+0000 7f578a000700 1 -- 172.21.15.79:0/4025253482 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5790005670 con 0x56071667f530 2024-06-03T02:25:06.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.206+0000 7f578a000700 1 -- 172.21.15.79:0/4025253482 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790016330 con 0x56071667f530 2024-06-03T02:25:06.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.207+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4025253482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071667f530 msgr2=0x560716682b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:06.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.207+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4025253482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071667f530 0x560716682b20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5790098d30 tx=0x7f57900cdf90 comp rx=0 tx=0).stop 2024-06-03T02:25:06.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.207+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4025253482 shutdown_connections 2024-06-03T02:25:06.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.207+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4025253482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071667f530 0x560716682b20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.207+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4025253482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607166792c0 0x5607166801d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.207+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4025253482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071666e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.207+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4025253482 >> 172.21.15.79:0/4025253482 conn(0x5607165a51f0 msgr2=0x56071667df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:06.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.207+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4025253482 shutdown_connections 2024-06-03T02:25:06.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.207+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4025253482 wait complete. 2024-06-03T02:25:06.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.208+0000 7f57aabe0b80 1 -- start start 2024-06-03T02:25:06.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.208+0000 7f57aabe0b80 1 Processor -- start 2024-06-03T02:25:06.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x56071669eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071669f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f57aabe0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 0x5607166fed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5778010090 con 0x56071667f530 2024-06-03T02:25:06.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f577800f9c0 con 0x56071667ec10 2024-06-03T02:25:06.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f57aabe0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f577800cbc0 con 0x5607166792c0 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f57aabe0b80 10 client.0 _ll_register_callbacks cb 0x560716560400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f579d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x56071669eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f579d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x56071669eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41330/0 (socket says 172.21.15.79:41330) 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f579d800700 1 -- 172.21.15.79:0/4067445968 learned_addr learned my addr 172.21.15.79:0/4067445968 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f579e200700 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 0x5607166fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.209+0000 7f579ce00700 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071669f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.210+0000 7f579d800700 1 -- 172.21.15.79:0/4067445968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 msgr2=0x56071669f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.210+0000 7f579d800700 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071669f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.210+0000 7f579d800700 1 -- 172.21.15.79:0/4067445968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 msgr2=0x5607166fed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:06.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.210+0000 7f579d800700 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 0x5607166fed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.210+0000 7f579d800700 1 -- 172.21.15.79:0/4067445968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607166ff340 con 0x5607166792c0 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.210+0000 7f579e200700 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 0x5607166fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.211+0000 7f579d800700 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x56071669eb00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f578c00ce00 tx=0x7f578c00c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=670f9434435924fd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.211+0000 7f577f400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.211+0000 7f577f400700 1 -- 172.21.15.79:0/4067445968 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c03f020 con 0x5607166792c0 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.211+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4067445968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5790005670 con 0x5607166792c0 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.211+0000 7f577f400700 1 -- 172.21.15.79:0/4067445968 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f578c0198d0 con 0x5607166792c0 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.211+0000 7f577f400700 1 -- 172.21.15.79:0/4067445968 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f578c04a050 con 0x5607166792c0 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.211+0000 7f57aabe0b80 10 client.25954 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.211+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4067445968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560716706720 con 0x5607166792c0 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.211+0000 7f577f400700 1 -- 172.21.15.79:0/4067445968 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5790005670 con 0x5607166792c0 2024-06-03T02:25:06.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.212+0000 7f577f400700 1 -- 172.21.15.79:0/4067445968 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f578c02a960 con 0x5607166792c0 2024-06-03T02:25:06.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.212+0000 7f57aabe0b80 10 client.25954 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:06.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.212+0000 7f57aabe0b80 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:06.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.212+0000 7f57aabe0b80 20 client.25954 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:06.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.212+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560716706d00 0x560716709120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:06.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.212+0000 7f57aabe0b80 4 client.25954 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:25:06.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.212+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4067445968 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560716681920 con 0x560716706d00 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.213+0000 7f579ce00700 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560716706d00 0x560716709120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.214+0000 7f579ce00700 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560716706d00 0x560716709120 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e707e1a1d0cc682 server_cookie=f2f6f1d59925f392 in_seq=0 out_seq=0 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.214+0000 7f577f400700 10 client.25954 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f577f400700 1 -- 172.21.15.79:0/4067445968 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560716681920 con 0x560716706d00 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f577f400700 10 client.25954 handle_command_reply: tid=0 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f57aabe0b80 1 client.25954 shutdown 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4067445968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 msgr2=0x56071669eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x56071669eb00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f578c00ce00 tx=0x7f578c00c9a0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4067445968 shutdown_connections 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560716706d00 0x560716709120 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56071667f530 0x5607166fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607166792c0 0x56071669eb00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.216+0000 7f577f400700 0 client.25954 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:06.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.215+0000 7f57aabe0b80 1 --2- 172.21.15.79:0/4067445968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56071667ec10 0x56071669f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.216+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4067445968 >> 172.21.15.79:0/4067445968 conn(0x5607165a51f0 msgr2=0x560716696ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:06.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.216+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4067445968 shutdown_connections 2024-06-03T02:25:06.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.216+0000 7f57aabe0b80 1 -- 172.21.15.79:0/4067445968 wait complete. 2024-06-03T02:25:06.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.216+0000 7f57aabe0b80 20 client.25954 trim_cache size 0 max 16384 2024-06-03T02:25:06.220 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:25:06.221 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:25:06.221 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "f90dc1f1-928b-440b-81ea-739ee098ea96", 2024-06-03T02:25:06.221 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:25:06.221 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:25:06.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.216+0000 7f57a0200700 1 -- 172.21.15.79:0/3606296692 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57800a2e60 msgr2=0x7f57800a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:06.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.216+0000 7f57a0200700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57800a2e60 0x7f57800a5320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f578c00ab90 tx=0x7f578c009250 comp rx=0 tx=0).stop 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 -- 172.21.15.79:0/3606296692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57981085f0 msgr2=0x7f5798075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57981085f0 0x7f5798075210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f579000e970 tx=0x7f579000ec80 comp rx=0 tx=0).stop 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 -- 172.21.15.79:0/3606296692 shutdown_connections 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57981085f0 0x7f5798075210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f57800a2e60 0x7f57800a5320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57981024e0 0x7f5798074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 --2- 172.21.15.79:0/3606296692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102ea0 0x7f5798074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 -- 172.21.15.79:0/3606296692 >> 172.21.15.79:0/3606296692 conn(0x7f5798079640 msgr2=0x7f5798100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 -- 172.21.15.79:0/3606296692 shutdown_connections 2024-06-03T02:25:06.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:06.217+0000 7f57a0200700 1 -- 172.21.15.79:0/3606296692 wait complete. 2024-06-03T02:25:07.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:25:07.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:07 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:25:07.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:07 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:07.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:07 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:25:07.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:07 smithi079 ceph-mon[31819]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 568 op/s 2024-06-03T02:25:07.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:07 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:07 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:25:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:07 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:07 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:25:07.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:07 smithi135 ceph-mon[36653]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 568 op/s 2024-06-03T02:25:07.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:07 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:07.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:07 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:25:07.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:07 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:07.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:07 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:25:07.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:07 smithi123 ceph-mon[36640]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 568 op/s 2024-06-03T02:25:07.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:07 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 300.0 s, 612.8 tps, lat 170.297 ms stddev 240.524 2024-06-03T02:25:08.494 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:09.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:09 smithi079 ceph-mon[31819]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 644 op/s 2024-06-03T02:25:09.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:09 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:25:09.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:09 smithi135 ceph-mon[36653]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 644 op/s 2024-06-03T02:25:09.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:09 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:25:09.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:09 smithi123 ceph-mon[36640]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 644 op/s 2024-06-03T02:25:09.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:09 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:25:11.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.742+0000 7f2254c00700 1 -- 172.21.15.79:0/2097753914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2250105090 msgr2=0x7f22501115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:11.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.742+0000 7f2254c00700 1 --2- 172.21.15.79:0/2097753914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2250105090 0x7f22501115a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2238009990 tx=0x7f2238009ca0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.744 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:11 smithi079 ceph-mon[31819]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 581 op/s 2024-06-03T02:25:11.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.744+0000 7f2254c00700 1 -- 172.21.15.79:0/2097753914 shutdown_connections 2024-06-03T02:25:11.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.744+0000 7f2254c00700 1 --2- 172.21.15.79:0/2097753914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2250105090 0x7f22501115a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.744+0000 7f2254c00700 1 --2- 172.21.15.79:0/2097753914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225010a6d0 0x7f225010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.744+0000 7f2254c00700 1 --2- 172.21.15.79:0/2097753914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22501046d0 0x7f2250104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.744+0000 7f2254c00700 1 -- 172.21.15.79:0/2097753914 >> 172.21.15.79:0/2097753914 conn(0x7f22501001c0 msgr2=0x7f22501025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:11.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.746+0000 7f2254c00700 1 -- 172.21.15.79:0/2097753914 shutdown_connections 2024-06-03T02:25:11.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.746+0000 7f2254c00700 1 -- 172.21.15.79:0/2097753914 wait complete. 2024-06-03T02:25:11.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.749+0000 7f2254c00700 1 Processor -- start 2024-06-03T02:25:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.750+0000 7f2254c00700 1 -- start start 2024-06-03T02:25:11.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f2254c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22501046d0 0x7f225019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f2254c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2250105090 0x7f225019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f2254c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f225010a6d0 0x7f225019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f2254c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2250114ab0 con 0x7f2250105090 2024-06-03T02:25:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f2254c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2250114930 con 0x7f22501046d0 2024-06-03T02:25:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f2254c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2250114c30 con 0x7f225010a6d0 2024-06-03T02:25:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f224de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22501046d0 0x7f225019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f224de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22501046d0 0x7f225019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33472/0 (socket says 172.21.15.79:33472) 2024-06-03T02:25:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f224de00700 1 -- 172.21.15.79:0/3259469595 learned_addr learned my addr 172.21.15.79:0/3259469595 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:11.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.751+0000 7f224e800700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f225010a6d0 0x7f225019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.752+0000 7f224de00700 1 -- 172.21.15.79:0/3259469595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f225010a6d0 msgr2=0x7f225019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.752+0000 7f224de00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f225010a6d0 0x7f225019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.752+0000 7f224de00700 1 -- 172.21.15.79:0/3259469595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2250105090 msgr2=0x7f225019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.752+0000 7f224de00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2250105090 0x7f225019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.752+0000 7f224de00700 1 -- 172.21.15.79:0/3259469595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225019fe00 con 0x7f22501046d0 2024-06-03T02:25:11.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.752+0000 7f224de00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22501046d0 0x7f225019ae40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f224800eb30 tx=0x7f224800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.753+0000 7f2246a00700 1 -- 172.21.15.79:0/3259469595 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224800cc90 con 0x7f22501046d0 2024-06-03T02:25:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.753+0000 7f2246a00700 1 -- 172.21.15.79:0/3259469595 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2248004500 con 0x7f22501046d0 2024-06-03T02:25:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.753+0000 7f2254c00700 1 -- 172.21.15.79:0/3259469595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22501a00f0 con 0x7f22501046d0 2024-06-03T02:25:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.753+0000 7f2246a00700 1 -- 172.21.15.79:0/3259469595 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248010430 con 0x7f22501046d0 2024-06-03T02:25:11.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.754+0000 7f2254c00700 1 -- 172.21.15.79:0/3259469595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22501a7970 con 0x7f22501046d0 2024-06-03T02:25:11.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.760+0000 7f2246a00700 1 -- 172.21.15.79:0/3259469595 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2248003f00 con 0x7f22501046d0 2024-06-03T02:25:11.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.762+0000 7f2246a00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22340a2cf0 0x7f22340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.762+0000 7f2246a00700 1 -- 172.21.15.79:0/3259469595 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f22480cf690 con 0x7f22501046d0 2024-06-03T02:25:11.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.763+0000 7f224d400700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22340a2cf0 0x7f22340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.763+0000 7f225f501b80 1 Processor -- start 2024-06-03T02:25:11.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.763+0000 7f225f501b80 1 -- start start 2024-06-03T02:25:11.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.764+0000 7f225f501b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d40f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.764+0000 7f225f501b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 0x556898d3ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.764+0000 7f225f501b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d3b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.764+0000 7f225f501b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556898c49a10 con 0x556898d414b0 2024-06-03T02:25:11.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.764+0000 7f225f501b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556898c8cb00 con 0x556898d3b240 2024-06-03T02:25:11.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.764+0000 7f225f501b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556898c478f0 con 0x556898d40b90 2024-06-03T02:25:11.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.764+0000 7f224de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d40f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.764+0000 7f224de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d40f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41358/0 (socket says 172.21.15.79:41358) 2024-06-03T02:25:11.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.764+0000 7f224de00700 1 -- 172.21.15.79:0/2824870461 learned_addr learned my addr 172.21.15.79:0/2824870461 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:11.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.765+0000 7f224de00700 1 -- 172.21.15.79:0/2824870461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 msgr2=0x556898d3b680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:25:11.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.765+0000 7f224de00700 1 --2- 172.21.15.79:0/2824870461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d3b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.765+0000 7f224de00700 1 -- 172.21.15.79:0/2824870461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 msgr2=0x556898d3ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:11.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.765+0000 7f224de00700 1 --2- 172.21.15.79:0/2824870461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 0x556898d3ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.765+0000 7f224de00700 1 -- 172.21.15.79:0/2824870461 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556898c34f90 con 0x556898d40b90 2024-06-03T02:25:11.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.766+0000 7f224de00700 1 --2- 172.21.15.79:0/2824870461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d40f70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2248009230 tx=0x7f224809b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba0efb63732cdf8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:11.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.766+0000 7f224d400700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22340a2cf0 0x7f22340a51b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22400060b0 tx=0x7f2240006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:11.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.768+0000 7f223f400700 1 -- 172.21.15.79:0/2824870461 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224809bae0 con 0x556898d40b90 2024-06-03T02:25:11.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.768+0000 7f223f400700 1 -- 172.21.15.79:0/2824870461 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22480b4070 con 0x556898d40b90 2024-06-03T02:25:11.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.768+0000 7f223f400700 1 -- 172.21.15.79:0/2824870461 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480cf360 con 0x556898d40b90 2024-06-03T02:25:11.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.769+0000 7f225f501b80 1 -- 172.21.15.79:0/2824870461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 msgr2=0x556898d40f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:11.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.769+0000 7f225f501b80 1 --2- 172.21.15.79:0/2824870461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d40f70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2248009230 tx=0x7f224809b620 comp rx=0 tx=0).stop 2024-06-03T02:25:11.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.769+0000 7f225f501b80 1 -- 172.21.15.79:0/2824870461 shutdown_connections 2024-06-03T02:25:11.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.769+0000 7f225f501b80 1 --2- 172.21.15.79:0/2824870461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d3b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.769+0000 7f225f501b80 1 --2- 172.21.15.79:0/2824870461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d40f70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.769+0000 7f225f501b80 1 --2- 172.21.15.79:0/2824870461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 0x556898d3ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.769+0000 7f225f501b80 1 -- 172.21.15.79:0/2824870461 >> 172.21.15.79:0/2824870461 conn(0x556898c67170 msgr2=0x556898d35ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:11.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.769+0000 7f225f501b80 1 -- 172.21.15.79:0/2824870461 shutdown_connections 2024-06-03T02:25:11.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.769+0000 7f225f501b80 1 -- 172.21.15.79:0/2824870461 wait complete. 2024-06-03T02:25:11.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.770+0000 7f225f501b80 1 -- start start 2024-06-03T02:25:11.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.770+0000 7f225f501b80 1 Processor -- start 2024-06-03T02:25:11.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.773+0000 7f225f501b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d47210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.774+0000 7f225f501b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d40b90 0x556898d44560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.774+0000 7f225f501b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d414b0 0x556898d44aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.774+0000 7f224de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d47210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.774+0000 7f225f501b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556898c28f60 con 0x556898d40b90 2024-06-03T02:25:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.774+0000 7f225f501b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556898c88ac0 con 0x556898d3b240 2024-06-03T02:25:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.774+0000 7f225f501b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556898d47e30 con 0x556898d414b0 2024-06-03T02:25:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.774+0000 7f225f501b80 10 client.0 _ll_register_callbacks cb 0x556898c22380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.774+0000 7f224d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d40b90 0x556898d44560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.775+0000 7f224d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d40b90 0x556898d44560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35934/0 (socket says 172.21.15.79:35934) 2024-06-03T02:25:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.775+0000 7f224d400700 1 -- 172.21.15.79:0/1093192039 learned_addr learned my addr 172.21.15.79:0/1093192039 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.776+0000 7f224de00700 1 -- 172.21.15.79:0/1093192039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d414b0 msgr2=0x556898d44aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.776+0000 7f224de00700 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d414b0 0x556898d44aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.776+0000 7f224de00700 1 -- 172.21.15.79:0/1093192039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d40b90 msgr2=0x556898d44560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.776+0000 7f224de00700 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d40b90 0x556898d44560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.776+0000 7f224de00700 1 -- 172.21.15.79:0/1093192039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556898bdb8d0 con 0x556898d3b240 2024-06-03T02:25:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.776+0000 7f224de00700 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d47210 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2248003840 tx=0x7f224809c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d8a419da9c34a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.776+0000 7f2233e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:25:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.777+0000 7f2233e00700 1 -- 172.21.15.79:0/1093192039 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224809bae0 con 0x556898d3b240 2024-06-03T02:25:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.777+0000 7f2233e00700 1 -- 172.21.15.79:0/1093192039 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f22480cf360 con 0x556898d3b240 2024-06-03T02:25:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f225f501b80 1 -- 172.21.15.79:0/1093192039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22480b4070 con 0x556898d3b240 2024-06-03T02:25:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f2233e00700 1 -- 172.21.15.79:0/1093192039 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2248098150 con 0x556898d3b240 2024-06-03T02:25:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f225f501b80 10 client.25976 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f225f501b80 1 -- 172.21.15.79:0/1093192039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556898d52580 con 0x556898d3b240 2024-06-03T02:25:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f2233e00700 1 -- 172.21.15.79:0/1093192039 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22480b4070 con 0x556898d3b240 2024-06-03T02:25:11.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f2233e00700 1 -- 172.21.15.79:0/1093192039 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f22480b4ce0 con 0x556898d3b240 2024-06-03T02:25:11.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f225f501b80 10 client.25976 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:11.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f225f501b80 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:11.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f225f501b80 20 client.25976 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:11.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.778+0000 7f225f501b80 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x556898d54a50 0x556898d56e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.779+0000 7f225f501b80 4 client.25976 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:25:11.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.779+0000 7f225f501b80 1 -- 172.21.15.79:0/1093192039 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556898c9a000 con 0x556898d54a50 2024-06-03T02:25:11.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.779+0000 7f224d400700 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x556898d54a50 0x556898d56e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.791+0000 7f2233e00700 10 client.25976 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:11.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.791+0000 7f224d400700 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x556898d54a50 0x556898d56e40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eaeac37609ee1ba server_cookie=d8f50127966cfaf3 in_seq=0 out_seq=0 2024-06-03T02:25:11.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f2233e00700 1 -- 172.21.15.79:0/1093192039 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556898c9a000 con 0x556898d54a50 2024-06-03T02:25:11.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f2233e00700 10 client.25976 handle_command_reply: tid=0 2024-06-03T02:25:11.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f225f501b80 1 client.25976 shutdown 2024-06-03T02:25:11.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f225f501b80 1 -- 172.21.15.79:0/1093192039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 msgr2=0x556898d47210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:11.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f225f501b80 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d47210 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2248003840 tx=0x7f224809c340 comp rx=0 tx=0).stop 2024-06-03T02:25:11.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f225f501b80 1 -- 172.21.15.79:0/1093192039 shutdown_connections 2024-06-03T02:25:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f225f501b80 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d414b0 0x556898d44aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f225f501b80 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d47210 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f225f501b80 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x556898d54a50 0x556898d56e40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f225f501b80 1 --2- 172.21.15.79:0/1093192039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d40b90 0x556898d44560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f2233e00700 0 client.25976 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.793+0000 7f225f501b80 1 -- 172.21.15.79:0/1093192039 >> 172.21.15.79:0/1093192039 conn(0x556898c67170 msgr2=0x556898d2f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.794+0000 7f225f501b80 1 -- 172.21.15.79:0/1093192039 shutdown_connections 2024-06-03T02:25:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.794+0000 7f225f501b80 1 -- 172.21.15.79:0/1093192039 wait complete. 2024-06-03T02:25:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.794+0000 7f225f501b80 20 client.25976 trim_cache size 0 max 16384 2024-06-03T02:25:11.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.837+0000 7f225f501b80 1 Processor -- start 2024-06-03T02:25:11.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.838+0000 7f225f501b80 1 -- start start 2024-06-03T02:25:11.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.839+0000 7f225f501b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d3b240 0x556898d3ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.839+0000 7f225f501b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d30130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.839+0000 7f225f501b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d414b0 0x556898d3e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.839+0000 7f225f501b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b06b0 con 0x556898d3b240 2024-06-03T02:25:11.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.839+0000 7f225f501b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b21f0 con 0x556898d414b0 2024-06-03T02:25:11.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.839+0000 7f225f501b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b0d80 con 0x556898d40b90 2024-06-03T02:25:11.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.840+0000 7f224d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d30130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.840+0000 7f224d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d30130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41376/0 (socket says 172.21.15.79:41376) 2024-06-03T02:25:11.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.840+0000 7f224e800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d414b0 0x556898d3e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.840+0000 7f224d400700 1 -- 172.21.15.79:0/3250156112 learned_addr learned my addr 172.21.15.79:0/3250156112 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:11.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.840+0000 7f224de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d3b240 0x556898d3ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.841+0000 7f224d400700 1 -- 172.21.15.79:0/3250156112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d414b0 msgr2=0x556898d3e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.841+0000 7f224d400700 1 --2- 172.21.15.79:0/3250156112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d414b0 0x556898d3e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.841+0000 7f224d400700 1 -- 172.21.15.79:0/3250156112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d3b240 msgr2=0x556898d3ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.841+0000 7f224d400700 1 --2- 172.21.15.79:0/3250156112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d3b240 0x556898d3ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.841+0000 7f224d400700 1 -- 172.21.15.79:0/3250156112 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556898d42ad0 con 0x556898d40b90 2024-06-03T02:25:11.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.842+0000 7f224e800700 1 --2- 172.21.15.79:0/3250156112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d414b0 0x556898d3e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:11.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.842+0000 7f224de00700 1 --2- 172.21.15.79:0/3250156112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d3b240 0x556898d3ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:11.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.843+0000 7f224d400700 1 --2- 172.21.15.79:0/3250156112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d30130 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x556898d47f40 tx=0x7f2240009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8aa5ec58fc14033 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:11.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.843+0000 7f223ea00700 1 -- 172.21.15.79:0/3250156112 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240023460 con 0x556898d40b90 2024-06-03T02:25:11.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.843+0000 7f223ea00700 1 -- 172.21.15.79:0/3250156112 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2240023ae0 con 0x556898d40b90 2024-06-03T02:25:11.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.843+0000 7f223ea00700 1 -- 172.21.15.79:0/3250156112 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22400187a0 con 0x556898d40b90 2024-06-03T02:25:11.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.846+0000 7f225f501b80 1 -- 172.21.15.79:0/3250156112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 msgr2=0x556898d30130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:11.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.846+0000 7f225f501b80 1 --2- 172.21.15.79:0/3250156112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d30130 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x556898d47f40 tx=0x7f2240009620 comp rx=0 tx=0).stop 2024-06-03T02:25:11.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.846+0000 7f225f501b80 1 -- 172.21.15.79:0/3250156112 shutdown_connections 2024-06-03T02:25:11.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.846+0000 7f225f501b80 1 --2- 172.21.15.79:0/3250156112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d414b0 0x556898d3e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.846+0000 7f225f501b80 1 --2- 172.21.15.79:0/3250156112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d3b240 0x556898d3ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.846+0000 7f225f501b80 1 --2- 172.21.15.79:0/3250156112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d30130 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.846+0000 7f225f501b80 1 -- 172.21.15.79:0/3250156112 >> 172.21.15.79:0/3250156112 conn(0x556898c67170 msgr2=0x556898d41f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:11.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.846+0000 7f225f501b80 1 -- 172.21.15.79:0/3250156112 shutdown_connections 2024-06-03T02:25:11.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.847+0000 7f225f501b80 1 -- 172.21.15.79:0/3250156112 wait complete. 2024-06-03T02:25:11.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.847+0000 7f225f501b80 1 -- start start 2024-06-03T02:25:11.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.848+0000 7f225f501b80 1 Processor -- start 2024-06-03T02:25:11.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.849+0000 7f225f501b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d5a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.849+0000 7f225f501b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d57950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.849+0000 7f224de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d5a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.849+0000 7f224de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d5a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33506/0 (socket says 172.21.15.79:33506) 2024-06-03T02:25:11.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.849+0000 7f225f501b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 0x556898d57e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.850+0000 7f224de00700 1 -- 172.21.15.79:0/1896190238 learned_addr learned my addr 172.21.15.79:0/1896190238 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:11.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.850+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b1b20 con 0x556898d414b0 2024-06-03T02:25:11.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.850+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22501b1450 con 0x556898d3b240 2024-06-03T02:25:11.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.850+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22501ae650 con 0x556898d40b90 2024-06-03T02:25:11.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.850+0000 7f225f501b80 10 client.0 _ll_register_callbacks cb 0x556898c22380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:11.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.850+0000 7f224d400700 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d57950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.850+0000 7f224e800700 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 0x556898d57e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.851+0000 7f224e800700 1 -- 172.21.15.79:0/1896190238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 msgr2=0x556898d57950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.851+0000 7f224e800700 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d57950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.851+0000 7f224e800700 1 -- 172.21.15.79:0/1896190238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 msgr2=0x556898d5a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.851+0000 7f224e800700 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d5a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.851+0000 7f224e800700 1 -- 172.21.15.79:0/1896190238 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556898d58690 con 0x556898d414b0 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.852+0000 7f224e800700 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 0x556898d57e90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x556898d306d0 tx=0x7f223800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a87a1ec331247bc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.852+0000 7f2233e00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.853+0000 7f2233e00700 1 -- 172.21.15.79:0/1896190238 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238005620 con 0x556898d414b0 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.853+0000 7f2233e00700 1 -- 172.21.15.79:0/1896190238 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2238003680 con 0x556898d414b0 2024-06-03T02:25:11.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.853+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2240023ae0 con 0x556898d414b0 2024-06-03T02:25:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.854+0000 7f2233e00700 1 -- 172.21.15.79:0/1896190238 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2238003880 con 0x556898d414b0 2024-06-03T02:25:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.854+0000 7f225f501b80 10 client.15672 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.854+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556898dc7b80 con 0x556898d414b0 2024-06-03T02:25:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.854+0000 7f2233e00700 1 -- 172.21.15.79:0/1896190238 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2240023ae0 con 0x556898d414b0 2024-06-03T02:25:11.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.855+0000 7f2233e00700 1 -- 172.21.15.79:0/1896190238 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f223800fb40 con 0x556898d414b0 2024-06-03T02:25:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.855+0000 7f225f501b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.855+0000 7f225f501b80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.855+0000 7f225f501b80 20 client.15672 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.855+0000 7f225f501b80 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x556898dc8310 0x556898dca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.855+0000 7f225f501b80 4 client.15672 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:25:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.855+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556898c94580 con 0x556898dc8310 2024-06-03T02:25:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.855+0000 7f224de00700 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x556898dc8310 0x556898dca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:11.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.856+0000 7f2233e00700 10 client.15672 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:11.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.858+0000 7f224de00700 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x556898dc8310 0x556898dca730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f80c906a24fc29 server_cookie=af3678d68597726b in_seq=0 out_seq=0 2024-06-03T02:25:11.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.858+0000 7f2233e00700 1 -- 172.21.15.79:0/1896190238 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556898c94580 con 0x556898dc8310 2024-06-03T02:25:11.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.858+0000 7f2233e00700 10 client.15672 handle_command_reply: tid=0 2024-06-03T02:25:11.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.858+0000 7f225f501b80 1 client.15672 shutdown 2024-06-03T02:25:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.860+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 msgr2=0x556898d57e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.860+0000 7f225f501b80 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 0x556898d57e90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x556898d306d0 tx=0x7f223800be30 comp rx=0 tx=0).stop 2024-06-03T02:25:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.860+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 shutdown_connections 2024-06-03T02:25:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.861+0000 7f225f501b80 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x556898dc8310 0x556898dca730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.861+0000 7f225f501b80 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556898d414b0 0x556898d57e90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.861+0000 7f225f501b80 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556898d3b240 0x556898d5a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.861+0000 7f225f501b80 1 --2- 172.21.15.79:0/1896190238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556898d40b90 0x556898d57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.861+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 >> 172.21.15.79:0/1896190238 conn(0x556898c67170 msgr2=0x556898d3f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:11.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.861+0000 7f2233e00700 0 client.15672 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:11.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.861+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 shutdown_connections 2024-06-03T02:25:11.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.861+0000 7f225f501b80 1 -- 172.21.15.79:0/1896190238 wait complete. 2024-06-03T02:25:11.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.861+0000 7f225f501b80 20 client.15672 trim_cache size 0 max 16384 2024-06-03T02:25:11.865 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:25:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:25:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:25:11.866 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:25:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 -- 172.21.15.79:0/3259469595 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22340a2cf0 msgr2=0x7f22340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:11.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22340a2cf0 0x7f22340a51b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22400060b0 tx=0x7f2240006040 comp rx=0 tx=0).stop 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 -- 172.21.15.79:0/3259469595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22501046d0 msgr2=0x7f225019ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22501046d0 0x7f225019ae40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f224800eb30 tx=0x7f224800ee40 comp rx=0 tx=0).stop 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 -- 172.21.15.79:0/3259469595 shutdown_connections 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f225010a6d0 0x7f225019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22501046d0 0x7f225019ae40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f22340a2cf0 0x7f22340a51b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 --2- 172.21.15.79:0/3259469595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2250105090 0x7f225019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.862+0000 7f2254c00700 1 -- 172.21.15.79:0/3259469595 >> 172.21.15.79:0/3259469595 conn(0x7f22501001c0 msgr2=0x7f2250100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.863+0000 7f2254c00700 1 -- 172.21.15.79:0/3259469595 shutdown_connections 2024-06-03T02:25:11.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:11.863+0000 7f2254c00700 1 -- 172.21.15.79:0/3259469595 wait complete. 2024-06-03T02:25:11.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:11 smithi123 ceph-mon[36640]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 581 op/s 2024-06-03T02:25:12.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:11 smithi135 ceph-mon[36653]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 581 op/s 2024-06-03T02:25:12.764 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:25:12.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:25:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 305.0 s, 707.2 tps, lat 137.705 ms stddev 153.189 2024-06-03T02:25:13.464 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:13.957 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:13 smithi079 ceph-mon[31819]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 636 op/s 2024-06-03T02:25:13.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:13 smithi123 ceph-mon[36640]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 636 op/s 2024-06-03T02:25:14.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:13 smithi135 ceph-mon[36653]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 636 op/s 2024-06-03T02:25:15.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.952+0000 7fd4d1a00700 1 -- 172.21.15.79:0/3616632554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4cc1073c0 msgr2=0x7fd4cc06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:15.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.952+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/3616632554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4cc1073c0 0x7fd4cc06a1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400b0a0 tx=0x7fd4c400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:25:15.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.954+0000 7fd4d1a00700 1 -- 172.21.15.79:0/3616632554 shutdown_connections 2024-06-03T02:25:15.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.954+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/3616632554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4cc104ed0 0x7fd4cc105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:15.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.954+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/3616632554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4cc1073c0 0x7fd4cc06a1a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:15.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.954+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/3616632554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc06a6e0 0x7fd4cc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:15.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.954+0000 7fd4d1a00700 1 -- 172.21.15.79:0/3616632554 >> 172.21.15.79:0/3616632554 conn(0x7fd4cc078340 msgr2=0x7fd4cc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:15.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:15 smithi079 ceph-mon[31819]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 666 op/s 2024-06-03T02:25:15.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.954+0000 7fd4d1a00700 1 -- 172.21.15.79:0/3616632554 shutdown_connections 2024-06-03T02:25:15.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.954+0000 7fd4d1a00700 1 -- 172.21.15.79:0/3616632554 wait complete. 2024-06-03T02:25:15.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.956+0000 7fd4d1a00700 1 Processor -- start 2024-06-03T02:25:15.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.956+0000 7fd4d1a00700 1 -- start start 2024-06-03T02:25:15.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.957+0000 7fd4d1a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4cc06a6e0 0x7fd4cc19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:15.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.957+0000 7fd4d1a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4cc104ed0 0x7fd4cc19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:15.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.957+0000 7fd4d1a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc1073c0 0x7fd4cc1a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:15.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.957+0000 7fd4d1a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc118e40 con 0x7fd4cc06a6e0 2024-06-03T02:25:15.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.957+0000 7fd4d1a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc118cc0 con 0x7fd4cc104ed0 2024-06-03T02:25:15.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.957+0000 7fd4d1a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc118fc0 con 0x7fd4cc1073c0 2024-06-03T02:25:15.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.957+0000 7fd4cb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc1073c0 0x7fd4cc1a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:15.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.957+0000 7fd4cb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc1073c0 0x7fd4cc1a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:48206/0 (socket says 172.21.15.79:48206) 2024-06-03T02:25:15.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.957+0000 7fd4cb400700 1 -- 172.21.15.79:0/1963996995 learned_addr learned my addr 172.21.15.79:0/1963996995 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:15.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.958+0000 7fd4ca000700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4cc104ed0 0x7fd4cc19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:15.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.958+0000 7fd4cb400700 1 -- 172.21.15.79:0/1963996995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4cc104ed0 msgr2=0x7fd4cc19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:15.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.958+0000 7fd4cb400700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4cc104ed0 0x7fd4cc19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:15.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.958+0000 7fd4cb400700 1 -- 172.21.15.79:0/1963996995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4cc06a6e0 msgr2=0x7fd4cc19f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:15.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.958+0000 7fd4cb400700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4cc06a6e0 0x7fd4cc19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:15.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.958+0000 7fd4cb400700 1 -- 172.21.15.79:0/1963996995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc1a4280 con 0x7fd4cc1073c0 2024-06-03T02:25:15.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.958+0000 7fd4cb400700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc1073c0 0x7fd4cc1a3b40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00e940 tx=0x7fd4bc00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.959+0000 7fd4c3400700 1 -- 172.21.15.79:0/1963996995 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00cbd0 con 0x7fd4cc1073c0 2024-06-03T02:25:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.959+0000 7fd4c3400700 1 -- 172.21.15.79:0/1963996995 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc00ee90 con 0x7fd4cc1073c0 2024-06-03T02:25:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.959+0000 7fd4c3400700 1 -- 172.21.15.79:0/1963996995 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0056a0 con 0x7fd4cc1073c0 2024-06-03T02:25:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.959+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc1a4570 con 0x7fd4cc1073c0 2024-06-03T02:25:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.959+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc1abdf0 con 0x7fd4cc1073c0 2024-06-03T02:25:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.961+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc069180 con 0x7fd4cc1073c0 2024-06-03T02:25:15.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.967+0000 7fd4c3400700 1 -- 172.21.15.79:0/1963996995 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd4bc004830 con 0x7fd4cc1073c0 2024-06-03T02:25:15.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:15 smithi123 ceph-mon[36640]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 666 op/s 2024-06-03T02:25:15.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.972+0000 7fd4c3400700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4b00a2ca0 0x7fd4b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:15.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.973+0000 7fd4c3400700 1 -- 172.21.15.79:0/1963996995 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd4bc0cf670 con 0x7fd4cc1073c0 2024-06-03T02:25:15.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.973+0000 7fd4caa00700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4b00a2ca0 0x7fd4b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:15.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.973+0000 7fd4c3400700 1 -- 172.21.15.79:0/1963996995 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd4bc0d1c20 con 0x7fd4cc1073c0 2024-06-03T02:25:15.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:15.974+0000 7fd4caa00700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4b00a2ca0 0x7fd4b00a5160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009e60 tx=0x7fd4c400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:15 smithi135 ceph-mon[36653]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 666 op/s 2024-06-03T02:25:16.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.465+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4cc1ac090 con 0x7fd4cc1073c0 2024-06-03T02:25:16.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.466+0000 7fd4c3400700 1 -- 172.21.15.79:0/1963996995 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7fd4bc00b2c0 con 0x7fd4cc1073c0 2024-06-03T02:25:16.467 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:25:16.467 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:25:16.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.471+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4b00a2ca0 msgr2=0x7fd4b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:16.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.471+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4b00a2ca0 0x7fd4b00a5160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009e60 tx=0x7fd4c400b700 comp rx=0 tx=0).stop 2024-06-03T02:25:16.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.471+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc1073c0 msgr2=0x7fd4cc1a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:16.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.471+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc1073c0 0x7fd4cc1a3b40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00e940 tx=0x7fd4bc00ec50 comp rx=0 tx=0).stop 2024-06-03T02:25:16.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.474+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 shutdown_connections 2024-06-03T02:25:16.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.474+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc1073c0 0x7fd4cc1a3b40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:16.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.474+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4b00a2ca0 0x7fd4b00a5160 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:16.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.474+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4cc06a6e0 0x7fd4cc19f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:16.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.475+0000 7fd4d1a00700 1 --2- 172.21.15.79:0/1963996995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4cc104ed0 0x7fd4cc19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:16.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.475+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 >> 172.21.15.79:0/1963996995 conn(0x7fd4cc078340 msgr2=0x7fd4cc100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:16.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.476+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 shutdown_connections 2024-06-03T02:25:16.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:16.476+0000 7fd4d1a00700 1 -- 172.21.15.79:0/1963996995 wait complete. 2024-06-03T02:25:16.482 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:25:16.785 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:16 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1963996995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:25:16.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:16 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1963996995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:25:17.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:16 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1963996995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:25:17.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:25:17.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:17 smithi079 ceph-mon[31819]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 575 op/s 2024-06-03T02:25:17.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:17 smithi123 ceph-mon[36640]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 575 op/s 2024-06-03T02:25:17.989 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:18.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:17 smithi135 ceph-mon[36653]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 575 op/s 2024-06-03T02:25:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 310.0 s, 492.6 tps, lat 199.587 ms stddev 287.110 2024-06-03T02:25:19.895 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:19 smithi079 ceph-mon[31819]: pgmap v674: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 655 op/s 2024-06-03T02:25:19.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:19 smithi123 ceph-mon[36640]: pgmap v674: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 655 op/s 2024-06-03T02:25:20.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:19 smithi135 ceph-mon[36653]: pgmap v674: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 655 op/s 2024-06-03T02:25:21.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.900+0000 7fa700c00700 1 -- 172.21.15.79:0/939270437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 msgr2=0x7fa6fc06abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:21.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.900+0000 7fa700c00700 1 --2- 172.21.15.79:0/939270437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 0x7fa6fc06abc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009990 tx=0x7fa6ec009ca0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.902 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:21 smithi079 ceph-mon[31819]: pgmap v675: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 561 op/s 2024-06-03T02:25:21.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.902+0000 7fa700c00700 1 -- 172.21.15.79:0/939270437 shutdown_connections 2024-06-03T02:25:21.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.902+0000 7fa700c00700 1 --2- 172.21.15.79:0/939270437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6fc104ec0 0x7fa6fc105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.902+0000 7fa700c00700 1 --2- 172.21.15.79:0/939270437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 0x7fa6fc06abc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.902+0000 7fa700c00700 1 --2- 172.21.15.79:0/939270437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6fc06b100 0x7fa6fc06b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.902+0000 7fa700c00700 1 -- 172.21.15.79:0/939270437 >> 172.21.15.79:0/939270437 conn(0x7fa6fc078340 msgr2=0x7fa6fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:21.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.902+0000 7fa700c00700 1 -- 172.21.15.79:0/939270437 shutdown_connections 2024-06-03T02:25:21.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.902+0000 7fa700c00700 1 -- 172.21.15.79:0/939270437 wait complete. 2024-06-03T02:25:21.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.904+0000 7fa700c00700 1 Processor -- start 2024-06-03T02:25:21.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.904+0000 7fa700c00700 1 -- start start 2024-06-03T02:25:21.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.905+0000 7fa700c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6fc06b100 0x7fa6fc1156e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.905+0000 7fa700c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6fc104ec0 0x7fa6fc115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.905+0000 7fa700c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 0x7fa6fc10e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.905+0000 7fa700c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc118db0 con 0x7fa6fc104ec0 2024-06-03T02:25:21.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.905+0000 7fa700c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc118c30 con 0x7fa6fc06b100 2024-06-03T02:25:21.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.905+0000 7fa700c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc118f30 con 0x7fa6fc1073b0 2024-06-03T02:25:21.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.905+0000 7fa6fa800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 0x7fa6fc10e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.905+0000 7fa6fa800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 0x7fa6fc10e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:48230/0 (socket says 172.21.15.79:48230) 2024-06-03T02:25:21.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.905+0000 7fa6fa800700 1 -- 172.21.15.79:0/1286729583 learned_addr learned my addr 172.21.15.79:0/1286729583 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:21.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.906+0000 7fa6f9400700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6fc104ec0 0x7fa6fc115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.906+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6fc06b100 0x7fa6fc1156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.907+0000 7fa6fa800700 1 -- 172.21.15.79:0/1286729583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6fc06b100 msgr2=0x7fa6fc1156e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:21.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.908+0000 7fa6fa800700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6fc06b100 0x7fa6fc1156e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.908+0000 7fa6fa800700 1 -- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6fc104ec0 msgr2=0x7fa6fc115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:21.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.908+0000 7fa6fa800700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6fc104ec0 0x7fa6fc115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.908+0000 7fa6fa800700 1 -- 172.21.15.79:0/1286729583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc10f200 con 0x7fa6fc1073b0 2024-06-03T02:25:21.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.908+0000 7fa6f9400700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6fc104ec0 0x7fa6fc115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:21.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.908+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6fc06b100 0x7fa6fc1156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:21.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.908+0000 7fa6fa800700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 0x7fa6fc10e900 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400ef30 tx=0x7fa6f400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:21.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.909+0000 7fa6f2a00700 1 -- 172.21.15.79:0/1286729583 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40092e0 con 0x7fa6fc1073b0 2024-06-03T02:25:21.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.909+0000 7fa6f2a00700 1 -- 172.21.15.79:0/1286729583 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f4004bb0 con 0x7fa6fc1073b0 2024-06-03T02:25:21.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.909+0000 7fa6f2a00700 1 -- 172.21.15.79:0/1286729583 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4004280 con 0x7fa6fc1073b0 2024-06-03T02:25:21.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.909+0000 7fa700c00700 1 -- 172.21.15.79:0/1286729583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc10f490 con 0x7fa6fc1073b0 2024-06-03T02:25:21.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.909+0000 7fa700c00700 1 -- 172.21.15.79:0/1286729583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc1133e0 con 0x7fa6fc1073b0 2024-06-03T02:25:21.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.919+0000 7fa6f2a00700 1 -- 172.21.15.79:0/1286729583 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa6f401b030 con 0x7fa6fc1073b0 2024-06-03T02:25:21.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.920+0000 7fa70b662b80 1 Processor -- start 2024-06-03T02:25:21.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.921+0000 7fa70b662b80 1 -- start start 2024-06-03T02:25:21.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.921+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e8786df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.922+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87867d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.922+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 0x559e87868690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.922+0000 7fa70b662b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559e87765a10 con 0x559e8786e4c0 2024-06-03T02:25:21.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.922+0000 7fa70b662b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559e877a8b00 con 0x559e8786dba0 2024-06-03T02:25:21.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.922+0000 7fa70b662b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559e877638f0 con 0x559e87868250 2024-06-03T02:25:21.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.922+0000 7fa6f9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87867d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.923+0000 7fa6f9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87867d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51012/0 (socket says 172.21.15.79:51012) 2024-06-03T02:25:21.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.923+0000 7fa6f9e00700 1 -- 172.21.15.79:0/172928213 learned_addr learned my addr 172.21.15.79:0/172928213 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:21.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.923+0000 7fa6f9400700 1 --2- 172.21.15.79:0/172928213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e8786df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.923+0000 7fa6f9400700 1 --2- 172.21.15.79:0/172928213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 0x559e87868690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.923+0000 7fa6f2a00700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa6e00a2ca0 0x7fa6e00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.923+0000 7fa6f2a00700 1 -- 172.21.15.79:0/1286729583 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa6f40cef50 con 0x7fa6fc1073b0 2024-06-03T02:25:21.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.924+0000 7fa6f9e00700 1 -- 172.21.15.79:0/172928213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 msgr2=0x559e87868690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:21.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.924+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/172928213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 0x559e87868690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.924+0000 7fa6f9e00700 1 -- 172.21.15.79:0/172928213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 msgr2=0x559e8786df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:21.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.924+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/172928213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e8786df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.924+0000 7fa6f9e00700 1 -- 172.21.15.79:0/172928213 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e87750f90 con 0x559e8786e4c0 2024-06-03T02:25:21.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.925+0000 7fa6fa800700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa6e00a2ca0 0x7fa6e00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.926+0000 7fa6fa800700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa6e00a2ca0 0x7fa6e00a5160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa6f409b190 tx=0x7fa6f4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:21.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.929+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/172928213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87867d10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00b2e0 tx=0x7fa6ec005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2bc47708327045 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:21.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.929+0000 7fa6e7400700 1 -- 172.21.15.79:0/172928213 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec009510 con 0x559e8786e4c0 2024-06-03T02:25:21.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.931+0000 7fa6e7400700 1 -- 172.21.15.79:0/172928213 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec004d10 con 0x559e8786e4c0 2024-06-03T02:25:21.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.932+0000 7fa70b662b80 1 -- 172.21.15.79:0/172928213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 msgr2=0x559e87867d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:21.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.932+0000 7fa6e7400700 1 -- 172.21.15.79:0/172928213 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec009510 con 0x559e8786e4c0 2024-06-03T02:25:21.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.932+0000 7fa70b662b80 1 --2- 172.21.15.79:0/172928213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87867d10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00b2e0 tx=0x7fa6ec005c80 comp rx=0 tx=0).stop 2024-06-03T02:25:21.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.933+0000 7fa70b662b80 1 -- 172.21.15.79:0/172928213 shutdown_connections 2024-06-03T02:25:21.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.933+0000 7fa70b662b80 1 --2- 172.21.15.79:0/172928213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 0x559e87868690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.933+0000 7fa70b662b80 1 --2- 172.21.15.79:0/172928213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e8786df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.933+0000 7fa70b662b80 1 --2- 172.21.15.79:0/172928213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87867d10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.933+0000 7fa70b662b80 1 -- 172.21.15.79:0/172928213 >> 172.21.15.79:0/172928213 conn(0x559e87783170 msgr2=0x559e87851ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:21.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.933+0000 7fa70b662b80 1 -- 172.21.15.79:0/172928213 shutdown_connections 2024-06-03T02:25:21.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.935+0000 7fa70b662b80 1 -- 172.21.15.79:0/172928213 wait complete. 2024-06-03T02:25:21.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.935+0000 7fa70b662b80 1 -- start start 2024-06-03T02:25:21.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.937+0000 7fa70b662b80 1 Processor -- start 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.938+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e87868250 0x559e87874150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.938+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e87871450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.938+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786e4c0 0x559e87871990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.938+0000 7fa6f9e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e87871450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.938+0000 7fa6f9e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e87871450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:52886/0 (socket says 172.21.15.79:52886) 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.938+0000 7fa6f9e00700 1 -- 172.21.15.79:0/1265438441 learned_addr learned my addr 172.21.15.79:0/1265438441 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.938+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559e87744f60 con 0x559e87868250 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.938+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559e877a4ac0 con 0x559e8786dba0 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.938+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559e87874e30 con 0x559e8786e4c0 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.939+0000 7fa6f9400700 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786e4c0 0x559e87871990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.939+0000 7fa70b662b80 10 client.0 _ll_register_callbacks cb 0x559e8773e380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.939+0000 7fa6f9400700 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e87868250 0x559e87874150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.939+0000 7fa6f9e00700 1 -- 172.21.15.79:0/1265438441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786e4c0 msgr2=0x559e87871990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.939+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786e4c0 0x559e87871990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.939+0000 7fa6f9e00700 1 -- 172.21.15.79:0/1265438441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e87868250 msgr2=0x559e87874150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.939+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e87868250 0x559e87874150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.939+0000 7fa6f9e00700 1 -- 172.21.15.79:0/1265438441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e876f78d0 con 0x559e8786dba0 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.940+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e87871450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec005b40 tx=0x7fa6ec004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94bf3283d6208a47 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.940+0000 7fa6dbe00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.940+0000 7fa6dbe00700 1 -- 172.21.15.79:0/1265438441 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec030020 con 0x559e8786dba0 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.940+0000 7fa6dbe00700 1 -- 172.21.15.79:0/1265438441 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa6ec01e040 con 0x559e8786dba0 2024-06-03T02:25:21.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.940+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6ec004d10 con 0x559e8786dba0 2024-06-03T02:25:21.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.941+0000 7fa6dbe00700 1 -- 172.21.15.79:0/1265438441 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6ec004ec0 con 0x559e8786dba0 2024-06-03T02:25:21.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.941+0000 7fa70b662b80 10 client.25994 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:21.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.941+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e8787f300 con 0x559e8786dba0 2024-06-03T02:25:21.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.941+0000 7fa6dbe00700 1 -- 172.21.15.79:0/1265438441 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec004d10 con 0x559e8786dba0 2024-06-03T02:25:21.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.941+0000 7fa6dbe00700 1 -- 172.21.15.79:0/1265438441 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa6ec00f460 con 0x559e8786dba0 2024-06-03T02:25:21.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.941+0000 7fa70b662b80 10 client.25994 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:21.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.941+0000 7fa70b662b80 10 client.25994 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:25:21.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.942+0000 7fa70b662b80 20 client.25994 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:21.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.942+0000 7fa70b662b80 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559e878817d0 0x559e87883bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.942+0000 7fa70b662b80 4 client.25994 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:25:21.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.942+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e877b6000 con 0x559e878817d0 2024-06-03T02:25:21.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.943+0000 7fa6f9400700 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559e878817d0 0x559e87883bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.945+0000 7fa6f9400700 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559e878817d0 0x559e87883bc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c1a2c3704314b60 server_cookie=59c926c1e2fbdcc6 in_seq=0 out_seq=0 2024-06-03T02:25:21.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.945+0000 7fa6dbe00700 10 client.25994 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:25:21.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.946+0000 7fa6dbe00700 1 -- 172.21.15.79:0/1265438441 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559e877b6000 con 0x559e878817d0 2024-06-03T02:25:21.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.946+0000 7fa6dbe00700 10 client.25994 handle_command_reply: tid=0 2024-06-03T02:25:21.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.946+0000 7fa70b662b80 1 client.25994 shutdown 2024-06-03T02:25:21.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.947+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 msgr2=0x559e87871450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:21.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.947+0000 7fa70b662b80 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e87871450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec005b40 tx=0x7fa6ec004af0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.947+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 shutdown_connections 2024-06-03T02:25:21.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.947+0000 7fa70b662b80 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559e878817d0 0x559e87883bc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.947+0000 7fa70b662b80 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786e4c0 0x559e87871990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.947+0000 7fa70b662b80 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e87868250 0x559e87874150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.947+0000 7fa6dbe00700 0 client.25994 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:25:21.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.947+0000 7fa70b662b80 1 --2- 172.21.15.79:0/1265438441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e87871450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.947+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 >> 172.21.15.79:0/1265438441 conn(0x559e87783170 msgr2=0x559e8784b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:21.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.948+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 shutdown_connections 2024-06-03T02:25:21.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.948+0000 7fa70b662b80 1 -- 172.21.15.79:0/1265438441 wait complete. 2024-06-03T02:25:21.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.949+0000 7fa70b662b80 20 client.25994 trim_cache size 0 max 16384 2024-06-03T02:25:21.975 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:21 smithi123 ceph-mon[36640]: pgmap v675: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 561 op/s 2024-06-03T02:25:21.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.984+0000 7fa70b662b80 1 Processor -- start 2024-06-03T02:25:21.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.984+0000 7fa70b662b80 1 -- start start 2024-06-03T02:25:21.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.984+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 0x559e87853430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.985+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e8784c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.985+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87851a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.985+0000 7fa70b662b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc073e00 con 0x559e8786e4c0 2024-06-03T02:25:21.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.985+0000 7fa70b662b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1b6540 con 0x559e8786dba0 2024-06-03T02:25:21.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.985+0000 7fa70b662b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc0744d0 con 0x559e87868250 2024-06-03T02:25:21.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.985+0000 7fa6f9400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87851a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.985+0000 7fa6f9400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87851a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51020/0 (socket says 172.21.15.79:51020) 2024-06-03T02:25:21.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.985+0000 7fa6f9400700 1 -- 172.21.15.79:0/4292948501 learned_addr learned my addr 172.21.15.79:0/4292948501 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:21.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.985+0000 7fa6f9400700 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 0x559e87853430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.986+0000 7fa6f9400700 1 -- 172.21.15.79:0/4292948501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 msgr2=0x559e87853430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:21.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.986+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e8784c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:21.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.986+0000 7fa6f9400700 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 0x559e87853430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.987+0000 7fa6f9400700 1 -- 172.21.15.79:0/4292948501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 msgr2=0x559e8784c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:21.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.987+0000 7fa6f9400700 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e8784c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.988+0000 7fa6f9400700 1 -- 172.21.15.79:0/4292948501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e8787e150 con 0x559e8786e4c0 2024-06-03T02:25:21.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.988+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e8784c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:21.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.989+0000 7fa6f9400700 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87851a90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8005ea0 tx=0x7fa6e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d72fde3bb42c29 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:21.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.993+0000 7fa6e6a00700 1 -- 172.21.15.79:0/4292948501 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8004830 con 0x559e8786e4c0 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.993+0000 7fa6e6a00700 1 -- 172.21.15.79:0/4292948501 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e8005020 con 0x559e8786e4c0 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.994+0000 7fa70b662b80 1 -- 172.21.15.79:0/4292948501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 msgr2=0x559e87851a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.994+0000 7fa70b662b80 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87851a90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8005ea0 tx=0x7fa6e8004750 comp rx=0 tx=0).stop 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.994+0000 7fa70b662b80 1 -- 172.21.15.79:0/4292948501 shutdown_connections 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.994+0000 7fa70b662b80 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e8786e4c0 0x559e87851a90 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8005ea0 tx=0x7fa6e8004750 comp rx=0 tx=0).stop 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.994+0000 7fa70b662b80 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e87868250 0x559e87853430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.994+0000 7fa70b662b80 1 --2- 172.21.15.79:0/4292948501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e8786dba0 0x559e8784c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.994+0000 7fa70b662b80 1 -- 172.21.15.79:0/4292948501 >> 172.21.15.79:0/4292948501 conn(0x559e87783170 msgr2=0x559e8786fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.995+0000 7fa70b662b80 1 -- 172.21.15.79:0/4292948501 shutdown_connections 2024-06-03T02:25:21.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.995+0000 7fa70b662b80 1 -- 172.21.15.79:0/4292948501 wait complete. 2024-06-03T02:25:21.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.996+0000 7fa70b662b80 1 -- start start 2024-06-03T02:25:21.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.996+0000 7fa70b662b80 1 Processor -- start 2024-06-03T02:25:21.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.997+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e87868250 0x559e87886f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.997+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786dba0 0x559e878874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.997+0000 7fa70b662b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e878843e0 0x559e87884860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:21.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.997+0000 7fa70b662b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1b5e70 con 0x559e87868250 2024-06-03T02:25:21.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.997+0000 7fa70b662b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1b57a0 con 0x559e878843e0 2024-06-03T02:25:21.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.997+0000 7fa70b662b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc071da0 con 0x559e8786dba0 2024-06-03T02:25:21.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.997+0000 7fa70b662b80 10 client.0 _ll_register_callbacks cb 0x559e8773e380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:22.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.999+0000 7fa6f9400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e878843e0 0x559e87884860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:22.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.999+0000 7fa6f9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786dba0 0x559e878874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:22.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.999+0000 7fa6f9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786dba0 0x559e878874c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:48266/0 (socket says 172.21.15.79:48266) 2024-06-03T02:25:22.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:21.999+0000 7fa6f9e00700 1 -- 172.21.15.79:0/2523518281 learned_addr learned my addr 172.21.15.79:0/2523518281 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:22.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.000+0000 7fa6f9e00700 1 -- 172.21.15.79:0/2523518281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e878843e0 msgr2=0x559e87884860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:22.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.000+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e878843e0 0x559e87884860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.000+0000 7fa6f9e00700 1 -- 172.21.15.79:0/2523518281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e87868250 msgr2=0x559e87886f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.000+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e87868250 0x559e87886f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.000+0000 7fa6f9e00700 1 -- 172.21.15.79:0/2523518281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e878850a0 con 0x559e8786dba0 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.000+0000 7fa6f9e00700 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786dba0 0x559e878874c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00f9f0 tx=0x7fa6ec0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e825a7175f66d1e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.000+0000 7fa6dbe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.000+0000 7fa6dbe00700 1 -- 172.21.15.79:0/2523518281 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec030040 con 0x559e8786dba0 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.000+0000 7fa6dbe00700 1 -- 172.21.15.79:0/2523518281 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa6ec0052d0 con 0x559e8786dba0 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.001+0000 7fa70b662b80 1 -- 172.21.15.79:0/2523518281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e877b0580 con 0x559e8786dba0 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.001+0000 7fa6dbe00700 1 -- 172.21.15.79:0/2523518281 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6ec00b340 con 0x559e8786dba0 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.001+0000 7fa70b662b80 10 client.25990 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.001+0000 7fa70b662b80 1 -- 172.21.15.79:0/2523518281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e878ec160 con 0x559e8786dba0 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.001+0000 7fa6dbe00700 1 -- 172.21.15.79:0/2523518281 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559e877b0580 con 0x559e8786dba0 2024-06-03T02:25:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.002+0000 7fa6dbe00700 1 -- 172.21.15.79:0/2523518281 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa6ec0054d0 con 0x559e8786dba0 2024-06-03T02:25:22.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.003+0000 7fa70b662b80 10 client.25990 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:22.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.003+0000 7fa70b662b80 10 client.25990 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:25:22.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.003+0000 7fa70b662b80 20 client.25990 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.003+0000 7fa70b662b80 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559e878ec630 0x559e878eea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.003+0000 7fa70b662b80 4 client.25990 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.003+0000 7fa70b662b80 1 -- 172.21.15.79:0/2523518281 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e8762a250 con 0x559e878ec630 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.003+0000 7fa6f9400700 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559e878ec630 0x559e878eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.004+0000 7fa6f9400700 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559e878ec630 0x559e878eea20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ecdab1cae0360ac5 server_cookie=7b814007d722613e in_seq=0 out_seq=0 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.004+0000 7fa6dbe00700 10 client.25990 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.004+0000 7fa6dbe00700 1 -- 172.21.15.79:0/2523518281 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e8762a250 con 0x559e878ec630 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.004+0000 7fa6dbe00700 10 client.25990 handle_command_reply: tid=0 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.004+0000 7fa70b662b80 1 client.25990 shutdown 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.004+0000 7fa70b662b80 1 -- 172.21.15.79:0/2523518281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786dba0 msgr2=0x559e878874c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.004+0000 7fa70b662b80 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786dba0 0x559e878874c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00f9f0 tx=0x7fa6ec0058e0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.005+0000 7fa70b662b80 1 -- 172.21.15.79:0/2523518281 shutdown_connections 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.005+0000 7fa70b662b80 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x559e878ec630 0x559e878eea20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.005+0000 7fa70b662b80 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e878843e0 0x559e87884860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.005+0000 7fa70b662b80 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e87868250 0x559e87886f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.005+0000 7fa6dbe00700 0 client.25990 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:25:22.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.005+0000 7fa70b662b80 1 --2- 172.21.15.79:0/2523518281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e8786dba0 0x559e878874c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.005+0000 7fa70b662b80 1 -- 172.21.15.79:0/2523518281 >> 172.21.15.79:0/2523518281 conn(0x559e87783170 msgr2=0x559e87851a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:22.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.006+0000 7fa70b662b80 1 -- 172.21.15.79:0/2523518281 shutdown_connections 2024-06-03T02:25:22.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.006+0000 7fa70b662b80 1 -- 172.21.15.79:0/2523518281 wait complete. 2024-06-03T02:25:22.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.006+0000 7fa70b662b80 20 client.25990 trim_cache size 0 max 16384 2024-06-03T02:25:22.008 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:25:22.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.007+0000 7fa700c00700 1 -- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa6e00a2ca0 msgr2=0x7fa6e00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:22.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.007+0000 7fa700c00700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa6e00a2ca0 0x7fa6e00a5160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa6f409b190 tx=0x7fa6f4009040 comp rx=0 tx=0).stop 2024-06-03T02:25:22.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.008+0000 7fa700c00700 1 -- 172.21.15.79:0/1286729583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 msgr2=0x7fa6fc10e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:22.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.008+0000 7fa700c00700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 0x7fa6fc10e900 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400ef30 tx=0x7fa6f400c450 comp rx=0 tx=0).stop 2024-06-03T02:25:22.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.008+0000 7fa700c00700 1 -- 172.21.15.79:0/1286729583 shutdown_connections 2024-06-03T02:25:22.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.008+0000 7fa700c00700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc1073b0 0x7fa6fc10e900 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.008+0000 7fa700c00700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6fc06b100 0x7fa6fc1156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.008+0000 7fa700c00700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa6e00a2ca0 0x7fa6e00a5160 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.008+0000 7fa700c00700 1 --2- 172.21.15.79:0/1286729583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6fc104ec0 0x7fa6fc115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:22.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.008+0000 7fa700c00700 1 -- 172.21.15.79:0/1286729583 >> 172.21.15.79:0/1286729583 conn(0x7fa6fc078340 msgr2=0x7fa6fc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:22.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.008+0000 7fa700c00700 1 -- 172.21.15.79:0/1286729583 shutdown_connections 2024-06-03T02:25:22.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:22.009+0000 7fa700c00700 1 -- 172.21.15.79:0/1286729583 wait complete. 2024-06-03T02:25:22.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:21 smithi135 ceph-mon[36653]: pgmap v675: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 561 op/s 2024-06-03T02:25:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 315.0 s, 574.0 tps, lat 178.949 ms stddev 241.048 2024-06-03T02:25:23.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:25:23.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:23 smithi123 ceph-mon[36640]: pgmap v676: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 566 op/s 2024-06-03T02:25:24.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:23 smithi079 ceph-mon[31819]: pgmap v676: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 566 op/s 2024-06-03T02:25:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:23 smithi135 ceph-mon[36653]: pgmap v676: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 566 op/s 2024-06-03T02:25:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:25.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:25 smithi123 ceph-mon[36640]: pgmap v677: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 561 op/s 2024-06-03T02:25:26.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:25 smithi079 ceph-mon[31819]: pgmap v677: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 561 op/s 2024-06-03T02:25:26.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:25 smithi135 ceph-mon[36653]: pgmap v677: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 561 op/s 2024-06-03T02:25:27.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:27 smithi123 ceph-mon[36640]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 490 op/s 2024-06-03T02:25:28.069 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:27 smithi079 ceph-mon[31819]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 490 op/s 2024-06-03T02:25:28.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:27 smithi135 ceph-mon[36653]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 490 op/s 2024-06-03T02:25:28.314 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 320.0 s, 718.3 tps, lat 128.811 ms stddev 161.579 2024-06-03T02:25:28.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:28 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:25:28.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.946+0000 7f4445e00700 1 -- 172.21.15.79:0/2301760538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4440074830 msgr2=0x7f4440074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:28.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.946+0000 7f4445e00700 1 --2- 172.21.15.79:0/2301760538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4440074830 0x7f4440074c10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4434009990 tx=0x7f4434009ca0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.947+0000 7f4445e00700 1 -- 172.21.15.79:0/2301760538 shutdown_connections 2024-06-03T02:25:28.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.947+0000 7f4445e00700 1 --2- 172.21.15.79:0/2301760538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44401133d0 0x7f44401157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.947+0000 7f4445e00700 1 --2- 172.21.15.79:0/2301760538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4440074830 0x7f4440074c10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.947+0000 7f4445e00700 1 --2- 172.21.15.79:0/2301760538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4440075150 0x7f4440112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.947+0000 7f4445e00700 1 -- 172.21.15.79:0/2301760538 >> 172.21.15.79:0/2301760538 conn(0x7f44400fdf10 msgr2=0x7f4440100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:28.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.947+0000 7f4445e00700 1 -- 172.21.15.79:0/2301760538 shutdown_connections 2024-06-03T02:25:28.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.947+0000 7f4445e00700 1 -- 172.21.15.79:0/2301760538 wait complete. 2024-06-03T02:25:28.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.949+0000 7f4445e00700 1 Processor -- start 2024-06-03T02:25:28.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.949+0000 7f4445e00700 1 -- start start 2024-06-03T02:25:28.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f4445e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4440074830 0x7f444019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f4445e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4440075150 0x7f444019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f4445e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44401133d0 0x7f44401a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f4445e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4440118da0 con 0x7f4440074830 2024-06-03T02:25:28.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f4445e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4440118c20 con 0x7f4440075150 2024-06-03T02:25:28.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f4445e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4440118f20 con 0x7f44401133d0 2024-06-03T02:25:28.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f443fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4440075150 0x7f444019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:28.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f443fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4440075150 0x7f444019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40566/0 (socket says 172.21.15.79:40566) 2024-06-03T02:25:28.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f443fe00700 1 -- 172.21.15.79:0/3478160046 learned_addr learned my addr 172.21.15.79:0/3478160046 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:28.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.950+0000 7f4445400700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44401133d0 0x7f44401a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:28.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f443fe00700 1 -- 172.21.15.79:0/3478160046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44401133d0 msgr2=0x7f44401a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f443fe00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44401133d0 0x7f44401a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f443fe00700 1 -- 172.21.15.79:0/3478160046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4440074830 msgr2=0x7f444019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f443fe00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4440074830 0x7f444019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f443fe00700 1 -- 172.21.15.79:0/3478160046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44401a41e0 con 0x7f4440075150 2024-06-03T02:25:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f4445400700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44401133d0 0x7f44401a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f443fe00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4440075150 0x7f444019f710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e9a0 tx=0x7f442c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f443d600700 1 -- 172.21.15.79:0/3478160046 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c00cbd0 con 0x7f4440075150 2024-06-03T02:25:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f443d600700 1 -- 172.21.15.79:0/3478160046 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f442c00cd70 con 0x7f4440075150 2024-06-03T02:25:28.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f443d600700 1 -- 172.21.15.79:0/3478160046 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c0187c0 con 0x7f4440075150 2024-06-03T02:25:28.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.951+0000 7f4445e00700 1 -- 172.21.15.79:0/3478160046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44401a44d0 con 0x7f4440075150 2024-06-03T02:25:28.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.952+0000 7f4445e00700 1 -- 172.21.15.79:0/3478160046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44401abd50 con 0x7f4440075150 2024-06-03T02:25:28.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.955+0000 7f443d600700 1 -- 172.21.15.79:0/3478160046 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f442c010430 con 0x7f4440075150 2024-06-03T02:25:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.958+0000 7f443d600700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44280a2cf0 0x7f44280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.958+0000 7f443d600700 1 -- 172.21.15.79:0/3478160046 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f442c0cf650 con 0x7f4440075150 2024-06-03T02:25:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.958+0000 7f4444a00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44280a2cf0 0x7f44280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.959+0000 7f4451d95b80 1 Processor -- start 2024-06-03T02:25:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.959+0000 7f4451d95b80 1 -- start start 2024-06-03T02:25:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.959+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 0x558c01d3f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.959+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3f680 0x558c01d49ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.959+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d4a530 0x558c01d4c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.959+0000 7f4451d95b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558c01c4ea10 con 0x558c01d3ed60 2024-06-03T02:25:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.959+0000 7f4451d95b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c01c91b00 con 0x558c01d3f680 2024-06-03T02:25:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.959+0000 7f4451d95b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c01c4c8f0 con 0x558c01d4a530 2024-06-03T02:25:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4445400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d4a530 0x558c01d4c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4445400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d4a530 0x558c01d4c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52896/0 (socket says 172.21.15.79:52896) 2024-06-03T02:25:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4445400700 1 -- 172.21.15.79:0/1810511794 learned_addr learned my addr 172.21.15.79:0/1810511794 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f443fe00700 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3f680 0x558c01d49ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4445400700 1 -- 172.21.15.79:0/1810511794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3f680 msgr2=0x558c01d49ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4445400700 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3f680 0x558c01d49ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4444a00700 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 0x558c01d3f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4445400700 1 -- 172.21.15.79:0/1810511794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 msgr2=0x558c01d3f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4445400700 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 0x558c01d3f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4445400700 1 -- 172.21.15.79:0/1810511794 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c01c39f90 con 0x558c01d4a530 2024-06-03T02:25:28.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f443fe00700 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3f680 0x558c01d49ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:28.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.960+0000 7f4444a00700 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 0x558c01d3f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:28.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.961+0000 7f4444a00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44280a2cf0 0x7f44280a51b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f44340098e0 tx=0x7f4434009610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:28.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.961+0000 7f4445400700 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d4a530 0x558c01d4c920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4438009d80 tx=0x7f44380077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ba6a9c787232cf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:28.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.961+0000 7f4432000700 1 -- 172.21.15.79:0/1810511794 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438004bb0 con 0x558c01d4a530 2024-06-03T02:25:28.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.961+0000 7f4432000700 1 -- 172.21.15.79:0/1810511794 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4438004d50 con 0x558c01d4a530 2024-06-03T02:25:28.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.961+0000 7f4432000700 1 -- 172.21.15.79:0/1810511794 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438014700 con 0x558c01d4a530 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.962+0000 7f4451d95b80 1 -- 172.21.15.79:0/1810511794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d4a530 msgr2=0x558c01d4c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.962+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d4a530 0x558c01d4c920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4438009d80 tx=0x7f44380077e0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.962+0000 7f4451d95b80 1 -- 172.21.15.79:0/1810511794 shutdown_connections 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.962+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d4a530 0x558c01d4c920 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.962+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 0x558c01d3f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.962+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1810511794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3f680 0x558c01d49ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.962+0000 7f4451d95b80 1 -- 172.21.15.79:0/1810511794 >> 172.21.15.79:0/1810511794 conn(0x558c01c6c170 msgr2=0x558c01d3b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.962+0000 7f4451d95b80 1 -- 172.21.15.79:0/1810511794 shutdown_connections 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.963+0000 7f4451d95b80 1 -- 172.21.15.79:0/1810511794 wait complete. 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.963+0000 7f4451d95b80 1 -- start start 2024-06-03T02:25:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.963+0000 7f4451d95b80 1 Processor -- start 2024-06-03T02:25:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.964+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 0x558c01d49ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.964+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d46fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.964+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d47520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.964+0000 7f4451d95b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558c01c2df60 con 0x558c01d3ed60 2024-06-03T02:25:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.964+0000 7f4451d95b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c01c8dac0 con 0x558c01d4a530 2024-06-03T02:25:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.964+0000 7f4451d95b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c01d4cfc0 con 0x558c01d3f680 2024-06-03T02:25:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.964+0000 7f4451d95b80 10 client.0 _ll_register_callbacks cb 0x558c01c27380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.965+0000 7f4445400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d47520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.965+0000 7f443fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d46fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.965+0000 7f4445400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d47520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40576/0 (socket says 172.21.15.79:40576) 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.965+0000 7f4445400700 1 -- 172.21.15.79:0/282649828 learned_addr learned my addr 172.21.15.79:0/282649828 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.965+0000 7f4445400700 1 -- 172.21.15.79:0/282649828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 msgr2=0x558c01d46fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.965+0000 7f4445400700 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d46fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.965+0000 7f4445400700 1 -- 172.21.15.79:0/282649828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 msgr2=0x558c01d49ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.965+0000 7f4445400700 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 0x558c01d49ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.965+0000 7f4445400700 1 -- 172.21.15.79:0/282649828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c01be08d0 con 0x558c01d4a530 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.966+0000 7f443fe00700 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d46fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.966+0000 7f4445400700 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d47520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4438007f50 tx=0x7f4438006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf9aca483856a9b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.966+0000 7f4426a00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.966+0000 7f4426a00700 1 -- 172.21.15.79:0/282649828 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438022020 con 0x558c01d4a530 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.966+0000 7f4426a00700 1 -- 172.21.15.79:0/282649828 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f443801e420 con 0x558c01d4a530 2024-06-03T02:25:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.967+0000 7f4451d95b80 1 -- 172.21.15.79:0/282649828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4438004d50 con 0x558c01d4a530 2024-06-03T02:25:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.967+0000 7f4426a00700 1 -- 172.21.15.79:0/282649828 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f443801e620 con 0x558c01d4a530 2024-06-03T02:25:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.967+0000 7f4451d95b80 10 client.26009 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.967+0000 7f4451d95b80 1 -- 172.21.15.79:0/282649828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c01d57680 con 0x558c01d4a530 2024-06-03T02:25:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.968+0000 7f4426a00700 1 -- 172.21.15.79:0/282649828 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4438004d50 con 0x558c01d4a530 2024-06-03T02:25:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.968+0000 7f4426a00700 1 -- 172.21.15.79:0/282649828 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4438014850 con 0x558c01d4a530 2024-06-03T02:25:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.968+0000 7f4451d95b80 10 client.26009 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.968+0000 7f4451d95b80 10 client.26009 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:25:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.968+0000 7f4451d95b80 20 client.26009 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.968+0000 7f4451d95b80 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x558c01d59c10 0x558c01d5c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.968+0000 7f4451d95b80 4 client.26009 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:25:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.968+0000 7f4451d95b80 1 -- 172.21.15.79:0/282649828 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c01c9f000 con 0x558c01d59c10 2024-06-03T02:25:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.969+0000 7f4444a00700 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x558c01d59c10 0x558c01d5c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.970+0000 7f4444a00700 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x558c01d59c10 0x558c01d5c000 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4ce08f9f229f10a server_cookie=32400126c718011e in_seq=0 out_seq=0 2024-06-03T02:25:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.970+0000 7f4426a00700 10 client.26009 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:25:28.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.971+0000 7f4426a00700 1 -- 172.21.15.79:0/282649828 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558c01c9f000 con 0x558c01d59c10 2024-06-03T02:25:28.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.971+0000 7f4426a00700 10 client.26009 handle_command_reply: tid=0 2024-06-03T02:25:28.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.971+0000 7f4451d95b80 1 client.26009 shutdown 2024-06-03T02:25:28.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.971+0000 7f4451d95b80 1 -- 172.21.15.79:0/282649828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 msgr2=0x558c01d47520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:28.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.971+0000 7f4451d95b80 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d47520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4438007f50 tx=0x7f4438006000 comp rx=0 tx=0).stop 2024-06-03T02:25:28.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.972+0000 7f4451d95b80 1 -- 172.21.15.79:0/282649828 shutdown_connections 2024-06-03T02:25:28.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.972+0000 7f4451d95b80 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d47520 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.972+0000 7f4451d95b80 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x558c01d59c10 0x558c01d5c000 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.972+0000 7f4451d95b80 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3ed60 0x558c01d49ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.972+0000 7f4451d95b80 1 --2- 172.21.15.79:0/282649828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d46fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.972+0000 7f4451d95b80 1 -- 172.21.15.79:0/282649828 >> 172.21.15.79:0/282649828 conn(0x558c01c6c170 msgr2=0x558c01d3ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:28.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.972+0000 7f4426a00700 0 client.26009 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:25:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:28 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:25:28.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.972+0000 7f4451d95b80 1 -- 172.21.15.79:0/282649828 shutdown_connections 2024-06-03T02:25:28.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.976+0000 7f4451d95b80 1 -- 172.21.15.79:0/282649828 wait complete. 2024-06-03T02:25:28.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:28.976+0000 7f4451d95b80 20 client.26009 trim_cache size 0 max 16384 2024-06-03T02:25:29.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.009+0000 7f4451d95b80 1 Processor -- start 2024-06-03T02:25:29.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.009+0000 7f4451d95b80 1 -- start start 2024-06-03T02:25:29.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.010+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3ed60 0x558c01d3b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:29.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.010+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.010+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d4a530 0x558c01d49820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.010+0000 7f4451d95b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f44401b49a0 con 0x558c01d4a530 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.010+0000 7f4451d95b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44401b64e0 con 0x558c01d3ed60 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.010+0000 7f4451d95b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44401b5070 con 0x558c01d3f680 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.010+0000 7f443fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.010+0000 7f4445400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d4a530 0x558c01d49820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.011+0000 7f4444a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3ed60 0x558c01d3b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.011+0000 7f443fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52906/0 (socket says 172.21.15.79:52906) 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.011+0000 7f443fe00700 1 -- 172.21.15.79:0/3958027285 learned_addr learned my addr 172.21.15.79:0/3958027285 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.011+0000 7f4445400700 1 -- 172.21.15.79:0/3958027285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 msgr2=0x558c01d427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.011+0000 7f4445400700 1 --2- 172.21.15.79:0/3958027285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.011+0000 7f4445400700 1 -- 172.21.15.79:0/3958027285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3ed60 msgr2=0x558c01d3b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:29.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.011+0000 7f4445400700 1 --2- 172.21.15.79:0/3958027285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3ed60 0x558c01d3b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.011+0000 7f4445400700 1 -- 172.21.15.79:0/3958027285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c01d3fcd0 con 0x558c01d4a530 2024-06-03T02:25:29.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.012+0000 7f4445400700 1 --2- 172.21.15.79:0/3958027285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d4a530 0x558c01d49820 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4438029040 tx=0x7f4438009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d6681da9418600 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:29.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.012+0000 7f4431600700 1 -- 172.21.15.79:0/3958027285 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438022070 con 0x558c01d4a530 2024-06-03T02:25:29.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.012+0000 7f4431600700 1 -- 172.21.15.79:0/3958027285 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f443801e420 con 0x558c01d4a530 2024-06-03T02:25:29.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.012+0000 7f4431600700 1 -- 172.21.15.79:0/3958027285 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438022070 con 0x558c01d4a530 2024-06-03T02:25:29.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.013+0000 7f4451d95b80 1 -- 172.21.15.79:0/3958027285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d4a530 msgr2=0x558c01d49820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:29.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.013+0000 7f4451d95b80 1 --2- 172.21.15.79:0/3958027285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d4a530 0x558c01d49820 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4438029040 tx=0x7f4438009270 comp rx=0 tx=0).stop 2024-06-03T02:25:29.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.014+0000 7f4451d95b80 1 -- 172.21.15.79:0/3958027285 shutdown_connections 2024-06-03T02:25:29.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.014+0000 7f4451d95b80 1 --2- 172.21.15.79:0/3958027285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d4a530 0x558c01d49820 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.014+0000 7f4451d95b80 1 --2- 172.21.15.79:0/3958027285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d3ed60 0x558c01d3b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.014+0000 7f4451d95b80 1 --2- 172.21.15.79:0/3958027285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3f680 0x558c01d427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.014+0000 7f4451d95b80 1 -- 172.21.15.79:0/3958027285 >> 172.21.15.79:0/3958027285 conn(0x558c01c6c170 msgr2=0x558c01d40c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:29.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.017+0000 7f4451d95b80 1 -- 172.21.15.79:0/3958027285 shutdown_connections 2024-06-03T02:25:29.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.017+0000 7f4451d95b80 1 -- 172.21.15.79:0/3958027285 wait complete. 2024-06-03T02:25:29.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.017+0000 7f4451d95b80 1 -- start start 2024-06-03T02:25:29.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.018+0000 7f4451d95b80 1 Processor -- start 2024-06-03T02:25:29.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.020+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3ed60 0x558c01d4d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:29.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.020+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3f680 0x558c01d4d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:29.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.020+0000 7f4451d95b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d4dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:29.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.020+0000 7f4451d95b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f44401b5e10 con 0x558c01d3f680 2024-06-03T02:25:29.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.020+0000 7f4451d95b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44401b5740 con 0x558c01d4a530 2024-06-03T02:25:29.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.020+0000 7f4451d95b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44401b2940 con 0x558c01d3ed60 2024-06-03T02:25:29.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.020+0000 7f4451d95b80 10 client.0 _ll_register_callbacks cb 0x558c01c27380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:29.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.021+0000 7f4444a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3ed60 0x558c01d4d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:29.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.021+0000 7f4445400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d4dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:29.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.021+0000 7f4444a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3ed60 0x558c01d4d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52908/0 (socket says 172.21.15.79:52908) 2024-06-03T02:25:29.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.021+0000 7f4444a00700 1 -- 172.21.15.79:0/1968676258 learned_addr learned my addr 172.21.15.79:0/1968676258 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:29.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.021+0000 7f4444a00700 1 -- 172.21.15.79:0/1968676258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 msgr2=0x558c01d4dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:29.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.021+0000 7f4444a00700 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d4dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.021+0000 7f4444a00700 1 -- 172.21.15.79:0/1968676258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3f680 msgr2=0x558c01d4d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:29.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.021+0000 7f4444a00700 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3f680 0x558c01d4d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.021+0000 7f4444a00700 1 -- 172.21.15.79:0/1968676258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c01dd3570 con 0x558c01d3ed60 2024-06-03T02:25:29.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.022+0000 7f4445400700 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d4dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:25:29.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.022+0000 7f4444a00700 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3ed60 0x558c01d4d200 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4434015000 tx=0x7f4434000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56332ff29c56529d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:29.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.022+0000 7f4426a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:25:29.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.023+0000 7f4426a00700 1 -- 172.21.15.79:0/1968676258 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434033060 con 0x558c01d3ed60 2024-06-03T02:25:29.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.023+0000 7f4426a00700 1 -- 172.21.15.79:0/1968676258 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f443400bdb0 con 0x558c01d3ed60 2024-06-03T02:25:29.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.023+0000 7f4451d95b80 1 -- 172.21.15.79:0/1968676258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f443801e420 con 0x558c01d3ed60 2024-06-03T02:25:29.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.023+0000 7f4426a00700 1 -- 172.21.15.79:0/1968676258 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f443403a050 con 0x558c01d3ed60 2024-06-03T02:25:29.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.023+0000 7f4451d95b80 10 client.26008 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:29.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.024+0000 7f4451d95b80 1 -- 172.21.15.79:0/1968676258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c01dd4820 con 0x558c01d3ed60 2024-06-03T02:25:29.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.024+0000 7f4426a00700 1 -- 172.21.15.79:0/1968676258 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4434003b60 con 0x558c01d3ed60 2024-06-03T02:25:29.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.024+0000 7f4426a00700 1 -- 172.21.15.79:0/1968676258 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f443402e970 con 0x558c01d3ed60 2024-06-03T02:25:29.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.025+0000 7f4451d95b80 10 client.26008 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:29.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.025+0000 7f4451d95b80 10 client.26008 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:25:29.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.025+0000 7f4451d95b80 20 client.26008 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:29.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.025+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x558c01dd4ee0 0x558c01dd7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:29.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.025+0000 7f4451d95b80 4 client.26008 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:25:29.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.025+0000 7f4451d95b80 1 -- 172.21.15.79:0/1968676258 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f443801e420 con 0x558c01dd4ee0 2024-06-03T02:25:29.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.026+0000 7f443fe00700 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x558c01dd4ee0 0x558c01dd7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:29.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.027+0000 7f4426a00700 10 client.26008 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:25:29.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.027+0000 7f443fe00700 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x558c01dd4ee0 0x558c01dd7300 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=398fe3f5453f43cf server_cookie=1297f722a3646c32 in_seq=0 out_seq=0 2024-06-03T02:25:29.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.027+0000 7f4426a00700 1 -- 172.21.15.79:0/1968676258 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f443801e420 con 0x558c01dd4ee0 2024-06-03T02:25:29.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.027+0000 7f4426a00700 10 client.26008 handle_command_reply: tid=0 2024-06-03T02:25:29.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.027+0000 7f4451d95b80 1 client.26008 shutdown 2024-06-03T02:25:29.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 -- 172.21.15.79:0/1968676258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3ed60 msgr2=0x558c01d4d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3ed60 0x558c01d4d200 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4434015000 tx=0x7f4434000bc0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 -- 172.21.15.79:0/1968676258 shutdown_connections 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c01d4a530 0x558c01d4dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c01d3ed60 0x558c01d4d200 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x558c01dd4ee0 0x558c01dd7300 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 --2- 172.21.15.79:0/1968676258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c01d3f680 0x558c01d4d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 -- 172.21.15.79:0/1968676258 >> 172.21.15.79:0/1968676258 conn(0x558c01c6c170 msgr2=0x558c01d42660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4426a00700 0 client.26008 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 -- 172.21.15.79:0/1968676258 shutdown_connections 2024-06-03T02:25:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 1 -- 172.21.15.79:0/1968676258 wait complete. 2024-06-03T02:25:29.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.029+0000 7f4451d95b80 20 client.26008 trim_cache size 0 max 16384 2024-06-03T02:25:29.031 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:25:29.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 -- 172.21.15.79:0/3478160046 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44280a2cf0 msgr2=0x7f44280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:29.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44280a2cf0 0x7f44280a51b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f44340098e0 tx=0x7f4434009610 comp rx=0 tx=0).stop 2024-06-03T02:25:29.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 -- 172.21.15.79:0/3478160046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4440075150 msgr2=0x7f444019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:29.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4440075150 0x7f444019f710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e9a0 tx=0x7f442c00ed60 comp rx=0 tx=0).stop 2024-06-03T02:25:29.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 -- 172.21.15.79:0/3478160046 shutdown_connections 2024-06-03T02:25:29.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44401133d0 0x7f44401a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f44280a2cf0 0x7f44280a51b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4440074830 0x7f444019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 --2- 172.21.15.79:0/3478160046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4440075150 0x7f444019f710 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:29.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 -- 172.21.15.79:0/3478160046 >> 172.21.15.79:0/3478160046 conn(0x7f44400fdf10 msgr2=0x7f44400feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:29.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 -- 172.21.15.79:0/3478160046 shutdown_connections 2024-06-03T02:25:29.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:29.032+0000 7f4445e00700 1 -- 172.21.15.79:0/3478160046 wait complete. 2024-06-03T02:25:29.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:28 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:25:29.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:29 smithi079 ceph-mon[31819]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 534 op/s 2024-06-03T02:25:29.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:29 smithi123 ceph-mon[36640]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 534 op/s 2024-06-03T02:25:30.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:25:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:29 smithi135 ceph-mon[36653]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 534 op/s 2024-06-03T02:25:30.789 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:30.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:25:30.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:25:30.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:30 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:25:30.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:25:30.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:25:30.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:30 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:25:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:25:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:25:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:30 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:25:32.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:31 smithi079 ceph-mon[31819]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 467 op/s 2024-06-03T02:25:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:31 smithi135 ceph-mon[36653]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 467 op/s 2024-06-03T02:25:32.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:31 smithi123 ceph-mon[36640]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 467 op/s 2024-06-03T02:25:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 325.0 s, 667.4 tps, lat 165.179 ms stddev 247.777 2024-06-03T02:25:33.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.866+0000 7fb15ce00700 1 -- 172.21.15.79:0/330092809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb15810eac0 msgr2=0x7fb15810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:33.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.866+0000 7fb15ce00700 1 --2- 172.21.15.79:0/330092809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb15810eac0 0x7fb15810eea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb148009990 tx=0x7fb148009ca0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.866+0000 7fb15ce00700 1 -- 172.21.15.79:0/330092809 shutdown_connections 2024-06-03T02:25:33.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.866+0000 7fb15ce00700 1 --2- 172.21.15.79:0/330092809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb158109480 0x7fb158115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.866+0000 7fb15ce00700 1 --2- 172.21.15.79:0/330092809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb15810eac0 0x7fb15810eea0 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb148009990 tx=0x7fb148009ca0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.866+0000 7fb15ce00700 1 --2- 172.21.15.79:0/330092809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb158108ac0 0x7fb158108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.866+0000 7fb15ce00700 1 -- 172.21.15.79:0/330092809 >> 172.21.15.79:0/330092809 conn(0x7fb158075c60 msgr2=0x7fb158078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:33.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.867+0000 7fb15ce00700 1 -- 172.21.15.79:0/330092809 shutdown_connections 2024-06-03T02:25:33.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.867+0000 7fb15ce00700 1 -- 172.21.15.79:0/330092809 wait complete. 2024-06-03T02:25:33.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.868+0000 7fb15ce00700 1 Processor -- start 2024-06-03T02:25:33.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.872+0000 7fb15ce00700 1 -- start start 2024-06-03T02:25:33.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.875+0000 7fb15ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb158108ac0 0x7fb158073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.876+0000 7fb15ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb158109480 0x7fb15806cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.876+0000 7fb15ce00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb15806d450 0x7fb15806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.876+0000 7fb15ce00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb158118ea0 con 0x7fb158109480 2024-06-03T02:25:33.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.876+0000 7fb15ce00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb158118d20 con 0x7fb158108ac0 2024-06-03T02:25:33.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.876+0000 7fb155400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb158109480 0x7fb15806cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.876+0000 7fb155400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb158109480 0x7fb15806cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:48034/0 (socket says 172.21.15.79:48034) 2024-06-03T02:25:33.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.876+0000 7fb155400700 1 -- 172.21.15.79:0/2741728112 learned_addr learned my addr 172.21.15.79:0/2741728112 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:33.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.876+0000 7fb156800700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb15806d450 0x7fb15806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.878+0000 7fb155e00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb158108ac0 0x7fb158073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.878+0000 7fb15ce00700 1 -- 172.21.15.79:0/2741728112 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb158119020 con 0x7fb15806d450 2024-06-03T02:25:33.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.879+0000 7fb155e00700 1 -- 172.21.15.79:0/2741728112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb15806d450 msgr2=0x7fb15806d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:33.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.879+0000 7fb155e00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb15806d450 0x7fb15806d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.879+0000 7fb155e00700 1 -- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb158109480 msgr2=0x7fb15806cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:33.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.879+0000 7fb155e00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb158109480 0x7fb15806cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.879+0000 7fb155e00700 1 -- 172.21.15.79:0/2741728112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1580719c0 con 0x7fb158108ac0 2024-06-03T02:25:33.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.879+0000 7fb155400700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb158109480 0x7fb15806cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:25:33.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.880+0000 7fb155e00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb158108ac0 0x7fb158073de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb148008000 tx=0x7fb1480057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:33.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.880+0000 7fb14ea00700 1 -- 172.21.15.79:0/2741728112 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148003b00 con 0x7fb158108ac0 2024-06-03T02:25:33.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.880+0000 7fb14ea00700 1 -- 172.21.15.79:0/2741728112 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb148004180 con 0x7fb158108ac0 2024-06-03T02:25:33.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.881+0000 7fb14ea00700 1 -- 172.21.15.79:0/2741728112 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14800fe10 con 0x7fb158108ac0 2024-06-03T02:25:33.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.881+0000 7fb15ce00700 1 -- 172.21.15.79:0/2741728112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb158071c50 con 0x7fb158108ac0 2024-06-03T02:25:33.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.881+0000 7fb15ce00700 1 -- 172.21.15.79:0/2741728112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1580720e0 con 0x7fb158108ac0 2024-06-03T02:25:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.890+0000 7fb1676a7b80 1 Processor -- start 2024-06-03T02:25:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.890+0000 7fb1676a7b80 1 -- start start 2024-06-03T02:25:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.890+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4972f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.891+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b49734c0 0x55d8b496cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.891+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b496d250 0x55d8b496d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.891+0000 7fb156800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4972f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.891+0000 7fb156800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4972f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52926/0 (socket says 172.21.15.79:52926) 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.891+0000 7fb156800700 1 -- 172.21.15.79:0/1794595624 learned_addr learned my addr 172.21.15.79:0/1794595624 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.891+0000 7fb1676a7b80 1 -- 172.21.15.79:0/1794595624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d8b486aa10 con 0x55d8b49734c0 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.891+0000 7fb1676a7b80 1 -- 172.21.15.79:0/1794595624 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8b48adb00 con 0x55d8b496d250 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.891+0000 7fb1676a7b80 1 -- 172.21.15.79:0/1794595624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8b48688f0 con 0x55d8b4972ba0 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.891+0000 7fb155400700 1 --2- 172.21.15.79:0/1794595624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b496d250 0x55d8b496d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.892+0000 7fb155400700 1 -- 172.21.15.79:0/1794595624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 msgr2=0x55d8b4972f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.892+0000 7fb155400700 1 --2- 172.21.15.79:0/1794595624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4972f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.892+0000 7fb155400700 1 -- 172.21.15.79:0/1794595624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b49734c0 msgr2=0x55d8b496cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:25:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.892+0000 7fb155400700 1 --2- 172.21.15.79:0/1794595624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b49734c0 0x55d8b496cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.892+0000 7fb155400700 1 -- 172.21.15.79:0/1794595624 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b4855f90 con 0x55d8b496d250 2024-06-03T02:25:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.892+0000 7fb155400700 1 --2- 172.21.15.79:0/1794595624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b496d250 0x55d8b496d690 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb15806e6c0 tx=0x7fb14400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9abec3825970412 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.892+0000 7fb143400700 1 -- 172.21.15.79:0/1794595624 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144017070 con 0x55d8b496d250 2024-06-03T02:25:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.892+0000 7fb156800700 1 --2- 172.21.15.79:0/1794595624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4972f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:33.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.896+0000 7fb14ea00700 1 -- 172.21.15.79:0/2741728112 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb14800f4f0 con 0x7fb158108ac0 2024-06-03T02:25:33.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.897+0000 7fb143400700 1 -- 172.21.15.79:0/1794595624 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14400ecd0 con 0x55d8b496d250 2024-06-03T02:25:33.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.898+0000 7fb1676a7b80 1 -- 172.21.15.79:0/1794595624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b496d250 msgr2=0x55d8b496d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:33.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.899+0000 7fb143400700 1 -- 172.21.15.79:0/1794595624 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144017070 con 0x55d8b496d250 2024-06-03T02:25:33.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.899+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/1794595624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b496d250 0x55d8b496d690 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb15806e6c0 tx=0x7fb14400e4b0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.903+0000 7fb1676a7b80 1 -- 172.21.15.79:0/1794595624 shutdown_connections 2024-06-03T02:25:33.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.903+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/1794595624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b496d250 0x55d8b496d690 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.903+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/1794595624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4972f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.903+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/1794595624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b49734c0 0x55d8b496cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.903+0000 7fb1676a7b80 1 -- 172.21.15.79:0/1794595624 >> 172.21.15.79:0/1794595624 conn(0x55d8b4888170 msgr2=0x55d8b4956ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.903+0000 7fb14ea00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb13c0a2c50 0x7fb13c0a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.903+0000 7fb1676a7b80 1 -- 172.21.15.79:0/1794595624 shutdown_connections 2024-06-03T02:25:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.904+0000 7fb14ea00700 1 -- 172.21.15.79:0/2741728112 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb1480d0140 con 0x7fb158108ac0 2024-06-03T02:25:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.904+0000 7fb155400700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb13c0a2c50 0x7fb13c0a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.904+0000 7fb1676a7b80 1 -- 172.21.15.79:0/1794595624 wait complete. 2024-06-03T02:25:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.905+0000 7fb1676a7b80 1 -- start start 2024-06-03T02:25:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.905+0000 7fb155400700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb13c0a2c50 0x7fb13c0a5110 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb14400efc0 tx=0x7fb144009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:33.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.905+0000 7fb1676a7b80 1 Processor -- start 2024-06-03T02:25:33.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.906+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b496d250 0x55d8b4979110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.906+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4976480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.906+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 0x55d8b49769c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.906+0000 7fb1676a7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d8b4849f60 con 0x55d8b496d250 2024-06-03T02:25:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.906+0000 7fb1676a7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8b48a9ac0 con 0x55d8b49734c0 2024-06-03T02:25:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.906+0000 7fb1676a7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8b4979ef0 con 0x55d8b4972ba0 2024-06-03T02:25:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.906+0000 7fb1676a7b80 10 client.0 _ll_register_callbacks cb 0x55d8b4843380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.907+0000 7fb156800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 0x55d8b49769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.907+0000 7fb156800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 0x55d8b49769c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40628/0 (socket says 172.21.15.79:40628) 2024-06-03T02:25:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.907+0000 7fb156800700 1 -- 172.21.15.79:0/367447470 learned_addr learned my addr 172.21.15.79:0/367447470 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:33.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.907+0000 7fb155e00700 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b496d250 0x55d8b4979110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.908+0000 7fb156800700 1 -- 172.21.15.79:0/367447470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 msgr2=0x55d8b4976480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:33.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.908+0000 7fb155400700 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4976480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.908+0000 7fb156800700 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4976480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.908+0000 7fb156800700 1 -- 172.21.15.79:0/367447470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b496d250 msgr2=0x55d8b4979110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:33.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.908+0000 7fb156800700 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b496d250 0x55d8b4979110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.908+0000 7fb156800700 1 -- 172.21.15.79:0/367447470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b47fc8d0 con 0x55d8b49734c0 2024-06-03T02:25:33.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.910+0000 7fb156800700 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 0x55d8b49769c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb150010af0 tx=0x7fb150010eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad966914256e769 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:33.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.910+0000 7fb137e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:25:33.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.910+0000 7fb137e00700 1 -- 172.21.15.79:0/367447470 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15001a020 con 0x55d8b49734c0 2024-06-03T02:25:33.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.910+0000 7fb1676a7b80 1 -- 172.21.15.79:0/367447470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb14400ecd0 con 0x55d8b49734c0 2024-06-03T02:25:33.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.910+0000 7fb137e00700 1 -- 172.21.15.79:0/367447470 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb15000ea30 con 0x55d8b49734c0 2024-06-03T02:25:33.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.911+0000 7fb137e00700 1 -- 172.21.15.79:0/367447470 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb15000ec30 con 0x55d8b49734c0 2024-06-03T02:25:33.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.911+0000 7fb1676a7b80 10 client.26024 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:33.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.911+0000 7fb1676a7b80 1 -- 172.21.15.79:0/367447470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8b49843e0 con 0x55d8b49734c0 2024-06-03T02:25:33.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.911+0000 7fb137e00700 1 -- 172.21.15.79:0/367447470 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14400ecd0 con 0x55d8b49734c0 2024-06-03T02:25:33.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.913+0000 7fb137e00700 1 -- 172.21.15.79:0/367447470 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb150027020 con 0x55d8b49734c0 2024-06-03T02:25:33.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.913+0000 7fb1676a7b80 10 client.26024 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:33.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.913+0000 7fb1676a7b80 10 client.26024 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:25:33.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.914+0000 7fb1676a7b80 20 client.26024 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:33.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.914+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d8b49869c0 0x55d8b4988df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.914+0000 7fb1676a7b80 4 client.26024 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:25:33.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.914+0000 7fb1676a7b80 1 -- 172.21.15.79:0/367447470 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8b48bb000 con 0x55d8b49869c0 2024-06-03T02:25:33.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.914+0000 7fb155e00700 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d8b49869c0 0x55d8b4988df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.916+0000 7fb137e00700 10 client.26024 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:25:33.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.921+0000 7fb155e00700 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d8b49869c0 0x55d8b4988df0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73b89ab6ea8a1cd8 server_cookie=cfe267baf28a12c8 in_seq=0 out_seq=0 2024-06-03T02:25:33.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.922+0000 7fb137e00700 1 -- 172.21.15.79:0/367447470 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d8b48bb000 con 0x55d8b49869c0 2024-06-03T02:25:33.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.922+0000 7fb137e00700 10 client.26024 handle_command_reply: tid=0 2024-06-03T02:25:33.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.923+0000 7fb1676a7b80 1 client.26024 shutdown 2024-06-03T02:25:33.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.923+0000 7fb1676a7b80 1 -- 172.21.15.79:0/367447470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 msgr2=0x55d8b49769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:33.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.923+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 0x55d8b49769c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb150010af0 tx=0x7fb150010eb0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.924+0000 7fb1676a7b80 1 -- 172.21.15.79:0/367447470 shutdown_connections 2024-06-03T02:25:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.924+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d8b49869c0 0x55d8b4988df0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.924+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 0x55d8b49769c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.924+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b496d250 0x55d8b4979110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.924+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/367447470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b4972ba0 0x55d8b4976480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.924+0000 7fb1676a7b80 1 -- 172.21.15.79:0/367447470 >> 172.21.15.79:0/367447470 conn(0x55d8b4888170 msgr2=0x55d8b4950790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.924+0000 7fb137e00700 0 client.26024 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:25:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.924+0000 7fb1676a7b80 1 -- 172.21.15.79:0/367447470 shutdown_connections 2024-06-03T02:25:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.924+0000 7fb1676a7b80 1 -- 172.21.15.79:0/367447470 wait complete. 2024-06-03T02:25:33.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.925+0000 7fb1676a7b80 20 client.26024 trim_cache size 0 max 16384 2024-06-03T02:25:33.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.957+0000 7fb1676a7b80 1 Processor -- start 2024-06-03T02:25:33.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.958+0000 7fb1676a7b80 1 -- start start 2024-06-03T02:25:33.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.958+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b4951130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.958+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b4972ba0 0x55d8b4974db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.958+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b49734c0 0x55d8b49752f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.958+0000 7fb1676a7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1580830a0 con 0x55d8b49734c0 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.958+0000 7fb1676a7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b6a80 con 0x55d8b4972ba0 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.958+0000 7fb1676a7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b5610 con 0x55d8b496d250 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b4951130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b4972ba0 0x55d8b4974db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b4951130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52934/0 (socket says 172.21.15.79:52934) 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b4972ba0 0x55d8b4974db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40634/0 (socket says 172.21.15.79:40634) 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155e00700 1 -- 172.21.15.79:0/388851731 learned_addr learned my addr 172.21.15.79:0/388851731 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb156800700 1 --2- 172.21.15.79:0/388851731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b49734c0 0x55d8b49752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155e00700 1 -- 172.21.15.79:0/388851731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b4972ba0 msgr2=0x55d8b4974db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:33.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155e00700 1 --2- 172.21.15.79:0/388851731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b4972ba0 0x55d8b4974db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155e00700 1 -- 172.21.15.79:0/388851731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b49734c0 msgr2=0x55d8b49752f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:33.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155e00700 1 --2- 172.21.15.79:0/388851731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b49734c0 0x55d8b49752f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.959+0000 7fb155e00700 1 -- 172.21.15.79:0/388851731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b4979470 con 0x55d8b496d250 2024-06-03T02:25:33.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.960+0000 7fb155e00700 1 --2- 172.21.15.79:0/388851731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b4951130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb1480a4cd0 tx=0x7fb148087040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9faa6c325ca38d6b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:33.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.960+0000 7fb142a00700 1 -- 172.21.15.79:0/388851731 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1480988c0 con 0x55d8b496d250 2024-06-03T02:25:33.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.960+0000 7fb142a00700 1 -- 172.21.15.79:0/388851731 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14809cd20 con 0x55d8b496d250 2024-06-03T02:25:33.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.960+0000 7fb142a00700 1 -- 172.21.15.79:0/388851731 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14809bf80 con 0x55d8b496d250 2024-06-03T02:25:33.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.961+0000 7fb1676a7b80 1 -- 172.21.15.79:0/388851731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 msgr2=0x55d8b4951130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:33.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.961+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/388851731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b4951130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb1480a4cd0 tx=0x7fb148087040 comp rx=0 tx=0).stop 2024-06-03T02:25:33.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.962+0000 7fb1676a7b80 1 -- 172.21.15.79:0/388851731 shutdown_connections 2024-06-03T02:25:33.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.962+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/388851731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b49734c0 0x55d8b49752f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.962+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/388851731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b4951130 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.962+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/388851731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b4972ba0 0x55d8b4974db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.962+0000 7fb1676a7b80 1 -- 172.21.15.79:0/388851731 >> 172.21.15.79:0/388851731 conn(0x55d8b4888170 msgr2=0x55d8b4976440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:33.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.962+0000 7fb1676a7b80 1 -- 172.21.15.79:0/388851731 shutdown_connections 2024-06-03T02:25:33.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.963+0000 7fb1676a7b80 1 -- 172.21.15.79:0/388851731 wait complete. 2024-06-03T02:25:33.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.964+0000 7fb1676a7b80 1 -- start start 2024-06-03T02:25:33.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.964+0000 7fb1676a7b80 1 Processor -- start 2024-06-03T02:25:33.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.965+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b497f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.965+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b4972ba0 0x55d8b497f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.965+0000 7fb155e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b497f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.965+0000 7fb155e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b497f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52940/0 (socket says 172.21.15.79:52940) 2024-06-03T02:25:33.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb1676a7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 0x55d8b498d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb1676a7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b63b0 con 0x55d8b4972ba0 2024-06-03T02:25:33.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb1676a7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b5ce0 con 0x55d8b49734c0 2024-06-03T02:25:33.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb1676a7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb158080fe0 con 0x55d8b496d250 2024-06-03T02:25:33.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb155e00700 1 -- 172.21.15.79:0/4197728072 learned_addr learned my addr 172.21.15.79:0/4197728072 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:33.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb1676a7b80 10 client.0 _ll_register_callbacks cb 0x55d8b4843380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:33.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb155e00700 1 -- 172.21.15.79:0/4197728072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 msgr2=0x55d8b498d890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:25:33.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb155e00700 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 0x55d8b498d890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb155e00700 1 -- 172.21.15.79:0/4197728072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b4972ba0 msgr2=0x55d8b497f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:33.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb155e00700 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b4972ba0 0x55d8b497f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.967+0000 7fb155e00700 1 -- 172.21.15.79:0/4197728072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b498de50 con 0x55d8b496d250 2024-06-03T02:25:33.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.968+0000 7fb155e00700 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b497f200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb1480093f0 tx=0x7fb1480982c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31845abf8ab2eff2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:33.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.969+0000 7fb137e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:25:33.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.970+0000 7fb137e00700 1 -- 172.21.15.79:0/4197728072 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14809c160 con 0x55d8b496d250 2024-06-03T02:25:33.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.971+0000 7fb137e00700 1 -- 172.21.15.79:0/4197728072 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb1480a4d70 con 0x55d8b496d250 2024-06-03T02:25:33.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.971+0000 7fb1676a7b80 1 -- 172.21.15.79:0/4197728072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb14809cd20 con 0x55d8b496d250 2024-06-03T02:25:33.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.972+0000 7fb137e00700 1 -- 172.21.15.79:0/4197728072 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb14809c3f0 con 0x55d8b496d250 2024-06-03T02:25:33.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.972+0000 7fb1676a7b80 10 client.26020 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:33.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.972+0000 7fb1676a7b80 1 -- 172.21.15.79:0/4197728072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8b49f0f30 con 0x55d8b496d250 2024-06-03T02:25:33.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.972+0000 7fb137e00700 1 -- 172.21.15.79:0/4197728072 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14809cd20 con 0x55d8b496d250 2024-06-03T02:25:33.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.974+0000 7fb137e00700 1 -- 172.21.15.79:0/4197728072 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb148085cc0 con 0x55d8b496d250 2024-06-03T02:25:33.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.975+0000 7fb1676a7b80 10 client.26020 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:33.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.975+0000 7fb1676a7b80 10 client.26020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:25:33.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.975+0000 7fb1676a7b80 20 client.26020 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:33.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.975+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d8b49f14e0 0x55d8b49f3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:33.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.975+0000 7fb1676a7b80 4 client.26020 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:25:33.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.975+0000 7fb1676a7b80 1 -- 172.21.15.79:0/4197728072 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8b48b5580 con 0x55d8b49f14e0 2024-06-03T02:25:33.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.976+0000 7fb155400700 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d8b49f14e0 0x55d8b49f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:33.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.979+0000 7fb137e00700 10 client.26020 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:25:33.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.982+0000 7fb155400700 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d8b49f14e0 0x55d8b49f3900 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=241098beeaf64adb server_cookie=5bf30df7746914f5 in_seq=0 out_seq=0 2024-06-03T02:25:33.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.982+0000 7fb137e00700 1 -- 172.21.15.79:0/4197728072 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8b48b5580 con 0x55d8b49f14e0 2024-06-03T02:25:33.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.982+0000 7fb137e00700 10 client.26020 handle_command_reply: tid=0 2024-06-03T02:25:33.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.982+0000 7fb1676a7b80 1 client.26020 shutdown 2024-06-03T02:25:33.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.982+0000 7fb1676a7b80 1 -- 172.21.15.79:0/4197728072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 msgr2=0x55d8b497f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:33.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.982+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b497f200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb1480093f0 tx=0x7fb1480982c0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb1676a7b80 1 -- 172.21.15.79:0/4197728072 shutdown_connections 2024-06-03T02:25:33.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55d8b49f14e0 0x55d8b49f3900 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8b49734c0 0x55d8b498d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b496d250 0x55d8b497f200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb137e00700 0 client.26020 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:25:33.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb1676a7b80 1 --2- 172.21.15.79:0/4197728072 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8b4972ba0 0x55d8b497f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb1676a7b80 1 -- 172.21.15.79:0/4197728072 >> 172.21.15.79:0/4197728072 conn(0x55d8b4888170 msgr2=0x55d8b4956a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:33.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb1676a7b80 1 -- 172.21.15.79:0/4197728072 shutdown_connections 2024-06-03T02:25:33.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb1676a7b80 1 -- 172.21.15.79:0/4197728072 wait complete. 2024-06-03T02:25:33.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.983+0000 7fb1676a7b80 20 client.26020 trim_cache size 0 max 16384 2024-06-03T02:25:33.985 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:25:33.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 -- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb13c0a2c50 msgr2=0x7fb13c0a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:33.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb13c0a2c50 0x7fb13c0a5110 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb14400efc0 tx=0x7fb144009290 comp rx=0 tx=0).stop 2024-06-03T02:25:33.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 -- 172.21.15.79:0/2741728112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb158108ac0 msgr2=0x7fb158073de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:33.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb158108ac0 0x7fb158073de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb148008000 tx=0x7fb1480057c0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 -- 172.21.15.79:0/2741728112 shutdown_connections 2024-06-03T02:25:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb15806d450 0x7fb15806d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb158108ac0 0x7fb158073de0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb13c0a2c50 0x7fb13c0a5110 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 --2- 172.21.15.79:0/2741728112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb158109480 0x7fb15806cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 -- 172.21.15.79:0/2741728112 >> 172.21.15.79:0/2741728112 conn(0x7fb158075c60 msgr2=0x7fb158076490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 -- 172.21.15.79:0/2741728112 shutdown_connections 2024-06-03T02:25:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:33.986+0000 7fb15ce00700 1 -- 172.21.15.79:0/2741728112 wait complete. 2024-06-03T02:25:34.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:33 smithi079 ceph-mon[31819]: pgmap v681: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 506 op/s 2024-06-03T02:25:34.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:33 smithi135 ceph-mon[36653]: pgmap v681: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 506 op/s 2024-06-03T02:25:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:33 smithi123 ceph-mon[36640]: pgmap v681: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 506 op/s 2024-06-03T02:25:35.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:25:36.023 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:36.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:35 smithi079 ceph-mon[31819]: pgmap v682: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 540 op/s 2024-06-03T02:25:36.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:35 smithi135 ceph-mon[36653]: pgmap v682: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 540 op/s 2024-06-03T02:25:36.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:35 smithi123 ceph-mon[36640]: pgmap v682: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 540 op/s 2024-06-03T02:25:37.807 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:37 smithi079 ceph-mon[31819]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 525 op/s 2024-06-03T02:25:38.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:37 smithi135 ceph-mon[36653]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 525 op/s 2024-06-03T02:25:38.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:37 smithi123 ceph-mon[36640]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 525 op/s 2024-06-03T02:25:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 330.0 s, 587.6 tps, lat 163.296 ms stddev 188.270 2024-06-03T02:25:38.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.496+0000 7f65b1800700 1 -- 172.21.15.79:0/3806610970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac104ec0 msgr2=0x7f65ac105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:38.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.496+0000 7f65b1800700 1 --2- 172.21.15.79:0/3806610970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac104ec0 0x7f65ac105350 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f65a40098f0 tx=0x7f65a4009c00 comp rx=0 tx=0).stop 2024-06-03T02:25:38.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.497+0000 7f65b1800700 1 -- 172.21.15.79:0/3806610970 shutdown_connections 2024-06-03T02:25:38.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.497+0000 7f65b1800700 1 --2- 172.21.15.79:0/3806610970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac104ec0 0x7f65ac105350 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.497+0000 7f65b1800700 1 --2- 172.21.15.79:0/3806610970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65ac1073b0 0x7f65ac06aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.497+0000 7f65b1800700 1 --2- 172.21.15.79:0/3806610970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65ac06af70 0x7f65ac06b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.497+0000 7f65b1800700 1 -- 172.21.15.79:0/3806610970 >> 172.21.15.79:0/3806610970 conn(0x7f65ac078300 msgr2=0x7f65ac078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:38.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.497+0000 7f65b1800700 1 -- 172.21.15.79:0/3806610970 shutdown_connections 2024-06-03T02:25:38.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.497+0000 7f65b1800700 1 -- 172.21.15.79:0/3806610970 wait complete. 2024-06-03T02:25:38.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.499+0000 7f65b1800700 1 Processor -- start 2024-06-03T02:25:38.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.500+0000 7f65b1800700 1 -- start start 2024-06-03T02:25:38.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.500+0000 7f65b1800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac06af70 0x7f65ac1156d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.500+0000 7f65b1800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65ac104ec0 0x7f65ac115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.500+0000 7f65b1800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65ac1073b0 0x7f65ac10e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.500+0000 7f65b1800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac118dc0 con 0x7f65ac104ec0 2024-06-03T02:25:38.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.500+0000 7f65b1800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac118c40 con 0x7f65ac06af70 2024-06-03T02:25:38.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.500+0000 7f65b1800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac118f40 con 0x7f65ac1073b0 2024-06-03T02:25:38.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.501+0000 7f65aaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac06af70 0x7f65ac1156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.501+0000 7f65aaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac06af70 0x7f65ac1156d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35352/0 (socket says 172.21.15.79:35352) 2024-06-03T02:25:38.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.501+0000 7f65aaa00700 1 -- 172.21.15.79:0/4084395242 learned_addr learned my addr 172.21.15.79:0/4084395242 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:38.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.501+0000 7f65aa000700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65ac104ec0 0x7f65ac115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.501+0000 7f65ab400700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65ac1073b0 0x7f65ac10e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.501+0000 7f65aaa00700 1 -- 172.21.15.79:0/4084395242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65ac1073b0 msgr2=0x7f65ac10e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:38.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.501+0000 7f65aaa00700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65ac1073b0 0x7f65ac10e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.502+0000 7f65aaa00700 1 -- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65ac104ec0 msgr2=0x7f65ac115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:38.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.502+0000 7f65aaa00700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65ac104ec0 0x7f65ac115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.502+0000 7f65aaa00700 1 -- 172.21.15.79:0/4084395242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac10f220 con 0x7f65ac06af70 2024-06-03T02:25:38.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.502+0000 7f65aa000700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65ac104ec0 0x7f65ac115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:38.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.502+0000 7f65ab400700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65ac1073b0 0x7f65ac10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:38.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.502+0000 7f65aaa00700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac06af70 0x7f65ac1156d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f659c00e3c0 tx=0x7f659c00e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:38.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.502+0000 7f65a3400700 1 -- 172.21.15.79:0/4084395242 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0090d0 con 0x7f65ac06af70 2024-06-03T02:25:38.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.502+0000 7f65a3400700 1 -- 172.21.15.79:0/4084395242 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c009270 con 0x7f65ac06af70 2024-06-03T02:25:38.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.502+0000 7f65a3400700 1 -- 172.21.15.79:0/4084395242 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0048a0 con 0x7f65ac06af70 2024-06-03T02:25:38.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.503+0000 7f65b1800700 1 -- 172.21.15.79:0/4084395242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac10f4b0 con 0x7f65ac06af70 2024-06-03T02:25:38.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.503+0000 7f65b1800700 1 -- 172.21.15.79:0/4084395242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac113430 con 0x7f65ac06af70 2024-06-03T02:25:38.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.507+0000 7f65a3400700 1 -- 172.21.15.79:0/4084395242 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f659c01b030 con 0x7f65ac06af70 2024-06-03T02:25:38.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.509+0000 7f65a3400700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65900a2ec0 0x7f65900a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.510+0000 7f65a3400700 1 -- 172.21.15.79:0/4084395242 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f659c0cedb0 con 0x7f65ac06af70 2024-06-03T02:25:38.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.510+0000 7f65aa000700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65900a2ec0 0x7f65900a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.510+0000 7f65bc198b80 1 Processor -- start 2024-06-03T02:25:38.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.511+0000 7f65bc198b80 1 -- start start 2024-06-03T02:25:38.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.511+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.511+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccec5b0 0x55e52cceca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.511+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccf7300 0x55e52ccf96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65bc198b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e52cbfba10 con 0x55e52ccec5b0 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65bc198b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e52cc3eb00 con 0x55e52ccf7300 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65bc198b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e52cbf98f0 con 0x55e52cc68e40 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccec070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46532/0 (socket says 172.21.15.79:46532) 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aaa00700 1 -- 172.21.15.79:0/3177141875 learned_addr learned my addr 172.21.15.79:0/3177141875 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aa000700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65900a2ec0 0x7f65900a5380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f65ac10ff20 tx=0x7f6594009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65ab400700 1 --2- 172.21.15.79:0/3177141875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccf7300 0x55e52ccf96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aaa00700 1 -- 172.21.15.79:0/3177141875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccf7300 msgr2=0x55e52ccf96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aaa00700 1 --2- 172.21.15.79:0/3177141875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccf7300 0x55e52ccf96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aaa00700 1 -- 172.21.15.79:0/3177141875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccec5b0 msgr2=0x55e52cceca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:38.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aaa00700 1 --2- 172.21.15.79:0/3177141875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccec5b0 0x55e52cceca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aaa00700 1 -- 172.21.15.79:0/3177141875 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52cbe6f90 con 0x55e52cc68e40 2024-06-03T02:25:38.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65ab400700 1 --2- 172.21.15.79:0/3177141875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccf7300 0x55e52ccf96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:38.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.512+0000 7f65aaa00700 1 --2- 172.21.15.79:0/3177141875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccec070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f659c09b220 tx=0x7f659c009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54988960826574c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:38.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.513+0000 7f659be00700 1 -- 172.21.15.79:0/3177141875 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c09add0 con 0x55e52cc68e40 2024-06-03T02:25:38.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.513+0000 7f659be00700 1 -- 172.21.15.79:0/3177141875 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c09b990 con 0x55e52cc68e40 2024-06-03T02:25:38.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.513+0000 7f659be00700 1 -- 172.21.15.79:0/3177141875 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c09abf0 con 0x55e52cc68e40 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.514+0000 7f65bc198b80 1 -- 172.21.15.79:0/3177141875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 msgr2=0x55e52ccec070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.514+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3177141875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccec070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f659c09b220 tx=0x7f659c009e20 comp rx=0 tx=0).stop 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.514+0000 7f65bc198b80 1 -- 172.21.15.79:0/3177141875 shutdown_connections 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.514+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3177141875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccf7300 0x55e52ccf96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.514+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3177141875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccec070 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.514+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3177141875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccec5b0 0x55e52cceca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.514+0000 7f65bc198b80 1 -- 172.21.15.79:0/3177141875 >> 172.21.15.79:0/3177141875 conn(0x55e52cc19170 msgr2=0x55e52cce84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.514+0000 7f65bc198b80 1 -- 172.21.15.79:0/3177141875 shutdown_connections 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.514+0000 7f65bc198b80 1 -- 172.21.15.79:0/3177141875 wait complete. 2024-06-03T02:25:38.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.515+0000 7f65bc198b80 1 -- start start 2024-06-03T02:25:38.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.515+0000 7f65bc198b80 1 Processor -- start 2024-06-03T02:25:38.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52cc68e40 0x55e52ccf6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccec5b0 0x55e52ccf3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 0x55e52ccf42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65bc198b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e52cbdaf60 con 0x55e52ccec5b0 2024-06-03T02:25:38.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65bc198b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e52cc3aac0 con 0x55e52cc68e40 2024-06-03T02:25:38.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65bc198b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e52ccf9fc0 con 0x55e52ccf7300 2024-06-03T02:25:38.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65bc198b80 10 client.0 _ll_register_callbacks cb 0x55e52cbd8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:38.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65aaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52cc68e40 0x55e52ccf6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65aaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52cc68e40 0x55e52ccf6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35378/0 (socket says 172.21.15.79:35378) 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65aaa00700 1 -- 172.21.15.79:0/1829422042 learned_addr learned my addr 172.21.15.79:0/1829422042 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.516+0000 7f65ab400700 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 0x55e52ccf42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f65aaa00700 1 -- 172.21.15.79:0/1829422042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 msgr2=0x55e52ccf42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f65aaa00700 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 0x55e52ccf42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f65aaa00700 1 -- 172.21.15.79:0/1829422042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccec5b0 msgr2=0x55e52ccf3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f65aaa00700 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccec5b0 0x55e52ccf3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f65aaa00700 1 -- 172.21.15.79:0/1829422042 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52cb8d8d0 con 0x55e52cc68e40 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f65aaa00700 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52cc68e40 0x55e52ccf6a70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f659c0a3940 tx=0x7f659c0bb430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb3129cec4c79cc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f6598c00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f6598c00700 1 -- 172.21.15.79:0/1829422042 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0b8030 con 0x55e52cc68e40 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f6598c00700 1 -- 172.21.15.79:0/1829422042 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f659c0cc520 con 0x55e52cc68e40 2024-06-03T02:25:38.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.517+0000 7f65bc198b80 1 -- 172.21.15.79:0/1829422042 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c09b990 con 0x55e52cc68e40 2024-06-03T02:25:38.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.518+0000 7f6598c00700 1 -- 172.21.15.79:0/1829422042 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f659c09add0 con 0x55e52cc68e40 2024-06-03T02:25:38.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.519+0000 7f65bc198b80 10 client.26036 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:38.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.519+0000 7f65bc198b80 1 -- 172.21.15.79:0/1829422042 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e52cd04660 con 0x55e52cc68e40 2024-06-03T02:25:38.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.520+0000 7f6598c00700 1 -- 172.21.15.79:0/1829422042 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c09b990 con 0x55e52cc68e40 2024-06-03T02:25:38.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.520+0000 7f6598c00700 1 -- 172.21.15.79:0/1829422042 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f659c0a39e0 con 0x55e52cc68e40 2024-06-03T02:25:38.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.520+0000 7f65bc198b80 10 client.26036 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:38.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.520+0000 7f65bc198b80 10 client.26036 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:25:38.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.520+0000 7f65bc198b80 20 client.26036 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:38.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.520+0000 7f65bc198b80 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e52cd06b80 0x55e52cd08f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.520+0000 7f65bc198b80 4 client.26036 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:25:38.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.520+0000 7f65bc198b80 1 -- 172.21.15.79:0/1829422042 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e52cc4c000 con 0x55e52cd06b80 2024-06-03T02:25:38.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.521+0000 7f65aa000700 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e52cd06b80 0x55e52cd08f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.522+0000 7f65aa000700 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e52cd06b80 0x55e52cd08f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34bf726336e0b4ab server_cookie=a8ab2b5ad8ac91dc in_seq=0 out_seq=0 2024-06-03T02:25:38.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.522+0000 7f6598c00700 10 client.26036 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:25:38.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.523+0000 7f6598c00700 1 -- 172.21.15.79:0/1829422042 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e52cc4c000 con 0x55e52cd06b80 2024-06-03T02:25:38.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.523+0000 7f6598c00700 10 client.26036 handle_command_reply: tid=0 2024-06-03T02:25:38.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.523+0000 7f65bc198b80 1 client.26036 shutdown 2024-06-03T02:25:38.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.523+0000 7f65bc198b80 1 -- 172.21.15.79:0/1829422042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52cc68e40 msgr2=0x55e52ccf6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:38.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.523+0000 7f65bc198b80 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52cc68e40 0x55e52ccf6a70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f659c0a3940 tx=0x7f659c0bb430 comp rx=0 tx=0).stop 2024-06-03T02:25:38.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f65bc198b80 1 -- 172.21.15.79:0/1829422042 shutdown_connections 2024-06-03T02:25:38.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f65bc198b80 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e52cd06b80 0x55e52cd08f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f65bc198b80 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 0x55e52ccf42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f65bc198b80 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52cc68e40 0x55e52ccf6a70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f65bc198b80 1 --2- 172.21.15.79:0/1829422042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccec5b0 0x55e52ccf3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f65bc198b80 1 -- 172.21.15.79:0/1829422042 >> 172.21.15.79:0/1829422042 conn(0x55e52cc19170 msgr2=0x55e52cce7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:38.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f6598c00700 0 client.26036 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:25:38.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f65bc198b80 1 -- 172.21.15.79:0/1829422042 shutdown_connections 2024-06-03T02:25:38.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f65bc198b80 1 -- 172.21.15.79:0/1829422042 wait complete. 2024-06-03T02:25:38.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.524+0000 7f65bc198b80 20 client.26036 trim_cache size 0 max 16384 2024-06-03T02:25:38.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.546+0000 7f65bc198b80 1 Processor -- start 2024-06-03T02:25:38.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.546+0000 7f65bc198b80 1 -- start start 2024-06-03T02:25:38.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.546+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccf65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.546+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.546+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccf7300 0x55e52ccf8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.546+0000 7f65bc198b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1bd3b0 con 0x55e52ccf7300 2024-06-03T02:25:38.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.546+0000 7f65bc198b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1beef0 con 0x55e52ccec5b0 2024-06-03T02:25:38.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.546+0000 7f65bc198b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1bda80 con 0x55e52cc68e40 2024-06-03T02:25:38.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.547+0000 7f65aaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccf65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.547+0000 7f65aa000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.547+0000 7f65aaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccf65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46552/0 (socket says 172.21.15.79:46552) 2024-06-03T02:25:38.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.547+0000 7f65aaa00700 1 -- 172.21.15.79:0/304998040 learned_addr learned my addr 172.21.15.79:0/304998040 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:38.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.547+0000 7f65ab400700 1 --2- 172.21.15.79:0/304998040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccf7300 0x55e52ccf8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.547+0000 7f65aa000700 1 -- 172.21.15.79:0/304998040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 msgr2=0x55e52ccf65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:38.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.547+0000 7f65aa000700 1 --2- 172.21.15.79:0/304998040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccf65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.548+0000 7f65aa000700 1 -- 172.21.15.79:0/304998040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccf7300 msgr2=0x55e52ccf8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:38.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.548+0000 7f65aa000700 1 --2- 172.21.15.79:0/304998040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccf7300 0x55e52ccf8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.548+0000 7f65aa000700 1 -- 172.21.15.79:0/304998040 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52ccdfe10 con 0x55e52ccec5b0 2024-06-03T02:25:38.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.548+0000 7f65aaa00700 1 --2- 172.21.15.79:0/304998040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccf65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:25:38.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.548+0000 7f65aa000700 1 --2- 172.21.15.79:0/304998040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf86f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55e52cd09740 tx=0x7f6594014070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae8bb331f14860 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:38.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.548+0000 7f659b400700 1 -- 172.21.15.79:0/304998040 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659402c070 con 0x55e52ccec5b0 2024-06-03T02:25:38.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.548+0000 7f659b400700 1 -- 172.21.15.79:0/304998040 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6594012c90 con 0x55e52ccec5b0 2024-06-03T02:25:38.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.549+0000 7f659b400700 1 -- 172.21.15.79:0/304998040 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594027c10 con 0x55e52ccec5b0 2024-06-03T02:25:38.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.549+0000 7f65bc198b80 1 -- 172.21.15.79:0/304998040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 msgr2=0x55e52ccf86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:38.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.549+0000 7f65bc198b80 1 --2- 172.21.15.79:0/304998040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf86f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55e52cd09740 tx=0x7f6594014070 comp rx=0 tx=0).stop 2024-06-03T02:25:38.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.550+0000 7f65bc198b80 1 -- 172.21.15.79:0/304998040 shutdown_connections 2024-06-03T02:25:38.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.550+0000 7f65bc198b80 1 --2- 172.21.15.79:0/304998040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52ccf7300 0x55e52ccf8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.550+0000 7f65bc198b80 1 --2- 172.21.15.79:0/304998040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52cc68e40 0x55e52ccf65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.550+0000 7f65bc198b80 1 --2- 172.21.15.79:0/304998040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf86f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.550+0000 7f65bc198b80 1 -- 172.21.15.79:0/304998040 >> 172.21.15.79:0/304998040 conn(0x55e52cc19170 msgr2=0x55e52ccf2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:38.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.550+0000 7f65bc198b80 1 -- 172.21.15.79:0/304998040 shutdown_connections 2024-06-03T02:25:38.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.550+0000 7f65bc198b80 1 -- 172.21.15.79:0/304998040 wait complete. 2024-06-03T02:25:38.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.551+0000 7f65bc198b80 1 -- start start 2024-06-03T02:25:38.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.551+0000 7f65bc198b80 1 Processor -- start 2024-06-03T02:25:38.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.552+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52cc68e40 0x55e52cd0ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.552+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.552+0000 7f65bc198b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 0x55e52ccf1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.552+0000 7f65bc198b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1be820 con 0x55e52cc68e40 2024-06-03T02:25:38.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.552+0000 7f65bc198b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1be150 con 0x55e52ccec5b0 2024-06-03T02:25:38.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.552+0000 7f65bc198b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1bb350 con 0x55e52ccf7300 2024-06-03T02:25:38.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.552+0000 7f65bc198b80 10 client.0 _ll_register_callbacks cb 0x55e52cbd8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:38.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.552+0000 7f65aa000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.552+0000 7f65aa000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35396/0 (socket says 172.21.15.79:35396) 2024-06-03T02:25:38.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.553+0000 7f65aa000700 1 -- 172.21.15.79:0/3183871581 learned_addr learned my addr 172.21.15.79:0/3183871581 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:38.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.553+0000 7f65ab400700 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 0x55e52ccf1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.554+0000 7f65aaa00700 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52cc68e40 0x55e52cd0ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.554+0000 7f65aa000700 1 -- 172.21.15.79:0/3183871581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 msgr2=0x55e52ccf1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:38.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.554+0000 7f65aa000700 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 0x55e52ccf1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.554+0000 7f65aa000700 1 -- 172.21.15.79:0/3183871581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52cc68e40 msgr2=0x55e52cd0ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:38.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.554+0000 7f65aa000700 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52cc68e40 0x55e52cd0ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.554+0000 7f65aa000700 1 -- 172.21.15.79:0/3183871581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52ccf1d20 con 0x55e52ccec5b0 2024-06-03T02:25:38.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.555+0000 7f65aa000700 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf1010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6594027750 tx=0x7f6594011dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae17cdec014409fc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:38.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.555+0000 7f6598c00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:25:38.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.556+0000 7f6598c00700 1 -- 172.21.15.79:0/3183871581 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659402c070 con 0x55e52ccec5b0 2024-06-03T02:25:38.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.556+0000 7f65bc198b80 1 -- 172.21.15.79:0/3183871581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6594012c90 con 0x55e52ccec5b0 2024-06-03T02:25:38.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.556+0000 7f6598c00700 1 -- 172.21.15.79:0/3183871581 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f65940274e0 con 0x55e52ccec5b0 2024-06-03T02:25:38.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.556+0000 7f6598c00700 1 -- 172.21.15.79:0/3183871581 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f659403d050 con 0x55e52ccec5b0 2024-06-03T02:25:38.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.556+0000 7f65bc198b80 10 client.26042 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:38.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.556+0000 7f65bc198b80 1 -- 172.21.15.79:0/3183871581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e52cd0d820 con 0x55e52ccec5b0 2024-06-03T02:25:38.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.557+0000 7f6598c00700 1 -- 172.21.15.79:0/3183871581 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6594012c90 con 0x55e52ccec5b0 2024-06-03T02:25:38.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.557+0000 7f6598c00700 1 -- 172.21.15.79:0/3183871581 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6594038030 con 0x55e52ccec5b0 2024-06-03T02:25:38.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.557+0000 7f65bc198b80 10 client.26042 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:38.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.557+0000 7f65bc198b80 10 client.26042 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:25:38.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.557+0000 7f65bc198b80 20 client.26042 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:38.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.557+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e52cd0deb0 0x55e52ccfe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:38.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.557+0000 7f65bc198b80 4 client.26042 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:25:38.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.557+0000 7f65bc198b80 1 -- 172.21.15.79:0/3183871581 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e52cc46580 con 0x55e52cd0deb0 2024-06-03T02:25:38.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.559+0000 7f65aaa00700 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e52cd0deb0 0x55e52ccfe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:38.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.565+0000 7f65aaa00700 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e52cd0deb0 0x55e52ccfe240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f84bd1469217d97 server_cookie=6a3b2a1afba7b204 in_seq=0 out_seq=0 2024-06-03T02:25:38.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.565+0000 7f6598c00700 10 client.26042 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:25:38.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.566+0000 7f6598c00700 1 -- 172.21.15.79:0/3183871581 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e52cc46580 con 0x55e52cd0deb0 2024-06-03T02:25:38.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.566+0000 7f6598c00700 10 client.26042 handle_command_reply: tid=0 2024-06-03T02:25:38.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.567+0000 7f65bc198b80 1 client.26042 shutdown 2024-06-03T02:25:38.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f65bc198b80 1 -- 172.21.15.79:0/3183871581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 msgr2=0x55e52ccf1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:38.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf1010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6594027750 tx=0x7f6594011dc0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f65bc198b80 1 -- 172.21.15.79:0/3183871581 shutdown_connections 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e52cd0deb0 0x55e52ccfe240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e52ccf7300 0x55e52ccf1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e52cc68e40 0x55e52cd0ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f65bc198b80 1 --2- 172.21.15.79:0/3183871581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e52ccec5b0 0x55e52ccf1010 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f6598c00700 0 client.26042 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f65bc198b80 1 -- 172.21.15.79:0/3183871581 >> 172.21.15.79:0/3183871581 conn(0x55e52cc19170 msgr2=0x55e52cce6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.568+0000 7f65bc198b80 1 -- 172.21.15.79:0/3183871581 shutdown_connections 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.569+0000 7f65bc198b80 1 -- 172.21.15.79:0/3183871581 wait complete. 2024-06-03T02:25:38.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.569+0000 7f65bc198b80 20 client.26042 trim_cache size 0 max 16384 2024-06-03T02:25:38.573 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:25:38.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 -- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65900a2ec0 msgr2=0x7f65900a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:38.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65900a2ec0 0x7f65900a5380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f65ac10ff20 tx=0x7f6594009260 comp rx=0 tx=0).stop 2024-06-03T02:25:38.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 -- 172.21.15.79:0/4084395242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac06af70 msgr2=0x7f65ac1156d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:38.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac06af70 0x7f65ac1156d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f659c00e3c0 tx=0x7f659c00e780 comp rx=0 tx=0).stop 2024-06-03T02:25:38.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 -- 172.21.15.79:0/4084395242 shutdown_connections 2024-06-03T02:25:38.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65ac1073b0 0x7f65ac10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac06af70 0x7f65ac1156d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65900a2ec0 0x7f65900a5380 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 --2- 172.21.15.79:0/4084395242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65ac104ec0 0x7f65ac115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:38.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.579+0000 7f65b1800700 1 -- 172.21.15.79:0/4084395242 >> 172.21.15.79:0/4084395242 conn(0x7f65ac078300 msgr2=0x7f65ac103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:38.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.580+0000 7f65b1800700 1 -- 172.21.15.79:0/4084395242 shutdown_connections 2024-06-03T02:25:38.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:38.581+0000 7f65b1800700 1 -- 172.21.15.79:0/4084395242 wait complete. 2024-06-03T02:25:40.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:39 smithi079 ceph-mon[31819]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 543 op/s 2024-06-03T02:25:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:39 smithi135 ceph-mon[36653]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 543 op/s 2024-06-03T02:25:40.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:39 smithi123 ceph-mon[36640]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 543 op/s 2024-06-03T02:25:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:41 smithi079 ceph-mon[31819]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 500 op/s 2024-06-03T02:25:42.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:41 smithi135 ceph-mon[36653]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 500 op/s 2024-06-03T02:25:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:41 smithi123 ceph-mon[36640]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 500 op/s 2024-06-03T02:25:42.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:25:43.077 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 335.0 s, 488.8 tps, lat 201.384 ms stddev 255.967 2024-06-03T02:25:44.091 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:43 smithi079 ceph-mon[31819]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 561 op/s 2024-06-03T02:25:44.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:43 smithi135 ceph-mon[36653]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 561 op/s 2024-06-03T02:25:44.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:43 smithi123 ceph-mon[36640]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 561 op/s 2024-06-03T02:25:45.976 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:45 smithi079 ceph-mon[31819]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 575 op/s 2024-06-03T02:25:46.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:45 smithi135 ceph-mon[36653]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 575 op/s 2024-06-03T02:25:46.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:45 smithi123 ceph-mon[36640]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 575 op/s 2024-06-03T02:25:46.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.223+0000 7f0684200700 1 -- 172.21.15.79:0/2497273128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067c06ab30 msgr2=0x7f067c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:46.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.223+0000 7f0684200700 1 --2- 172.21.15.79:0/2497273128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067c06ab30 0x7f067c10f320 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0670009990 tx=0x7f0670009ca0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.225+0000 7f0684200700 1 -- 172.21.15.79:0/2497273128 shutdown_connections 2024-06-03T02:25:46.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.225+0000 7f0684200700 1 --2- 172.21.15.79:0/2497273128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067c06ab30 0x7f067c10f320 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.225+0000 7f0684200700 1 --2- 172.21.15.79:0/2497273128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f067c108500 0x7f067c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.225+0000 7f0684200700 1 --2- 172.21.15.79:0/2497273128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067c06a170 0x7f067c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.225+0000 7f0684200700 1 -- 172.21.15.79:0/2497273128 >> 172.21.15.79:0/2497273128 conn(0x7f067c078340 msgr2=0x7f067c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:46.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.226+0000 7f0684200700 1 -- 172.21.15.79:0/2497273128 shutdown_connections 2024-06-03T02:25:46.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.226+0000 7f0684200700 1 -- 172.21.15.79:0/2497273128 wait complete. 2024-06-03T02:25:46.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.227+0000 7f0684200700 1 Processor -- start 2024-06-03T02:25:46.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.228+0000 7f0684200700 1 -- start start 2024-06-03T02:25:46.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.228+0000 7f0684200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067c06a170 0x7f067c19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.229+0000 7f0684200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f067c06ab30 0x7f067c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.229+0000 7f0684200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067c108500 0x7f067c10d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.230+0000 7f0680e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f067c06ab30 0x7f067c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.230+0000 7f0680e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f067c06ab30 0x7f067c19bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55166/0 (socket says 172.21.15.79:55166) 2024-06-03T02:25:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.230+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f067c112830 con 0x7f067c108500 2024-06-03T02:25:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.230+0000 7f0680e00700 1 -- 172.21.15.79:0/337925069 learned_addr learned my addr 172.21.15.79:0/337925069 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.230+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1126b0 con 0x7f067c06a170 2024-06-03T02:25:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.230+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1129b0 con 0x7f067c06ab30 2024-06-03T02:25:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.230+0000 7f0682200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067c108500 0x7f067c10d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.231+0000 7f0682200700 1 -- 172.21.15.79:0/337925069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f067c06ab30 msgr2=0x7f067c19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:46.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.231+0000 7f0682200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f067c06ab30 0x7f067c19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.231+0000 7f0682200700 1 -- 172.21.15.79:0/337925069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067c06a170 msgr2=0x7f067c19b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.231+0000 7f0681800700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067c06a170 0x7f067c19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.232+0000 7f0682200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067c06a170 0x7f067c19b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.232+0000 7f0682200700 1 -- 172.21.15.79:0/337925069 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c10dae0 con 0x7f067c108500 2024-06-03T02:25:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.232+0000 7f0680e00700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f067c06ab30 0x7f067c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:46.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.232+0000 7f0681800700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067c06a170 0x7f067c19b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:46.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.238+0000 7f0682200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067c108500 0x7f067c10d1e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f06700098e0 tx=0x7f067000bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.239+0000 7f067a000700 1 -- 172.21.15.79:0/337925069 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0670027070 con 0x7f067c108500 2024-06-03T02:25:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.239+0000 7f067a000700 1 -- 172.21.15.79:0/337925069 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0670003d70 con 0x7f067c108500 2024-06-03T02:25:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.239+0000 7f067a000700 1 -- 172.21.15.79:0/337925069 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067000f460 con 0x7f067c108500 2024-06-03T02:25:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.239+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c10dd70 con 0x7f067c108500 2024-06-03T02:25:46.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.239+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c19f620 con 0x7f067c108500 2024-06-03T02:25:46.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.248+0000 7f067a000700 1 -- 172.21.15.79:0/337925069 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0670009d10 con 0x7f067c108500 2024-06-03T02:25:46.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.249+0000 7f067a000700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06640a2ca0 0x7f06640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.249+0000 7f067a000700 1 -- 172.21.15.79:0/337925069 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f06700d01b0 con 0x7f067c108500 2024-06-03T02:25:46.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.252+0000 7f0681800700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06640a2ca0 0x7f06640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.252+0000 7f068eb31b80 1 Processor -- start 2024-06-03T02:25:46.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.253+0000 7f068eb31b80 1 -- start start 2024-06-03T02:25:46.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.253+0000 7f0681800700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06640a2ca0 0x7f06640a5160 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0674007eb0 tx=0x7f06740073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:46.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.253+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72a51e40 0x55df72ad5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.253+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ad5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.253+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ae0300 0x55df72ae26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f068eb31b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df729e4a10 con 0x55df72ae0300 2024-06-03T02:25:46.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f068eb31b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df72a27b00 con 0x55df72a51e40 2024-06-03T02:25:46.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f068eb31b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55df729e28f0 con 0x55df72ad55b0 2024-06-03T02:25:46.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0681800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72a51e40 0x55df72ad5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0681800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72a51e40 0x55df72ad5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38702/0 (socket says 172.21.15.79:38702) 2024-06-03T02:25:46.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0681800700 1 -- 172.21.15.79:0/3923395708 learned_addr learned my addr 172.21.15.79:0/3923395708 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:46.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0680e00700 1 --2- 172.21.15.79:0/3923395708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ad5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0681800700 1 -- 172.21.15.79:0/3923395708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 msgr2=0x55df72ad5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:46.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0681800700 1 --2- 172.21.15.79:0/3923395708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ad5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0681800700 1 -- 172.21.15.79:0/3923395708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ae0300 msgr2=0x55df72ae26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:46.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0681800700 1 --2- 172.21.15.79:0/3923395708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ae0300 0x55df72ae26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0681800700 1 -- 172.21.15.79:0/3923395708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df729cff90 con 0x55df72a51e40 2024-06-03T02:25:46.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.254+0000 7f0680e00700 1 --2- 172.21.15.79:0/3923395708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ad5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:25:46.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.255+0000 7f0681800700 1 --2- 172.21.15.79:0/3923395708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72a51e40 0x55df72ad5070 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f067400f440 tx=0x7f067400f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11604b804c78e81f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:46.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.255+0000 7f066ea00700 1 -- 172.21.15.79:0/3923395708 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067400fde0 con 0x55df72a51e40 2024-06-03T02:25:46.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.255+0000 7f066ea00700 1 -- 172.21.15.79:0/3923395708 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0674021070 con 0x55df72a51e40 2024-06-03T02:25:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.255+0000 7f066ea00700 1 -- 172.21.15.79:0/3923395708 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067401b860 con 0x55df72a51e40 2024-06-03T02:25:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.256+0000 7f068eb31b80 1 -- 172.21.15.79:0/3923395708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72a51e40 msgr2=0x55df72ad5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.256+0000 7f068eb31b80 1 --2- 172.21.15.79:0/3923395708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72a51e40 0x55df72ad5070 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f067400f440 tx=0x7f067400f000 comp rx=0 tx=0).stop 2024-06-03T02:25:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.257+0000 7f068eb31b80 1 -- 172.21.15.79:0/3923395708 shutdown_connections 2024-06-03T02:25:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.257+0000 7f068eb31b80 1 --2- 172.21.15.79:0/3923395708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ae0300 0x55df72ae26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.257+0000 7f068eb31b80 1 --2- 172.21.15.79:0/3923395708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72a51e40 0x55df72ad5070 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.257+0000 7f068eb31b80 1 --2- 172.21.15.79:0/3923395708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ad5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.257+0000 7f068eb31b80 1 -- 172.21.15.79:0/3923395708 >> 172.21.15.79:0/3923395708 conn(0x55df72a02170 msgr2=0x55df72ad14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:46.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.259+0000 7f068eb31b80 1 -- 172.21.15.79:0/3923395708 shutdown_connections 2024-06-03T02:25:46.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.259+0000 7f068eb31b80 1 -- 172.21.15.79:0/3923395708 wait complete. 2024-06-03T02:25:46.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.261+0000 7f068eb31b80 1 -- start start 2024-06-03T02:25:46.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.261+0000 7f068eb31b80 1 Processor -- start 2024-06-03T02:25:46.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.262+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72a51e40 0x55df72adfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.263+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72adcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.263+0000 7f0681800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72a51e40 0x55df72adfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.263+0000 7f0681800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72a51e40 0x55df72adfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55184/0 (socket says 172.21.15.79:55184) 2024-06-03T02:25:46.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.263+0000 7f0681800700 1 -- 172.21.15.79:0/1568290039 learned_addr learned my addr 172.21.15.79:0/1568290039 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:46.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.264+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72add310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.264+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df729c3f60 con 0x55df72ad55b0 2024-06-03T02:25:46.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.264+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df72a23ac0 con 0x55df72ae0300 2024-06-03T02:25:46.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.264+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55df72ae2f60 con 0x55df72a51e40 2024-06-03T02:25:46.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.264+0000 7f068eb31b80 10 client.0 _ll_register_callbacks cb 0x55df729c1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:46.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.264+0000 7f0680e00700 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72adcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.264+0000 7f0682200700 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72add310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.265+0000 7f0681800700 1 -- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 msgr2=0x55df72add310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.265+0000 7f0681800700 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72add310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.265+0000 7f0681800700 1 -- 172.21.15.79:0/1568290039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 msgr2=0x55df72adcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.265+0000 7f0681800700 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72adcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.265+0000 7f0681800700 1 -- 172.21.15.79:0/1568290039 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df729768d0 con 0x55df72a51e40 2024-06-03T02:25:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.266+0000 7f0682200700 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72add310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.266+0000 7f0680e00700 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72adcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.266+0000 7f0681800700 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72a51e40 0x55df72adfad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0674004c20 tx=0x7f067401c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f85ad57b4583b916 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:46.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.266+0000 7f0663400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:25:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.269+0000 7f0663400700 1 -- 172.21.15.79:0/1568290039 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674031680 con 0x55df72a51e40 2024-06-03T02:25:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.269+0000 7f0663400700 1 -- 172.21.15.79:0/1568290039 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0674031820 con 0x55df72a51e40 2024-06-03T02:25:46.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.270+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0674021070 con 0x55df72a51e40 2024-06-03T02:25:46.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.270+0000 7f0663400700 1 -- 172.21.15.79:0/1568290039 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0674031a20 con 0x55df72a51e40 2024-06-03T02:25:46.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.270+0000 7f068eb31b80 10 client.26044 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:46.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.270+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df72aed6c0 con 0x55df72a51e40 2024-06-03T02:25:46.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.271+0000 7f0663400700 1 -- 172.21.15.79:0/1568290039 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0674021070 con 0x55df72a51e40 2024-06-03T02:25:46.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.271+0000 7f0663400700 1 -- 172.21.15.79:0/1568290039 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f067401bae0 con 0x55df72a51e40 2024-06-03T02:25:46.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.271+0000 7f068eb31b80 10 client.26044 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:46.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.271+0000 7f068eb31b80 10 client.26044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:46.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.272+0000 7f068eb31b80 20 client.26044 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:46.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.272+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55df72aefbf0 0x55df72af1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.272+0000 7f068eb31b80 4 client.26044 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:25:46.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.272+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df72a35000 con 0x55df72aefbf0 2024-06-03T02:25:46.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.273+0000 7f0680e00700 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55df72aefbf0 0x55df72af1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.273+0000 7f0680e00700 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55df72aefbf0 0x55df72af1fe0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=304496bc0cf67b27 server_cookie=76f50652c5120f42 in_seq=0 out_seq=0 2024-06-03T02:25:46.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.273+0000 7f0663400700 10 client.26044 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:46.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.274+0000 7f0663400700 1 -- 172.21.15.79:0/1568290039 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55df72a35000 con 0x55df72aefbf0 2024-06-03T02:25:46.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.274+0000 7f0663400700 10 client.26044 handle_command_reply: tid=0 2024-06-03T02:25:46.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.274+0000 7f068eb31b80 1 client.26044 shutdown 2024-06-03T02:25:46.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.278+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72a51e40 msgr2=0x55df72adfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:46.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.278+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72a51e40 0x55df72adfad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0674004c20 tx=0x7f067401c7a0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.278+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 shutdown_connections 2024-06-03T02:25:46.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.278+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72add310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.278+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72a51e40 0x55df72adfad0 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0674004c20 tx=0x7f067401c7a0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.278+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55df72aefbf0 0x55df72af1fe0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.278+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1568290039 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72adcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.278+0000 7f0663400700 0 client.26044 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:46.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.278+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 >> 172.21.15.79:0/1568290039 conn(0x55df72a02170 msgr2=0x55df72ad0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:46.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.279+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 shutdown_connections 2024-06-03T02:25:46.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.279+0000 7f068eb31b80 1 -- 172.21.15.79:0/1568290039 wait complete. 2024-06-03T02:25:46.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.279+0000 7f068eb31b80 20 client.26044 trim_cache size 0 max 16384 2024-06-03T02:25:46.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.312+0000 7f068eb31b80 1 Processor -- start 2024-06-03T02:25:46.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.312+0000 7f068eb31b80 1 -- start start 2024-06-03T02:25:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72ad8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ad85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72adbe80 0x55df72adcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f068eb31b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1bd120 con 0x55df72ad55b0 2024-06-03T02:25:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f068eb31b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1bec60 con 0x55df72ae0300 2024-06-03T02:25:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f068eb31b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1bd7f0 con 0x55df72adbe80 2024-06-03T02:25:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f0680e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ad85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f0680e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ad85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38718/0 (socket says 172.21.15.79:38718) 2024-06-03T02:25:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f0680e00700 1 -- 172.21.15.79:0/984086332 learned_addr learned my addr 172.21.15.79:0/984086332 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.313+0000 7f0681800700 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72ad8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.314+0000 7f0682200700 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72adbe80 0x55df72adcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.314+0000 7f0680e00700 1 -- 172.21.15.79:0/984086332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72adbe80 msgr2=0x55df72adcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.314+0000 7f0680e00700 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72adbe80 0x55df72adcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.314+0000 7f0680e00700 1 -- 172.21.15.79:0/984086332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 msgr2=0x55df72ad8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.314+0000 7f0680e00700 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72ad8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.314+0000 7f0680e00700 1 -- 172.21.15.79:0/984086332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df72ac8e10 con 0x55df72ae0300 2024-06-03T02:25:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.315+0000 7f0682200700 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72adbe80 0x55df72adcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:25:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.315+0000 7f0681800700 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72ad8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.315+0000 7f0680e00700 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ad85b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55df72ad2920 tx=0x7f0668014750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf24c51db1b78d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:46.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.316+0000 7f066e000700 1 -- 172.21.15.79:0/984086332 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668023070 con 0x55df72ae0300 2024-06-03T02:25:46.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.316+0000 7f066e000700 1 -- 172.21.15.79:0/984086332 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0668012930 con 0x55df72ae0300 2024-06-03T02:25:46.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.316+0000 7f066e000700 1 -- 172.21.15.79:0/984086332 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066801d960 con 0x55df72ae0300 2024-06-03T02:25:46.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.317+0000 7f068eb31b80 1 -- 172.21.15.79:0/984086332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 msgr2=0x55df72ad85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:46.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.317+0000 7f068eb31b80 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ad85b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55df72ad2920 tx=0x7f0668014750 comp rx=0 tx=0).stop 2024-06-03T02:25:46.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.317+0000 7f068eb31b80 1 -- 172.21.15.79:0/984086332 shutdown_connections 2024-06-03T02:25:46.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.317+0000 7f068eb31b80 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72adbe80 0x55df72adcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.317+0000 7f068eb31b80 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72ad55b0 0x55df72ad8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.317+0000 7f068eb31b80 1 --2- 172.21.15.79:0/984086332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ad85b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.317+0000 7f068eb31b80 1 -- 172.21.15.79:0/984086332 >> 172.21.15.79:0/984086332 conn(0x55df72a02170 msgr2=0x55df72adc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:46.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.317+0000 7f068eb31b80 1 -- 172.21.15.79:0/984086332 shutdown_connections 2024-06-03T02:25:46.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.318+0000 7f068eb31b80 1 -- 172.21.15.79:0/984086332 wait complete. 2024-06-03T02:25:46.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.319+0000 7f068eb31b80 1 -- start start 2024-06-03T02:25:46.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.319+0000 7f068eb31b80 1 Processor -- start 2024-06-03T02:25:46.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.323+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ae8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.323+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72adbe80 0x55df72ad9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.323+0000 7f068eb31b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ada250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f068eb31b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1be590 con 0x55df72adbe80 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f0681800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ae8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f0681800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ae8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55202/0 (socket says 172.21.15.79:55202) 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f0681800700 1 -- 172.21.15.79:0/1319145891 learned_addr learned my addr 172.21.15.79:0/1319145891 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f0682200700 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ada250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f0680e00700 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72adbe80 0x55df72ad9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1bdec0 con 0x55df72ae0300 2024-06-03T02:25:46.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1bb0c0 con 0x55df72ad55b0 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f0682200700 1 -- 172.21.15.79:0/1319145891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 msgr2=0x55df72ae8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f0682200700 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ae8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f0682200700 1 -- 172.21.15.79:0/1319145891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72adbe80 msgr2=0x55df72ad9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.324+0000 7f0682200700 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72adbe80 0x55df72ad9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.325+0000 7f0682200700 1 -- 172.21.15.79:0/1319145891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df72adaa50 con 0x55df72ae0300 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.325+0000 7f0680e00700 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72adbe80 0x55df72ad9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.325+0000 7f0682200700 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ada250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0670005f50 tx=0x7f067009c590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262e7d7253a2436a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.325+0000 7f068eb31b80 10 client.0 _ll_register_callbacks cb 0x55df729c1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.325+0000 7f0663400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.325+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0668012930 con 0x55df72ae0300 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.325+0000 7f0663400700 1 -- 172.21.15.79:0/1319145891 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067009c6f0 con 0x55df72ae0300 2024-06-03T02:25:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.325+0000 7f0663400700 1 -- 172.21.15.79:0/1319145891 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0670022470 con 0x55df72ae0300 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.325+0000 7f0663400700 1 -- 172.21.15.79:0/1319145891 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06700a0aa0 con 0x55df72ae0300 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.326+0000 7f068eb31b80 10 client.26054 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.326+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55df72b5a560 con 0x55df72ae0300 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.326+0000 7f0663400700 1 -- 172.21.15.79:0/1319145891 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f067009c950 con 0x55df72ae0300 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.326+0000 7f068eb31b80 10 client.26054 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.326+0000 7f068eb31b80 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.326+0000 7f068eb31b80 20 client.26054 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.326+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55df72b5aa10 0x55df72b5ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.326+0000 7f068eb31b80 4 client.26054 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:25:46.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.326+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df72a2f580 con 0x55df72b5aa10 2024-06-03T02:25:46.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.327+0000 7f0681800700 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55df72b5aa10 0x55df72b5ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:46.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.327+0000 7f0663400700 10 client.26054 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:46.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.327+0000 7f0681800700 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55df72b5aa10 0x55df72b5ce00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42dd23607d43236 server_cookie=7bb2359cf2f49d82 in_seq=0 out_seq=0 2024-06-03T02:25:46.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.328+0000 7f0663400700 1 -- 172.21.15.79:0/1319145891 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df72a2f580 con 0x55df72b5aa10 2024-06-03T02:25:46.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.328+0000 7f0663400700 10 client.26054 handle_command_reply: tid=0 2024-06-03T02:25:46.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.328+0000 7f068eb31b80 1 client.26054 shutdown 2024-06-03T02:25:46.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.330+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 msgr2=0x55df72ada250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:46.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.330+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ada250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0670005f50 tx=0x7f067009c590 comp rx=0 tx=0).stop 2024-06-03T02:25:46.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.330+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 shutdown_connections 2024-06-03T02:25:46.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.330+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df72ae0300 0x55df72ada250 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.330+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df72ad55b0 0x55df72ae8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.330+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55df72b5aa10 0x55df72b5ce00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.330+0000 7f068eb31b80 1 --2- 172.21.15.79:0/1319145891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df72adbe80 0x55df72ad9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.330+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 >> 172.21.15.79:0/1319145891 conn(0x55df72a02170 msgr2=0x55df72adcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:46.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.330+0000 7f0663400700 0 client.26054 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:46.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.331+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 shutdown_connections 2024-06-03T02:25:46.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.331+0000 7f068eb31b80 1 -- 172.21.15.79:0/1319145891 wait complete. 2024-06-03T02:25:46.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.331+0000 7f068eb31b80 20 client.26054 trim_cache size 0 max 16384 2024-06-03T02:25:46.331 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:25:46.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06640a2ca0 msgr2=0x7f06640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:46.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06640a2ca0 0x7f06640a5160 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0674007eb0 tx=0x7f06740073c0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067c108500 msgr2=0x7f067c10d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:46.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067c108500 0x7f067c10d1e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f06700098e0 tx=0x7f067000bb60 comp rx=0 tx=0).stop 2024-06-03T02:25:46.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 shutdown_connections 2024-06-03T02:25:46.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f06640a2ca0 0x7f06640a5160 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f067c108500 0x7f067c10d1e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f067c06a170 0x7f067c19b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 --2- 172.21.15.79:0/337925069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f067c06ab30 0x7f067c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:46.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.332+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 >> 172.21.15.79:0/337925069 conn(0x7f067c078340 msgr2=0x7f067c101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:46.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.333+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 shutdown_connections 2024-06-03T02:25:46.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:46.333+0000 7f0684200700 1 -- 172.21.15.79:0/337925069 wait complete. 2024-06-03T02:25:48.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:47 smithi123 ceph-mon[36640]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 567 op/s 2024-06-03T02:25:48.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 340.0 s, 572.8 tps, lat 179.844 ms stddev 237.151 2024-06-03T02:25:48.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:47 smithi079 ceph-mon[31819]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 567 op/s 2024-06-03T02:25:48.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:47 smithi135 ceph-mon[36653]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 567 op/s 2024-06-03T02:25:49.812 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:25:49.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:25:50.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:49 smithi123 ceph-mon[36640]: pgmap v689: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 562 op/s 2024-06-03T02:25:50.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:49 smithi079 ceph-mon[31819]: pgmap v689: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 562 op/s 2024-06-03T02:25:50.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:49 smithi135 ceph-mon[36653]: pgmap v689: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 562 op/s 2024-06-03T02:25:50.531 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:52.140 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:51 smithi079 ceph-mon[31819]: pgmap v690: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 535 op/s 2024-06-03T02:25:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:51 smithi123 ceph-mon[36640]: pgmap v690: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 535 op/s 2024-06-03T02:25:52.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:51 smithi135 ceph-mon[36653]: pgmap v690: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 535 op/s 2024-06-03T02:25:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 345.0 s, 751.6 tps, lat 131.767 ms stddev 154.859 2024-06-03T02:25:53.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.644+0000 7f8575a00700 1 -- 172.21.15.79:0/3852191901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8570108520 msgr2=0x7f8570108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:53.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.644+0000 7f8575a00700 1 --2- 172.21.15.79:0/3852191901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8570108520 0x7f8570108900 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8560009990 tx=0x7f8560009ca0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.645+0000 7f8575a00700 1 -- 172.21.15.79:0/3852191901 shutdown_connections 2024-06-03T02:25:53.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.645+0000 7f8575a00700 1 --2- 172.21.15.79:0/3852191901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8570102ed0 0x7f857010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.645+0000 7f8575a00700 1 --2- 172.21.15.79:0/3852191901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8570108520 0x7f8570108900 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.645+0000 7f8575a00700 1 --2- 172.21.15.79:0/3852191901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570102510 0x7f8570102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.645+0000 7f8575a00700 1 -- 172.21.15.79:0/3852191901 >> 172.21.15.79:0/3852191901 conn(0x7f8570079560 msgr2=0x7f8570079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:53.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.646+0000 7f8575a00700 1 -- 172.21.15.79:0/3852191901 shutdown_connections 2024-06-03T02:25:53.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.646+0000 7f8575a00700 1 -- 172.21.15.79:0/3852191901 wait complete. 2024-06-03T02:25:53.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.649+0000 7f8575a00700 1 Processor -- start 2024-06-03T02:25:53.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.649+0000 7f8575a00700 1 -- start start 2024-06-03T02:25:53.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.649+0000 7f8575a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570102510 0x7f857010ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f8575a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8570102ed0 0x7f857010c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f8575a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8570108520 0x7f857010c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f8575a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8570075f00 con 0x7f8570108520 2024-06-03T02:25:53.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f8575a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8570075d80 con 0x7f8570102ed0 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f8575a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8570076080 con 0x7f8570102510 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f856ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570102510 0x7f857010ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f856ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570102510 0x7f857010ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55230/0 (socket says 172.21.15.79:55230) 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f856ea00700 1 -- 172.21.15.79:0/526963664 learned_addr learned my addr 172.21.15.79:0/526963664 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f856ea00700 1 -- 172.21.15.79:0/526963664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8570102ed0 msgr2=0x7f857010c030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f856ea00700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8570102ed0 0x7f857010c030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f856ea00700 1 -- 172.21.15.79:0/526963664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8570108520 msgr2=0x7f857010c570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f856ea00700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8570108520 0x7f857010c570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.650+0000 7f856ea00700 1 -- 172.21.15.79:0/526963664 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857010cc70 con 0x7f8570102510 2024-06-03T02:25:53.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.651+0000 7f856ea00700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570102510 0x7f857010ef80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f85600098e0 tx=0x7f8560005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:53.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.652+0000 7f8567400700 1 -- 172.21.15.79:0/526963664 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560026070 con 0x7f8570102510 2024-06-03T02:25:53.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.652+0000 7f8567400700 1 -- 172.21.15.79:0/526963664 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8560004050 con 0x7f8570102510 2024-06-03T02:25:53.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.652+0000 7f8575a00700 1 -- 172.21.15.79:0/526963664 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85701aba80 con 0x7f8570102510 2024-06-03T02:25:53.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.652+0000 7f8567400700 1 -- 172.21.15.79:0/526963664 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856000fe20 con 0x7f8570102510 2024-06-03T02:25:53.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.654+0000 7f8575a00700 1 -- 172.21.15.79:0/526963664 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701abe20 con 0x7f8570102510 2024-06-03T02:25:53.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.658+0000 7f8567400700 1 -- 172.21.15.79:0/526963664 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8560003b70 con 0x7f8570102510 2024-06-03T02:25:53.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.662+0000 7f8567400700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f85540a2da0 0x7f85540a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.662+0000 7f8567400700 1 -- 172.21.15.79:0/526963664 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f85600d0170 con 0x7f8570102510 2024-06-03T02:25:53.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.664+0000 7f856e000700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f85540a2da0 0x7f85540a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.664+0000 7f8580477b80 1 Processor -- start 2024-06-03T02:25:53.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.665+0000 7f8580477b80 1 -- start start 2024-06-03T02:25:53.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.665+0000 7f856e000700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f85540a2da0 0x7f85540a5260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f857010d7f0 tx=0x7f855c006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:53.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f8580477b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad800eec0 0x560ad80920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f8580477b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8092ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f856ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad800eec0 0x560ad80920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f8580477b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad809f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f8580477b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560ad7fa1a90 con 0x560ad8092630 2024-06-03T02:25:53.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f856ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad800eec0 0x560ad80920f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55244/0 (socket says 172.21.15.79:55244) 2024-06-03T02:25:53.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f856ea00700 1 -- 172.21.15.79:0/978639643 learned_addr learned my addr 172.21.15.79:0/978639643 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:53.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f8580477b80 1 -- 172.21.15.79:0/978639643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560ad7fe4b80 con 0x560ad809d380 2024-06-03T02:25:53.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f8580477b80 1 -- 172.21.15.79:0/978639643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560ad7f9f970 con 0x560ad800eec0 2024-06-03T02:25:53.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.666+0000 7f856e000700 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8092ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.667+0000 7f856f400700 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad809f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.668+0000 7f856e000700 1 -- 172.21.15.79:0/978639643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad800eec0 msgr2=0x560ad80920f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:53.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.668+0000 7f856e000700 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad800eec0 0x560ad80920f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.668+0000 7f856e000700 1 -- 172.21.15.79:0/978639643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 msgr2=0x560ad809f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:53.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.668+0000 7f856e000700 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad809f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.668+0000 7f856e000700 1 -- 172.21.15.79:0/978639643 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad7f8d010 con 0x560ad8092630 2024-06-03T02:25:53.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.668+0000 7f856f400700 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad809f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:25:53.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.669+0000 7f856e000700 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8092ab0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f855c015820 tx=0x7f855c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f29dc84ae146b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:53.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.672+0000 7f855be00700 1 -- 172.21.15.79:0/978639643 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c023070 con 0x560ad8092630 2024-06-03T02:25:53.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.672+0000 7f855be00700 1 -- 172.21.15.79:0/978639643 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855c013900 con 0x560ad8092630 2024-06-03T02:25:53.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.672+0000 7f855be00700 1 -- 172.21.15.79:0/978639643 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c01e8a0 con 0x560ad8092630 2024-06-03T02:25:53.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.673+0000 7f8580477b80 1 -- 172.21.15.79:0/978639643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 msgr2=0x560ad8092ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.673+0000 7f8580477b80 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8092ab0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f855c015820 tx=0x7f855c00e000 comp rx=0 tx=0).stop 2024-06-03T02:25:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.673+0000 7f8580477b80 1 -- 172.21.15.79:0/978639643 shutdown_connections 2024-06-03T02:25:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.673+0000 7f8580477b80 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad809f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.673+0000 7f8580477b80 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad800eec0 0x560ad80920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.673+0000 7f8580477b80 1 --2- 172.21.15.79:0/978639643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8092ab0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.673+0000 7f8580477b80 1 -- 172.21.15.79:0/978639643 >> 172.21.15.79:0/978639643 conn(0x560ad7fbf1f0 msgr2=0x560ad808e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.674+0000 7f8580477b80 1 -- 172.21.15.79:0/978639643 shutdown_connections 2024-06-03T02:25:53.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.674+0000 7f8580477b80 1 -- 172.21.15.79:0/978639643 wait complete. 2024-06-03T02:25:53.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.675+0000 7f8580477b80 1 -- start start 2024-06-03T02:25:53.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.675+0000 7f8580477b80 1 Processor -- start 2024-06-03T02:25:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.677+0000 7f8580477b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 0x560ad809cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.677+0000 7f8580477b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.677+0000 7f8580477b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 0x560ad809a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.677+0000 7f8580477b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560ad7f80fe0 con 0x560ad8092630 2024-06-03T02:25:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.677+0000 7f8580477b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560ad7fe0b40 con 0x560ad800eec0 2024-06-03T02:25:53.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.677+0000 7f8580477b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560ad80a0040 con 0x560ad809d380 2024-06-03T02:25:53.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.677+0000 7f8580477b80 10 client.0 _ll_register_callbacks cb 0x560ad7f7e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:53.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.677+0000 7f856e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.678+0000 7f856e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46762/0 (socket says 172.21.15.79:46762) 2024-06-03T02:25:53.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.678+0000 7f856e000700 1 -- 172.21.15.79:0/2685721767 learned_addr learned my addr 172.21.15.79:0/2685721767 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.678+0000 7f856f400700 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 0x560ad809a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.678+0000 7f856e000700 1 -- 172.21.15.79:0/2685721767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 msgr2=0x560ad809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.679+0000 7f856e000700 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 0x560ad809a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.679+0000 7f856e000700 1 -- 172.21.15.79:0/2685721767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 msgr2=0x560ad809cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.679+0000 7f856e000700 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 0x560ad809cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.679+0000 7f856e000700 1 -- 172.21.15.79:0/2685721767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad7f33950 con 0x560ad8092630 2024-06-03T02:25:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.679+0000 7f856f400700 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 0x560ad809a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:25:53.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.679+0000 7f856e000700 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099eb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f855c000c00 tx=0x7f855c013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe153c8e502650b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:53.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.679+0000 7f8558c00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:25:53.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.680+0000 7f8558c00700 1 -- 172.21.15.79:0/2685721767 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c02c020 con 0x560ad8092630 2024-06-03T02:25:53.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.680+0000 7f8558c00700 1 -- 172.21.15.79:0/2685721767 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f855c01e730 con 0x560ad8092630 2024-06-03T02:25:53.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.680+0000 7f8580477b80 1 -- 172.21.15.79:0/2685721767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855c013900 con 0x560ad8092630 2024-06-03T02:25:53.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.681+0000 7f8558c00700 1 -- 172.21.15.79:0/2685721767 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f855c023050 con 0x560ad8092630 2024-06-03T02:25:53.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.681+0000 7f8580477b80 10 client.15735 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:53.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.681+0000 7f8580477b80 1 -- 172.21.15.79:0/2685721767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ad80bb840 con 0x560ad8092630 2024-06-03T02:25:53.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.682+0000 7f8558c00700 1 -- 172.21.15.79:0/2685721767 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855c013900 con 0x560ad8092630 2024-06-03T02:25:53.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.682+0000 7f8558c00700 1 -- 172.21.15.79:0/2685721767 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f855c028aa0 con 0x560ad8092630 2024-06-03T02:25:53.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.683+0000 7f8580477b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:53.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.683+0000 7f8580477b80 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:53.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.683+0000 7f8580477b80 20 client.15735 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:53.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.683+0000 7f8580477b80 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560ad80bdd70 0x560ad80c0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.683+0000 7f8580477b80 4 client.15735 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:25:53.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.683+0000 7f8580477b80 1 -- 172.21.15.79:0/2685721767 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ad7ff2080 con 0x560ad80bdd70 2024-06-03T02:25:53.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.684+0000 7f856ea00700 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560ad80bdd70 0x560ad80c0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.685+0000 7f856ea00700 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560ad80bdd70 0x560ad80c0160 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ada20f14225273 server_cookie=d8924636f439df89 in_seq=0 out_seq=0 2024-06-03T02:25:53.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.685+0000 7f8558c00700 10 client.15735 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:53.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.686+0000 7f8558c00700 1 -- 172.21.15.79:0/2685721767 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560ad7ff2080 con 0x560ad80bdd70 2024-06-03T02:25:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.686+0000 7f8558c00700 10 client.15735 handle_command_reply: tid=0 2024-06-03T02:25:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.686+0000 7f8580477b80 1 client.15735 shutdown 2024-06-03T02:25:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.687+0000 7f8580477b80 1 -- 172.21.15.79:0/2685721767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 msgr2=0x560ad8099eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.687+0000 7f8580477b80 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099eb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f855c000c00 tx=0x7f855c013ce0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.687+0000 7f8580477b80 1 -- 172.21.15.79:0/2685721767 shutdown_connections 2024-06-03T02:25:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.687+0000 7f8580477b80 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 0x560ad809a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.687+0000 7f8580477b80 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 0x560ad809cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.687+0000 7f8580477b80 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560ad80bdd70 0x560ad80c0160 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.687+0000 7f8580477b80 1 --2- 172.21.15.79:0/2685721767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099eb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.687+0000 7f8558c00700 0 client.15735 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.687+0000 7f8580477b80 1 -- 172.21.15.79:0/2685721767 >> 172.21.15.79:0/2685721767 conn(0x560ad7fbf1f0 msgr2=0x560ad808d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:53.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.689+0000 7f8580477b80 1 -- 172.21.15.79:0/2685721767 shutdown_connections 2024-06-03T02:25:53.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.691+0000 7f8580477b80 1 -- 172.21.15.79:0/2685721767 wait complete. 2024-06-03T02:25:53.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.691+0000 7f8580477b80 20 client.15735 trim_cache size 0 max 16384 2024-06-03T02:25:53.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.730+0000 7f8580477b80 1 Processor -- start 2024-06-03T02:25:53.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.730+0000 7f8580477b80 1 -- start start 2024-06-03T02:25:53.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.731+0000 7f8580477b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 0x560ad808e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.731+0000 7f8580477b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.732+0000 7f8580477b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 0x560ad808d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.732+0000 7f856ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 0x560ad808e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.732+0000 7f856ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 0x560ad808e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38782/0 (socket says 172.21.15.79:38782) 2024-06-03T02:25:53.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.732+0000 7f856ea00700 1 -- 172.21.15.79:0/3873154919 learned_addr learned my addr 172.21.15.79:0/3873154919 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:53.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.732+0000 7f8580477b80 1 -- 172.21.15.79:0/3873154919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f85701ac220 con 0x560ad8092630 2024-06-03T02:25:53.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.732+0000 7f8580477b80 1 -- 172.21.15.79:0/3873154919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85701add60 con 0x560ad800eec0 2024-06-03T02:25:53.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.732+0000 7f8580477b80 1 -- 172.21.15.79:0/3873154919 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85701ac8f0 con 0x560ad809d380 2024-06-03T02:25:53.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.732+0000 7f856e000700 1 --2- 172.21.15.79:0/3873154919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.732+0000 7f856ea00700 1 -- 172.21.15.79:0/3873154919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 msgr2=0x560ad808d640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:25:53.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.733+0000 7f856ea00700 1 --2- 172.21.15.79:0/3873154919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 0x560ad808d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.733+0000 7f856ea00700 1 -- 172.21.15.79:0/3873154919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 msgr2=0x560ad8099030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:53.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.733+0000 7f856ea00700 1 --2- 172.21.15.79:0/3873154919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.733+0000 7f856ea00700 1 -- 172.21.15.79:0/3873154919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad8099c80 con 0x560ad800eec0 2024-06-03T02:25:53.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.733+0000 7f856e000700 1 --2- 172.21.15.79:0/3873154919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:53.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.733+0000 7f856ea00700 1 --2- 172.21.15.79:0/3873154919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 0x560ad808e6a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f85600cdc30 tx=0x7f85600cd790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43ba2ad057ba1c31 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:53.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.733+0000 7f855b400700 1 -- 172.21.15.79:0/3873154919 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856009c010 con 0x560ad800eec0 2024-06-03T02:25:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.734+0000 7f855b400700 1 -- 172.21.15.79:0/3873154919 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856009cdb0 con 0x560ad800eec0 2024-06-03T02:25:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.734+0000 7f855b400700 1 -- 172.21.15.79:0/3873154919 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856009a7e0 con 0x560ad800eec0 2024-06-03T02:25:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.735+0000 7f8580477b80 1 -- 172.21.15.79:0/3873154919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 msgr2=0x560ad808e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.735+0000 7f8580477b80 1 --2- 172.21.15.79:0/3873154919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 0x560ad808e6a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f85600cdc30 tx=0x7f85600cd790 comp rx=0 tx=0).stop 2024-06-03T02:25:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.736+0000 7f8580477b80 1 -- 172.21.15.79:0/3873154919 shutdown_connections 2024-06-03T02:25:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.736+0000 7f8580477b80 1 --2- 172.21.15.79:0/3873154919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad809d380 0x560ad808d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.736+0000 7f8580477b80 1 --2- 172.21.15.79:0/3873154919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad800eec0 0x560ad808e6a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.736+0000 7f8580477b80 1 --2- 172.21.15.79:0/3873154919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad8092630 0x560ad8099030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.736+0000 7f8580477b80 1 -- 172.21.15.79:0/3873154919 >> 172.21.15.79:0/3873154919 conn(0x560ad7fbf1f0 msgr2=0x560ad808cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.736+0000 7f8580477b80 1 -- 172.21.15.79:0/3873154919 shutdown_connections 2024-06-03T02:25:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.739+0000 7f8580477b80 1 -- 172.21.15.79:0/3873154919 wait complete. 2024-06-03T02:25:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.739+0000 7f8580477b80 1 -- start start 2024-06-03T02:25:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.740+0000 7f8580477b80 1 Processor -- start 2024-06-03T02:25:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.741+0000 7f8580477b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad800eec0 0x560ad80c3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.741+0000 7f8580477b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad8092630 0x560ad80c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.742+0000 7f8580477b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad80c0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.742+0000 7f8580477b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f85701ad690 con 0x560ad800eec0 2024-06-03T02:25:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.742+0000 7f8580477b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85701acfc0 con 0x560ad809d380 2024-06-03T02:25:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.742+0000 7f8580477b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8570072880 con 0x560ad8092630 2024-06-03T02:25:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.743+0000 7f856e000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad8092630 0x560ad80c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.743+0000 7f856ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad800eec0 0x560ad80c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.743+0000 7f856ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad800eec0 0x560ad80c3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46770/0 (socket says 172.21.15.79:46770) 2024-06-03T02:25:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.743+0000 7f856ea00700 1 -- 172.21.15.79:0/6112061 learned_addr learned my addr 172.21.15.79:0/6112061 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.744+0000 7f856ea00700 1 -- 172.21.15.79:0/6112061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad8092630 msgr2=0x560ad80c38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.743+0000 7f856e000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad8092630 0x560ad80c38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55282/0 (socket says 172.21.15.79:55282) 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.746+0000 7f856ea00700 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad8092630 0x560ad80c38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.746+0000 7f856ea00700 1 -- 172.21.15.79:0/6112061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 msgr2=0x560ad80c0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.746+0000 7f8580477b80 10 client.0 _ll_register_callbacks cb 0x560ad7f7e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.746+0000 7f856f400700 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad80c0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.746+0000 7f856ea00700 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad80c0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.746+0000 7f856ea00700 1 -- 172.21.15.79:0/6112061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad80c0880 con 0x560ad800eec0 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.746+0000 7f856e000700 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad8092630 0x560ad80c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.747+0000 7f8580477b80 1 -- 172.21.15.79:0/6112061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f856009cdb0 con 0x560ad800eec0 2024-06-03T02:25:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.747+0000 7f856f400700 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad80c0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.747+0000 7f856ea00700 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad800eec0 0x560ad80c3370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f85600cd910 tx=0x7f8560009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a8a1e7759a0874 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.747+0000 7f8558c00700 10 client.15738 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:25:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.748+0000 7f8558c00700 1 -- 172.21.15.79:0/6112061 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600b9070 con 0x560ad800eec0 2024-06-03T02:25:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.748+0000 7f8558c00700 1 -- 172.21.15.79:0/6112061 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f85600a4e00 con 0x560ad800eec0 2024-06-03T02:25:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.748+0000 7f8558c00700 1 -- 172.21.15.79:0/6112061 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85600a5000 con 0x560ad800eec0 2024-06-03T02:25:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.750+0000 7f8580477b80 10 client.15738 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.750+0000 7f8580477b80 1 -- 172.21.15.79:0/6112061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({fsmap=30}) -- 0x560ad8128f70 con 0x560ad800eec0 2024-06-03T02:25:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.751+0000 7f8558c00700 1 -- 172.21.15.79:0/6112061 <== mon.0 v2:172.21.15.79:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f85600b0b40 con 0x560ad800eec0 2024-06-03T02:25:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.751+0000 7f8580477b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.751+0000 7f8580477b80 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.751+0000 7f8580477b80 20 client.15738 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.751+0000 7f8580477b80 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560ad8129160 0x560ad812b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.751+0000 7f8580477b80 4 client.15738 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:25:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.751+0000 7f8580477b80 1 -- 172.21.15.79:0/6112061 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560ad7fec600 con 0x560ad8129160 2024-06-03T02:25:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.751+0000 7f856e000700 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560ad8129160 0x560ad812b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:53.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.752+0000 7f856e000700 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560ad8129160 0x560ad812b580 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b55b0373ff3c16c server_cookie=73b6531abf3c95be in_seq=0 out_seq=0 2024-06-03T02:25:53.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.752+0000 7f8558c00700 10 client.15738 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:53.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.756+0000 7f8558c00700 1 -- 172.21.15.79:0/6112061 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ad7fec600 con 0x560ad8129160 2024-06-03T02:25:53.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.756+0000 7f8558c00700 10 client.15738 handle_command_reply: tid=0 2024-06-03T02:25:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.756+0000 7f8580477b80 1 client.15738 shutdown 2024-06-03T02:25:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.756+0000 7f8580477b80 1 -- 172.21.15.79:0/6112061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad800eec0 msgr2=0x560ad80c3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.756+0000 7f8580477b80 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad800eec0 0x560ad80c3370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f85600cd910 tx=0x7f8560009710 comp rx=0 tx=0).stop 2024-06-03T02:25:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.757+0000 7f8580477b80 1 -- 172.21.15.79:0/6112061 shutdown_connections 2024-06-03T02:25:53.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.757+0000 7f8580477b80 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ad809d380 0x560ad80c0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.758+0000 7f8580477b80 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560ad8129160 0x560ad812b580 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.758+0000 7f8580477b80 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ad800eec0 0x560ad80c3370 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.758+0000 7f8580477b80 1 --2- 172.21.15.79:0/6112061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad8092630 0x560ad80c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.758+0000 7f8558c00700 0 client.15738 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.758+0000 7f8580477b80 1 -- 172.21.15.79:0/6112061 >> 172.21.15.79:0/6112061 conn(0x560ad7fbf1f0 msgr2=0x560ad80a82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.758+0000 7f8580477b80 1 -- 172.21.15.79:0/6112061 shutdown_connections 2024-06-03T02:25:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.758+0000 7f8580477b80 1 -- 172.21.15.79:0/6112061 wait complete. 2024-06-03T02:25:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.758+0000 7f8580477b80 20 client.15738 trim_cache size 0 max 16384 2024-06-03T02:25:53.760 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:25:53.760 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:25:53.760 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "30d8bf3e-e3f2-439d-ae88-c0b323a691cd", 2024-06-03T02:25:53.760 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:25:53.760 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:25:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.759+0000 7f8575a00700 1 -- 172.21.15.79:0/526963664 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f85540a2da0 msgr2=0x7f85540a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.759+0000 7f8575a00700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f85540a2da0 0x7f85540a5260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f857010d7f0 tx=0x7f855c006cb0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.759+0000 7f8575a00700 1 -- 172.21.15.79:0/526963664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570102510 msgr2=0x7f857010ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.759+0000 7f8575a00700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570102510 0x7f857010ef80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f85600098e0 tx=0x7f8560005640 comp rx=0 tx=0).stop 2024-06-03T02:25:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.760+0000 7f8575a00700 1 -- 172.21.15.79:0/526963664 shutdown_connections 2024-06-03T02:25:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.760+0000 7f8575a00700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f85540a2da0 0x7f85540a5260 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.760+0000 7f8575a00700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8570108520 0x7f857010c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.760+0000 7f8575a00700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570102510 0x7f857010ef80 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f85600098e0 tx=0x7f8560005640 comp rx=0 tx=0).stop 2024-06-03T02:25:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.760+0000 7f8575a00700 1 --2- 172.21.15.79:0/526963664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8570102ed0 0x7f857010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.760+0000 7f8575a00700 1 -- 172.21.15.79:0/526963664 >> 172.21.15.79:0/526963664 conn(0x7f8570079560 msgr2=0x7f8570100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.760+0000 7f8575a00700 1 -- 172.21.15.79:0/526963664 shutdown_connections 2024-06-03T02:25:53.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:53.760+0000 7f8575a00700 1 -- 172.21.15.79:0/526963664 wait complete. 2024-06-03T02:25:54.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:53 smithi079 ceph-mon[31819]: pgmap v691: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 569 op/s 2024-06-03T02:25:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:53 smithi123 ceph-mon[36640]: pgmap v691: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 569 op/s 2024-06-03T02:25:54.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:53 smithi135 ceph-mon[36653]: pgmap v691: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 569 op/s 2024-06-03T02:25:54.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:25:55.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:54 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:25:55.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:54 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:55.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:54 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:25:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:54 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:25:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:54 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:55.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:54 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:25:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:54 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:25:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:54 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:55.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:54 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:25:55.482 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:25:55.971 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:55 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:55.971 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:55 smithi079 ceph-mon[31819]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 541 op/s 2024-06-03T02:25:56.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:55 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:56.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:55 smithi123 ceph-mon[36640]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 541 op/s 2024-06-03T02:25:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:55 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:25:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:55 smithi135 ceph-mon[36653]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 541 op/s 2024-06-03T02:25:58.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:57 smithi079 ceph-mon[31819]: pgmap v693: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 535 op/s 2024-06-03T02:25:58.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:57 smithi123 ceph-mon[36640]: pgmap v693: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 535 op/s 2024-06-03T02:25:58.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 350.0 s, 649.4 tps, lat 154.084 ms stddev 178.905 2024-06-03T02:25:58.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:57 smithi135 ceph-mon[36653]: pgmap v693: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 535 op/s 2024-06-03T02:25:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.645+0000 7f58f3200700 1 -- 172.21.15.79:0/1904532330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec101d50 msgr2=0x7f58ec06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.645+0000 7f58f3200700 1 --2- 172.21.15.79:0/1904532330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec101d50 0x7f58ec06a170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f58e400b0a0 tx=0x7f58e400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.646+0000 7f58f3200700 1 -- 172.21.15.79:0/1904532330 shutdown_connections 2024-06-03T02:25:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.646+0000 7f58f3200700 1 --2- 172.21.15.79:0/1904532330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec101d50 0x7f58ec06a170 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.646+0000 7f58f3200700 1 --2- 172.21.15.79:0/1904532330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58ec107400 0x7f58ec1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.646+0000 7f58f3200700 1 --2- 172.21.15.79:0/1904532330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58ec101390 0x7f58ec101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.646+0000 7f58f3200700 1 -- 172.21.15.79:0/1904532330 >> 172.21.15.79:0/1904532330 conn(0x7f58ec078340 msgr2=0x7f58ec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.646+0000 7f58f3200700 1 -- 172.21.15.79:0/1904532330 shutdown_connections 2024-06-03T02:25:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.646+0000 7f58f3200700 1 -- 172.21.15.79:0/1904532330 wait complete. 2024-06-03T02:25:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.648+0000 7f58f3200700 1 Processor -- start 2024-06-03T02:25:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.648+0000 7f58f3200700 1 -- start start 2024-06-03T02:25:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.649+0000 7f58f3200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58ec101390 0x7f58ec10d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.649+0000 7f58f3200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58ec101d50 0x7f58ec10dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.649+0000 7f58f3200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec107400 0x7f58ec071f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.650+0000 7f58ebe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58ec101390 0x7f58ec10d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.650+0000 7f58eb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58ec101d50 0x7f58ec10dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.650+0000 7f58f1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec107400 0x7f58ec071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.650+0000 7f58eb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58ec101d50 0x7f58ec10dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:50388/0 (socket says 172.21.15.79:50388) 2024-06-03T02:25:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.650+0000 7f58f1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec107400 0x7f58ec071f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53656/0 (socket says 172.21.15.79:53656) 2024-06-03T02:25:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.650+0000 7f58f1200700 1 -- 172.21.15.79:0/2758344242 learned_addr learned my addr 172.21.15.79:0/2758344242 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.650+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec114a80 con 0x7f58ec101d50 2024-06-03T02:25:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.650+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec114900 con 0x7f58ec101390 2024-06-03T02:25:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.650+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec114c00 con 0x7f58ec107400 2024-06-03T02:25:58.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.651+0000 7f58f1200700 1 -- 172.21.15.79:0/2758344242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58ec101390 msgr2=0x7f58ec10d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:58.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.651+0000 7f58f1200700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58ec101390 0x7f58ec10d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.651+0000 7f58f1200700 1 -- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58ec101d50 msgr2=0x7f58ec10dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:58.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.651+0000 7f58f1200700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58ec101d50 0x7f58ec10dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.651+0000 7f58f1200700 1 -- 172.21.15.79:0/2758344242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec072830 con 0x7f58ec107400 2024-06-03T02:25:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.651+0000 7f58ebe00700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58ec101390 0x7f58ec10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.651+0000 7f58eb400700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58ec101d50 0x7f58ec10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:25:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.652+0000 7f58f1200700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec107400 0x7f58ec071f80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f58e400d010 tx=0x7f58e400bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.652+0000 7f58e8c00700 1 -- 172.21.15.79:0/2758344242 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4021070 con 0x7f58ec107400 2024-06-03T02:25:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.652+0000 7f58e8c00700 1 -- 172.21.15.79:0/2758344242 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e4007c80 con 0x7f58ec107400 2024-06-03T02:25:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.652+0000 7f58e8c00700 1 -- 172.21.15.79:0/2758344242 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e401d450 con 0x7f58ec107400 2024-06-03T02:25:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.652+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ec072ac0 con 0x7f58ec107400 2024-06-03T02:25:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.653+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ec1136f0 con 0x7f58ec107400 2024-06-03T02:25:58.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.660+0000 7f58fdb38b80 1 Processor -- start 2024-06-03T02:25:58.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.661+0000 7f58e8c00700 1 -- 172.21.15.79:0/2758344242 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f58e40040b0 con 0x7f58ec107400 2024-06-03T02:25:58.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.661+0000 7f58fdb38b80 1 -- start start 2024-06-03T02:25:58.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.662+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014685ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.662+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146859d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.662+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685a240 0x56014685a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.662+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560146768a10 con 0x56014685a240 2024-06-03T02:25:58.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.662+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5601467abb00 con 0x56014685fb90 2024-06-03T02:25:58.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.662+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601467668f0 con 0x5601468604b0 2024-06-03T02:25:58.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.662+0000 7f58ebe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146859d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.662+0000 7f58ebe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146859d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53668/0 (socket says 172.21.15.79:53668) 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.662+0000 7f58ebe00700 1 -- 172.21.15.79:0/4122976137 learned_addr learned my addr 172.21.15.79:0/4122976137 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.663+0000 7f58eb400700 1 --2- 172.21.15.79:0/4122976137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014685ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.663+0000 7f58ebe00700 1 -- 172.21.15.79:0/4122976137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 msgr2=0x56014685ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.663+0000 7f58ebe00700 1 --2- 172.21.15.79:0/4122976137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014685ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.663+0000 7f58ebe00700 1 -- 172.21.15.79:0/4122976137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685a240 msgr2=0x56014685a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.663+0000 7f58ebe00700 1 --2- 172.21.15.79:0/4122976137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685a240 0x56014685a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.663+0000 7f58ebe00700 1 -- 172.21.15.79:0/4122976137 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560146753f90 con 0x5601468604b0 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.663+0000 7f58e8c00700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f58d00a2d70 0x7f58d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.664+0000 7f58ebe00700 1 --2- 172.21.15.79:0/4122976137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146859d00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00aa90 tx=0x7f58dc00ada0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aceb9fb38ed7c2f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:58.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.664+0000 7f58e8c00700 1 -- 172.21.15.79:0/2758344242 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f58e400f030 con 0x7f58ec107400 2024-06-03T02:25:58.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.665+0000 7f58e1600700 1 -- 172.21.15.79:0/4122976137 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00ca90 con 0x5601468604b0 2024-06-03T02:25:58.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.665+0000 7f58e1600700 1 -- 172.21.15.79:0/4122976137 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58dc00cc30 con 0x5601468604b0 2024-06-03T02:25:58.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.665+0000 7f58e1600700 1 -- 172.21.15.79:0/4122976137 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0106d0 con 0x5601468604b0 2024-06-03T02:25:58.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.665+0000 7f58eb400700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f58d00a2d70 0x7f58d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58fdb38b80 1 -- 172.21.15.79:0/4122976137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 msgr2=0x560146859d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:58.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/4122976137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146859d00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00aa90 tx=0x7f58dc00ada0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58eb400700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f58d00a2d70 0x7f58d00a5230 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58d800b010 tx=0x7f58d8012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:58.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58fdb38b80 1 -- 172.21.15.79:0/4122976137 shutdown_connections 2024-06-03T02:25:58.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/4122976137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685a240 0x56014685a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/4122976137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014685ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/4122976137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146859d00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58fdb38b80 1 -- 172.21.15.79:0/4122976137 >> 172.21.15.79:0/4122976137 conn(0x560146786170 msgr2=0x560146854ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:58.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58fdb38b80 1 -- 172.21.15.79:0/4122976137 shutdown_connections 2024-06-03T02:25:58.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.667+0000 7f58fdb38b80 1 -- 172.21.15.79:0/4122976137 wait complete. 2024-06-03T02:25:58.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.668+0000 7f58fdb38b80 1 -- start start 2024-06-03T02:25:58.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.668+0000 7f58fdb38b80 1 Processor -- start 2024-06-03T02:25:58.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.669+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685a240 0x560146866200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.669+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685fb90 0x560146863500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.669+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146863a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.669+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560146747f60 con 0x56014685fb90 2024-06-03T02:25:58.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.669+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5601467a7ac0 con 0x56014685a240 2024-06-03T02:25:58.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.669+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560146866ee0 con 0x5601468604b0 2024-06-03T02:25:58.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.669+0000 7f58fdb38b80 10 client.0 _ll_register_callbacks cb 0x560146741380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:58.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.669+0000 7f58ebe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685a240 0x560146866200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.670+0000 7f58f1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146863a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.670+0000 7f58eb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685fb90 0x560146863500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.670+0000 7f58f1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146863a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53674/0 (socket says 172.21.15.79:53674) 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.670+0000 7f58f1200700 1 -- 172.21.15.79:0/2959011635 learned_addr learned my addr 172.21.15.79:0/2959011635 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.670+0000 7f58eb400700 1 -- 172.21.15.79:0/2959011635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 msgr2=0x560146863a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.670+0000 7f58eb400700 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146863a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.670+0000 7f58eb400700 1 -- 172.21.15.79:0/2959011635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685a240 msgr2=0x560146866200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.670+0000 7f58eb400700 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685a240 0x560146866200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.670+0000 7f58eb400700 1 -- 172.21.15.79:0/2959011635 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601466fa8d0 con 0x56014685fb90 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.671+0000 7f58eb400700 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685fb90 0x560146863500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f58ec073780 tx=0x7f58d800a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acd27729a7f5884 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.672+0000 7f58d6000700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.672+0000 7f58d6000700 1 -- 172.21.15.79:0/2959011635 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d801ce10 con 0x56014685fb90 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.672+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2959011635 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58dc00cc30 con 0x56014685fb90 2024-06-03T02:25:58.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.672+0000 7f58d6000700 1 -- 172.21.15.79:0/2959011635 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f58d8017070 con 0x56014685fb90 2024-06-03T02:25:58.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.673+0000 7f58d6000700 1 -- 172.21.15.79:0/2959011635 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58d801ce10 con 0x56014685fb90 2024-06-03T02:25:58.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.673+0000 7f58fdb38b80 10 client.15744 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.673+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2959011635 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601468716f0 con 0x56014685fb90 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.673+0000 7f58d6000700 1 -- 172.21.15.79:0/2959011635 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58dc00cc30 con 0x56014685fb90 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.673+0000 7f58d6000700 1 -- 172.21.15.79:0/2959011635 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f58d8026510 con 0x56014685fb90 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.674+0000 7f58fdb38b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.674+0000 7f58fdb38b80 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.674+0000 7f58fdb38b80 20 client.15744 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.674+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560146873bc0 0x560146875fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.674+0000 7f58fdb38b80 4 client.15744 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.674+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2959011635 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601467b9000 con 0x560146873bc0 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.674+0000 7f58ebe00700 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560146873bc0 0x560146875fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.675+0000 7f58ebe00700 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560146873bc0 0x560146875fb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a2d3fd7b8dc4ca server_cookie=a6e98f3f43d35d9c in_seq=0 out_seq=0 2024-06-03T02:25:58.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.675+0000 7f58d6000700 10 client.15744 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:58.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.676+0000 7f58d6000700 1 -- 172.21.15.79:0/2959011635 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5601467b9000 con 0x560146873bc0 2024-06-03T02:25:58.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.676+0000 7f58d6000700 10 client.15744 handle_command_reply: tid=0 2024-06-03T02:25:58.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.676+0000 7f58fdb38b80 1 client.15744 shutdown 2024-06-03T02:25:58.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.677+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2959011635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685fb90 msgr2=0x560146863500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:58.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.677+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685fb90 0x560146863500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f58ec073780 tx=0x7f58d800a380 comp rx=0 tx=0).stop 2024-06-03T02:25:58.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.677+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2959011635 shutdown_connections 2024-06-03T02:25:58.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.677+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601468604b0 0x560146863a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.677+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685a240 0x560146866200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.677+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560146873bc0 0x560146875fb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.678+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2959011635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56014685fb90 0x560146863500 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.678+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2959011635 >> 172.21.15.79:0/2959011635 conn(0x560146786170 msgr2=0x56014684e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:58.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.678+0000 7f58d6000700 0 client.15744 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:58.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.678+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2959011635 shutdown_connections 2024-06-03T02:25:58.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.678+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2959011635 wait complete. 2024-06-03T02:25:58.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.679+0000 7f58fdb38b80 20 client.15744 trim_cache size 0 max 16384 2024-06-03T02:25:58.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.710+0000 7f58fdb38b80 1 Processor -- start 2024-06-03T02:25:58.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.710+0000 7f58fdb38b80 1 -- start start 2024-06-03T02:25:58.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 0x560146859d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014684f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 0x560146861060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1c5920 con 0x5601468604b0 2024-06-03T02:25:58.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1c7460 con 0x56014685fb90 2024-06-03T02:25:58.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1c5ff0 con 0x56014685a240 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58f1200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 0x560146861060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58eb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014684f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58eb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014684f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33448/0 (socket says 172.21.15.79:33448) 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58eb400700 1 -- 172.21.15.79:0/2860053985 learned_addr learned my addr 172.21.15.79:0/2860053985 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.711+0000 7f58ebe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 0x560146859d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.712+0000 7f58eb400700 1 -- 172.21.15.79:0/2860053985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 msgr2=0x560146859d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.712+0000 7f58eb400700 1 --2- 172.21.15.79:0/2860053985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 0x560146859d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.712+0000 7f58eb400700 1 -- 172.21.15.79:0/2860053985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 msgr2=0x560146861060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.712+0000 7f58eb400700 1 --2- 172.21.15.79:0/2860053985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 0x560146861060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.712+0000 7f58eb400700 1 -- 172.21.15.79:0/2860053985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56014685f690 con 0x56014685fb90 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.712+0000 7f58f1200700 1 --2- 172.21.15.79:0/2860053985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 0x560146861060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:25:58.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.712+0000 7f58eb400700 1 --2- 172.21.15.79:0/2860053985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014684f130 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f58d8012010 tx=0x7f58d800a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57099fab957e49f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:58.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.712+0000 7f58e0c00700 1 -- 172.21.15.79:0/2860053985 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d803f070 con 0x56014685fb90 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.713+0000 7f58e0c00700 1 -- 172.21.15.79:0/2860053985 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58d8005520 con 0x56014685fb90 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.713+0000 7f58e0c00700 1 -- 172.21.15.79:0/2860053985 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8004710 con 0x56014685fb90 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.713+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2860053985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 msgr2=0x56014684f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.713+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2860053985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014684f130 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f58d8012010 tx=0x7f58d800a4d0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.714+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2860053985 shutdown_connections 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.714+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2860053985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 0x560146861060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.714+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2860053985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 0x560146859d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.714+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/2860053985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014684f130 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.714+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2860053985 >> 172.21.15.79:0/2860053985 conn(0x560146786170 msgr2=0x560146860950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.714+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2860053985 shutdown_connections 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.714+0000 7f58fdb38b80 1 -- 172.21.15.79:0/2860053985 wait complete. 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.714+0000 7f58fdb38b80 1 -- start start 2024-06-03T02:25:58.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.714+0000 7f58fdb38b80 1 Processor -- start 2024-06-03T02:25:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 0x56014686c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014687af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58fdb38b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 0x56014687b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1c6d90 con 0x5601468604b0 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1c66c0 con 0x56014685fb90 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58fdb38b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1c3860 con 0x56014685a240 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58fdb38b80 10 client.0 _ll_register_callbacks cb 0x560146741380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58eb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014687af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58eb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014687af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33462/0 (socket says 172.21.15.79:33462) 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58eb400700 1 -- 172.21.15.79:0/3214855826 learned_addr learned my addr 172.21.15.79:0/3214855826 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.715+0000 7f58f1200700 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 0x56014687b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58ebe00700 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 0x56014686c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58eb400700 1 -- 172.21.15.79:0/3214855826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 msgr2=0x56014686c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58eb400700 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 0x56014686c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58eb400700 1 -- 172.21.15.79:0/3214855826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 msgr2=0x56014687b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58eb400700 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 0x56014687b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58eb400700 1 -- 172.21.15.79:0/3214855826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56014687ba50 con 0x56014685fb90 2024-06-03T02:25:58.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58ebe00700 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 0x56014686c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:25:58.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58eb400700 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014687af50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f58d800afe0 tx=0x7f58d800a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=976436e00b0ee395 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:25:58.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58d6000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:25:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58d6000700 1 -- 172.21.15.79:0/3214855826 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8048060 con 0x56014685fb90 2024-06-03T02:25:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58d6000700 1 -- 172.21.15.79:0/3214855826 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f58d8030630 con 0x56014685fb90 2024-06-03T02:25:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.716+0000 7f58fdb38b80 1 -- 172.21.15.79:0/3214855826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d8005520 con 0x56014685fb90 2024-06-03T02:25:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58d6000700 1 -- 172.21.15.79:0/3214855826 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58d803f070 con 0x56014685fb90 2024-06-03T02:25:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58fdb38b80 10 client.26075 fetch_fsmap learned FSMap version 30 2024-06-03T02:25:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58fdb38b80 1 -- 172.21.15.79:0/3214855826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601468de600 con 0x56014685fb90 2024-06-03T02:25:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58d6000700 1 -- 172.21.15.79:0/3214855826 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58d8005520 con 0x56014685fb90 2024-06-03T02:25:58.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58d6000700 1 -- 172.21.15.79:0/3214855826 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f58d803a870 con 0x56014685fb90 2024-06-03T02:25:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58fdb38b80 10 client.26075 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:25:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58fdb38b80 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:25:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58fdb38b80 20 client.26075 populate_metadata read hostname 'smithi079' 2024-06-03T02:25:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601468debd0 0x5601468e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:25:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58fdb38b80 4 client.26075 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:25:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.717+0000 7f58fdb38b80 1 -- 172.21.15.79:0/3214855826 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5601467b3580 con 0x5601468debd0 2024-06-03T02:25:58.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.718+0000 7f58ebe00700 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601468debd0 0x5601468e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:25:58.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.719+0000 7f58ebe00700 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601468debd0 0x5601468e0ff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f95653993fe6001 server_cookie=6f3f228a76ee95e4 in_seq=0 out_seq=0 2024-06-03T02:25:58.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.719+0000 7f58d6000700 10 client.26075 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:58.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58d6000700 1 -- 172.21.15.79:0/3214855826 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5601467b3580 con 0x5601468debd0 2024-06-03T02:25:58.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58d6000700 10 client.26075 handle_command_reply: tid=0 2024-06-03T02:25:58.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58fdb38b80 1 client.26075 shutdown 2024-06-03T02:25:58.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58fdb38b80 1 -- 172.21.15.79:0/3214855826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 msgr2=0x56014687af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:58.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014687af50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f58d800afe0 tx=0x7f58d800a380 comp rx=0 tx=0).stop 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58fdb38b80 1 -- 172.21.15.79:0/3214855826 shutdown_connections 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601468debd0 0x5601468e0ff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601468604b0 0x56014687b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56014685a240 0x56014686c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58fdb38b80 1 --2- 172.21.15.79:0/3214855826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56014685fb90 0x56014687af50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58fdb38b80 1 -- 172.21.15.79:0/3214855826 >> 172.21.15.79:0/3214855826 conn(0x560146786170 msgr2=0x56014685e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.720+0000 7f58d6000700 0 client.26075 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.721+0000 7f58fdb38b80 1 -- 172.21.15.79:0/3214855826 shutdown_connections 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.721+0000 7f58fdb38b80 1 -- 172.21.15.79:0/3214855826 wait complete. 2024-06-03T02:25:58.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.721+0000 7f58fdb38b80 20 client.26075 trim_cache size 0 max 16384 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": { 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: "30d8bf3e-e3f2-439d-ae88-c0b323a691cd": { 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: "path": "/", 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: "tag": "30d8bf3e-e3f2-439d-ae88-c0b323a691cd", 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: "options": "recursive,force" 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:25:58.723 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:25:58.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f58d00a2d70 msgr2=0x7f58d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:58.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f58d00a2d70 0x7f58d00a5230 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58d800b010 tx=0x7f58d8012040 comp rx=0 tx=0).stop 2024-06-03T02:25:58.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec107400 msgr2=0x7f58ec071f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:25:58.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec107400 0x7f58ec071f80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f58e400d010 tx=0x7f58e400bde0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 shutdown_connections 2024-06-03T02:25:58.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec107400 0x7f58ec071f80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58ec101390 0x7f58ec10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f58d00a2d70 0x7f58d00a5230 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 --2- 172.21.15.79:0/2758344242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58ec101d50 0x7f58ec10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:25:58.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.722+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 >> 172.21.15.79:0/2758344242 conn(0x7f58ec078340 msgr2=0x7f58ec104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:25:58.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.723+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 shutdown_connections 2024-06-03T02:25:58.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:25:58.723+0000 7f58f3200700 1 -- 172.21.15.79:0/2758344242 wait complete. 2024-06-03T02:25:59.535 INFO:tasks.cephfs.filesystem:scrub status for tag:30d8bf3e-e3f2-439d-ae88-c0b323a691cd - {'path': '/', 'tag': '30d8bf3e-e3f2-439d-ae88-c0b323a691cd', 'options': 'recursive,force'} 2024-06-03T02:26:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:59 smithi123 ceph-mon[36640]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 528 op/s 2024-06-03T02:26:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:25:59 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:26:00.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:59 smithi079 ceph-mon[31819]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 528 op/s 2024-06-03T02:26:00.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:25:59 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:26:00.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:59 smithi135 ceph-mon[36653]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 528 op/s 2024-06-03T02:26:00.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:25:59 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:26:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:01 smithi123 ceph-mon[36640]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 528 op/s 2024-06-03T02:26:02.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:01 smithi079 ceph-mon[31819]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 528 op/s 2024-06-03T02:26:02.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:01 smithi135 ceph-mon[36653]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 528 op/s 2024-06-03T02:26:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 355.0 s, 608.2 tps, lat 160.140 ms stddev 193.475 2024-06-03T02:26:04.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:03 smithi123 ceph-mon[36640]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 548 op/s 2024-06-03T02:26:04.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:03 smithi079 ceph-mon[31819]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 548 op/s 2024-06-03T02:26:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:03 smithi135 ceph-mon[36653]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 548 op/s 2024-06-03T02:26:06.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:05 smithi123 ceph-mon[36640]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 561 op/s 2024-06-03T02:26:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:05 smithi079 ceph-mon[31819]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 561 op/s 2024-06-03T02:26:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:05 smithi135 ceph-mon[36653]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 561 op/s 2024-06-03T02:26:08.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:07 smithi123 ceph-mon[36640]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 554 op/s 2024-06-03T02:26:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 360.0 s, 736.8 tps, lat 139.444 ms stddev 182.078 2024-06-03T02:26:08.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:07 smithi079 ceph-mon[31819]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 554 op/s 2024-06-03T02:26:08.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:07 smithi135 ceph-mon[36653]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 554 op/s 2024-06-03T02:26:10.228 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:09 smithi123 ceph-mon[36640]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 558 op/s 2024-06-03T02:26:10.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:09 smithi079 ceph-mon[31819]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 558 op/s 2024-06-03T02:26:10.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:09 smithi135 ceph-mon[36653]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 558 op/s 2024-06-03T02:26:12.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:11 smithi123 ceph-mon[36640]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.9 MiB/s wr, 537 op/s 2024-06-03T02:26:12.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:11 smithi079 ceph-mon[31819]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.9 MiB/s wr, 537 op/s 2024-06-03T02:26:12.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:11 smithi135 ceph-mon[36653]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.9 MiB/s wr, 537 op/s 2024-06-03T02:26:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 365.0 s, 517.2 tps, lat 192.180 ms stddev 258.200 2024-06-03T02:26:14.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:13 smithi123 ceph-mon[36640]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.2 MiB/s wr, 543 op/s 2024-06-03T02:26:14.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:13 smithi079 ceph-mon[31819]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.2 MiB/s wr, 543 op/s 2024-06-03T02:26:14.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:13 smithi135 ceph-mon[36653]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.2 MiB/s wr, 543 op/s 2024-06-03T02:26:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:15 smithi123 ceph-mon[36640]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 607 op/s 2024-06-03T02:26:16.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:15 smithi079 ceph-mon[31819]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 607 op/s 2024-06-03T02:26:16.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:15 smithi135 ceph-mon[36653]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 607 op/s 2024-06-03T02:26:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 370.0 s, 674.6 tps, lat 152.623 ms stddev 175.302 2024-06-03T02:26:18.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:17 smithi079 ceph-mon[31819]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 580 op/s 2024-06-03T02:26:18.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:17 smithi135 ceph-mon[36653]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 580 op/s 2024-06-03T02:26:18.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:17 smithi123 ceph-mon[36640]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 580 op/s 2024-06-03T02:26:20.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:19 smithi079 ceph-mon[31819]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.7 MiB/s wr, 570 op/s 2024-06-03T02:26:20.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:19 smithi135 ceph-mon[36653]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.7 MiB/s wr, 570 op/s 2024-06-03T02:26:20.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:19 smithi123 ceph-mon[36640]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.7 MiB/s wr, 570 op/s 2024-06-03T02:26:22.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:21 smithi079 ceph-mon[31819]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 557 op/s 2024-06-03T02:26:22.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:21 smithi135 ceph-mon[36653]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 557 op/s 2024-06-03T02:26:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:21 smithi123 ceph-mon[36640]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 557 op/s 2024-06-03T02:26:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 375.0 s, 610.2 tps, lat 162.296 ms stddev 191.501 2024-06-03T02:26:24.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:24 smithi079 ceph-mon[31819]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 574 op/s 2024-06-03T02:26:24.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:24 smithi135 ceph-mon[36653]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 574 op/s 2024-06-03T02:26:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:24 smithi123 ceph-mon[36640]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 574 op/s 2024-06-03T02:26:26.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:26 smithi079 ceph-mon[31819]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 MiB/s wr, 603 op/s 2024-06-03T02:26:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:26 smithi135 ceph-mon[36653]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 MiB/s wr, 603 op/s 2024-06-03T02:26:26.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:26 smithi123 ceph-mon[36640]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 MiB/s wr, 603 op/s 2024-06-03T02:26:28.275 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:28 smithi123 ceph-mon[36640]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 MiB/s wr, 538 op/s 2024-06-03T02:26:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 380.0 s, 854.8 tps, lat 118.007 ms stddev 147.475 2024-06-03T02:26:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:28 smithi079 ceph-mon[31819]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 MiB/s wr, 538 op/s 2024-06-03T02:26:28.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:28 smithi135 ceph-mon[36653]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 MiB/s wr, 538 op/s 2024-06-03T02:26:29.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:26:30.227 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:26:30.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:30 smithi079 ceph-mon[31819]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 562 op/s 2024-06-03T02:26:30.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:30 smithi135 ceph-mon[36653]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 562 op/s 2024-06-03T02:26:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:30 smithi123 ceph-mon[36640]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 562 op/s 2024-06-03T02:26:31.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:31 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:26:31.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:31 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:26:31.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:31 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:26:32.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:32 smithi079 ceph-mon[31819]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 MiB/s wr, 542 op/s 2024-06-03T02:26:32.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:32 smithi135 ceph-mon[36653]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 MiB/s wr, 542 op/s 2024-06-03T02:26:32.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:32 smithi123 ceph-mon[36640]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 MiB/s wr, 542 op/s 2024-06-03T02:26:32.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.804+0000 7f94f2000700 1 -- 172.21.15.79:0/312410484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94ec1073c0 msgr2=0x7f94ec06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:32.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.805+0000 7f94f2000700 1 --2- 172.21.15.79:0/312410484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94ec1073c0 0x7f94ec06a1a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00b0a0 tx=0x7f94dc00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.806+0000 7f94f2000700 1 -- 172.21.15.79:0/312410484 shutdown_connections 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.807+0000 7f94f2000700 1 --2- 172.21.15.79:0/312410484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94ec104ed0 0x7f94ec105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.807+0000 7f94f2000700 1 --2- 172.21.15.79:0/312410484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94ec1073c0 0x7f94ec06a1a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.807+0000 7f94f2000700 1 --2- 172.21.15.79:0/312410484 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94ec06a6e0 0x7f94ec06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.807+0000 7f94f2000700 1 -- 172.21.15.79:0/312410484 >> 172.21.15.79:0/312410484 conn(0x7f94ec078340 msgr2=0x7f94ec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.807+0000 7f94f2000700 1 -- 172.21.15.79:0/312410484 shutdown_connections 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.808+0000 7f94f2000700 1 -- 172.21.15.79:0/312410484 wait complete. 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.810+0000 7f94f2000700 1 Processor -- start 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.812+0000 7f94f2000700 1 -- start start 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.816+0000 7f94f2000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94ec06a6e0 0x7f94ec19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.816+0000 7f94f2000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94ec104ed0 0x7f94ec19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.816+0000 7f94f2000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94ec1073c0 0x7f94ec1a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.817+0000 7f94f2000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec118e40 con 0x7f94ec06a6e0 2024-06-03T02:26:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.817+0000 7f94f2000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec118cc0 con 0x7f94ec1073c0 2024-06-03T02:26:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.817+0000 7f94f2000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec118fc0 con 0x7f94ec104ed0 2024-06-03T02:26:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.817+0000 7f94ebe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94ec1073c0 0x7f94ec1a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.817+0000 7f94ebe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94ec1073c0 0x7f94ec1a3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58210/0 (socket says 172.21.15.79:58210) 2024-06-03T02:26:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.817+0000 7f94ebe00700 1 -- 172.21.15.79:0/1282170339 learned_addr learned my addr 172.21.15.79:0/1282170339 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.817+0000 7f94ebe00700 1 -- 172.21.15.79:0/1282170339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94ec104ed0 msgr2=0x7f94ec19f650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:26:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.817+0000 7f94ebe00700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94ec104ed0 0x7f94ec19f650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.817+0000 7f94ebe00700 1 -- 172.21.15.79:0/1282170339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94ec06a6e0 msgr2=0x7f94ec19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.818+0000 7f94ebe00700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94ec06a6e0 0x7f94ec19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.818+0000 7f94ebe00700 1 -- 172.21.15.79:0/1282170339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec1a40d0 con 0x7f94ec1073c0 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.818+0000 7f94ebe00700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94ec1073c0 0x7f94ec1a3990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94e400c970 tx=0x7f94e400cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.819+0000 7f94e8c00700 1 -- 172.21.15.79:0/1282170339 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4007b00 con 0x7f94ec1073c0 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.819+0000 7f94f2000700 1 -- 172.21.15.79:0/1282170339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec1a43c0 con 0x7f94ec1073c0 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.819+0000 7f94f2000700 1 -- 172.21.15.79:0/1282170339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec1abc40 con 0x7f94ec1073c0 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.819+0000 7f94e8c00700 1 -- 172.21.15.79:0/1282170339 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e4004d10 con 0x7f94ec1073c0 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.819+0000 7f94e8c00700 1 -- 172.21.15.79:0/1282170339 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e40056a0 con 0x7f94ec1073c0 2024-06-03T02:26:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.829+0000 7f94e8c00700 1 -- 172.21.15.79:0/1282170339 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f94e400f470 con 0x7f94ec1073c0 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.831+0000 7f94e8c00700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94d00a2ca0 0x7f94d00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.831+0000 7f94fc9f2b80 1 Processor -- start 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.831+0000 7f94e8c00700 1 -- 172.21.15.79:0/1282170339 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f94e40cffb0 con 0x7f94ec1073c0 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.831+0000 7f94fc9f2b80 1 -- start start 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.832+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 0x564fe6c67070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.832+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c675b0 0x564fe6c67a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.832+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c83310 0x564fe6c85700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.832+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564fe6b76a10 con 0x564fe6be3e40 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.832+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564fe6bb9b00 con 0x564fe6c83310 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.832+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564fe6b748f0 con 0x564fe6c675b0 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.832+0000 7f94ebe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c83310 0x564fe6c85700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.832+0000 7f94ebe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c83310 0x564fe6c85700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58218/0 (socket says 172.21.15.79:58218) 2024-06-03T02:26:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.832+0000 7f94ebe00700 1 -- 172.21.15.79:0/3353823338 learned_addr learned my addr 172.21.15.79:0/3353823338 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.833+0000 7f94eb400700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94d00a2ca0 0x7f94d00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.833+0000 7f94e2a00700 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c675b0 0x564fe6c67a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.834+0000 7f94eb400700 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 0x564fe6c67070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.834+0000 7f94ebe00700 1 -- 172.21.15.79:0/3353823338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c675b0 msgr2=0x564fe6c67a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.835+0000 7f94ebe00700 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c675b0 0x564fe6c67a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.835+0000 7f94ebe00700 1 -- 172.21.15.79:0/3353823338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 msgr2=0x564fe6c67070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.835+0000 7f94ebe00700 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 0x564fe6c67070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.835+0000 7f94ebe00700 1 -- 172.21.15.79:0/3353823338 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe6b61f90 con 0x564fe6c83310 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.836+0000 7f94e2a00700 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c675b0 0x564fe6c67a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.837+0000 7f94ebe00700 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c83310 0x564fe6c85700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f94e409c3f0 tx=0x7f94e409c0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=224baa16a1e9547b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.838+0000 7f94e0c00700 1 -- 172.21.15.79:0/3353823338 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e401d040 con 0x564fe6c83310 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.838+0000 7f94e0c00700 1 -- 172.21.15.79:0/3353823338 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e40a0810 con 0x564fe6c83310 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.839+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3353823338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c83310 msgr2=0x564fe6c85700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.839+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c83310 0x564fe6c85700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f94e409c3f0 tx=0x7f94e409c0b0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.839+0000 7f94eb400700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94d00a2ca0 0x7f94d00a5160 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d010 tx=0x7f94dc00cfa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.840+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3353823338 shutdown_connections 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.840+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c83310 0x564fe6c85700 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.840+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 0x564fe6c67070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.840+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3353823338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c675b0 0x564fe6c67a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.840+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3353823338 >> 172.21.15.79:0/3353823338 conn(0x564fe6b94170 msgr2=0x564fe6c634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.840+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3353823338 shutdown_connections 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.841+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3353823338 wait complete. 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.841+0000 7f94fc9f2b80 1 -- start start 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.842+0000 7f94fc9f2b80 1 Processor -- start 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.843+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 0x564fe6c82a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.843+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c7fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.843+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c83310 0x564fe6c80260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.844+0000 7f94eb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 0x564fe6c82a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.844+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564fe6b55f60 con 0x564fe6be3e40 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.844+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564fe6bb5ac0 con 0x564fe6c675b0 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.844+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564fe6c85fd0 con 0x564fe6c83310 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.844+0000 7f94fc9f2b80 10 client.0 _ll_register_callbacks cb 0x564fe6b53720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.844+0000 7f94ebe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c83310 0x564fe6c80260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.844+0000 7f94e2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c7fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.844+0000 7f94e2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c7fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58224/0 (socket says 172.21.15.79:58224) 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.844+0000 7f94e2a00700 1 -- 172.21.15.79:0/3495879746 learned_addr learned my addr 172.21.15.79:0/3495879746 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.845+0000 7f94e2a00700 1 -- 172.21.15.79:0/3495879746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c83310 msgr2=0x564fe6c80260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.845+0000 7f94e2a00700 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c83310 0x564fe6c80260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.845+0000 7f94e2a00700 1 -- 172.21.15.79:0/3495879746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 msgr2=0x564fe6c82a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.845+0000 7f94e2a00700 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 0x564fe6c82a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.845+0000 7f94e2a00700 1 -- 172.21.15.79:0/3495879746 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe6b088d0 con 0x564fe6c675b0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.846+0000 7f94d9600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.846+0000 7f94e2a00700 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c7fd20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94d400afd0 tx=0x7f94d400c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d25e6ecfa2039ec1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.849+0000 7f94d9600700 1 -- 172.21.15.79:0/3495879746 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4026070 con 0x564fe6c675b0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.849+0000 7f94d9600700 1 -- 172.21.15.79:0/3495879746 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f94d4022400 con 0x564fe6c675b0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.849+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3495879746 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fe6bc7000 con 0x564fe6c675b0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.849+0000 7f94d9600700 1 -- 172.21.15.79:0/3495879746 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94d4032050 con 0x564fe6c675b0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.849+0000 7f94fc9f2b80 10 client.26084 fetch_fsmap learned FSMap version 30 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.849+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3495879746 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fe6c907d0 con 0x564fe6c675b0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.851+0000 7f94d9600700 1 -- 172.21.15.79:0/3495879746 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564fe6bc7000 con 0x564fe6c675b0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.851+0000 7f94d9600700 1 -- 172.21.15.79:0/3495879746 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f94d402a7b0 con 0x564fe6c675b0 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.851+0000 7f94fc9f2b80 10 client.26084 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.851+0000 7f94fc9f2b80 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:26:32.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.852+0000 7f94fc9f2b80 20 client.26084 populate_metadata read hostname 'smithi079' 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.852+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564fe6c92d60 0x564fe6c951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.852+0000 7f94fc9f2b80 4 client.26084 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.852+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3495879746 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fe6bc1580 con 0x564fe6c92d60 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.853+0000 7f94eb400700 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564fe6c92d60 0x564fe6c951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.855+0000 7f94eb400700 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564fe6c92d60 0x564fe6c951b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b5841e97161be2 server_cookie=1e9501eb170e7199 in_seq=0 out_seq=0 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.855+0000 7f94d9600700 10 client.26084 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.857+0000 7f94d9600700 1 -- 172.21.15.79:0/3495879746 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564fe6bc1580 con 0x564fe6c92d60 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.857+0000 7f94d9600700 10 client.26084 handle_command_reply: tid=0 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.857+0000 7f94fc9f2b80 1 client.26084 shutdown 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.857+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3495879746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 msgr2=0x564fe6c7fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.857+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c7fd20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94d400afd0 tx=0x7f94d400c560 comp rx=0 tx=0).stop 2024-06-03T02:26:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.857+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3495879746 shutdown_connections 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.857+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6c83310 0x564fe6c80260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.858+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564fe6c92d60 0x564fe6c951b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.858+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6be3e40 0x564fe6c82a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.858+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3495879746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c7fd20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.858+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3495879746 >> 172.21.15.79:0/3495879746 conn(0x564fe6b94170 msgr2=0x564fe6c62740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.858+0000 7f94d9600700 0 client.26084 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.858+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3495879746 shutdown_connections 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.858+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3495879746 wait complete. 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.858+0000 7f94fc9f2b80 20 client.26084 trim_cache size 0 max 16384 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.900+0000 7f94fc9f2b80 1 Processor -- start 2024-06-03T02:26:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.900+0000 7f94fc9f2b80 1 -- start start 2024-06-03T02:26:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.900+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c80290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.900+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c84700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.900+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 0x564fe6c61eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.900+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800eb90 con 0x564fe6c83310 2024-06-03T02:26:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.900+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94c80106d0 con 0x564fe6c675b0 2024-06-03T02:26:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.900+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800f260 con 0x564fe6be3e40 2024-06-03T02:26:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.901+0000 7f94eb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c80290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.901+0000 7f94e2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c84700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.901+0000 7f94ebe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 0x564fe6c61eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.901+0000 7f94e2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c84700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58234/0 (socket says 172.21.15.79:58234) 2024-06-03T02:26:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.901+0000 7f94e2a00700 1 -- 172.21.15.79:0/3263664858 learned_addr learned my addr 172.21.15.79:0/3263664858 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.902+0000 7f94ebe00700 1 -- 172.21.15.79:0/3263664858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 msgr2=0x564fe6c80290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.902+0000 7f94ebe00700 1 --2- 172.21.15.79:0/3263664858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c80290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.902+0000 7f94ebe00700 1 -- 172.21.15.79:0/3263664858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 msgr2=0x564fe6c84700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.902+0000 7f94ebe00700 1 --2- 172.21.15.79:0/3263664858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c84700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.902+0000 7f94ebe00700 1 -- 172.21.15.79:0/3263664858 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe6c5ae10 con 0x564fe6c83310 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.903+0000 7f94ebe00700 1 --2- 172.21.15.79:0/3263664858 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 0x564fe6c61eb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f94e409c3f0 tx=0x7f94e409ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900448b6d275092b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.903+0000 7f94dbe00700 1 -- 172.21.15.79:0/3263664858 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e401d040 con 0x564fe6c83310 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.904+0000 7f94dbe00700 1 -- 172.21.15.79:0/3263664858 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e40a50d0 con 0x564fe6c83310 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.904+0000 7f94dbe00700 1 -- 172.21.15.79:0/3263664858 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e40a4c40 con 0x564fe6c83310 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.906+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3263664858 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 msgr2=0x564fe6c61eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.906+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3263664858 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 0x564fe6c61eb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f94e409c3f0 tx=0x7f94e409ccd0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.907+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3263664858 shutdown_connections 2024-06-03T02:26:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.907+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3263664858 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 0x564fe6c61eb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.907+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3263664858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c80290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.907+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/3263664858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c84700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.907+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3263664858 >> 172.21.15.79:0/3263664858 conn(0x564fe6b94170 msgr2=0x564fe6c7e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.908+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3263664858 shutdown_connections 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.908+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/3263664858 wait complete. 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.909+0000 7f94fc9f2b80 1 -- start start 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.909+0000 7f94fc9f2b80 1 Processor -- start 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c8b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c8bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94fc9f2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 0x564fe6c7c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8010000 con 0x564fe6c83310 2024-06-03T02:26:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800f930 con 0x564fe6c675b0 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94fc9f2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800cb30 con 0x564fe6be3e40 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94fc9f2b80 10 client.0 _ll_register_callbacks cb 0x564fe6b53720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94eb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c8b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94eb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c8b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56658/0 (socket says 172.21.15.79:56658) 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.912+0000 7f94eb400700 1 -- 172.21.15.79:0/593693568 learned_addr learned my addr 172.21.15.79:0/593693568 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.913+0000 7f94eb400700 1 -- 172.21.15.79:0/593693568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 msgr2=0x564fe6c8bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.913+0000 7f94ebe00700 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 0x564fe6c7c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.913+0000 7f94e2a00700 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c8bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.913+0000 7f94eb400700 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c8bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.913+0000 7f94eb400700 1 -- 172.21.15.79:0/593693568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 msgr2=0x564fe6c7c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.913+0000 7f94eb400700 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 0x564fe6c7c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.913+0000 7f94eb400700 1 -- 172.21.15.79:0/593693568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe6c7cd80 con 0x564fe6be3e40 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.913+0000 7f94e2a00700 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c8bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.914+0000 7f94eb400700 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c8b570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009fd0 tx=0x7f94dc006210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65dc8e8a4eb3829e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.914+0000 7f94d9600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.914+0000 7f94d9600700 1 -- 172.21.15.79:0/593693568 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc03a060 con 0x564fe6be3e40 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.914+0000 7f94d9600700 1 -- 172.21.15.79:0/593693568 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f94dc0078b0 con 0x564fe6be3e40 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.915+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/593693568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94e40a50d0 con 0x564fe6be3e40 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.915+0000 7f94d9600700 1 -- 172.21.15.79:0/593693568 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94dc0094f0 con 0x564fe6be3e40 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.915+0000 7f94fc9f2b80 10 client.26083 fetch_fsmap learned FSMap version 30 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.915+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/593693568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fe6c8ae60 con 0x564fe6be3e40 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.917+0000 7f94d9600700 1 -- 172.21.15.79:0/593693568 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e40a50d0 con 0x564fe6be3e40 2024-06-03T02:26:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.917+0000 7f94d9600700 1 -- 172.21.15.79:0/593693568 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f94dc028040 con 0x564fe6be3e40 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.917+0000 7f94fc9f2b80 10 client.26083 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.917+0000 7f94fc9f2b80 10 client.26083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.917+0000 7f94fc9f2b80 20 client.26083 populate_metadata read hostname 'smithi079' 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.917+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564fe6cfdc40 0x564fe6d00060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.918+0000 7f94fc9f2b80 4 client.26083 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.918+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/593693568 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564fe6a3b250 con 0x564fe6cfdc40 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.918+0000 7f94e2a00700 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564fe6cfdc40 0x564fe6d00060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.920+0000 7f94e2a00700 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564fe6cfdc40 0x564fe6d00060 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dee90065aaaae63 server_cookie=c068d4cbc334fb2d in_seq=0 out_seq=0 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.920+0000 7f94d9600700 10 client.26083 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94d9600700 1 -- 172.21.15.79:0/593693568 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564fe6a3b250 con 0x564fe6cfdc40 2024-06-03T02:26:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94d9600700 10 client.26083 handle_command_reply: tid=0 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94fc9f2b80 1 client.26083 shutdown 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/593693568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 msgr2=0x564fe6c8b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c8b570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009fd0 tx=0x7f94dc006210 comp rx=0 tx=0).stop 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/593693568 shutdown_connections 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x564fe6cfdc40 0x564fe6d00060 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564fe6c83310 0x564fe6c7c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe6be3e40 0x564fe6c8b570 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94fc9f2b80 1 --2- 172.21.15.79:0/593693568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe6c675b0 0x564fe6c8bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/593693568 >> 172.21.15.79:0/593693568 conn(0x564fe6b94170 msgr2=0x564fe6c62740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.921+0000 7f94d9600700 0 client.26083 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:26:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.922+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/593693568 shutdown_connections 2024-06-03T02:26:33.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.922+0000 7f94fc9f2b80 1 -- 172.21.15.79:0/593693568 wait complete. 2024-06-03T02:26:33.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.922+0000 7f94fc9f2b80 20 client.26083 trim_cache size 0 max 16384 2024-06-03T02:26:33.008 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:26:33.008 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:26:33.008 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:26:33.008 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:26:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 -- 172.21.15.79:0/1282170339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94d00a2ca0 msgr2=0x7f94d00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94d00a2ca0 0x7f94d00a5160 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d010 tx=0x7f94dc00cfa0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 -- 172.21.15.79:0/1282170339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94ec1073c0 msgr2=0x7f94ec1a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94ec1073c0 0x7f94ec1a3990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94e400c970 tx=0x7f94e400cd30 comp rx=0 tx=0).stop 2024-06-03T02:26:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 -- 172.21.15.79:0/1282170339 shutdown_connections 2024-06-03T02:26:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94ec1073c0 0x7f94ec1a3990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f94d00a2ca0 0x7f94d00a5160 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94ec06a6e0 0x7f94ec19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 --2- 172.21.15.79:0/1282170339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94ec104ed0 0x7f94ec19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.924+0000 7f94f2000700 1 -- 172.21.15.79:0/1282170339 >> 172.21.15.79:0/1282170339 conn(0x7f94ec078340 msgr2=0x7f94ec103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.925+0000 7f94f2000700 1 -- 172.21.15.79:0/1282170339 shutdown_connections 2024-06-03T02:26:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:32.925+0000 7f94f2000700 1 -- 172.21.15.79:0/1282170339 wait complete. 2024-06-03T02:26:33.153 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.154 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.154 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.154 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.154 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.154 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:33 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 385.0 s, 567.2 tps, lat 164.120 ms stddev 179.906 2024-06-03T02:26:33.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:33 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:33 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:33.831 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:26:33.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:26:34.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:34 smithi079 ceph-mon[31819]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 560 op/s 2024-06-03T02:26:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:34 smithi135 ceph-mon[36653]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 560 op/s 2024-06-03T02:26:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:34 smithi123 ceph-mon[36640]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 560 op/s 2024-06-03T02:26:35.902 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:26:36.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:36 smithi135 ceph-mon[36653]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 589 op/s 2024-06-03T02:26:36.358 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:36 smithi079 ceph-mon[31819]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 589 op/s 2024-06-03T02:26:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:36 smithi123 ceph-mon[36640]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 589 op/s 2024-06-03T02:26:38.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 390.0 s, 526.0 tps, lat 194.155 ms stddev 212.969 2024-06-03T02:26:38.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:38 smithi079 ceph-mon[31819]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 561 op/s 2024-06-03T02:26:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:38 smithi123 ceph-mon[36640]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 561 op/s 2024-06-03T02:26:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:38 smithi135 ceph-mon[36653]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 561 op/s 2024-06-03T02:26:39.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.716+0000 7f01b1200700 1 -- 172.21.15.79:0/2638589444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac102ed0 msgr2=0x7f01ac10f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:39.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.716+0000 7f01b1200700 1 --2- 172.21.15.79:0/2638589444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac102ed0 0x7f01ac10f360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f01a401c0d0 tx=0x7f01a401c3e0 comp rx=0 tx=0).stop 2024-06-03T02:26:39.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.717+0000 7f01b1200700 1 -- 172.21.15.79:0/2638589444 shutdown_connections 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.717+0000 7f01b1200700 1 --2- 172.21.15.79:0/2638589444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac102ed0 0x7f01ac10f360 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.718+0000 7f01b1200700 1 --2- 172.21.15.79:0/2638589444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac108520 0x7f01ac108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.718+0000 7f01b1200700 1 --2- 172.21.15.79:0/2638589444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac102510 0x7f01ac102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.718+0000 7f01b1200700 1 -- 172.21.15.79:0/2638589444 >> 172.21.15.79:0/2638589444 conn(0x7f01ac079560 msgr2=0x7f01ac079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.718+0000 7f01b1200700 1 -- 172.21.15.79:0/2638589444 shutdown_connections 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.718+0000 7f01b1200700 1 -- 172.21.15.79:0/2638589444 wait complete. 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.719+0000 7f01b1200700 1 Processor -- start 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.719+0000 7f01b1200700 1 -- start start 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.720+0000 7f01b1200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac102510 0x7f01ac10b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.720+0000 7f01b1200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac102ed0 0x7f01ac10b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:39.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.720+0000 7f01b1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac108520 0x7f01ac1b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.720+0000 7f01b1200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac06e6c0 con 0x7f01ac102510 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.720+0000 7f01b1200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac06e540 con 0x7f01ac102ed0 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.720+0000 7f01b1200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac06e840 con 0x7f01ac108520 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.722+0000 7f01a9400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac102ed0 0x7f01ac10b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.722+0000 7f01a9400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac102ed0 0x7f01ac10b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:51840/0 (socket says 172.21.15.79:51840) 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.722+0000 7f01a9400700 1 -- 172.21.15.79:0/1947273140 learned_addr learned my addr 172.21.15.79:0/1947273140 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.722+0000 7f01aa800700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac108520 0x7f01ac1b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.722+0000 7f01a9e00700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac102510 0x7f01ac10b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.723+0000 7f01aa800700 1 -- 172.21.15.79:0/1947273140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac102ed0 msgr2=0x7f01ac10b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.723+0000 7f01aa800700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac102ed0 0x7f01ac10b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.723+0000 7f01aa800700 1 -- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac102510 msgr2=0x7f01ac10b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.723+0000 7f01aa800700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac102510 0x7f01ac10b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.723+0000 7f01aa800700 1 -- 172.21.15.79:0/1947273140 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac1b7490 con 0x7f01ac108520 2024-06-03T02:26:39.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.723+0000 7f01a9400700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac102ed0 0x7f01ac10b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:26:39.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.723+0000 7f01aa800700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac108520 0x7f01ac1b6cd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01a401c0d0 tx=0x7f01a401ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:39.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.725+0000 7f01a2a00700 1 -- 172.21.15.79:0/1947273140 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4019e10 con 0x7f01ac108520 2024-06-03T02:26:39.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.725+0000 7f01a2a00700 1 -- 172.21.15.79:0/1947273140 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01a40055a0 con 0x7f01ac108520 2024-06-03T02:26:39.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.725+0000 7f01a2a00700 1 -- 172.21.15.79:0/1947273140 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4007a60 con 0x7f01ac108520 2024-06-03T02:26:39.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.725+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac1b7720 con 0x7f01ac108520 2024-06-03T02:26:39.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.726+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac0746c0 con 0x7f01ac108520 2024-06-03T02:26:39.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.729+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac103c00 con 0x7f01ac108520 2024-06-03T02:26:39.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.732+0000 7f01a2a00700 1 -- 172.21.15.79:0/1947273140 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f01a4016020 con 0x7f01ac108520 2024-06-03T02:26:39.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.735+0000 7f01a2a00700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f01900a70b0 0x7f01900a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:39.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.735+0000 7f01a2a00700 1 -- 172.21.15.79:0/1947273140 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f01a4020020 con 0x7f01ac108520 2024-06-03T02:26:39.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.739+0000 7f01a9e00700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f01900a70b0 0x7f01900a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:39.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.740+0000 7f01a9e00700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f01900a70b0 0x7f01900a9570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f019c006fd0 tx=0x7f019c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:39.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:39.747+0000 7f01a2a00700 1 -- 172.21.15.79:0/1947273140 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f01a400c070 con 0x7f01ac108520 2024-06-03T02:26:40.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.200+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01ac10c290 con 0x7f01ac108520 2024-06-03T02:26:40.273 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:26:40.273 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:26:40.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.270+0000 7f01a2a00700 1 -- 172.21.15.79:0/1947273140 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f01a409d000 con 0x7f01ac108520 2024-06-03T02:26:40.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.275+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f01900a70b0 msgr2=0x7f01900a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:40.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.275+0000 7f01b1200700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f01900a70b0 0x7f01900a9570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f019c006fd0 tx=0x7f019c008040 comp rx=0 tx=0).stop 2024-06-03T02:26:40.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.275+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac108520 msgr2=0x7f01ac1b6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:40.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.275+0000 7f01b1200700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac108520 0x7f01ac1b6cd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01a401c0d0 tx=0x7f01a401ac00 comp rx=0 tx=0).stop 2024-06-03T02:26:40.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.277+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 shutdown_connections 2024-06-03T02:26:40.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.277+0000 7f01b1200700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac108520 0x7f01ac1b6cd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:40.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.277+0000 7f01b1200700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f01900a70b0 0x7f01900a9570 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:40.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.277+0000 7f01b1200700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac102510 0x7f01ac10b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:40.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.277+0000 7f01b1200700 1 --2- 172.21.15.79:0/1947273140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac102ed0 0x7f01ac10b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:40.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.277+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 >> 172.21.15.79:0/1947273140 conn(0x7f01ac079560 msgr2=0x7f01ac101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:40.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.277+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 shutdown_connections 2024-06-03T02:26:40.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:40.278+0000 7f01b1200700 1 -- 172.21.15.79:0/1947273140 wait complete. 2024-06-03T02:26:40.289 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:26:40.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:40 smithi135 ceph-mon[36653]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 557 op/s 2024-06-03T02:26:40.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:40.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:40 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:40.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:40 smithi123 ceph-mon[36640]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 557 op/s 2024-06-03T02:26:40.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:40.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:40 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:40.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:40 smithi079 ceph-mon[31819]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 557 op/s 2024-06-03T02:26:40.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:40.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:40 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:41.448 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:41 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1947273140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:26:41.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:41 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1947273140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:26:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:41 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1947273140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:26:41.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:26:42.139 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:42 smithi079 ceph-mon[31819]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 520 op/s 2024-06-03T02:26:42.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:42 smithi123 ceph-mon[36640]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 520 op/s 2024-06-03T02:26:42.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:42 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:26:42.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:42 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:26:42.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:42 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:42 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:26:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:42 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:26:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:42 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:42.590 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:26:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:42 smithi135 ceph-mon[36653]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 520 op/s 2024-06-03T02:26:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:42 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:26:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:42 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:26:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:42 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:26:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 395.0 s, 509.6 tps, lat 200.429 ms stddev 219.667 2024-06-03T02:26:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:44 smithi123 ceph-mon[36640]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-06-03T02:26:44.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:44 smithi079 ceph-mon[31819]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-06-03T02:26:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:44 smithi135 ceph-mon[36653]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-06-03T02:26:46.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:46 smithi079 ceph-mon[31819]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 528 op/s 2024-06-03T02:26:46.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:46 smithi135 ceph-mon[36653]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 528 op/s 2024-06-03T02:26:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:46 smithi123 ceph-mon[36640]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 528 op/s 2024-06-03T02:26:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.185+0000 7fa66ee00700 1 -- 172.21.15.79:0/3823369271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66810dee0 msgr2=0x7fa66810e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.185+0000 7fa66ee00700 1 --2- 172.21.15.79:0/3823369271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66810dee0 0x7fa66810e2c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa65400b600 tx=0x7fa65400b910 comp rx=0 tx=0).stop 2024-06-03T02:26:48.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.188+0000 7fa66ee00700 1 -- 172.21.15.79:0/3823369271 shutdown_connections 2024-06-03T02:26:48.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.188+0000 7fa66ee00700 1 --2- 172.21.15.79:0/3823369271 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa668109e50 0x7fa668115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.188+0000 7fa66ee00700 1 --2- 172.21.15.79:0/3823369271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66810dee0 0x7fa66810e2c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.188+0000 7fa66ee00700 1 --2- 172.21.15.79:0/3823369271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa668109490 0x7fa668109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.188+0000 7fa66ee00700 1 -- 172.21.15.79:0/3823369271 >> 172.21.15.79:0/3823369271 conn(0x7fa66806d9e0 msgr2=0x7fa66806ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:48.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.188+0000 7fa66ee00700 1 -- 172.21.15.79:0/3823369271 shutdown_connections 2024-06-03T02:26:48.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.188+0000 7fa66ee00700 1 -- 172.21.15.79:0/3823369271 wait complete. 2024-06-03T02:26:48.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.190+0000 7fa66ee00700 1 Processor -- start 2024-06-03T02:26:48.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.190+0000 7fa66ee00700 1 -- start start 2024-06-03T02:26:48.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.190+0000 7fa66ee00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa668109490 0x7fa6681a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.190+0000 7fa66ee00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa668109e50 0x7fa6681a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.190+0000 7fa66ee00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66810dee0 0x7fa6681ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.191+0000 7fa667400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa668109e50 0x7fa6681a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.191+0000 7fa667400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa668109e50 0x7fa6681a2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58960/0 (socket says 172.21.15.79:58960) 2024-06-03T02:26:48.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.191+0000 7fa667e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa668109490 0x7fa6681a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.191+0000 7fa667e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa668109490 0x7fa6681a21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38572/0 (socket says 172.21.15.79:38572) 2024-06-03T02:26:48.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.191+0000 7fa667400700 1 -- 172.21.15.79:0/1946404818 learned_addr learned my addr 172.21.15.79:0/1946404818 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:48.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.191+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa668119f80 con 0x7fa66810dee0 2024-06-03T02:26:48.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.191+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa668119e00 con 0x7fa668109490 2024-06-03T02:26:48.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.191+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa66811a100 con 0x7fa668109e50 2024-06-03T02:26:48.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.191+0000 7fa66ce00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66810dee0 0x7fa6681ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.192+0000 7fa667400700 1 -- 172.21.15.79:0/1946404818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa668109490 msgr2=0x7fa6681a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:48.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.192+0000 7fa667400700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa668109490 0x7fa6681a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.192+0000 7fa667400700 1 -- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66810dee0 msgr2=0x7fa6681ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:48.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.192+0000 7fa667400700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66810dee0 0x7fa6681ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.192+0000 7fa667400700 1 -- 172.21.15.79:0/1946404818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66819ffd0 con 0x7fa668109e50 2024-06-03T02:26:48.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.192+0000 7fa667400700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa668109e50 0x7fa6681a2720 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa65800b550 tx=0x7fa65800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:48.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.193+0000 7fa664c00700 1 -- 172.21.15.79:0/1946404818 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658015020 con 0x7fa668109e50 2024-06-03T02:26:48.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.193+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6681a02c0 con 0x7fa668109e50 2024-06-03T02:26:48.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.193+0000 7fa664c00700 1 -- 172.21.15.79:0/1946404818 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa658011d10 con 0x7fa668109e50 2024-06-03T02:26:48.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.193+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6681a0850 con 0x7fa668109e50 2024-06-03T02:26:48.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.193+0000 7fa664c00700 1 -- 172.21.15.79:0/1946404818 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6580193b0 con 0x7fa668109e50 2024-06-03T02:26:48.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.205+0000 7fa679850b80 1 Processor -- start 2024-06-03T02:26:48.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.205+0000 7fa664c00700 1 -- 172.21.15.79:0/1946404818 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa658018020 con 0x7fa668109e50 2024-06-03T02:26:48.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.208+0000 7fa664c00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa64c0a2ca0 0x7fa64c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.208+0000 7fa66ce00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa64c0a2ca0 0x7fa64c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.209+0000 7fa66ce00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa64c0a2ca0 0x7fa64c0a5160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa66810eb20 tx=0x7fa660009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:48.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.210+0000 7fa664c00700 1 -- 172.21.15.79:0/1946404818 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa658098ff0 con 0x7fa668109e50 2024-06-03T02:26:48.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.210+0000 7fa679850b80 1 -- start start 2024-06-03T02:26:48.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.211+0000 7fa679850b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 0x561e5ea97f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.211+0000 7fa679850b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea91d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.211+0000 7fa679850b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea92240 0x561e5ea92680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.211+0000 7fa679850b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561e5e9a0a10 con 0x561e5ea92240 2024-06-03T02:26:48.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.211+0000 7fa679850b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e5e9e3b00 con 0x561e5ea984b0 2024-06-03T02:26:48.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.211+0000 7fa679850b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e5e99e8f0 con 0x561e5ea97b90 2024-06-03T02:26:48.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.211+0000 7fa667400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea91d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.211+0000 7fa667400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea91d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38578/0 (socket says 172.21.15.79:38578) 2024-06-03T02:26:48.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.211+0000 7fa667400700 1 -- 172.21.15.79:0/1731371956 learned_addr learned my addr 172.21.15.79:0/1731371956 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:48.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.212+0000 7fa667400700 1 -- 172.21.15.79:0/1731371956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 msgr2=0x561e5ea97f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:26:48.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.212+0000 7fa667400700 1 --2- 172.21.15.79:0/1731371956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 0x561e5ea97f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.212+0000 7fa667400700 1 -- 172.21.15.79:0/1731371956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea92240 msgr2=0x561e5ea92680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:48.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.212+0000 7fa667400700 1 --2- 172.21.15.79:0/1731371956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea92240 0x561e5ea92680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.212+0000 7fa667400700 1 -- 172.21.15.79:0/1731371956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5e98bf90 con 0x561e5ea984b0 2024-06-03T02:26:48.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.212+0000 7fa667400700 1 --2- 172.21.15.79:0/1731371956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea91d00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa65801aae0 tx=0x7fa6580a2610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b7fb35e4bb59fdd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:48.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.212+0000 7fa65d600700 1 -- 172.21.15.79:0/1731371956 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658097980 con 0x561e5ea984b0 2024-06-03T02:26:48.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.213+0000 7fa65d600700 1 -- 172.21.15.79:0/1731371956 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65809b650 con 0x561e5ea984b0 2024-06-03T02:26:48.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.213+0000 7fa65d600700 1 -- 172.21.15.79:0/1731371956 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658097b20 con 0x561e5ea984b0 2024-06-03T02:26:48.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.214+0000 7fa679850b80 1 -- 172.21.15.79:0/1731371956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 msgr2=0x561e5ea91d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:48.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.214+0000 7fa679850b80 1 --2- 172.21.15.79:0/1731371956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea91d00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa65801aae0 tx=0x7fa6580a2610 comp rx=0 tx=0).stop 2024-06-03T02:26:48.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.214+0000 7fa679850b80 1 -- 172.21.15.79:0/1731371956 shutdown_connections 2024-06-03T02:26:48.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.214+0000 7fa679850b80 1 --2- 172.21.15.79:0/1731371956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea92240 0x561e5ea92680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.214+0000 7fa679850b80 1 --2- 172.21.15.79:0/1731371956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 0x561e5ea97f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.214+0000 7fa679850b80 1 --2- 172.21.15.79:0/1731371956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea91d00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.214+0000 7fa679850b80 1 -- 172.21.15.79:0/1731371956 >> 172.21.15.79:0/1731371956 conn(0x561e5e9be170 msgr2=0x561e5ea8cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:48.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.214+0000 7fa679850b80 1 -- 172.21.15.79:0/1731371956 shutdown_connections 2024-06-03T02:26:48.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.214+0000 7fa679850b80 1 -- 172.21.15.79:0/1731371956 wait complete. 2024-06-03T02:26:48.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.215+0000 7fa679850b80 1 -- start start 2024-06-03T02:26:48.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.215+0000 7fa679850b80 1 Processor -- start 2024-06-03T02:26:48.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.218+0000 7fa679850b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea92240 0x561e5ea9e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.220+0000 7fa667e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea92240 0x561e5ea9e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.220+0000 7fa667e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea92240 0x561e5ea9e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38588/0 (socket says 172.21.15.79:38588) 2024-06-03T02:26:48.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.220+0000 7fa667e00700 1 -- 172.21.15.79:0/1577155760 learned_addr learned my addr 172.21.15.79:0/1577155760 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:48.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.220+0000 7fa679850b80 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 0x561e5ea9b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.220+0000 7fa679850b80 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea984b0 0x561e5ea9ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.220+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561e5e97ff60 con 0x561e5ea984b0 2024-06-03T02:26:48.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.220+0000 7fa667400700 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 0x561e5ea9b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.221+0000 7fa66ce00700 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea984b0 0x561e5ea9ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.221+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e5e9dfac0 con 0x561e5ea92240 2024-06-03T02:26:48.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.221+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e5ea9ee00 con 0x561e5ea97b90 2024-06-03T02:26:48.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.223+0000 7fa679850b80 10 client.0 _ll_register_callbacks cb 0x561e5e979380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:26:48.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.223+0000 7fa667400700 1 -- 172.21.15.79:0/1577155760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea92240 msgr2=0x561e5ea9e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:48.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.223+0000 7fa667400700 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea92240 0x561e5ea9e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.223+0000 7fa667400700 1 -- 172.21.15.79:0/1577155760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea984b0 msgr2=0x561e5ea9ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:48.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.223+0000 7fa667400700 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea984b0 0x561e5ea9ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.223+0000 7fa667400700 1 -- 172.21.15.79:0/1577155760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5e9328d0 con 0x561e5ea97b90 2024-06-03T02:26:48.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.223+0000 7fa667e00700 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea92240 0x561e5ea9e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:26:48.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.225+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65809b650 con 0x561e5ea97b90 2024-06-03T02:26:48.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.225+0000 7fa66ce00700 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea984b0 0x561e5ea9ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:26:48.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.225+0000 7fa667400700 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 0x561e5ea9b530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa658097e60 tx=0x7fa6580a3bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923c7858371274a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:48.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.225+0000 7fa652000700 10 client.26095 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:26:48.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.226+0000 7fa652000700 1 -- 172.21.15.79:0/1577155760 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658097cc0 con 0x561e5ea97b90 2024-06-03T02:26:48.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.226+0000 7fa652000700 1 -- 172.21.15.79:0/1577155760 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa6580b9b90 con 0x561e5ea97b90 2024-06-03T02:26:48.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.226+0000 7fa652000700 1 -- 172.21.15.79:0/1577155760 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa658097e90 con 0x561e5ea97b90 2024-06-03T02:26:48.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.226+0000 7fa679850b80 10 client.26095 fetch_fsmap learned FSMap version 30 2024-06-03T02:26:48.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.226+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=30}) -- 0x561e5eaba450 con 0x561e5ea97b90 2024-06-03T02:26:48.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.227+0000 7fa652000700 1 -- 172.21.15.79:0/1577155760 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa65809a3a0 con 0x561e5ea97b90 2024-06-03T02:26:48.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.227+0000 7fa679850b80 10 client.26095 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:26:48.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.227+0000 7fa679850b80 10 client.26095 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:26:48.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.227+0000 7fa679850b80 20 client.26095 populate_metadata read hostname 'smithi079' 2024-06-03T02:26:48.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.227+0000 7fa679850b80 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x561e5eabc920 0x561e5eabed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.227+0000 7fa679850b80 4 client.26095 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:26:48.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.227+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e5e9f1000 con 0x561e5eabc920 2024-06-03T02:26:48.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.227+0000 7fa667e00700 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x561e5eabc920 0x561e5eabed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.230+0000 7fa667e00700 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x561e5eabc920 0x561e5eabed10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=245d25f12d57f343 server_cookie=5c6a007cf4e58086 in_seq=0 out_seq=0 2024-06-03T02:26:48.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.230+0000 7fa652000700 10 client.26095 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:26:48.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.231+0000 7fa652000700 1 -- 172.21.15.79:0/1577155760 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e5e9f1000 con 0x561e5eabc920 2024-06-03T02:26:48.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.231+0000 7fa652000700 10 client.26095 handle_command_reply: tid=0 2024-06-03T02:26:48.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.231+0000 7fa679850b80 1 client.26095 shutdown 2024-06-03T02:26:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 msgr2=0x561e5ea9b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa679850b80 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 0x561e5ea9b530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa658097e60 tx=0x7fa6580a3bc0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 shutdown_connections 2024-06-03T02:26:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa679850b80 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x561e5eabc920 0x561e5eabed10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa652000700 0 client.26095 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:26:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa679850b80 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea984b0 0x561e5ea9ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa679850b80 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea92240 0x561e5ea9e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa679850b80 1 --2- 172.21.15.79:0/1577155760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea97b90 0x561e5ea9b530 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 >> 172.21.15.79:0/1577155760 conn(0x561e5e9be170 msgr2=0x561e5ea86790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:48.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.233+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 shutdown_connections 2024-06-03T02:26:48.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.236+0000 7fa679850b80 1 -- 172.21.15.79:0/1577155760 wait complete. 2024-06-03T02:26:48.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.236+0000 7fa679850b80 20 client.26095 trim_cache size 0 max 16384 2024-06-03T02:26:48.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.270+0000 7fa679850b80 1 Processor -- start 2024-06-03T02:26:48.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.270+0000 7fa679850b80 1 -- start start 2024-06-03T02:26:48.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.271+0000 7fa679850b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea92240 0x561e5ea91d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.271+0000 7fa679850b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea97b90 0x561e5ea8d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.271+0000 7fa679850b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea87130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.271+0000 7fa679850b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa64800eb40 con 0x561e5ea97b90 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.271+0000 7fa679850b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa648010680 con 0x561e5ea984b0 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.271+0000 7fa679850b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa64800f210 con 0x561e5ea92240 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.272+0000 7fa667e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea92240 0x561e5ea91d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.272+0000 7fa667e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea92240 0x561e5ea91d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58984/0 (socket says 172.21.15.79:58984) 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.272+0000 7fa667e00700 1 -- 172.21.15.79:0/4072096737 learned_addr learned my addr 172.21.15.79:0/4072096737 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.272+0000 7fa667400700 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea97b90 0x561e5ea8d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.272+0000 7fa66ce00700 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea87130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.272+0000 7fa667e00700 1 -- 172.21.15.79:0/4072096737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 msgr2=0x561e5ea87130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.273+0000 7fa667e00700 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea87130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.273+0000 7fa667e00700 1 -- 172.21.15.79:0/4072096737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea97b90 msgr2=0x561e5ea8d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.273+0000 7fa667e00700 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea97b90 0x561e5ea8d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.273+0000 7fa667e00700 1 -- 172.21.15.79:0/4072096737 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5ea9b300 con 0x561e5ea92240 2024-06-03T02:26:48.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.273+0000 7fa66ce00700 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea87130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:26:48.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.273+0000 7fa667e00700 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea92240 0x561e5ea91d00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa654006010 tx=0x7fa65401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30eacf15cbd8d3d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:48.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.274+0000 7fa65cc00700 1 -- 172.21.15.79:0/4072096737 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654028020 con 0x561e5ea92240 2024-06-03T02:26:48.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.275+0000 7fa65cc00700 1 -- 172.21.15.79:0/4072096737 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6540098c0 con 0x561e5ea92240 2024-06-03T02:26:48.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.275+0000 7fa65cc00700 1 -- 172.21.15.79:0/4072096737 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654003680 con 0x561e5ea92240 2024-06-03T02:26:48.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.276+0000 7fa679850b80 1 -- 172.21.15.79:0/4072096737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea92240 msgr2=0x561e5ea91d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:48.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.276+0000 7fa679850b80 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea92240 0x561e5ea91d00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa654006010 tx=0x7fa65401a040 comp rx=0 tx=0).stop 2024-06-03T02:26:48.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.277+0000 7fa679850b80 1 -- 172.21.15.79:0/4072096737 shutdown_connections 2024-06-03T02:26:48.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.277+0000 7fa679850b80 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea984b0 0x561e5ea87130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.277+0000 7fa679850b80 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea92240 0x561e5ea91d00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.277+0000 7fa679850b80 1 --2- 172.21.15.79:0/4072096737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5ea97b90 0x561e5ea8d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.277+0000 7fa679850b80 1 -- 172.21.15.79:0/4072096737 >> 172.21.15.79:0/4072096737 conn(0x561e5e9be170 msgr2=0x561e5ea8ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:48.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.277+0000 7fa679850b80 1 -- 172.21.15.79:0/4072096737 shutdown_connections 2024-06-03T02:26:48.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.277+0000 7fa679850b80 1 -- 172.21.15.79:0/4072096737 wait complete. 2024-06-03T02:26:48.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.278+0000 7fa679850b80 1 -- start start 2024-06-03T02:26:48.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.278+0000 7fa679850b80 1 Processor -- start 2024-06-03T02:26:48.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.279+0000 7fa679850b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea97b90 0x561e5eabed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.280+0000 7fa679850b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea984b0 0x561e5eabf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.280+0000 7fa679850b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5eabc920 0x561e5eabf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.280+0000 7fa667e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea97b90 0x561e5eabed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.280+0000 7fa667e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea97b90 0x561e5eabed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38598/0 (socket says 172.21.15.79:38598) 2024-06-03T02:26:48.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.280+0000 7fa667e00700 1 -- 172.21.15.79:0/3609901177 learned_addr learned my addr 172.21.15.79:0/3609901177 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:48.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.281+0000 7fa66ce00700 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5eabc920 0x561e5eabf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.280+0000 7fa679850b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa64800ffb0 con 0x561e5eabc920 2024-06-03T02:26:48.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.282+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa64800f8e0 con 0x561e5ea97b90 2024-06-03T02:26:48.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.282+0000 7fa667400700 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea984b0 0x561e5eabf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.282+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa64800cae0 con 0x561e5ea984b0 2024-06-03T02:26:48.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.282+0000 7fa679850b80 10 client.0 _ll_register_callbacks cb 0x561e5e979380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:26:48.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.283+0000 7fa667400700 1 -- 172.21.15.79:0/3609901177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea97b90 msgr2=0x561e5eabed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:48.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.283+0000 7fa667400700 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea97b90 0x561e5eabed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.283+0000 7fa667400700 1 -- 172.21.15.79:0/3609901177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5eabc920 msgr2=0x561e5eabf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:48.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.283+0000 7fa667400700 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5eabc920 0x561e5eabf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.283+0000 7fa667400700 1 -- 172.21.15.79:0/3609901177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5eb254b0 con 0x561e5ea984b0 2024-06-03T02:26:48.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.284+0000 7fa667e00700 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea97b90 0x561e5eabed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:26:48.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.284+0000 7fa667400700 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea984b0 0x561e5eabf250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa6580981d0 tx=0x7fa6580a0420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea96d9b681071505 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:48.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.284+0000 7fa652000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:26:48.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.284+0000 7fa652000700 1 -- 172.21.15.79:0/3609901177 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658098030 con 0x561e5ea984b0 2024-06-03T02:26:48.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.284+0000 7fa652000700 1 -- 172.21.15.79:0/3609901177 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa6580a0490 con 0x561e5ea984b0 2024-06-03T02:26:48.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.285+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6540098c0 con 0x561e5ea984b0 2024-06-03T02:26:48.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.285+0000 7fa652000700 1 -- 172.21.15.79:0/3609901177 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa658098200 con 0x561e5ea984b0 2024-06-03T02:26:48.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.285+0000 7fa679850b80 10 client.26101 fetch_fsmap learned FSMap version 30 2024-06-03T02:26:48.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.285+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e5eb267b0 con 0x561e5ea984b0 2024-06-03T02:26:48.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.287+0000 7fa652000700 1 -- 172.21.15.79:0/3609901177 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6540098c0 con 0x561e5ea984b0 2024-06-03T02:26:48.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.287+0000 7fa652000700 1 -- 172.21.15.79:0/3609901177 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa6580af080 con 0x561e5ea984b0 2024-06-03T02:26:48.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.287+0000 7fa679850b80 10 client.26101 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:26:48.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.287+0000 7fa679850b80 10 client.26101 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:26:48.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.287+0000 7fa679850b80 20 client.26101 populate_metadata read hostname 'smithi079' 2024-06-03T02:26:48.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.287+0000 7fa679850b80 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x561e5eb26a80 0x561e5eb28e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:48.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.288+0000 7fa679850b80 4 client.26101 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:26:48.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.288+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e5e9eb580 con 0x561e5eb26a80 2024-06-03T02:26:48.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.288+0000 7fa667e00700 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x561e5eb26a80 0x561e5eb28e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:48.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.289+0000 7fa667e00700 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x561e5eb26a80 0x561e5eb28e70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=150b5be2654dce36 server_cookie=ceccdc581bfc1e0d in_seq=0 out_seq=0 2024-06-03T02:26:48.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.289+0000 7fa652000700 10 client.26101 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:26:48.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.289+0000 7fa652000700 1 -- 172.21.15.79:0/3609901177 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e5e9eb580 con 0x561e5eb26a80 2024-06-03T02:26:48.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.289+0000 7fa652000700 10 client.26101 handle_command_reply: tid=0 2024-06-03T02:26:48.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.289+0000 7fa679850b80 1 client.26101 shutdown 2024-06-03T02:26:48.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.290+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea984b0 msgr2=0x561e5eabf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:48.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.290+0000 7fa679850b80 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea984b0 0x561e5eabf250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa6580981d0 tx=0x7fa6580a0420 comp rx=0 tx=0).stop 2024-06-03T02:26:48.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.290+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 shutdown_connections 2024-06-03T02:26:48.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.290+0000 7fa679850b80 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x561e5eb26a80 0x561e5eb28e70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.290+0000 7fa679850b80 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e5eabc920 0x561e5eabf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.290+0000 7fa679850b80 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5ea97b90 0x561e5eabed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.290+0000 7fa652000700 0 client.26101 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:26:48.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.290+0000 7fa679850b80 1 --2- 172.21.15.79:0/3609901177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5ea984b0 0x561e5eabf250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.290+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 >> 172.21.15.79:0/3609901177 conn(0x561e5e9be170 msgr2=0x561e5ea98950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:48.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.291+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 shutdown_connections 2024-06-03T02:26:48.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.291+0000 7fa679850b80 1 -- 172.21.15.79:0/3609901177 wait complete. 2024-06-03T02:26:48.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.292+0000 7fa679850b80 20 client.26101 trim_cache size 0 max 16384 2024-06-03T02:26:48.294 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:26:48.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa64c0a2ca0 msgr2=0x7fa64c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:48.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa64c0a2ca0 0x7fa64c0a5160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa66810eb20 tx=0x7fa660009040 comp rx=0 tx=0).stop 2024-06-03T02:26:48.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa668109e50 msgr2=0x7fa6681a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:48.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa668109e50 0x7fa6681a2720 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa65800b550 tx=0x7fa65800b910 comp rx=0 tx=0).stop 2024-06-03T02:26:48.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 shutdown_connections 2024-06-03T02:26:48.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa64c0a2ca0 0x7fa64c0a5160 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66810dee0 0x7fa6681ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa668109490 0x7fa6681a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 --2- 172.21.15.79:0/1946404818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa668109e50 0x7fa6681a2720 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:48.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.294+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 >> 172.21.15.79:0/1946404818 conn(0x7fa66806d9e0 msgr2=0x7fa668071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:48.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.295+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 shutdown_connections 2024-06-03T02:26:48.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:48.295+0000 7fa66ee00700 1 -- 172.21.15.79:0/1946404818 wait complete. 2024-06-03T02:26:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 400.0 s, 374.0 tps, lat 200.912 ms stddev 265.833 2024-06-03T02:26:48.591 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:48 smithi079 ceph-mon[31819]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 506 op/s 2024-06-03T02:26:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:48 smithi135 ceph-mon[36653]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 506 op/s 2024-06-03T02:26:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:48 smithi123 ceph-mon[36640]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 506 op/s 2024-06-03T02:26:49.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:26:49.945 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:26:50.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:50 smithi079 ceph-mon[31819]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 675 op/s 2024-06-03T02:26:50.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:50 smithi135 ceph-mon[36653]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 675 op/s 2024-06-03T02:26:50.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:50 smithi123 ceph-mon[36640]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 675 op/s 2024-06-03T02:26:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:52 smithi135 ceph-mon[36653]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 833 op/s 2024-06-03T02:26:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:52 smithi123 ceph-mon[36640]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 833 op/s 2024-06-03T02:26:52.936 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:52 smithi079 ceph-mon[31819]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 833 op/s 2024-06-03T02:26:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 405.0 s, 479.0 tps, lat 260.223 ms stddev 467.550 2024-06-03T02:26:54.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:54 smithi079 ceph-mon[31819]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-06-03T02:26:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:54 smithi135 ceph-mon[36653]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-06-03T02:26:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:54 smithi123 ceph-mon[36640]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-06-03T02:26:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.162+0000 7fdc1ce00700 1 -- 172.21.15.79:0/98451028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc1806a170 msgr2=0x7fdc1806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.162+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/98451028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc1806a170 0x7fdc1806a5f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009990 tx=0x7fdc04009ca0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.162+0000 7fdc1ce00700 1 -- 172.21.15.79:0/98451028 shutdown_connections 2024-06-03T02:26:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.162+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/98451028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1806ab30 0x7fdc1810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.162+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/98451028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc18108500 0x7fdc181088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.162+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/98451028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc1806a170 0x7fdc1806a5f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.162+0000 7fdc1ce00700 1 -- 172.21.15.79:0/98451028 >> 172.21.15.79:0/98451028 conn(0x7fdc18078340 msgr2=0x7fdc18078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.162+0000 7fdc1ce00700 1 -- 172.21.15.79:0/98451028 shutdown_connections 2024-06-03T02:26:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.162+0000 7fdc1ce00700 1 -- 172.21.15.79:0/98451028 wait complete. 2024-06-03T02:26:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.164+0000 7fdc1ce00700 1 Processor -- start 2024-06-03T02:26:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.164+0000 7fdc1ce00700 1 -- start start 2024-06-03T02:26:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.164+0000 7fdc1ce00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc1806a170 0x7fdc18101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.164+0000 7fdc1ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1806ab30 0x7fdc181019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.164+0000 7fdc1ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc18108500 0x7fdc18101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.164+0000 7fdc1ce00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc18112760 con 0x7fdc18108500 2024-06-03T02:26:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.164+0000 7fdc1ce00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181125e0 con 0x7fdc1806ab30 2024-06-03T02:26:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.164+0000 7fdc1ce00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181128e0 con 0x7fdc1806a170 2024-06-03T02:26:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1806ab30 0x7fdc181019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc16800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc18108500 0x7fdc18101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc1806a170 0x7fdc18101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1806ab30 0x7fdc181019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35074/0 (socket says 172.21.15.79:35074) 2024-06-03T02:26:55.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15400700 1 -- 172.21.15.79:0/887003065 learned_addr learned my addr 172.21.15.79:0/887003065 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:55.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15e00700 1 -- 172.21.15.79:0/887003065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1806ab30 msgr2=0x7fdc181019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:55.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15e00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1806ab30 0x7fdc181019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15e00700 1 -- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc18108500 msgr2=0x7fdc18101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:55.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15e00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc18108500 0x7fdc18101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15e00700 1 -- 172.21.15.79:0/887003065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc181a7140 con 0x7fdc1806a170 2024-06-03T02:26:55.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15400700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1806ab30 0x7fdc181019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc16800700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc18108500 0x7fdc18101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.165+0000 7fdc15e00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc1806a170 0x7fdc18101460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e9f0 tx=0x7fdc0800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.166+0000 7fdc0ea00700 1 -- 172.21.15.79:0/887003065 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0800cb70 con 0x7fdc1806a170 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.166+0000 7fdc1ce00700 1 -- 172.21.15.79:0/887003065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc181a7430 con 0x7fdc1806a170 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.166+0000 7fdc0ea00700 1 -- 172.21.15.79:0/887003065 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0800ee90 con 0x7fdc1806a170 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.166+0000 7fdc0ea00700 1 -- 172.21.15.79:0/887003065 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08003dc0 con 0x7fdc1806a170 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.166+0000 7fdc1ce00700 1 -- 172.21.15.79:0/887003065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc181a7970 con 0x7fdc1806a170 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.169+0000 7fdc0ea00700 1 -- 172.21.15.79:0/887003065 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdc08003f60 con 0x7fdc1806a170 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.170+0000 7fdc0ea00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdbfc0a2c20 0x7fdbfc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.170+0000 7fdc27745b80 1 Processor -- start 2024-06-03T02:26:55.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.170+0000 7fdc0ea00700 1 -- 172.21.15.79:0/887003065 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fdc080cf560 con 0x7fdc1806a170 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.170+0000 7fdc27745b80 1 -- start start 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.170+0000 7fdc15400700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdbfc0a2c20 0x7fdbfc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 0x55b24bddb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bddbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc27745b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b24bce1a10 con 0x55b24bddba50 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc27745b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b24bd24b00 con 0x55b24bde13a0 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc27745b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b24bcdf8f0 con 0x55b24bde1cc0 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35088/0 (socket says 172.21.15.79:35088) 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15e00700 1 -- 172.21.15.79:0/3197225275 learned_addr learned my addr 172.21.15.79:0/3197225275 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15400700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdbfc0a2c20 0x7fdbfc0a50e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc18078940 tx=0x7fdc04009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15400700 1 --2- 172.21.15.79:0/3197225275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 0x55b24bddb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc16800700 1 --2- 172.21.15.79:0/3197225275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bddbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15e00700 1 -- 172.21.15.79:0/3197225275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 msgr2=0x55b24bddb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15e00700 1 --2- 172.21.15.79:0/3197225275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 0x55b24bddb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15e00700 1 -- 172.21.15.79:0/3197225275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 msgr2=0x55b24bddbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15e00700 1 --2- 172.21.15.79:0/3197225275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bddbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.171+0000 7fdc15e00700 1 -- 172.21.15.79:0/3197225275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b24bcccf90 con 0x55b24bde13a0 2024-06-03T02:26:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.172+0000 7fdc15e00700 1 --2- 172.21.15.79:0/3197225275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde1780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc08098380 tx=0x7fdc08003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c91509ab640aecd3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.172+0000 7fdc03400700 1 -- 172.21.15.79:0/3197225275 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0809b9b0 con 0x55b24bde13a0 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.172+0000 7fdc03400700 1 -- 172.21.15.79:0/3197225275 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0809fd60 con 0x55b24bde13a0 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.172+0000 7fdc03400700 1 -- 172.21.15.79:0/3197225275 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc080cf230 con 0x55b24bde13a0 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.172+0000 7fdc27745b80 1 -- 172.21.15.79:0/3197225275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 msgr2=0x55b24bde1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.172+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3197225275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde1780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc08098380 tx=0x7fdc08003680 comp rx=0 tx=0).stop 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.173+0000 7fdc27745b80 1 -- 172.21.15.79:0/3197225275 shutdown_connections 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.173+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3197225275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bddbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.173+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3197225275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde1780 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.173+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3197225275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 0x55b24bddb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.173+0000 7fdc27745b80 1 -- 172.21.15.79:0/3197225275 >> 172.21.15.79:0/3197225275 conn(0x55b24bcff170 msgr2=0x55b24bdcdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.173+0000 7fdc27745b80 1 -- 172.21.15.79:0/3197225275 shutdown_connections 2024-06-03T02:26:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.173+0000 7fdc27745b80 1 -- 172.21.15.79:0/3197225275 wait complete. 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.173+0000 7fdc27745b80 1 -- start start 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.173+0000 7fdc27745b80 1 Processor -- start 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bddba50 0x55b24bde7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde1cc0 0x55b24bde52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc27745b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b24bcc0f60 con 0x55b24bde1cc0 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc27745b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b24bd20ac0 con 0x55b24bde13a0 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc27745b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b24bde86a0 con 0x55b24bddba50 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc27745b80 10 client.0 _ll_register_callbacks cb 0x55b24bcba380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc15400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc15400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35098/0 (socket says 172.21.15.79:35098) 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc15400700 1 -- 172.21.15.79:0/3296963084 learned_addr learned my addr 172.21.15.79:0/3296963084 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc15e00700 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bddba50 0x55b24bde7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.174+0000 7fdc16800700 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde1cc0 0x55b24bde52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc15400700 1 -- 172.21.15.79:0/3296963084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bddba50 msgr2=0x55b24bde7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc15400700 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bddba50 0x55b24bde7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc15400700 1 -- 172.21.15.79:0/3296963084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde1cc0 msgr2=0x55b24bde52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc15400700 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde1cc0 0x55b24bde52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc15400700 1 -- 172.21.15.79:0/3296963084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b24bc738d0 con 0x55b24bde13a0 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc16800700 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde1cc0 0x55b24bde52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc15400700 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde4d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc040098e0 tx=0x7fdc04009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5cff60e790a59f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdbf7e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3296963084 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04034060 con 0x55b24bde13a0 2024-06-03T02:26:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc27745b80 1 -- 172.21.15.79:0/3296963084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc0809fd60 con 0x55b24bde13a0 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3296963084 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fdc0400f870 con 0x55b24bde13a0 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3296963084 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc0403b050 con 0x55b24bde13a0 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc27745b80 10 client.26120 fetch_fsmap learned FSMap version 30 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.175+0000 7fdc27745b80 1 -- 172.21.15.79:0/3296963084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b24bdf2e80 con 0x55b24bde13a0 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.176+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3296963084 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0809fd60 con 0x55b24bde13a0 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.176+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3296963084 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdc0402fb90 con 0x55b24bde13a0 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.176+0000 7fdc27745b80 10 client.26120 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.176+0000 7fdc27745b80 10 client.26120 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.176+0000 7fdc27745b80 20 client.26120 populate_metadata read hostname 'smithi079' 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.176+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b24bdf5780 0x55b24bdf7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.176+0000 7fdc27745b80 4 client.26120 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.176+0000 7fdc27745b80 1 -- 172.21.15.79:0/3296963084 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b24bd32000 con 0x55b24bdf5780 2024-06-03T02:26:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.177+0000 7fdc15e00700 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b24bdf5780 0x55b24bdf7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.177+0000 7fdc15e00700 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b24bdf5780 0x55b24bdf7b70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76da3dae08a7a074 server_cookie=b67078dfca605914 in_seq=0 out_seq=0 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.177+0000 7fdbf7e00700 10 client.26120 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3296963084 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b24bd32000 con 0x55b24bdf5780 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdbf7e00700 10 client.26120 handle_command_reply: tid=0 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdc27745b80 1 client.26120 shutdown 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdc27745b80 1 -- 172.21.15.79:0/3296963084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 msgr2=0x55b24bde4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde4d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc040098e0 tx=0x7fdc04009ca0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdc27745b80 1 -- 172.21.15.79:0/3296963084 shutdown_connections 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b24bdf5780 0x55b24bdf7b70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde1cc0 0x55b24bde52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bddba50 0x55b24bde7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3296963084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde13a0 0x55b24bde4d90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.179+0000 7fdc27745b80 1 -- 172.21.15.79:0/3296963084 >> 172.21.15.79:0/3296963084 conn(0x55b24bcff170 msgr2=0x55b24bdd1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.180+0000 7fdbf7e00700 0 client.26120 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.183+0000 7fdc27745b80 1 -- 172.21.15.79:0/3296963084 shutdown_connections 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.186+0000 7fdc27745b80 1 -- 172.21.15.79:0/3296963084 wait complete. 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.186+0000 7fdc27745b80 20 client.26120 trim_cache size 0 max 16384 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.203+0000 7fdc27745b80 1 Processor -- start 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.203+0000 7fdc27745b80 1 -- start start 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.203+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bddba50 0x55b24bdc8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.203+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde13a0 0x55b24bdd2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.203+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 0x55b24bde75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.203+0000 7fdc27745b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181ac250 con 0x55b24bde13a0 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.203+0000 7fdc27745b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181add90 con 0x55b24bddba50 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.203+0000 7fdc27745b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181ac920 con 0x55b24bde1cc0 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde13a0 0x55b24bdd2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde13a0 0x55b24bdd2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56686/0 (socket says 172.21.15.79:56686) 2024-06-03T02:26:56.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15400700 1 -- 172.21.15.79:0/609212197 learned_addr learned my addr 172.21.15.79:0/609212197 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc16800700 1 --2- 172.21.15.79:0/609212197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 0x55b24bde75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15e00700 1 --2- 172.21.15.79:0/609212197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bddba50 0x55b24bdc8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15400700 1 -- 172.21.15.79:0/609212197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 msgr2=0x55b24bde75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15400700 1 --2- 172.21.15.79:0/609212197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 0x55b24bde75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15400700 1 -- 172.21.15.79:0/609212197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bddba50 msgr2=0x55b24bdc8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15400700 1 --2- 172.21.15.79:0/609212197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bddba50 0x55b24bdc8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15400700 1 -- 172.21.15.79:0/609212197 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b24bde1af0 con 0x55b24bde13a0 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.204+0000 7fdc15400700 1 --2- 172.21.15.79:0/609212197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde13a0 0x55b24bdd2400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc04006010 tx=0x7fdc04005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4fef53f6968c58 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc02a00700 1 -- 172.21.15.79:0/609212197 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04034070 con 0x55b24bde13a0 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc02a00700 1 -- 172.21.15.79:0/609212197 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0400f670 con 0x55b24bde13a0 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc02a00700 1 -- 172.21.15.79:0/609212197 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0402f6a0 con 0x55b24bde13a0 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc27745b80 1 -- 172.21.15.79:0/609212197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde13a0 msgr2=0x55b24bdd2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc27745b80 1 --2- 172.21.15.79:0/609212197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde13a0 0x55b24bdd2400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc04006010 tx=0x7fdc04005c40 comp rx=0 tx=0).stop 2024-06-03T02:26:56.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc27745b80 1 -- 172.21.15.79:0/609212197 shutdown_connections 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc27745b80 1 --2- 172.21.15.79:0/609212197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde1cc0 0x55b24bde75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc27745b80 1 --2- 172.21.15.79:0/609212197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bddba50 0x55b24bdc8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc27745b80 1 --2- 172.21.15.79:0/609212197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bde13a0 0x55b24bdd2400 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc27745b80 1 -- 172.21.15.79:0/609212197 >> 172.21.15.79:0/609212197 conn(0x55b24bcff170 msgr2=0x55b24bdd1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.205+0000 7fdc27745b80 1 -- 172.21.15.79:0/609212197 shutdown_connections 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.206+0000 7fdc27745b80 1 -- 172.21.15.79:0/609212197 wait complete. 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.206+0000 7fdc27745b80 1 -- start start 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.206+0000 7fdc27745b80 1 Processor -- start 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bdfa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde13a0 0x55b24bdfa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc27745b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde1cc0 0x55b24be58100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc27745b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181ad6c0 con 0x55b24bddba50 2024-06-03T02:26:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc27745b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181acff0 con 0x55b24bde1cc0 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc27745b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181aa1f0 con 0x55b24bde13a0 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc27745b80 10 client.0 _ll_register_callbacks cb 0x55b24bcba380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bdfa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc16800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde1cc0 0x55b24be58100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bdfa250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56696/0 (socket says 172.21.15.79:56696) 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15e00700 1 -- 172.21.15.79:0/3003434284 learned_addr learned my addr 172.21.15.79:0/3003434284 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15400700 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde13a0 0x55b24bdfa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15e00700 1 -- 172.21.15.79:0/3003434284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde13a0 msgr2=0x55b24bdfa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15e00700 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde13a0 0x55b24bdfa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15e00700 1 -- 172.21.15.79:0/3003434284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde1cc0 msgr2=0x55b24be58100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15e00700 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde1cc0 0x55b24be58100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15e00700 1 -- 172.21.15.79:0/3003434284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b24be586f0 con 0x55b24bddba50 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc16800700 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde1cc0 0x55b24be58100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:26:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdc15e00700 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bdfa250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc0809b940 tx=0x7fdc0809b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce456f5aa810e3bb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.207+0000 7fdbf7e00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3003434284 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0809fd60 con 0x55b24bddba50 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3003434284 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fdc0800cda0 con 0x55b24bddba50 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdc27745b80 1 -- 172.21.15.79:0/3003434284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc0400f670 con 0x55b24bddba50 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3003434284 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc08099cb0 con 0x55b24bddba50 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdc27745b80 10 client.15786 fetch_fsmap learned FSMap version 30 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdc27745b80 1 -- 172.21.15.79:0/3003434284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b24be5fa90 con 0x55b24bddba50 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3003434284 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0400f670 con 0x55b24bddba50 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3003434284 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdc0809cce0 con 0x55b24bddba50 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdc27745b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdc27745b80 10 client.15786 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdc27745b80 20 client.15786 populate_metadata read hostname 'smithi079' 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b24be5fd20 0x55b24be62110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdc27745b80 4 client.15786 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:26:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.208+0000 7fdc27745b80 1 -- 172.21.15.79:0/3003434284 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b24bd2c580 con 0x55b24be5fd20 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.209+0000 7fdc15400700 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b24be5fd20 0x55b24be62110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.209+0000 7fdc15400700 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b24be5fd20 0x55b24be62110 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=351734d8b8c10b7c server_cookie=ab65cf142467d4c8 in_seq=0 out_seq=0 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.209+0000 7fdbf7e00700 10 client.15786 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.209+0000 7fdbf7e00700 1 -- 172.21.15.79:0/3003434284 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b24bd2c580 con 0x55b24be5fd20 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.209+0000 7fdbf7e00700 10 client.15786 handle_command_reply: tid=0 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.209+0000 7fdc27745b80 1 client.15786 shutdown 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.209+0000 7fdc27745b80 1 -- 172.21.15.79:0/3003434284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 msgr2=0x55b24bdfa250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.209+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bdfa250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc0809b940 tx=0x7fdc0809b440 comp rx=0 tx=0).stop 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.209+0000 7fdc27745b80 1 -- 172.21.15.79:0/3003434284 shutdown_connections 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b24bde1cc0 0x55b24be58100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b24be5fd20 0x55b24be62110 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdbf7e00700 0 client.15786 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b24bddba50 0x55b24bdfa250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc27745b80 1 --2- 172.21.15.79:0/3003434284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b24bde13a0 0x55b24bdfa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc27745b80 1 -- 172.21.15.79:0/3003434284 >> 172.21.15.79:0/3003434284 conn(0x55b24bcff170 msgr2=0x55b24bdd2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:56.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc27745b80 1 -- 172.21.15.79:0/3003434284 shutdown_connections 2024-06-03T02:26:56.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc27745b80 1 -- 172.21.15.79:0/3003434284 wait complete. 2024-06-03T02:26:56.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc27745b80 20 client.15786 trim_cache size 0 max 16384 2024-06-03T02:26:56.006 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:26:56.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc1ce00700 1 -- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdbfc0a2c20 msgr2=0x7fdbfc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdbfc0a2c20 0x7fdbfc0a50e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc18078940 tx=0x7fdc04009570 comp rx=0 tx=0).stop 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc1ce00700 1 -- 172.21.15.79:0/887003065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc1806a170 msgr2=0x7fdc18101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.210+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc1806a170 0x7fdc18101460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e9f0 tx=0x7fdc0800ed00 comp rx=0 tx=0).stop 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.211+0000 7fdc1ce00700 1 -- 172.21.15.79:0/887003065 shutdown_connections 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.211+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fdbfc0a2c20 0x7fdbfc0a50e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.211+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc18108500 0x7fdc18101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.211+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc1806a170 0x7fdc18101460 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.211+0000 7fdc1ce00700 1 --2- 172.21.15.79:0/887003065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1806ab30 0x7fdc181019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.211+0000 7fdc1ce00700 1 -- 172.21.15.79:0/887003065 >> 172.21.15.79:0/887003065 conn(0x7fdc18078340 msgr2=0x7fdc180fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.211+0000 7fdc1ce00700 1 -- 172.21.15.79:0/887003065 shutdown_connections 2024-06-03T02:26:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:26:55.211+0000 7fdc1ce00700 1 -- 172.21.15.79:0/887003065 wait complete. 2024-06-03T02:26:56.323 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:56 smithi079 ceph-mon[31819]: pgmap v722: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-06-03T02:26:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:56 smithi135 ceph-mon[36653]: pgmap v722: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-06-03T02:26:56.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:56 smithi123 ceph-mon[36640]: pgmap v722: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-06-03T02:26:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 410.0 s, 495.4 tps, lat 207.606 ms stddev 316.304 2024-06-03T02:26:58.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:26:58.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:26:58 smithi079 ceph-mon[31819]: pgmap v723: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-06-03T02:26:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:26:58 smithi135 ceph-mon[36653]: pgmap v723: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-06-03T02:26:58.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:26:58 smithi123 ceph-mon[36640]: pgmap v723: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-06-03T02:26:59.384 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:27:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:00 smithi123 ceph-mon[36640]: pgmap v724: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-06-03T02:27:00.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:00 smithi079 ceph-mon[31819]: pgmap v724: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-06-03T02:27:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:00 smithi135 ceph-mon[36653]: pgmap v724: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-06-03T02:27:02.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:02 smithi123 ceph-mon[36640]: pgmap v725: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.0 MiB/s wr, 903 op/s 2024-06-03T02:27:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:02 smithi135 ceph-mon[36653]: pgmap v725: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.0 MiB/s wr, 903 op/s 2024-06-03T02:27:03.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:02 smithi079 ceph-mon[31819]: pgmap v725: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.0 MiB/s wr, 903 op/s 2024-06-03T02:27:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 415.0 s, 494.8 tps, lat 195.901 ms stddev 270.817 2024-06-03T02:27:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:04 smithi123 ceph-mon[36640]: pgmap v726: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.1 MiB/s wr, 788 op/s 2024-06-03T02:27:04.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:04 smithi079 ceph-mon[31819]: pgmap v726: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.1 MiB/s wr, 788 op/s 2024-06-03T02:27:04.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:04 smithi135 ceph-mon[36653]: pgmap v726: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.1 MiB/s wr, 788 op/s 2024-06-03T02:27:05.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.382+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2998198051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e0108500 msgr2=0x7fa5e01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:05.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.382+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2998198051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e0108500 0x7fa5e01088e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40098f0 tx=0x7fa5d4009c00 comp rx=0 tx=0).stop 2024-06-03T02:27:05.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.383+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2998198051 shutdown_connections 2024-06-03T02:27:05.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.383+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2998198051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e006ab30 0x7fa5e010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.383+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2998198051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e0108500 0x7fa5e01088e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.383+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2998198051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e006a170 0x7fa5e006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.383+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2998198051 >> 172.21.15.79:0/2998198051 conn(0x7fa5e0078340 msgr2=0x7fa5e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:05.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.383+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2998198051 shutdown_connections 2024-06-03T02:27:05.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.383+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2998198051 wait complete. 2024-06-03T02:27:05.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.385+0000 7fa5e7c00700 1 Processor -- start 2024-06-03T02:27:05.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.385+0000 7fa5e7c00700 1 -- start start 2024-06-03T02:27:05.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.385+0000 7fa5e7c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e006a170 0x7fa5e0117b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.385+0000 7fa5e7c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e006ab30 0x7fa5e0118050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.385+0000 7fa5e7c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e0108500 0x7fa5e0111c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.385+0000 7fa5e7c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e01105c0 con 0x7fa5e006ab30 2024-06-03T02:27:05.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5e7c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0110440 con 0x7fa5e006a170 2024-06-03T02:27:05.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5e7c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0110740 con 0x7fa5e0108500 2024-06-03T02:27:05.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5e5c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e0108500 0x7fa5e0111c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5e5c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e0108500 0x7fa5e0111c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53026/0 (socket says 172.21.15.79:53026) 2024-06-03T02:27:05.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5e5c00700 1 -- 172.21.15.79:0/2664362588 learned_addr learned my addr 172.21.15.79:0/2664362588 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:05.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e006ab30 0x7fa5e0118050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5e5200700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e006a170 0x7fa5e0117b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5e5c00700 1 -- 172.21.15.79:0/2664362588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e006a170 msgr2=0x7fa5e0117b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:05.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5e5c00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e006a170 0x7fa5e0117b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.386+0000 7fa5e5c00700 1 -- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e006ab30 msgr2=0x7fa5e0118050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:05.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.387+0000 7fa5e5c00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e006ab30 0x7fa5e0118050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.387+0000 7fa5e5c00700 1 -- 172.21.15.79:0/2664362588 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e0112540 con 0x7fa5e0108500 2024-06-03T02:27:05.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.387+0000 7fa5e5c00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e0108500 0x7fa5e0111c40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c110 tx=0x7fa5d800c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:05.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.387+0000 7fa5dd600700 1 -- 172.21.15.79:0/2664362588 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800ce50 con 0x7fa5e0108500 2024-06-03T02:27:05.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.387+0000 7fa5dd600700 1 -- 172.21.15.79:0/2664362588 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d8004bb0 con 0x7fa5e0108500 2024-06-03T02:27:05.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.387+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2664362588 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e01127d0 con 0x7fa5e0108500 2024-06-03T02:27:05.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.387+0000 7fa5dd600700 1 -- 172.21.15.79:0/2664362588 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800ce50 con 0x7fa5e0108500 2024-06-03T02:27:05.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.388+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2664362588 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e01b4640 con 0x7fa5e0108500 2024-06-03T02:27:05.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.389+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e006ab30 0x7fa5e0118050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:27:05.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.396+0000 7fa5dd600700 1 -- 172.21.15.79:0/2664362588 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa5d801a020 con 0x7fa5e0108500 2024-06-03T02:27:05.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.398+0000 7fa5dd600700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa5c80a2f00 0x7fa5c80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.398+0000 7fa5f25fbb80 1 Processor -- start 2024-06-03T02:27:05.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.398+0000 7fa5dd600700 1 -- 172.21.15.79:0/2664362588 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa5d80cea60 con 0x7fa5e0108500 2024-06-03T02:27:05.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.398+0000 7fa5f25fbb80 1 -- start start 2024-06-03T02:27:05.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.398+0000 7fa5e5200700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa5c80a2f00 0x7fa5c80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.399+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636541f2e40 0x563654276070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.399+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636542765b0 0x563654276a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.399+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563654281300 0x5636542836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.399+0000 7fa5f25fbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563654185a10 con 0x5636541f2e40 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.399+0000 7fa5f25fbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5636541c8b00 con 0x5636542765b0 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.399+0000 7fa5f25fbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5636541838f0 con 0x563654281300 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.399+0000 7fa5e5c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563654281300 0x5636542836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.399+0000 7fa5e5c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563654281300 0x5636542836f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53042/0 (socket says 172.21.15.79:53042) 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.400+0000 7fa5e5c00700 1 -- 172.21.15.79:0/2288685445 learned_addr learned my addr 172.21.15.79:0/2288685445 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.400+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/2288685445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636542765b0 0x563654276a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.400+0000 7fa5e5c00700 1 -- 172.21.15.79:0/2288685445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636542765b0 msgr2=0x563654276a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.400+0000 7fa5e5c00700 1 --2- 172.21.15.79:0/2288685445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636542765b0 0x563654276a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.400+0000 7fa5e5c00700 1 -- 172.21.15.79:0/2288685445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636541f2e40 msgr2=0x563654276070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:05.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.400+0000 7fa5e5c00700 1 --2- 172.21.15.79:0/2288685445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636541f2e40 0x563654276070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.400+0000 7fa5e5c00700 1 -- 172.21.15.79:0/2288685445 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563654170f90 con 0x563654281300 2024-06-03T02:27:05.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.400+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/2288685445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636542765b0 0x563654276a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:05.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.400+0000 7fa5e5c00700 1 --2- 172.21.15.79:0/2288685445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563654281300 0x5636542836f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa5d809abb0 tx=0x7fa5d809aa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf115b4bbcc44a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:05.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.404+0000 7fa5e5200700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa5c80a2f00 0x7fa5c80a53c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009680 tx=0x7fa5d4009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:05.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.404+0000 7fa5d2000700 1 -- 172.21.15.79:0/2288685445 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d809b5b0 con 0x563654281300 2024-06-03T02:27:05.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.404+0000 7fa5d2000700 1 -- 172.21.15.79:0/2288685445 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d809aeb0 con 0x563654281300 2024-06-03T02:27:05.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.405+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/2288685445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563654281300 msgr2=0x5636542836f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:05.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.405+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/2288685445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563654281300 0x5636542836f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa5d809abb0 tx=0x7fa5d809aa70 comp rx=0 tx=0).stop 2024-06-03T02:27:05.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.407+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/2288685445 shutdown_connections 2024-06-03T02:27:05.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.407+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/2288685445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563654281300 0x5636542836f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.407+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/2288685445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636541f2e40 0x563654276070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.407+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/2288685445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636542765b0 0x563654276a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.407+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/2288685445 >> 172.21.15.79:0/2288685445 conn(0x5636541a3170 msgr2=0x5636542724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:05.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.407+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/2288685445 shutdown_connections 2024-06-03T02:27:05.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.408+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/2288685445 wait complete. 2024-06-03T02:27:05.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.408+0000 7fa5f25fbb80 1 -- start start 2024-06-03T02:27:05.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.409+0000 7fa5f25fbb80 1 Processor -- start 2024-06-03T02:27:05.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.412+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x5636542809d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.412+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x56365427dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.412+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x56365427e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.412+0000 7fa5e5200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x5636542809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.412+0000 7fa5e5200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x5636542809d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40824/0 (socket says 172.21.15.79:40824) 2024-06-03T02:27:05.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x56365427dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5e5200700 1 -- 172.21.15.79:0/3352431054 learned_addr learned my addr 172.21.15.79:0/3352431054 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563654164f60 con 0x563654281300 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5636541c4ac0 con 0x5636541f2e40 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563654283f60 con 0x5636542765b0 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5f25fbb80 10 client.0 _ll_register_callbacks cb 0x563654162720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5e5c00700 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x56365427e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5e5200700 1 -- 172.21.15.79:0/3352431054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 msgr2=0x56365427dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5e5200700 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x56365427dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5e5200700 1 -- 172.21.15.79:0/3352431054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 msgr2=0x56365427e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5e5200700 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x56365427e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5e5200700 1 -- 172.21.15.79:0/3352431054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636541178d0 con 0x5636541f2e40 2024-06-03T02:27:05.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5e5c00700 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x56365427e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:05.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.413+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x56365427dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:27:05.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.414+0000 7fa5e5200700 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x5636542809d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4016000 tx=0x7fa5d4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee53a0dc2f80da31 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:05.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.414+0000 7fa5c6a00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:27:05.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.414+0000 7fa5c6a00700 1 -- 172.21.15.79:0/3352431054 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4032060 con 0x5636541f2e40 2024-06-03T02:27:05.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.414+0000 7fa5c6a00700 1 -- 172.21.15.79:0/3352431054 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa5d40046d0 con 0x5636541f2e40 2024-06-03T02:27:05.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.414+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5d809aeb0 con 0x5636541f2e40 2024-06-03T02:27:05.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.415+0000 7fa5c6a00700 1 -- 172.21.15.79:0/3352431054 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa5d4039050 con 0x5636541f2e40 2024-06-03T02:27:05.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.415+0000 7fa5f25fbb80 10 client.26135 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:05.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.415+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56365428e5a0 con 0x5636541f2e40 2024-06-03T02:27:05.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.417+0000 7fa5c6a00700 1 -- 172.21.15.79:0/3352431054 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d809aeb0 con 0x5636541f2e40 2024-06-03T02:27:05.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.417+0000 7fa5c6a00700 1 -- 172.21.15.79:0/3352431054 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa5d400bbf0 con 0x5636541f2e40 2024-06-03T02:27:05.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.417+0000 7fa5f25fbb80 10 client.26135 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:05.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.417+0000 7fa5f25fbb80 10 client.26135 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:27:05.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.417+0000 7fa5f25fbb80 20 client.26135 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:05.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.417+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563654290be0 0x563654292fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.417+0000 7fa5f25fbb80 4 client.26135 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:27:05.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.417+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636541d6000 con 0x563654290be0 2024-06-03T02:27:05.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.418+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563654290be0 0x563654292fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.420+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563654290be0 0x563654292fd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33fa36d51063912f server_cookie=5428c97b32e5cd20 in_seq=0 out_seq=0 2024-06-03T02:27:05.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.420+0000 7fa5c6a00700 10 client.26135 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:27:05.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.421+0000 7fa5c6a00700 1 -- 172.21.15.79:0/3352431054 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5636541d6000 con 0x563654290be0 2024-06-03T02:27:05.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.421+0000 7fa5c6a00700 10 client.26135 handle_command_reply: tid=0 2024-06-03T02:27:05.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.421+0000 7fa5f25fbb80 1 client.26135 shutdown 2024-06-03T02:27:05.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.423+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 msgr2=0x5636542809d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:05.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.423+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x5636542809d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4016000 tx=0x7fa5d4009260 comp rx=0 tx=0).stop 2024-06-03T02:27:05.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.423+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 shutdown_connections 2024-06-03T02:27:05.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.423+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x56365427e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.423+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563654290be0 0x563654292fd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.424+0000 7fa5c6a00700 0 client.26135 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:27:05.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.424+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x5636542809d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.424+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/3352431054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x56365427dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.424+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 >> 172.21.15.79:0/3352431054 conn(0x5636541a3170 msgr2=0x563654271740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:05.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.424+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 shutdown_connections 2024-06-03T02:27:05.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.424+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/3352431054 wait complete. 2024-06-03T02:27:05.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.425+0000 7fa5f25fbb80 20 client.26135 trim_cache size 0 max 16384 2024-06-03T02:27:05.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.465+0000 7fa5f25fbb80 1 Processor -- start 2024-06-03T02:27:05.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.465+0000 7fa5f25fbb80 1 -- start start 2024-06-03T02:27:05.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.466+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636541f2e40 0x56365427e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.466+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x5636542826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.466+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563654281300 0x563654270eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.466+0000 7fa5f25fbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e01b4c40 con 0x5636541f2e40 2024-06-03T02:27:05.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.466+0000 7fa5f25fbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e01b6780 con 0x563654281300 2024-06-03T02:27:05.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.466+0000 7fa5f25fbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e01b5310 con 0x5636542765b0 2024-06-03T02:27:05.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.466+0000 7fa5dfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x5636542826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.466+0000 7fa5dfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x5636542826f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53072/0 (socket says 172.21.15.79:53072) 2024-06-03T02:27:05.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.466+0000 7fa5dfe00700 1 -- 172.21.15.79:0/564440604 learned_addr learned my addr 172.21.15.79:0/564440604 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.467+0000 7fa5e5c00700 1 --2- 172.21.15.79:0/564440604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563654281300 0x563654270eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.467+0000 7fa5e5200700 1 --2- 172.21.15.79:0/564440604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636541f2e40 0x56365427e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.467+0000 7fa5dfe00700 1 -- 172.21.15.79:0/564440604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563654281300 msgr2=0x563654270eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.467+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/564440604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563654281300 0x563654270eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.467+0000 7fa5dfe00700 1 -- 172.21.15.79:0/564440604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636541f2e40 msgr2=0x56365427e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.467+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/564440604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636541f2e40 0x56365427e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.467+0000 7fa5dfe00700 1 -- 172.21.15.79:0/564440604 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563654277bd0 con 0x5636542765b0 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.467+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/564440604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x5636542826f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0113350 tx=0x7fa5cc00b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96cb0fa9ffb8eb5c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.467+0000 7fa5d1600700 1 -- 172.21.15.79:0/564440604 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc004da0 con 0x5636542765b0 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.468+0000 7fa5d1600700 1 -- 172.21.15.79:0/564440604 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5cc013b20 con 0x5636542765b0 2024-06-03T02:27:05.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.468+0000 7fa5d1600700 1 -- 172.21.15.79:0/564440604 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc01ba70 con 0x5636542765b0 2024-06-03T02:27:05.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.469+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/564440604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 msgr2=0x5636542826f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:05.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.469+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/564440604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x5636542826f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0113350 tx=0x7fa5cc00b520 comp rx=0 tx=0).stop 2024-06-03T02:27:05.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.470+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/564440604 shutdown_connections 2024-06-03T02:27:05.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.470+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/564440604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563654281300 0x563654270eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.470+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/564440604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636541f2e40 0x56365427e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.470+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/564440604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x5636542826f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.470+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/564440604 >> 172.21.15.79:0/564440604 conn(0x5636541a3170 msgr2=0x56365427c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:05.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.470+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/564440604 shutdown_connections 2024-06-03T02:27:05.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.470+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/564440604 wait complete. 2024-06-03T02:27:05.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.471+0000 7fa5f25fbb80 1 -- start start 2024-06-03T02:27:05.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.471+0000 7fa5f25fbb80 1 Processor -- start 2024-06-03T02:27:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.472+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x563654297880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.472+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x563654297dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5f25fbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x563654298300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5e5200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x563654297880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5e5200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x563654297880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40850/0 (socket says 172.21.15.79:40850) 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5dfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x563654297dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5dfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x563654297dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53078/0 (socket says 172.21.15.79:53078) 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5e5c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x563654298300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5e5c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x563654298300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:48554/0 (socket says 172.21.15.79:48554) 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5e5200700 1 -- 172.21.15.79:0/239013500 learned_addr learned my addr 172.21.15.79:0/239013500 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e01b60b0 con 0x563654281300 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e01b59e0 con 0x5636541f2e40 2024-06-03T02:27:05.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0072a80 con 0x5636542765b0 2024-06-03T02:27:05.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.473+0000 7fa5f25fbb80 10 client.0 _ll_register_callbacks cb 0x563654162720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:05.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.474+0000 7fa5dfe00700 1 -- 172.21.15.79:0/239013500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 msgr2=0x563654297880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:05.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.474+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x563654297880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.474+0000 7fa5dfe00700 1 -- 172.21.15.79:0/239013500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 msgr2=0x563654298300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:05.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.474+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x563654298300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.474+0000 7fa5dfe00700 1 -- 172.21.15.79:0/239013500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636542882f0 con 0x5636542765b0 2024-06-03T02:27:05.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.475+0000 7fa5dfe00700 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x563654297dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c010 tx=0x7fa5cc005060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3698693c45e91c4a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:05.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.476+0000 7fa5c6a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:27:05.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.476+0000 7fa5c6a00700 1 -- 172.21.15.79:0/239013500 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc032060 con 0x5636542765b0 2024-06-03T02:27:05.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.476+0000 7fa5c6a00700 1 -- 172.21.15.79:0/239013500 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa5cc013d50 con 0x5636542765b0 2024-06-03T02:27:05.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.476+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5cc013b20 con 0x5636542765b0 2024-06-03T02:27:05.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.476+0000 7fa5c6a00700 1 -- 172.21.15.79:0/239013500 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa5cc039050 con 0x5636542765b0 2024-06-03T02:27:05.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.476+0000 7fa5f25fbb80 10 client.26137 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:05.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.476+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636542895b0 con 0x5636542765b0 2024-06-03T02:27:05.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.478+0000 7fa5c6a00700 1 -- 172.21.15.79:0/239013500 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5cc013b20 con 0x5636542765b0 2024-06-03T02:27:05.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.478+0000 7fa5c6a00700 1 -- 172.21.15.79:0/239013500 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa5cc00f040 con 0x5636542765b0 2024-06-03T02:27:05.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.478+0000 7fa5f25fbb80 10 client.26137 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:05.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.478+0000 7fa5f25fbb80 10 client.26137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:27:05.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.478+0000 7fa5f25fbb80 20 client.26137 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:05.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.478+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563654290be0 0x5636542fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:05.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.479+0000 7fa5f25fbb80 4 client.26137 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:27:05.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.479+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636541d0580 con 0x563654290be0 2024-06-03T02:27:05.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.479+0000 7fa5e5200700 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563654290be0 0x5636542fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:05.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.480+0000 7fa5e5200700 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563654290be0 0x5636542fd870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f423b186d62d3296 server_cookie=1640f874adec094d in_seq=0 out_seq=0 2024-06-03T02:27:05.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.480+0000 7fa5c6a00700 10 client.26137 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.480+0000 7fa5c6a00700 1 -- 172.21.15.79:0/239013500 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636541d0580 con 0x563654290be0 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.480+0000 7fa5c6a00700 10 client.26137 handle_command_reply: tid=0 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.480+0000 7fa5f25fbb80 1 client.26137 shutdown 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.480+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 msgr2=0x563654297dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.480+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x563654297dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c010 tx=0x7fa5cc005060 comp rx=0 tx=0).stop 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.481+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 shutdown_connections 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.481+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563654281300 0x563654298300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.481+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563654290be0 0x5636542fd870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.481+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5636541f2e40 0x563654297880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.481+0000 7fa5f25fbb80 1 --2- 172.21.15.79:0/239013500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5636542765b0 0x563654297dc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.481+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 >> 172.21.15.79:0/239013500 conn(0x5636541a3170 msgr2=0x563654270eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:05.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.481+0000 7fa5c6a00700 0 client.26137 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:27:05.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.481+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 shutdown_connections 2024-06-03T02:27:05.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.481+0000 7fa5f25fbb80 1 -- 172.21.15.79:0/239013500 wait complete. 2024-06-03T02:27:05.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.482+0000 7fa5f25fbb80 20 client.26137 trim_cache size 0 max 16384 2024-06-03T02:27:05.484 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:27:05.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.482+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa5c80a2f00 msgr2=0x7fa5c80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:05.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.482+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa5c80a2f00 0x7fa5c80a53c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009680 tx=0x7fa5d4009f90 comp rx=0 tx=0).stop 2024-06-03T02:27:05.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.482+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2664362588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e0108500 msgr2=0x7fa5e0111c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:05.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.482+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e0108500 0x7fa5e0111c40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c110 tx=0x7fa5d800c4d0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.483+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2664362588 shutdown_connections 2024-06-03T02:27:05.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.483+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e0108500 0x7fa5e0111c40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.483+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e006a170 0x7fa5e0117b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.483+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa5c80a2f00 0x7fa5c80a53c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.483+0000 7fa5e7c00700 1 --2- 172.21.15.79:0/2664362588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e006ab30 0x7fa5e0118050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:05.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.483+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2664362588 >> 172.21.15.79:0/2664362588 conn(0x7fa5e0078340 msgr2=0x7fa5e01001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:05.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.483+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2664362588 shutdown_connections 2024-06-03T02:27:05.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:05.483+0000 7fa5e7c00700 1 -- 172.21.15.79:0/2664362588 wait complete. 2024-06-03T02:27:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:06 smithi079 ceph-mon[31819]: pgmap v727: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.8 MiB/s wr, 585 op/s 2024-06-03T02:27:06.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:06 smithi123 ceph-mon[36640]: pgmap v727: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.8 MiB/s wr, 585 op/s 2024-06-03T02:27:06.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:06 smithi135 ceph-mon[36653]: pgmap v727: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.8 MiB/s wr, 585 op/s 2024-06-03T02:27:07.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:27:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 420.0 s, 508.8 tps, lat 196.628 ms stddev 250.055 2024-06-03T02:27:08.675 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:08 smithi079 ceph-mon[31819]: pgmap v728: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.9 MiB/s wr, 525 op/s 2024-06-03T02:27:08.720 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:27:08.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:08 smithi123 ceph-mon[36640]: pgmap v728: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.9 MiB/s wr, 525 op/s 2024-06-03T02:27:08.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:08 smithi135 ceph-mon[36653]: pgmap v728: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.9 MiB/s wr, 525 op/s 2024-06-03T02:27:10.644 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:10 smithi079 ceph-mon[31819]: pgmap v729: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.9 MiB/s wr, 539 op/s 2024-06-03T02:27:10.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:10 smithi135 ceph-mon[36653]: pgmap v729: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.9 MiB/s wr, 539 op/s 2024-06-03T02:27:10.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:10 smithi123 ceph-mon[36640]: pgmap v729: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.9 MiB/s wr, 539 op/s 2024-06-03T02:27:12.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:12 smithi079 ceph-mon[31819]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.5 MiB/s wr, 438 op/s 2024-06-03T02:27:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:12 smithi123 ceph-mon[36640]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.5 MiB/s wr, 438 op/s 2024-06-03T02:27:12.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:12 smithi135 ceph-mon[36653]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.5 MiB/s wr, 438 op/s 2024-06-03T02:27:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 425.0 s, 621.2 tps, lat 157.269 ms stddev 189.729 2024-06-03T02:27:14.065 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:27:14.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.375+0000 7f5523e00700 1 -- 172.21.15.79:0/750730045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c1096f0 msgr2=0x7f551c109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:14.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.375+0000 7f5523e00700 1 --2- 172.21.15.79:0/750730045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c1096f0 0x7f551c109ad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f55100098f0 tx=0x7f5510009c00 comp rx=0 tx=0).stop 2024-06-03T02:27:14.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.375+0000 7f5523e00700 1 -- 172.21.15.79:0/750730045 shutdown_connections 2024-06-03T02:27:14.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.375+0000 7f5523e00700 1 --2- 172.21.15.79:0/750730045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f551c10e220 0x7f551c115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.375+0000 7f5523e00700 1 --2- 172.21.15.79:0/750730045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c1096f0 0x7f551c109ad0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.375+0000 7f5523e00700 1 --2- 172.21.15.79:0/750730045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f551c10a0a0 0x7f551c10db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.375+0000 7f5523e00700 1 -- 172.21.15.79:0/750730045 >> 172.21.15.79:0/750730045 conn(0x7f551c06d600 msgr2=0x7f551c06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:14.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.376+0000 7f5523e00700 1 -- 172.21.15.79:0/750730045 shutdown_connections 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.376+0000 7f5523e00700 1 -- 172.21.15.79:0/750730045 wait complete. 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.378+0000 7f5523e00700 1 Processor -- start 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.379+0000 7f5523e00700 1 -- start start 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5523e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f551c1096f0 0x7f551c1148d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5523e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10a0a0 0x7f551c114e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5523e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f551c10e220 0x7f551c115350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5523e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f551c118510 con 0x7f551c1096f0 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5523e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f551c118390 con 0x7f551c10a0a0 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5523e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f551c118690 con 0x7f551c10e220 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5521e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f551c10e220 0x7f551c115350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5521e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f551c10e220 0x7f551c115350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53094/0 (socket says 172.21.15.79:53094) 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5520a00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10a0a0 0x7f551c114e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5521e00700 1 -- 172.21.15.79:0/2147665886 learned_addr learned my addr 172.21.15.79:0/2147665886 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.380+0000 7f5521400700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f551c1096f0 0x7f551c1148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.381+0000 7f5520a00700 1 -- 172.21.15.79:0/2147665886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f551c10e220 msgr2=0x7f551c115350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.381+0000 7f5520a00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f551c10e220 0x7f551c115350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.381+0000 7f5520a00700 1 -- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f551c1096f0 msgr2=0x7f551c1148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.381+0000 7f5520a00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f551c1096f0 0x7f551c1148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.381+0000 7f5520a00700 1 -- 172.21.15.79:0/2147665886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c1bc620 con 0x7f551c10a0a0 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.381+0000 7f5521e00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f551c10e220 0x7f551c115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.383+0000 7f5521400700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f551c1096f0 0x7f551c1148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.383+0000 7f5520a00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10a0a0 0x7f551c114e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5508005fd0 tx=0x7f5508008c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.383+0000 7f551a000700 1 -- 172.21.15.79:0/2147665886 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508017070 con 0x7f551c10a0a0 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.383+0000 7f551a000700 1 -- 172.21.15.79:0/2147665886 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5508004d10 con 0x7f551c10a0a0 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.383+0000 7f5523e00700 1 -- 172.21.15.79:0/2147665886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551c1bc910 con 0x7f551c10a0a0 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.384+0000 7f551a000700 1 -- 172.21.15.79:0/2147665886 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508005700 con 0x7f551c10a0a0 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.384+0000 7f5523e00700 1 -- 172.21.15.79:0/2147665886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551c1bcea0 con 0x7f551c10a0a0 2024-06-03T02:27:14.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.398+0000 7f551a000700 1 -- 172.21.15.79:0/2147665886 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f550800f040 con 0x7f551c10a0a0 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.400+0000 7f551a000700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f55040a2d20 0x7f55040a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.400+0000 7f551a000700 1 -- 172.21.15.79:0/2147665886 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f55080cea00 con 0x7f551c10a0a0 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.402+0000 7f5521400700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f55040a2d20 0x7f55040a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.403+0000 7f5521400700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f55040a2d20 0x7f55040a51e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5510009fd0 tx=0x7f5510019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.404+0000 7f552e687b80 1 Processor -- start 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.404+0000 7f552e687b80 1 -- start start 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.404+0000 7f552e687b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 0x5597a2088f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.404+0000 7f552e687b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a2082d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.404+0000 7f552e687b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2083680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.404+0000 7f552e687b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5597a1f91a10 con 0x5597a2088b90 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.404+0000 7f552e687b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597a1fd4b00 con 0x5597a20894b0 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.404+0000 7f552e687b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597a1f8f8f0 con 0x5597a2083240 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.405+0000 7f5521e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2083680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.405+0000 7f5521e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2083680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53106/0 (socket says 172.21.15.79:53106) 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.405+0000 7f5521e00700 1 -- 172.21.15.79:0/2345100222 learned_addr learned my addr 172.21.15.79:0/2345100222 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.405+0000 7f5520a00700 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a2082d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.406+0000 7f5521e00700 1 -- 172.21.15.79:0/2345100222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 msgr2=0x5597a2082d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.406+0000 7f5521e00700 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a2082d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.406+0000 7f5521e00700 1 -- 172.21.15.79:0/2345100222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 msgr2=0x5597a2088f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.407+0000 7f5521400700 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 0x5597a2088f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.407+0000 7f5521e00700 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 0x5597a2088f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.407+0000 7f5521e00700 1 -- 172.21.15.79:0/2345100222 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a1f7cf90 con 0x5597a2083240 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.407+0000 7f5521400700 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 0x5597a2088f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.409+0000 7f5521e00700 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2083680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f551400e5b0 tx=0x7f5514009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a3ee99fe7bd16e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.409+0000 7f550ea00700 1 -- 172.21.15.79:0/2345100222 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514017020 con 0x5597a2083240 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.409+0000 7f550ea00700 1 -- 172.21.15.79:0/2345100222 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5514013cf0 con 0x5597a2083240 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.409+0000 7f550ea00700 1 -- 172.21.15.79:0/2345100222 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514007d30 con 0x5597a2083240 2024-06-03T02:27:14.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.410+0000 7f552e687b80 1 -- 172.21.15.79:0/2345100222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 msgr2=0x5597a2083680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.410+0000 7f552e687b80 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2083680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f551400e5b0 tx=0x7f5514009710 comp rx=0 tx=0).stop 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.411+0000 7f552e687b80 1 -- 172.21.15.79:0/2345100222 shutdown_connections 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.411+0000 7f552e687b80 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2083680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.411+0000 7f552e687b80 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 0x5597a2088f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.411+0000 7f552e687b80 1 --2- 172.21.15.79:0/2345100222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a2082d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.411+0000 7f552e687b80 1 -- 172.21.15.79:0/2345100222 >> 172.21.15.79:0/2345100222 conn(0x5597a1faf170 msgr2=0x5597a207dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.412+0000 7f552e687b80 1 -- 172.21.15.79:0/2345100222 shutdown_connections 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.412+0000 7f552e687b80 1 -- 172.21.15.79:0/2345100222 wait complete. 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.412+0000 7f552e687b80 1 -- start start 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.413+0000 7f552e687b80 1 Processor -- start 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.416+0000 7f552e687b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2083240 0x5597a208f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.416+0000 7f552e687b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2088b90 0x5597a208c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.416+0000 7f552e687b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a208ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.416+0000 7f552e687b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5597a1f70f60 con 0x5597a2083240 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.416+0000 7f552e687b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597a1fd0ac0 con 0x5597a20894b0 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.416+0000 7f552e687b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597a208fe90 con 0x5597a2088b90 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.417+0000 7f5521e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a208ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.417+0000 7f5521e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a208ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40876/0 (socket says 172.21.15.79:40876) 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.417+0000 7f5521e00700 1 -- 172.21.15.79:0/1051996666 learned_addr learned my addr 172.21.15.79:0/1051996666 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.417+0000 7f5521e00700 1 -- 172.21.15.79:0/1051996666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2088b90 msgr2=0x5597a208c520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.417+0000 7f5521e00700 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2088b90 0x5597a208c520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.417+0000 7f5521e00700 1 -- 172.21.15.79:0/1051996666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2083240 msgr2=0x5597a208f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.417+0000 7f5521e00700 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2083240 0x5597a208f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.417+0000 7f5521e00700 1 -- 172.21.15.79:0/1051996666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a1f238d0 con 0x5597a20894b0 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.417+0000 7f552e687b80 10 client.0 _ll_register_callbacks cb 0x5597a1f6a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.418+0000 7f552e687b80 1 -- 172.21.15.79:0/1051996666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5514013cf0 con 0x5597a20894b0 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.418+0000 7f5521e00700 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a208ca60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f551400e230 tx=0x7f5514007070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9055538be26d2a10 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.418+0000 7f5503400700 10 client.26153 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.419+0000 7f5503400700 1 -- 172.21.15.79:0/1051996666 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551402b060 con 0x5597a20894b0 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.419+0000 7f5503400700 1 -- 172.21.15.79:0/1051996666 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5514013810 con 0x5597a20894b0 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.419+0000 7f5503400700 1 -- 172.21.15.79:0/1051996666 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5514017040 con 0x5597a20894b0 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.419+0000 7f552e687b80 10 client.26153 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.419+0000 7f552e687b80 1 -- 172.21.15.79:0/1051996666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5597a209a420 con 0x5597a20894b0 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.420+0000 7f5503400700 1 -- 172.21.15.79:0/1051996666 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f551401c420 con 0x5597a20894b0 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.420+0000 7f552e687b80 10 client.26153 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.420+0000 7f552e687b80 10 client.26153 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.420+0000 7f552e687b80 20 client.26153 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.420+0000 7f552e687b80 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5597a209cce0 0x5597a209f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.420+0000 7f552e687b80 4 client.26153 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.420+0000 7f552e687b80 1 -- 172.21.15.79:0/1051996666 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597a1fe2000 con 0x5597a209cce0 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.422+0000 7f5521400700 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5597a209cce0 0x5597a209f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.426+0000 7f5503400700 10 client.26153 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:27:14.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.426+0000 7f5521400700 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5597a209cce0 0x5597a209f1a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=259c08b9b7921bff server_cookie=7cdd4a1954e64d7 in_seq=0 out_seq=0 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.428+0000 7f5503400700 1 -- 172.21.15.79:0/1051996666 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5597a1fe2000 con 0x5597a209cce0 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.428+0000 7f5503400700 10 client.26153 handle_command_reply: tid=0 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.428+0000 7f552e687b80 1 client.26153 shutdown 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.429+0000 7f552e687b80 1 -- 172.21.15.79:0/1051996666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 msgr2=0x5597a208ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.429+0000 7f552e687b80 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a208ca60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f551400e230 tx=0x7f5514007070 comp rx=0 tx=0).stop 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.430+0000 7f552e687b80 1 -- 172.21.15.79:0/1051996666 shutdown_connections 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.430+0000 7f552e687b80 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a20894b0 0x5597a208ca60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.430+0000 7f552e687b80 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2083240 0x5597a208f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.430+0000 7f552e687b80 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5597a209cce0 0x5597a209f1a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.430+0000 7f552e687b80 1 --2- 172.21.15.79:0/1051996666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2088b90 0x5597a208c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.430+0000 7f552e687b80 1 -- 172.21.15.79:0/1051996666 >> 172.21.15.79:0/1051996666 conn(0x5597a1faf170 msgr2=0x5597a2077790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.430+0000 7f5503400700 0 client.26153 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.431+0000 7f552e687b80 1 -- 172.21.15.79:0/1051996666 shutdown_connections 2024-06-03T02:27:14.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.431+0000 7f552e687b80 1 -- 172.21.15.79:0/1051996666 wait complete. 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.432+0000 7f552e687b80 20 client.26153 trim_cache size 0 max 16384 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.464+0000 7f552e687b80 1 Processor -- start 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.466+0000 7f552e687b80 1 -- start start 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.466+0000 7f552e687b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2082d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.466+0000 7f552e687b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2088b90 0x5597a2078130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.466+0000 7f552e687b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a20894b0 0x5597a2086b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.466+0000 7f552e687b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1c5c90 con 0x5597a20894b0 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.466+0000 7f552e687b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1c77d0 con 0x5597a2088b90 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.466+0000 7f552e687b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1c6360 con 0x5597a2083240 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.467+0000 7f5521400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2082d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.467+0000 7f5521400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2082d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53126/0 (socket says 172.21.15.79:53126) 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.467+0000 7f5521400700 1 -- 172.21.15.79:0/113582911 learned_addr learned my addr 172.21.15.79:0/113582911 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.467+0000 7f5520a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2088b90 0x5597a2078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.467+0000 7f5520a00700 1 -- 172.21.15.79:0/113582911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 msgr2=0x5597a2082d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:14.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.467+0000 7f5520a00700 1 --2- 172.21.15.79:0/113582911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2082d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.467+0000 7f5520a00700 1 -- 172.21.15.79:0/113582911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a20894b0 msgr2=0x5597a2086b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.467+0000 7f5520a00700 1 --2- 172.21.15.79:0/113582911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a20894b0 0x5597a2086b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.467+0000 7f5520a00700 1 -- 172.21.15.79:0/113582911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a208aab0 con 0x5597a2088b90 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.468+0000 7f5521400700 1 --2- 172.21.15.79:0/113582911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2082d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.469+0000 7f5520a00700 1 --2- 172.21.15.79:0/113582911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2088b90 0x5597a2078130 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f55080cc0c0 tx=0x7f5508008f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4972824753c54f75 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.469+0000 7f550e000700 1 -- 172.21.15.79:0/113582911 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508020040 con 0x5597a2088b90 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.469+0000 7f550e000700 1 -- 172.21.15.79:0/113582911 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550809d070 con 0x5597a2088b90 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.469+0000 7f550e000700 1 -- 172.21.15.79:0/113582911 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550800ece0 con 0x5597a2088b90 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.471+0000 7f552e687b80 1 -- 172.21.15.79:0/113582911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2088b90 msgr2=0x5597a2078130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.471+0000 7f552e687b80 1 --2- 172.21.15.79:0/113582911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2088b90 0x5597a2078130 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f55080cc0c0 tx=0x7f5508008f40 comp rx=0 tx=0).stop 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.471+0000 7f552e687b80 1 -- 172.21.15.79:0/113582911 shutdown_connections 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.471+0000 7f552e687b80 1 --2- 172.21.15.79:0/113582911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a20894b0 0x5597a2086b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.471+0000 7f552e687b80 1 --2- 172.21.15.79:0/113582911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a2083240 0x5597a2082d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.471+0000 7f552e687b80 1 --2- 172.21.15.79:0/113582911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2088b90 0x5597a2078130 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.471+0000 7f552e687b80 1 -- 172.21.15.79:0/113582911 >> 172.21.15.79:0/113582911 conn(0x5597a1faf170 msgr2=0x5597a2086400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.473+0000 7f552e687b80 1 -- 172.21.15.79:0/113582911 shutdown_connections 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.473+0000 7f552e687b80 1 -- 172.21.15.79:0/113582911 wait complete. 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.473+0000 7f552e687b80 1 -- start start 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.475+0000 7f552e687b80 1 Processor -- start 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.476+0000 7f552e687b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2083240 0x5597a20a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.476+0000 7f552e687b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 0x5597a2095150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.476+0000 7f552e687b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a20894b0 0x5597a2095690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.476+0000 7f5521400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2083240 0x5597a20a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.476+0000 7f5521400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2083240 0x5597a20a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40896/0 (socket says 172.21.15.79:40896) 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.476+0000 7f5521400700 1 -- 172.21.15.79:0/657987848 learned_addr learned my addr 172.21.15.79:0/657987848 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.477+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1c7100 con 0x5597a2088b90 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.477+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1c6a30 con 0x5597a2083240 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.477+0000 7f5521e00700 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a20894b0 0x5597a2095690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.477+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1c3c30 con 0x5597a20894b0 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.477+0000 7f552e687b80 10 client.0 _ll_register_callbacks cb 0x5597a1f6a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.478+0000 7f5521e00700 1 -- 172.21.15.79:0/657987848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2083240 msgr2=0x5597a20a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.478+0000 7f5521e00700 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2083240 0x5597a20a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.478+0000 7f5521e00700 1 -- 172.21.15.79:0/657987848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 msgr2=0x5597a2095150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.478+0000 7f5521e00700 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 0x5597a2095150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.478+0000 7f5521e00700 1 -- 172.21.15.79:0/657987848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a2095bd0 con 0x5597a20894b0 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.478+0000 7f5521e00700 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a20894b0 0x5597a2095690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5514000f80 tx=0x7f5514009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af20bed86145538 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.478+0000 7f5503400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:27:14.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.478+0000 7f5503400700 1 -- 172.21.15.79:0/657987848 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514013810 con 0x5597a20894b0 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.479+0000 7f5503400700 1 -- 172.21.15.79:0/657987848 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5514009070 con 0x5597a20894b0 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.479+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550809d070 con 0x5597a20894b0 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.479+0000 7f5503400700 1 -- 172.21.15.79:0/657987848 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f551401c9f0 con 0x5597a20894b0 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.479+0000 7f552e687b80 10 client.26152 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.479+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597a2106a80 con 0x5597a20894b0 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.479+0000 7f5503400700 1 -- 172.21.15.79:0/657987848 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550809d070 con 0x5597a20894b0 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.481+0000 7f5503400700 1 -- 172.21.15.79:0/657987848 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5514026900 con 0x5597a20894b0 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.481+0000 7f552e687b80 10 client.26152 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.481+0000 7f552e687b80 10 client.26152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.481+0000 7f552e687b80 20 client.26152 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.481+0000 7f552e687b80 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5597a2106f20 0x5597a2109310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.481+0000 7f552e687b80 4 client.26152 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.481+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597a1fdc580 con 0x5597a2106f20 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.482+0000 7f5521400700 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5597a2106f20 0x5597a2109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:14.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.485+0000 7f5521400700 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5597a2106f20 0x5597a2109310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c32cb63c4c365e5 server_cookie=ae09a7963f8ef311 in_seq=0 out_seq=0 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.485+0000 7f5503400700 10 client.26152 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.486+0000 7f5503400700 1 -- 172.21.15.79:0/657987848 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597a1fdc580 con 0x5597a2106f20 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.486+0000 7f5503400700 10 client.26152 handle_command_reply: tid=0 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.486+0000 7f552e687b80 1 client.26152 shutdown 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a20894b0 msgr2=0x5597a2095690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a20894b0 0x5597a2095690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5514000f80 tx=0x7f5514009710 comp rx=0 tx=0).stop 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 shutdown_connections 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x5597a2106f20 0x5597a2109310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597a20894b0 0x5597a2095690 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a2083240 0x5597a20a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 --2- 172.21.15.79:0/657987848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5597a2088b90 0x5597a2095150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f5503400700 0 client.26152 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 >> 172.21.15.79:0/657987848 conn(0x5597a1faf170 msgr2=0x5597a2089890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 shutdown_connections 2024-06-03T02:27:14.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.487+0000 7f552e687b80 1 -- 172.21.15.79:0/657987848 wait complete. 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.488+0000 7f552e687b80 20 client.26152 trim_cache size 0 max 16384 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.488+0000 7f5523e00700 1 -- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f55040a2d20 msgr2=0x7f55040a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f55040a2d20 0x7f55040a51e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5510009fd0 tx=0x7f5510019040 comp rx=0 tx=0).stop 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 -- 172.21.15.79:0/2147665886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10a0a0 msgr2=0x7f551c114e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10a0a0 0x7f551c114e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5508005fd0 tx=0x7f5508008c50 comp rx=0 tx=0).stop 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 -- 172.21.15.79:0/2147665886 shutdown_connections 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f551c10e220 0x7f551c115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f55040a2d20 0x7f55040a51e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f551c1096f0 0x7f551c1148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 --2- 172.21.15.79:0/2147665886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f551c10a0a0 0x7f551c114e10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 -- 172.21.15.79:0/2147665886 >> 172.21.15.79:0/2147665886 conn(0x7f551c06d600 msgr2=0x7f551c119e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 -- 172.21.15.79:0/2147665886 shutdown_connections 2024-06-03T02:27:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:13.489+0000 7f5523e00700 1 -- 172.21.15.79:0/2147665886 wait complete. 2024-06-03T02:27:14.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:14 smithi123 ceph-mon[36640]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 538 op/s 2024-06-03T02:27:14.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:14 smithi079 ceph-mon[31819]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 538 op/s 2024-06-03T02:27:14.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:14 smithi135 ceph-mon[36653]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 538 op/s 2024-06-03T02:27:16.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:27:16.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:16 smithi123 ceph-mon[36640]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.6 MiB/s wr, 510 op/s 2024-06-03T02:27:16.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:16 smithi079 ceph-mon[31819]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.6 MiB/s wr, 510 op/s 2024-06-03T02:27:16.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:16 smithi135 ceph-mon[36653]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.6 MiB/s wr, 510 op/s 2024-06-03T02:27:16.991 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:27:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 430.0 s, 700.8 tps, lat 143.446 ms stddev 218.731 2024-06-03T02:27:18.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:18 smithi079 ceph-mon[31819]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2024-06-03T02:27:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:18 smithi123 ceph-mon[36640]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2024-06-03T02:27:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:18 smithi135 ceph-mon[36653]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2024-06-03T02:27:20.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.374+0000 7fca72600700 1 -- 172.21.15.79:0/2689313116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101390 msgr2=0x7fca6c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:20.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.375+0000 7fca72600700 1 --2- 172.21.15.79:0/2689313116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101390 0x7fca6c101810 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca58009990 tx=0x7fca58009ca0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.376+0000 7fca72600700 1 -- 172.21.15.79:0/2689313116 shutdown_connections 2024-06-03T02:27:20.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.376+0000 7fca72600700 1 --2- 172.21.15.79:0/2689313116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca6c101d50 0x7fca6c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.376+0000 7fca72600700 1 --2- 172.21.15.79:0/2689313116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c107400 0x7fca6c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.376+0000 7fca72600700 1 --2- 172.21.15.79:0/2689313116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101390 0x7fca6c101810 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.377+0000 7fca72600700 1 -- 172.21.15.79:0/2689313116 >> 172.21.15.79:0/2689313116 conn(0x7fca6c078340 msgr2=0x7fca6c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:20.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.377+0000 7fca72600700 1 -- 172.21.15.79:0/2689313116 shutdown_connections 2024-06-03T02:27:20.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.377+0000 7fca72600700 1 -- 172.21.15.79:0/2689313116 wait complete. 2024-06-03T02:27:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.381+0000 7fca72600700 1 Processor -- start 2024-06-03T02:27:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.381+0000 7fca72600700 1 -- start start 2024-06-03T02:27:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.382+0000 7fca72600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca6c101390 0x7fca6c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.382+0000 7fca72600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101d50 0x7fca6c0740b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.382+0000 7fca72600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c107400 0x7fca6c06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.382+0000 7fca72600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c110680 con 0x7fca6c101390 2024-06-03T02:27:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.382+0000 7fca72600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c110500 con 0x7fca6c107400 2024-06-03T02:27:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.382+0000 7fca72600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c110800 con 0x7fca6c101d50 2024-06-03T02:27:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.383+0000 7fca6aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101d50 0x7fca6c0740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.383+0000 7fca6be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c107400 0x7fca6c06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.383+0000 7fca6aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101d50 0x7fca6c0740b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37892/0 (socket says 172.21.15.79:37892) 2024-06-03T02:27:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.383+0000 7fca6aa00700 1 -- 172.21.15.79:0/2072057226 learned_addr learned my addr 172.21.15.79:0/2072057226 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.383+0000 7fca6aa00700 1 -- 172.21.15.79:0/2072057226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c107400 msgr2=0x7fca6c06cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.383+0000 7fca6aa00700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c107400 0x7fca6c06cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.383+0000 7fca6aa00700 1 -- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca6c101390 msgr2=0x7fca6c073b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:27:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.384+0000 7fca6b400700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca6c101390 0x7fca6c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.384+0000 7fca6aa00700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca6c101390 0x7fca6c073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.384+0000 7fca6aa00700 1 -- 172.21.15.79:0/2072057226 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c06d640 con 0x7fca6c101d50 2024-06-03T02:27:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.384+0000 7fca6b400700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca6c101390 0x7fca6c073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.385+0000 7fca6aa00700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101d50 0x7fca6c0740b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fca58000c00 tx=0x7fca5800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.385+0000 7fca63e00700 1 -- 172.21.15.79:0/2072057226 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58026070 con 0x7fca6c101d50 2024-06-03T02:27:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.386+0000 7fca63e00700 1 -- 172.21.15.79:0/2072057226 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca58003d70 con 0x7fca6c101d50 2024-06-03T02:27:20.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.386+0000 7fca72600700 1 -- 172.21.15.79:0/2072057226 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c06d8d0 con 0x7fca6c101d50 2024-06-03T02:27:20.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.386+0000 7fca63e00700 1 -- 172.21.15.79:0/2072057226 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58021410 con 0x7fca6c101d50 2024-06-03T02:27:20.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.386+0000 7fca72600700 1 -- 172.21.15.79:0/2072057226 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c071870 con 0x7fca6c101d50 2024-06-03T02:27:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.392+0000 7fca63e00700 1 -- 172.21.15.79:0/2072057226 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fca58009d10 con 0x7fca6c101d50 2024-06-03T02:27:20.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.393+0000 7fca7ceb2b80 1 Processor -- start 2024-06-03T02:27:20.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.396+0000 7fca7ceb2b80 1 -- start start 2024-06-03T02:27:20.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.397+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d69539ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.397+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 0x55d695399d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.397+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d69539a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.397+0000 7fca7ceb2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d6952a8a10 con 0x55d69539fb90 2024-06-03T02:27:20.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.397+0000 7fca7ceb2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6952ebb00 con 0x55d69539a240 2024-06-03T02:27:20.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.397+0000 7fca7ceb2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d6952a68f0 con 0x55d6953a04b0 2024-06-03T02:27:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.397+0000 7fca6aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d69539a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.398+0000 7fca6aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d69539a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59616/0 (socket says 172.21.15.79:59616) 2024-06-03T02:27:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.398+0000 7fca6aa00700 1 -- 172.21.15.79:0/435851350 learned_addr learned my addr 172.21.15.79:0/435851350 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.398+0000 7fca63e00700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca500a2d50 0x7fca500a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.399+0000 7fca63e00700 1 -- 172.21.15.79:0/2072057226 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fca580cfff0 con 0x7fca6c101d50 2024-06-03T02:27:20.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.399+0000 7fca6be00700 1 --2- 172.21.15.79:0/435851350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d69539ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.399+0000 7fca6aa00700 1 -- 172.21.15.79:0/435851350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 msgr2=0x55d695399d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:27:20.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.399+0000 7fca6aa00700 1 --2- 172.21.15.79:0/435851350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 0x55d695399d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.399+0000 7fca6aa00700 1 -- 172.21.15.79:0/435851350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 msgr2=0x55d69539ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:20.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.400+0000 7fca6aa00700 1 --2- 172.21.15.79:0/435851350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d69539ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.400+0000 7fca6be00700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca500a2d50 0x7fca500a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.400+0000 7fca6aa00700 1 -- 172.21.15.79:0/435851350 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d695293f90 con 0x55d69539a240 2024-06-03T02:27:20.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.400+0000 7fca6aa00700 1 --2- 172.21.15.79:0/435851350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d69539a680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fca5800fa70 tx=0x7fca58009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e3498d8056d13d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.400+0000 7fca60c00700 1 -- 172.21.15.79:0/435851350 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5809c010 con 0x55d69539a240 2024-06-03T02:27:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.400+0000 7fca60c00700 1 -- 172.21.15.79:0/435851350 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca580095c0 con 0x55d69539a240 2024-06-03T02:27:20.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.400+0000 7fca60c00700 1 -- 172.21.15.79:0/435851350 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5809be30 con 0x55d69539a240 2024-06-03T02:27:20.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.401+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/435851350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 msgr2=0x55d69539a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:20.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.401+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/435851350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d69539a680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fca5800fa70 tx=0x7fca58009ca0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.402+0000 7fca6be00700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca500a2d50 0x7fca500a5210 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fca64009900 tx=0x7fca64009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:20.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.402+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/435851350 shutdown_connections 2024-06-03T02:27:20.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.402+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/435851350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d69539a680 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.403+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/435851350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d69539ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.406+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/435851350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 0x55d695399d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.406+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/435851350 >> 172.21.15.79:0/435851350 conn(0x55d6952c6170 msgr2=0x55d695394ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:20.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.406+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/435851350 shutdown_connections 2024-06-03T02:27:20.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.406+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/435851350 wait complete. 2024-06-03T02:27:20.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.407+0000 7fca7ceb2b80 1 -- start start 2024-06-03T02:27:20.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.408+0000 7fca7ceb2b80 1 Processor -- start 2024-06-03T02:27:20.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.409+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539a240 0x55d6953a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.409+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d6953a3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.410+0000 7fca6b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539a240 0x55d6953a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.411+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6953a04b0 0x55d6953a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.411+0000 7fca6aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d6953a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.411+0000 7fca6b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539a240 0x55d6953a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37904/0 (socket says 172.21.15.79:37904) 2024-06-03T02:27:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.411+0000 7fca6aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d6953a3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33376/0 (socket says 172.21.15.79:33376) 2024-06-03T02:27:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.411+0000 7fca6b400700 1 -- 172.21.15.79:0/2183710641 learned_addr learned my addr 172.21.15.79:0/2183710641 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:20.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.412+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d695287f60 con 0x55d69539fb90 2024-06-03T02:27:20.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.412+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6952e7ac0 con 0x55d6953a04b0 2024-06-03T02:27:20.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.412+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d6953a6c40 con 0x55d69539a240 2024-06-03T02:27:20.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.412+0000 7fca7ceb2b80 10 client.0 _ll_register_callbacks cb 0x55d695281380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:20.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.412+0000 7fca6aa00700 1 -- 172.21.15.79:0/2183710641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539a240 msgr2=0x55d6953a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:20.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.412+0000 7fca6aa00700 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539a240 0x55d6953a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.412+0000 7fca6aa00700 1 -- 172.21.15.79:0/2183710641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6953a04b0 msgr2=0x55d6953a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:20.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.412+0000 7fca6aa00700 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6953a04b0 0x55d6953a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.412+0000 7fca6aa00700 1 -- 172.21.15.79:0/2183710641 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d69523a8d0 con 0x55d69539fb90 2024-06-03T02:27:20.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.413+0000 7fca6aa00700 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d6953a3540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca5800beb0 tx=0x7fca580d36c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98a756c3c1fe67f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:20.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.413+0000 7fca55600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:27:20.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.413+0000 7fca55600700 1 -- 172.21.15.79:0/2183710641 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5809be30 con 0x55d69539fb90 2024-06-03T02:27:20.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.413+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6952f9000 con 0x55d69539fb90 2024-06-03T02:27:20.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.414+0000 7fca55600700 1 -- 172.21.15.79:0/2183710641 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fca5800f6f0 con 0x55d69539fb90 2024-06-03T02:27:20.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.415+0000 7fca55600700 1 -- 172.21.15.79:0/2183710641 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca5809c010 con 0x55d69539fb90 2024-06-03T02:27:20.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.417+0000 7fca7ceb2b80 10 client.15804 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:20.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.417+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6953b13c0 con 0x55d69539fb90 2024-06-03T02:27:20.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.417+0000 7fca55600700 1 -- 172.21.15.79:0/2183710641 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d6952f9000 con 0x55d69539fb90 2024-06-03T02:27:20.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.417+0000 7fca55600700 1 -- 172.21.15.79:0/2183710641 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fca580b5190 con 0x55d69539fb90 2024-06-03T02:27:20.419 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.419+0000 7fca7ceb2b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.419+0000 7fca7ceb2b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:27:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.419+0000 7fca7ceb2b80 20 client.15804 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.419+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55d6953b3890 0x55d6953b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.419+0000 7fca7ceb2b80 4 client.15804 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:27:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.419+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6952f3580 con 0x55d6953b3890 2024-06-03T02:27:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.419+0000 7fca6b400700 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55d6953b3890 0x55d6953b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.420+0000 7fca6b400700 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55d6953b3890 0x55d6953b5c80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60284338663ecfc server_cookie=ea46b405e692de2e in_seq=0 out_seq=0 2024-06-03T02:27:20.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.420+0000 7fca55600700 10 client.15804 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:20.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.421+0000 7fca55600700 1 -- 172.21.15.79:0/2183710641 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d6952f3580 con 0x55d6953b3890 2024-06-03T02:27:20.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.421+0000 7fca55600700 10 client.15804 handle_command_reply: tid=0 2024-06-03T02:27:20.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.421+0000 7fca7ceb2b80 1 client.15804 shutdown 2024-06-03T02:27:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.424+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 msgr2=0x55d6953a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.424+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d6953a3540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca5800beb0 tx=0x7fca580d36c0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.424+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 shutdown_connections 2024-06-03T02:27:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.424+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6953a04b0 0x55d6953a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.424+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539a240 0x55d6953a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.424+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55d6953b3890 0x55d6953b5c80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.424+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2183710641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539fb90 0x55d6953a3540 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca5800beb0 tx=0x7fca580d36c0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.424+0000 7fca55600700 0 client.15804 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.425+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 >> 172.21.15.79:0/2183710641 conn(0x55d6952c6170 msgr2=0x55d69538e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.425+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 shutdown_connections 2024-06-03T02:27:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.425+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2183710641 wait complete. 2024-06-03T02:27:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.425+0000 7fca7ceb2b80 20 client.15804 trim_cache size 0 max 16384 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.461+0000 7fca7ceb2b80 1 Processor -- start 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.461+0000 7fca7ceb2b80 1 -- start start 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.461+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d695399d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.461+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6953a04b0 0x55d69538f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.461+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539f5a0 0x55d6953a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.461+0000 7fca7ceb2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1bd1c0 con 0x55d6953a04b0 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.461+0000 7fca7ceb2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1bed00 con 0x55d69539a240 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.461+0000 7fca7ceb2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1bd890 con 0x55d69539f5a0 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.462+0000 7fca6b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d695399d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.462+0000 7fca6be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539f5a0 0x55d6953a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.462+0000 7fca6be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539f5a0 0x55d6953a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37912/0 (socket says 172.21.15.79:37912) 2024-06-03T02:27:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.462+0000 7fca6be00700 1 -- 172.21.15.79:0/2543244021 learned_addr learned my addr 172.21.15.79:0/2543244021 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:20.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.462+0000 7fca6aa00700 1 --2- 172.21.15.79:0/2543244021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6953a04b0 0x55d69538f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.463+0000 7fca6be00700 1 -- 172.21.15.79:0/2543244021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 msgr2=0x55d695399d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:20.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.463+0000 7fca6be00700 1 --2- 172.21.15.79:0/2543244021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d695399d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.463+0000 7fca6be00700 1 -- 172.21.15.79:0/2543244021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6953a04b0 msgr2=0x55d69538f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:20.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.463+0000 7fca6be00700 1 --2- 172.21.15.79:0/2543244021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6953a04b0 0x55d69538f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.463+0000 7fca6be00700 1 -- 172.21.15.79:0/2543244021 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d695396510 con 0x55d69539f5a0 2024-06-03T02:27:20.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.463+0000 7fca6aa00700 1 --2- 172.21.15.79:0/2543244021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6953a04b0 0x55d69538f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:20.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.463+0000 7fca6be00700 1 --2- 172.21.15.79:0/2543244021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539f5a0 0x55d6953a1120 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fca64009a60 tx=0x7fca64004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4b6776d7be2e20 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:20.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.464+0000 7fca57e00700 1 -- 172.21.15.79:0/2543244021 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64029070 con 0x55d69539f5a0 2024-06-03T02:27:20.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.464+0000 7fca57e00700 1 -- 172.21.15.79:0/2543244021 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca64024470 con 0x55d69539f5a0 2024-06-03T02:27:20.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.465+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2543244021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539f5a0 msgr2=0x55d6953a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:20.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.465+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2543244021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539f5a0 0x55d6953a1120 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fca64009a60 tx=0x7fca64004ad0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.465+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2543244021 shutdown_connections 2024-06-03T02:27:20.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.465+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2543244021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d69539f5a0 0x55d6953a1120 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.465+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2543244021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539a240 0x55d695399d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.465+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2543244021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6953a04b0 0x55d69538f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.465+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2543244021 >> 172.21.15.79:0/2543244021 conn(0x55d6952c6170 msgr2=0x55d6953a1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:20.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.465+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2543244021 shutdown_connections 2024-06-03T02:27:20.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.465+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2543244021 wait complete. 2024-06-03T02:27:20.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.466+0000 7fca7ceb2b80 1 -- start start 2024-06-03T02:27:20.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.466+0000 7fca7ceb2b80 1 Processor -- start 2024-06-03T02:27:20.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.467+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539a240 0x55d6953ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.467+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539f5a0 0x55d6953baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca7ceb2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 0x55d6953bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca6aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539f5a0 0x55d6953baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca7ceb2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1be630 con 0x55d69539a240 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca7ceb2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1bdf60 con 0x55d69539f5a0 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca7ceb2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1bb100 con 0x55d6953a04b0 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca7ceb2b80 10 client.0 _ll_register_callbacks cb 0x55d695281380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca6be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 0x55d6953bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca6be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 0x55d6953bb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37914/0 (socket says 172.21.15.79:37914) 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca6be00700 1 -- 172.21.15.79:0/2985519208 learned_addr learned my addr 172.21.15.79:0/2985519208 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:20.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca6b400700 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539a240 0x55d6953ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.468+0000 7fca6aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539f5a0 0x55d6953baac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59654/0 (socket says 172.21.15.79:59654) 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.469+0000 7fca6be00700 1 -- 172.21.15.79:0/2985519208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539f5a0 msgr2=0x55d6953baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.469+0000 7fca6be00700 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539f5a0 0x55d6953baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.469+0000 7fca6be00700 1 -- 172.21.15.79:0/2985519208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539a240 msgr2=0x55d6953ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.469+0000 7fca6be00700 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539a240 0x55d6953ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.469+0000 7fca6be00700 1 -- 172.21.15.79:0/2985519208 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6953bb640 con 0x55d6953a04b0 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.469+0000 7fca6aa00700 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539f5a0 0x55d6953baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.469+0000 7fca6b400700 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539a240 0x55d6953ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.469+0000 7fca6be00700 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 0x55d6953bb000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fca64027040 tx=0x7fca64024e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a673b649d3bd05ac server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.470+0000 7fca55600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.470+0000 7fca55600700 1 -- 172.21.15.79:0/2985519208 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64029030 con 0x55d6953a04b0 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.470+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2985519208 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca64024470 con 0x55d6953a04b0 2024-06-03T02:27:20.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.470+0000 7fca55600700 1 -- 172.21.15.79:0/2985519208 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fca64020040 con 0x55d6953a04b0 2024-06-03T02:27:20.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.470+0000 7fca55600700 1 -- 172.21.15.79:0/2985519208 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca64014040 con 0x55d6953a04b0 2024-06-03T02:27:20.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.470+0000 7fca7ceb2b80 10 client.26167 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:20.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.470+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2985519208 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d69541e130 con 0x55d6953a04b0 2024-06-03T02:27:20.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.472+0000 7fca55600700 1 -- 172.21.15.79:0/2985519208 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca64024470 con 0x55d6953a04b0 2024-06-03T02:27:20.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.472+0000 7fca55600700 1 -- 172.21.15.79:0/2985519208 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fca64019ca0 con 0x55d6953a04b0 2024-06-03T02:27:20.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.472+0000 7fca7ceb2b80 10 client.26167 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:20.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.472+0000 7fca7ceb2b80 10 client.26167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:27:20.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.472+0000 7fca7ceb2b80 20 client.26167 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:20.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.472+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55d69541e850 0x55d695420c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:20.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.472+0000 7fca7ceb2b80 4 client.26167 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:27:20.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.472+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2985519208 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d69516d250 con 0x55d69541e850 2024-06-03T02:27:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.474+0000 7fca6b400700 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55d69541e850 0x55d695420c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.475+0000 7fca6b400700 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55d69541e850 0x55d695420c70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c125d0162e7fb90 server_cookie=cc240c3474adc690 in_seq=0 out_seq=0 2024-06-03T02:27:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.475+0000 7fca55600700 10 client.26167 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.475+0000 7fca55600700 1 -- 172.21.15.79:0/2985519208 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d69516d250 con 0x55d69541e850 2024-06-03T02:27:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.475+0000 7fca55600700 10 client.26167 handle_command_reply: tid=0 2024-06-03T02:27:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.475+0000 7fca7ceb2b80 1 client.26167 shutdown 2024-06-03T02:27:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.476+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2985519208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 msgr2=0x55d6953bb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.476+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 0x55d6953bb000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fca64027040 tx=0x7fca64024e00 comp rx=0 tx=0).stop 2024-06-03T02:27:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.476+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2985519208 shutdown_connections 2024-06-03T02:27:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.476+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6953a04b0 0x55d6953bb000 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.476+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55d69541e850 0x55d695420c70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.476+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d69539a240 0x55d6953ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.476+0000 7fca7ceb2b80 1 --2- 172.21.15.79:0/2985519208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d69539f5a0 0x55d6953baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.476+0000 7fca55600700 0 client.26167 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.476+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2985519208 >> 172.21.15.79:0/2985519208 conn(0x55d6952c6170 msgr2=0x55d6953a26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.477+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2985519208 shutdown_connections 2024-06-03T02:27:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.477+0000 7fca7ceb2b80 1 -- 172.21.15.79:0/2985519208 wait complete. 2024-06-03T02:27:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.477+0000 7fca7ceb2b80 20 client.26167 trim_cache size 0 max 16384 2024-06-03T02:27:20.478 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 -- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca500a2d50 msgr2=0x7fca500a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca500a2d50 0x7fca500a5210 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fca64009900 tx=0x7fca64009650 comp rx=0 tx=0).stop 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 -- 172.21.15.79:0/2072057226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101d50 msgr2=0x7fca6c0740b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101d50 0x7fca6c0740b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fca58000c00 tx=0x7fca5800bab0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 -- 172.21.15.79:0/2072057226 shutdown_connections 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c107400 0x7fca6c06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca500a2d50 0x7fca500a5210 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca6c101390 0x7fca6c073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 --2- 172.21.15.79:0/2072057226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6c101d50 0x7fca6c0740b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 -- 172.21.15.79:0/2072057226 >> 172.21.15.79:0/2072057226 conn(0x7fca6c078340 msgr2=0x7fca6c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:20.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 -- 172.21.15.79:0/2072057226 shutdown_connections 2024-06-03T02:27:20.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:20.480+0000 7fca72600700 1 -- 172.21.15.79:0/2072057226 wait complete. 2024-06-03T02:27:20.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:20 smithi123 ceph-mon[36640]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.8 MiB/s wr, 568 op/s 2024-06-03T02:27:20.841 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:20 smithi079 ceph-mon[31819]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.8 MiB/s wr, 568 op/s 2024-06-03T02:27:20.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:20 smithi135 ceph-mon[36653]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.8 MiB/s wr, 568 op/s 2024-06-03T02:27:22.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:22 smithi123 ceph-mon[36640]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 501 op/s 2024-06-03T02:27:22.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:22 smithi079 ceph-mon[31819]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 501 op/s 2024-06-03T02:27:22.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:22 smithi135 ceph-mon[36653]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 501 op/s 2024-06-03T02:27:23.238 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:27:23.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:27:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 435.0 s, 525.0 tps, lat 198.395 ms stddev 258.607 2024-06-03T02:27:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:27:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:24 smithi079 ceph-mon[31819]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 645 op/s 2024-06-03T02:27:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:24 smithi123 ceph-mon[36640]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 645 op/s 2024-06-03T02:27:24.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:24 smithi135 ceph-mon[36653]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 645 op/s 2024-06-03T02:27:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:26 smithi123 ceph-mon[36640]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 616 op/s 2024-06-03T02:27:26.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:26 smithi079 ceph-mon[31819]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 616 op/s 2024-06-03T02:27:26.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:26 smithi135 ceph-mon[36653]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 616 op/s 2024-06-03T02:27:27.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.489+0000 7f0a43000700 1 -- 172.21.15.79:0/64377714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c0748c0 msgr2=0x7f0a3c074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:27.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.489+0000 7f0a43000700 1 --2- 172.21.15.79:0/64377714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c0748c0 0x7f0a3c074ca0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009f90 tx=0x7f0a3000b330 comp rx=0 tx=0).stop 2024-06-03T02:27:27.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.489+0000 7f0a43000700 1 -- 172.21.15.79:0/64377714 shutdown_connections 2024-06-03T02:27:27.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.489+0000 7f0a43000700 1 --2- 172.21.15.79:0/64377714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a3c10ec00 0x7f0a3c111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.489+0000 7f0a43000700 1 --2- 172.21.15.79:0/64377714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c0748c0 0x7f0a3c074ca0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.489+0000 7f0a43000700 1 --2- 172.21.15.79:0/64377714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a3c0751e0 0x7f0a3c10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.489+0000 7f0a43000700 1 -- 172.21.15.79:0/64377714 >> 172.21.15.79:0/64377714 conn(0x7f0a3c0fdf10 msgr2=0x7f0a3c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:27.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.490+0000 7f0a43000700 1 -- 172.21.15.79:0/64377714 shutdown_connections 2024-06-03T02:27:27.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.491+0000 7f0a43000700 1 -- 172.21.15.79:0/64377714 wait complete. 2024-06-03T02:27:27.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.493+0000 7f0a43000700 1 Processor -- start 2024-06-03T02:27:27.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.494+0000 7f0a43000700 1 -- start start 2024-06-03T02:27:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.495+0000 7f0a43000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a3c0751e0 0x7f0a3c107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.495+0000 7f0a43000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a3c10ec00 0x7f0a3c108430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.496+0000 7f0a43000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c10dc10 0x7f0a3c10e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.496+0000 7f0a43000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c114a00 con 0x7f0a3c10ec00 2024-06-03T02:27:27.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.496+0000 7f0a43000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c114880 con 0x7f0a3c0751e0 2024-06-03T02:27:27.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.496+0000 7f0a43000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c114b80 con 0x7f0a3c10dc10 2024-06-03T02:27:27.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.496+0000 7f0a42600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c10dc10 0x7f0a3c10e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.496+0000 7f0a42600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c10dc10 0x7f0a3c10e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56110/0 (socket says 172.21.15.79:56110) 2024-06-03T02:27:27.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.496+0000 7f0a42600700 1 -- 172.21.15.79:0/3379160825 learned_addr learned my addr 172.21.15.79:0/3379160825 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:27.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.496+0000 7f0a41200700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a3c10ec00 0x7f0a3c108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.497+0000 7f0a42600700 1 -- 172.21.15.79:0/3379160825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a3c0751e0 msgr2=0x7f0a3c107ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:27.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.497+0000 7f0a42600700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a3c0751e0 0x7f0a3c107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.497+0000 7f0a42600700 1 -- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a3c10ec00 msgr2=0x7f0a3c108430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:27.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.497+0000 7f0a42600700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a3c10ec00 0x7f0a3c108430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.497+0000 7f0a42600700 1 -- 172.21.15.79:0/3379160825 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c10e900 con 0x7f0a3c10dc10 2024-06-03T02:27:27.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.498+0000 7f0a42600700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c10dc10 0x7f0a3c10e0c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400e970 tx=0x7f0a3400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:27.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.498+0000 7f0a3a000700 1 -- 172.21.15.79:0/3379160825 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34004a30 con 0x7f0a3c10dc10 2024-06-03T02:27:27.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.499+0000 7f0a43000700 1 -- 172.21.15.79:0/3379160825 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c103010 con 0x7f0a3c10dc10 2024-06-03T02:27:27.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.499+0000 7f0a3a000700 1 -- 172.21.15.79:0/3379160825 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3400cca0 con 0x7f0a3c10dc10 2024-06-03T02:27:27.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.499+0000 7f0a3a000700 1 -- 172.21.15.79:0/3379160825 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34004a30 con 0x7f0a3c10dc10 2024-06-03T02:27:27.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.499+0000 7f0a43000700 1 -- 172.21.15.79:0/3379160825 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c103550 con 0x7f0a3c10dc10 2024-06-03T02:27:27.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.510+0000 7f0a4efa5b80 1 Processor -- start 2024-06-03T02:27:27.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.513+0000 7f0a3a000700 1 -- 172.21.15.79:0/3379160825 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0a340054c0 con 0x7f0a3c10dc10 2024-06-03T02:27:27.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.513+0000 7f0a4efa5b80 1 -- start start 2024-06-03T02:27:27.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.514+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 0x562c8b9aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.514+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.515+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 0x562c8b9b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.515+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562c8b8b9a90 con 0x562c8b9b55b0 2024-06-03T02:27:27.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.515+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562c8b8fcb80 con 0x562c8b9a9de0 2024-06-03T02:27:27.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.515+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562c8b8b7970 con 0x562c8b9aa700 2024-06-03T02:27:27.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.515+0000 7f0a41c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.515+0000 7f0a41c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56126/0 (socket says 172.21.15.79:56126) 2024-06-03T02:27:27.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.515+0000 7f0a41c00700 1 -- 172.21.15.79:0/2279763215 learned_addr learned my addr 172.21.15.79:0/2279763215 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:27.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.515+0000 7f0a41200700 1 --2- 172.21.15.79:0/2279763215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 0x562c8b9aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.517+0000 7f0a3a000700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a240a2ee0 0x7f0a240a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.517+0000 7f0a3a000700 1 -- 172.21.15.79:0/3379160825 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0a340ced30 con 0x7f0a3c10dc10 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.518+0000 7f0a42600700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a240a2ee0 0x7f0a240a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.519+0000 7f0a41c00700 1 -- 172.21.15.79:0/2279763215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 msgr2=0x562c8b9aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.519+0000 7f0a41c00700 1 --2- 172.21.15.79:0/2279763215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 0x562c8b9aa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.519+0000 7f0a41c00700 1 -- 172.21.15.79:0/2279763215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 msgr2=0x562c8b9b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.519+0000 7f0a41c00700 1 --2- 172.21.15.79:0/2279763215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 0x562c8b9b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.519+0000 7f0a41c00700 1 -- 172.21.15.79:0/2279763215 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8b8a5010 con 0x562c8b9aa700 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.519+0000 7f0a41c00700 1 --2- 172.21.15.79:0/2279763215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9b5070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009040 tx=0x7f0a30009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d25ee529604fef server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.520+0000 7f0a2ea00700 1 -- 172.21.15.79:0/2279763215 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3001f030 con 0x562c8b9aa700 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.520+0000 7f0a2ea00700 1 -- 172.21.15.79:0/2279763215 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3000b9f0 con 0x562c8b9aa700 2024-06-03T02:27:27.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.520+0000 7f0a2ea00700 1 -- 172.21.15.79:0/2279763215 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a300056b0 con 0x562c8b9aa700 2024-06-03T02:27:27.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.520+0000 7f0a42600700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a240a2ee0 0x7f0a240a53a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a3409aee0 tx=0x7f0a340bb040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:27.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.521+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/2279763215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 msgr2=0x562c8b9b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:27.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.521+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/2279763215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9b5070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009040 tx=0x7f0a30009470 comp rx=0 tx=0).stop 2024-06-03T02:27:27.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.522+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/2279763215 shutdown_connections 2024-06-03T02:27:27.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.522+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/2279763215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 0x562c8b9b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.522+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/2279763215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 0x562c8b9aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.522+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/2279763215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9b5070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.522+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/2279763215 >> 172.21.15.79:0/2279763215 conn(0x562c8b8d71f0 msgr2=0x562c8b9a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:27.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.522+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/2279763215 shutdown_connections 2024-06-03T02:27:27.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.522+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/2279763215 wait complete. 2024-06-03T02:27:27.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.523+0000 7f0a4efa5b80 1 -- start start 2024-06-03T02:27:27.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.523+0000 7f0a4efa5b80 1 Processor -- start 2024-06-03T02:27:27.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.523+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 0x562c8b9b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9aa700 0x562c8b9b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9b55b0 0x562c8b9b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562c8b898fe0 con 0x562c8b9a9de0 2024-06-03T02:27:27.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562c8b8f8b40 con 0x562c8b9aa700 2024-06-03T02:27:27.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562c8b9b7fe0 con 0x562c8b9b55b0 2024-06-03T02:27:27.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a41c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9aa700 0x562c8b9b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a4efa5b80 10 client.0 _ll_register_callbacks cb 0x562c8b892400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:27.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a41c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9aa700 0x562c8b9b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48530/0 (socket says 172.21.15.79:48530) 2024-06-03T02:27:27.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a41c00700 1 -- 172.21.15.79:0/712199729 learned_addr learned my addr 172.21.15.79:0/712199729 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:27.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a41200700 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9b55b0 0x562c8b9b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.524+0000 7f0a41200700 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 0x562c8b9b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.525+0000 7f0a41c00700 1 -- 172.21.15.79:0/712199729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9b55b0 msgr2=0x562c8b9b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.525+0000 7f0a41c00700 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9b55b0 0x562c8b9b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.525+0000 7f0a41c00700 1 -- 172.21.15.79:0/712199729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 msgr2=0x562c8b9b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.525+0000 7f0a41c00700 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 0x562c8b9b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.525+0000 7f0a41c00700 1 -- 172.21.15.79:0/712199729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8b84b950 con 0x562c8b9aa700 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.525+0000 7f0a41c00700 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9aa700 0x562c8b9b2010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0a30005210 tx=0x7f0a30004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a0eb6af6fd2f70 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.525+0000 7f0a23400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.526+0000 7f0a23400700 1 -- 172.21.15.79:0/712199729 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3001f040 con 0x562c8b9aa700 2024-06-03T02:27:27.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.526+0000 7f0a23400700 1 -- 172.21.15.79:0/712199729 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0a30002030 con 0x562c8b9aa700 2024-06-03T02:27:27.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.526+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/712199729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3000b9f0 con 0x562c8b9aa700 2024-06-03T02:27:27.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.527+0000 7f0a23400700 1 -- 172.21.15.79:0/712199729 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a30032050 con 0x562c8b9aa700 2024-06-03T02:27:27.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.527+0000 7f0a4efa5b80 10 client.26180 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:27.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.527+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/712199729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c8b9c26b0 con 0x562c8b9aa700 2024-06-03T02:27:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.528+0000 7f0a23400700 1 -- 172.21.15.79:0/712199729 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a300057f0 con 0x562c8b9aa700 2024-06-03T02:27:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.528+0000 7f0a23400700 1 -- 172.21.15.79:0/712199729 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0a30011070 con 0x562c8b9aa700 2024-06-03T02:27:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.528+0000 7f0a4efa5b80 10 client.26180 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.528+0000 7f0a4efa5b80 10 client.26180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:27:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.528+0000 7f0a4efa5b80 20 client.26180 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.528+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562c8b9c4b80 0x562c8b9c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.528+0000 7f0a4efa5b80 4 client.26180 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:27:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.528+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/712199729 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0a3000b9f0 con 0x562c8b9c4b80 2024-06-03T02:27:27.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.529+0000 7f0a41200700 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562c8b9c4b80 0x562c8b9c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.529+0000 7f0a41200700 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562c8b9c4b80 0x562c8b9c6f70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76af60438cf61fff server_cookie=2a1f771cc70cb23e in_seq=0 out_seq=0 2024-06-03T02:27:27.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.529+0000 7f0a23400700 10 client.26180 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:27.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.530+0000 7f0a23400700 1 -- 172.21.15.79:0/712199729 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f0a3000b9f0 con 0x562c8b9c4b80 2024-06-03T02:27:27.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.530+0000 7f0a23400700 10 client.26180 handle_command_reply: tid=0 2024-06-03T02:27:27.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.530+0000 7f0a4efa5b80 1 client.26180 shutdown 2024-06-03T02:27:27.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/712199729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9aa700 msgr2=0x562c8b9b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:27.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9aa700 0x562c8b9b2010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0a30005210 tx=0x7f0a30004d50 comp rx=0 tx=0).stop 2024-06-03T02:27:27.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/712199729 shutdown_connections 2024-06-03T02:27:27.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9b55b0 0x562c8b9b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562c8b9c4b80 0x562c8b9c6f70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 0x562c8b9b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/712199729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9aa700 0x562c8b9b2010 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a23400700 0 client.26180 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:27.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/712199729 >> 172.21.15.79:0/712199729 conn(0x562c8b8d71f0 msgr2=0x562c8b9a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:27.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/712199729 shutdown_connections 2024-06-03T02:27:27.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.531+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/712199729 wait complete. 2024-06-03T02:27:27.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.532+0000 7f0a4efa5b80 20 client.26180 trim_cache size 0 max 16384 2024-06-03T02:27:27.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.592+0000 7f0a4efa5b80 1 Processor -- start 2024-06-03T02:27:27.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.596+0000 7f0a4efa5b80 1 -- start start 2024-06-03T02:27:27.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.596+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 0x562c8b9c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.596+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.596+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9b55b0 0x562c8b9c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.596+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1bd4b0 con 0x562c8b9a9de0 2024-06-03T02:27:27.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.596+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1beff0 con 0x562c8b9b55b0 2024-06-03T02:27:27.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.596+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1bdb80 con 0x562c8b9aa700 2024-06-03T02:27:27.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.597+0000 7f0a41200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 0x562c8b9c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.597+0000 7f0a41200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 0x562c8b9c0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34334/0 (socket says 172.21.15.79:34334) 2024-06-03T02:27:27.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.597+0000 7f0a41200700 1 -- 172.21.15.79:0/4123670556 learned_addr learned my addr 172.21.15.79:0/4123670556 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:27.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.597+0000 7f0a41200700 1 --2- 172.21.15.79:0/4123670556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9b55b0 0x562c8b9c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.597+0000 7f0a41200700 1 -- 172.21.15.79:0/4123670556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 msgr2=0x562c8b9c0e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:27:27.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.597+0000 7f0a41200700 1 --2- 172.21.15.79:0/4123670556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9c0e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.597+0000 7f0a41200700 1 -- 172.21.15.79:0/4123670556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 msgr2=0x562c8b9c0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:27.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.597+0000 7f0a41200700 1 --2- 172.21.15.79:0/4123670556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 0x562c8b9c0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.597+0000 7f0a41200700 1 -- 172.21.15.79:0/4123670556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8b9b0580 con 0x562c8b9b55b0 2024-06-03T02:27:27.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.598+0000 7f0a41200700 1 --2- 172.21.15.79:0/4123670556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9b55b0 0x562c8b9c9fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009b20 tx=0x7f0a28009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9a45dbf457ee378 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:27.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.599+0000 7f0a2e000700 1 -- 172.21.15.79:0/4123670556 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2801b280 con 0x562c8b9b55b0 2024-06-03T02:27:27.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.599+0000 7f0a2e000700 1 -- 172.21.15.79:0/4123670556 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a28019930 con 0x562c8b9b55b0 2024-06-03T02:27:27.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.599+0000 7f0a2e000700 1 -- 172.21.15.79:0/4123670556 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28017980 con 0x562c8b9b55b0 2024-06-03T02:27:27.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.600+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/4123670556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9b55b0 msgr2=0x562c8b9c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:27.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.600+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/4123670556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9b55b0 0x562c8b9c9fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009b20 tx=0x7f0a28009580 comp rx=0 tx=0).stop 2024-06-03T02:27:27.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.600+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/4123670556 shutdown_connections 2024-06-03T02:27:27.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.600+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/4123670556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9b55b0 0x562c8b9c9fa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.600+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/4123670556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9a9de0 0x562c8b9c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.600+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/4123670556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.600+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/4123670556 >> 172.21.15.79:0/4123670556 conn(0x562c8b8d71f0 msgr2=0x562c8b9c0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:27.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.603+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/4123670556 shutdown_connections 2024-06-03T02:27:27.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.603+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/4123670556 wait complete. 2024-06-03T02:27:27.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.604+0000 7f0a4efa5b80 1 -- start start 2024-06-03T02:27:27.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.604+0000 7f0a4efa5b80 1 Processor -- start 2024-06-03T02:27:27.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.605+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 0x562c8b9c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.605+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.605+0000 7f0a4efa5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 0x562c8ba28760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.606+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1be920 con 0x562c8b9b55b0 2024-06-03T02:27:27.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.606+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1be250 con 0x562c8b9a9de0 2024-06-03T02:27:27.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.606+0000 7f0a4efa5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1bb450 con 0x562c8b9aa700 2024-06-03T02:27:27.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.606+0000 7f0a4efa5b80 10 client.0 _ll_register_callbacks cb 0x562c8b892400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:27.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.606+0000 7f0a41200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 0x562c8b9c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.606+0000 7f0a41200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 0x562c8b9c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48546/0 (socket says 172.21.15.79:48546) 2024-06-03T02:27:27.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.606+0000 7f0a41200700 1 -- 172.21.15.79:0/150604158 learned_addr learned my addr 172.21.15.79:0/150604158 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:27.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.607+0000 7f0a41c00700 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.608+0000 7f0a41200700 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 0x562c8ba28760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.608+0000 7f0a41c00700 1 -- 172.21.15.79:0/150604158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 msgr2=0x562c8b9c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:27.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.608+0000 7f0a41c00700 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 0x562c8b9c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.609+0000 7f0a41c00700 1 -- 172.21.15.79:0/150604158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 msgr2=0x562c8ba28760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:27.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.609+0000 7f0a41c00700 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 0x562c8ba28760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.609+0000 7f0a41c00700 1 -- 172.21.15.79:0/150604158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8ba28e40 con 0x562c8b9aa700 2024-06-03T02:27:27.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.610+0000 7f0a41200700 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 0x562c8ba28760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:27.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.611+0000 7f0a41c00700 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9ca020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x562c8b8b9ab0 tx=0x7f0a30009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bbfafcd5c60c115 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:27.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.611+0000 7f0a23400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:27:27.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.613+0000 7f0a23400700 1 -- 172.21.15.79:0/150604158 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3001f040 con 0x562c8b9aa700 2024-06-03T02:27:27.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.613+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/150604158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a28019930 con 0x562c8b9aa700 2024-06-03T02:27:27.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.614+0000 7f0a23400700 1 -- 172.21.15.79:0/150604158 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0a3000f040 con 0x562c8b9aa700 2024-06-03T02:27:27.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.614+0000 7f0a23400700 1 -- 172.21.15.79:0/150604158 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a3001f040 con 0x562c8b9aa700 2024-06-03T02:27:27.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.615+0000 7f0a4efa5b80 10 client.26182 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:27.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.615+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/150604158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c8ba313a0 con 0x562c8b9aa700 2024-06-03T02:27:27.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.616+0000 7f0a23400700 1 -- 172.21.15.79:0/150604158 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a28019930 con 0x562c8b9aa700 2024-06-03T02:27:27.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.616+0000 7f0a23400700 1 -- 172.21.15.79:0/150604158 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0a30011070 con 0x562c8b9aa700 2024-06-03T02:27:27.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.617+0000 7f0a4efa5b80 10 client.26182 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:27.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.617+0000 7f0a4efa5b80 10 client.26182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:27:27.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.617+0000 7f0a4efa5b80 20 client.26182 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:27.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.617+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562c8ba31990 0x562c8ba33d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:27.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.617+0000 7f0a4efa5b80 4 client.26182 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:27:27.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.617+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/150604158 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c8b90a080 con 0x562c8ba31990 2024-06-03T02:27:27.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.618+0000 7f0a41200700 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562c8ba31990 0x562c8ba33d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:27.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.620+0000 7f0a23400700 10 client.26182 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:27.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.620+0000 7f0a41200700 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562c8ba31990 0x562c8ba33d80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7ca13f8c8b1edf server_cookie=9b37272d17000669 in_seq=0 out_seq=0 2024-06-03T02:27:27.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.624+0000 7f0a23400700 1 -- 172.21.15.79:0/150604158 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c8b90a080 con 0x562c8ba31990 2024-06-03T02:27:27.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.624+0000 7f0a23400700 10 client.26182 handle_command_reply: tid=0 2024-06-03T02:27:27.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.624+0000 7f0a4efa5b80 1 client.26182 shutdown 2024-06-03T02:27:27.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/150604158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 msgr2=0x562c8b9ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:27.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9ca020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x562c8b8b9ab0 tx=0x7f0a30009470 comp rx=0 tx=0).stop 2024-06-03T02:27:27.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/150604158 shutdown_connections 2024-06-03T02:27:27.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562c8ba31990 0x562c8ba33d80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562c8b9b55b0 0x562c8ba28760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a23400700 0 client.26182 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:27.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c8b9a9de0 0x562c8b9c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 --2- 172.21.15.79:0/150604158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c8b9aa700 0x562c8b9ca020 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/150604158 >> 172.21.15.79:0/150604158 conn(0x562c8b8d71f0 msgr2=0x562c8b9c0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:27.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/150604158 shutdown_connections 2024-06-03T02:27:27.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.625+0000 7f0a4efa5b80 1 -- 172.21.15.79:0/150604158 wait complete. 2024-06-03T02:27:27.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.626+0000 7f0a4efa5b80 20 client.26182 trim_cache size 0 max 16384 2024-06-03T02:27:27.627 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:27:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:27:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "24d00964-44d4-4e4a-b5b8-52fdb6602280", 2024-06-03T02:27:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:27:27.628 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:27:27.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 -- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a240a2ee0 msgr2=0x7f0a240a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a240a2ee0 0x7f0a240a53a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a3409aee0 tx=0x7f0a340bb040 comp rx=0 tx=0).stop 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 -- 172.21.15.79:0/3379160825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c10dc10 msgr2=0x7f0a3c10e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c10dc10 0x7f0a3c10e0c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400e970 tx=0x7f0a3400ec80 comp rx=0 tx=0).stop 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 -- 172.21.15.79:0/3379160825 shutdown_connections 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a3c10dc10 0x7f0a3c10e0c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a3c0751e0 0x7f0a3c107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0a240a2ee0 0x7f0a240a53a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 --2- 172.21.15.79:0/3379160825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a3c10ec00 0x7f0a3c108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 -- 172.21.15.79:0/3379160825 >> 172.21.15.79:0/3379160825 conn(0x7f0a3c0fdf10 msgr2=0x7f0a3c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.628+0000 7f0a43000700 1 -- 172.21.15.79:0/3379160825 shutdown_connections 2024-06-03T02:27:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:27.629+0000 7f0a43000700 1 -- 172.21.15.79:0/3379160825 wait complete. 2024-06-03T02:27:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 440.0 s, 671.2 tps, lat 141.173 ms stddev 226.417 2024-06-03T02:27:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:28 smithi123 ceph-mon[36640]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 612 op/s 2024-06-03T02:27:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:28 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:27:28.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:28 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:27:28.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:28 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:27:28.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:28 smithi079 ceph-mon[31819]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 612 op/s 2024-06-03T02:27:28.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:28 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:27:28.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:28 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:27:28.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:28 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:27:28.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:28 smithi135 ceph-mon[36653]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 612 op/s 2024-06-03T02:27:28.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:28 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:27:28.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:28 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:27:28.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:28 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:27:29.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:29 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:27:29.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:29 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:27:29.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:29 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:27:29.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:29 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:27:29.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:29 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:27:29.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:29 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:27:30.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:27:30.712 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:30 smithi079 ceph-mon[31819]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.6 MiB/s wr, 650 op/s 2024-06-03T02:27:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:30 smithi123 ceph-mon[36640]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.6 MiB/s wr, 650 op/s 2024-06-03T02:27:30.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:30 smithi135 ceph-mon[36653]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.6 MiB/s wr, 650 op/s 2024-06-03T02:27:31.546 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:27:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:32 smithi123 ceph-mon[36640]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 543 op/s 2024-06-03T02:27:32.841 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:32 smithi079 ceph-mon[31819]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 543 op/s 2024-06-03T02:27:32.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:32 smithi135 ceph-mon[36653]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 543 op/s 2024-06-03T02:27:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 445.0 s, 558.8 tps, lat 188.052 ms stddev 252.801 2024-06-03T02:27:34.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:34 smithi079 ceph-mon[31819]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 MiB/s wr, 693 op/s 2024-06-03T02:27:34.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:34 smithi135 ceph-mon[36653]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 MiB/s wr, 693 op/s 2024-06-03T02:27:34.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:34 smithi123 ceph-mon[36640]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 MiB/s wr, 693 op/s 2024-06-03T02:27:35.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:35 smithi079 ceph-mon[31819]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 612 op/s 2024-06-03T02:27:35.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:35 smithi135 ceph-mon[36653]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 612 op/s 2024-06-03T02:27:35.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:35 smithi123 ceph-mon[36640]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 612 op/s 2024-06-03T02:27:36.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.017+0000 7f03cc800700 1 -- 172.21.15.79:0/1446693092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03c410eac0 msgr2=0x7f03c410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:36.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.017+0000 7f03cc800700 1 --2- 172.21.15.79:0/1446693092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03c410eac0 0x7f03c410eea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009990 tx=0x7f03b8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.018+0000 7f03cc800700 1 -- 172.21.15.79:0/1446693092 shutdown_connections 2024-06-03T02:27:36.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.018+0000 7f03cc800700 1 --2- 172.21.15.79:0/1446693092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c4109480 0x7f03c4115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.018+0000 7f03cc800700 1 --2- 172.21.15.79:0/1446693092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03c410eac0 0x7f03c410eea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.018+0000 7f03cc800700 1 --2- 172.21.15.79:0/1446693092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c4108ac0 0x7f03c4108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.018+0000 7f03cc800700 1 -- 172.21.15.79:0/1446693092 >> 172.21.15.79:0/1446693092 conn(0x7f03c4075c60 msgr2=0x7f03c4078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:36.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.020+0000 7f03cc800700 1 -- 172.21.15.79:0/1446693092 shutdown_connections 2024-06-03T02:27:36.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.021+0000 7f03cc800700 1 -- 172.21.15.79:0/1446693092 wait complete. 2024-06-03T02:27:36.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.022+0000 7f03cc800700 1 Processor -- start 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.022+0000 7f03cc800700 1 -- start start 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.022+0000 7f03cc800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c4108ac0 0x7f03c419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.022+0000 7f03cc800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c4109480 0x7f03c419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.022+0000 7f03cc800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03c410eac0 0x7f03c41a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.022+0000 7f03cc800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4118e60 con 0x7f03c4108ac0 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.022+0000 7f03cc800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4118ce0 con 0x7f03c4109480 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.022+0000 7f03cc800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4118fe0 con 0x7f03c410eac0 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.023+0000 7f03c9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c4108ac0 0x7f03c419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.023+0000 7f03c9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c4108ac0 0x7f03c419f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34254/0 (socket says 172.21.15.79:34254) 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.023+0000 7f03c9e00700 1 -- 172.21.15.79:0/2881784811 learned_addr learned my addr 172.21.15.79:0/2881784811 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.023+0000 7f03c9400700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c4109480 0x7f03c419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.023+0000 7f03ca800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03c410eac0 0x7f03c41a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.023+0000 7f03c9e00700 1 -- 172.21.15.79:0/2881784811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03c410eac0 msgr2=0x7f03c41a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.023+0000 7f03c9e00700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03c410eac0 0x7f03c41a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.024+0000 7f03c9e00700 1 -- 172.21.15.79:0/2881784811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c4109480 msgr2=0x7f03c419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.024+0000 7f03c9e00700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c4109480 0x7f03c419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.024+0000 7f03c9e00700 1 -- 172.21.15.79:0/2881784811 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c41a42a0 con 0x7f03c4108ac0 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.024+0000 7f03c9e00700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c4108ac0 0x7f03c419f290 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f03b8006010 tx=0x7f03b8005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.025+0000 7f03ca800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03c410eac0 0x7f03c41a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.025+0000 7f03bea00700 1 -- 172.21.15.79:0/2881784811 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8027070 con 0x7f03c4108ac0 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.025+0000 7f03bea00700 1 -- 172.21.15.79:0/2881784811 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b8004050 con 0x7f03c4108ac0 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.025+0000 7f03cc800700 1 -- 172.21.15.79:0/2881784811 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c41a4530 con 0x7f03c4108ac0 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.025+0000 7f03bea00700 1 -- 172.21.15.79:0/2881784811 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b800f460 con 0x7f03c4108ac0 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.026+0000 7f03cc800700 1 -- 172.21.15.79:0/2881784811 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c41abe10 con 0x7f03c4108ac0 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.032+0000 7f03bea00700 1 -- 172.21.15.79:0/2881784811 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f03b802a070 con 0x7f03c4108ac0 2024-06-03T02:27:36.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.035+0000 7f03d71c7b80 1 Processor -- start 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.035+0000 7f03bea00700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03ac0a2df0 0x7f03ac0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.035+0000 7f03bea00700 1 -- 172.21.15.79:0/2881784811 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f03b80d0640 con 0x7f03c4108ac0 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.036+0000 7f03d71c7b80 1 -- start start 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.037+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbbc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.037+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbbca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.037+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.037+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5561cbacba10 con 0x5561cbbc7300 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.037+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5561cbb0eb00 con 0x5561cbbbc5b0 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.037+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5561cbac98f0 con 0x5561cbb38e40 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.037+0000 7f03c9400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbbca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.037+0000 7f03c9400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbbca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47742/0 (socket says 172.21.15.79:47742) 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.037+0000 7f03c9400700 1 -- 172.21.15.79:0/4064882182 learned_addr learned my addr 172.21.15.79:0/4064882182 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:36.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.038+0000 7f03ca800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03ac0a2df0 0x7f03ac0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.038+0000 7f03c9e00700 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbbc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.043+0000 7f03c9400700 1 -- 172.21.15.79:0/4064882182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 msgr2=0x5561cbbbc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.043+0000 7f03c9400700 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbbc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.043+0000 7f03c9400700 1 -- 172.21.15.79:0/4064882182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 msgr2=0x5561cbbc96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.040+0000 7f03ca800700 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.043+0000 7f03c9400700 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.043+0000 7f03c9400700 1 -- 172.21.15.79:0/4064882182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cbab6f90 con 0x5561cbbbc5b0 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.044+0000 7f03c9e00700 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbbc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.044+0000 7f03ca800700 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.045+0000 7f03c9400700 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbbca30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f03b400dc40 tx=0x7f03b400df50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b49320397c8f2c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.045+0000 7f03b3400700 1 -- 172.21.15.79:0/4064882182 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4015070 con 0x5561cbbbc5b0 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.046+0000 7f03b3400700 1 -- 172.21.15.79:0/4064882182 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b4004d10 con 0x5561cbbbc5b0 2024-06-03T02:27:36.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.046+0000 7f03b3400700 1 -- 172.21.15.79:0/4064882182 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4011440 con 0x5561cbbbc5b0 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.046+0000 7f03d71c7b80 1 -- 172.21.15.79:0/4064882182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 msgr2=0x5561cbbbca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.046+0000 7f03ca800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03ac0a2df0 0x7f03ac0a52b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f03c000e400 tx=0x7f03c0007ac0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.046+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbbca30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f03b400dc40 tx=0x7f03b400df50 comp rx=0 tx=0).stop 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.047+0000 7f03d71c7b80 1 -- 172.21.15.79:0/4064882182 shutdown_connections 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.047+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.047+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbbc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.047+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/4064882182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbbca30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.047+0000 7f03d71c7b80 1 -- 172.21.15.79:0/4064882182 >> 172.21.15.79:0/4064882182 conn(0x5561cbae9170 msgr2=0x5561cbbb84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.047+0000 7f03d71c7b80 1 -- 172.21.15.79:0/4064882182 shutdown_connections 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.049+0000 7f03d71c7b80 1 -- 172.21.15.79:0/4064882182 wait complete. 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.049+0000 7f03d71c7b80 1 -- start start 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.050+0000 7f03d71c7b80 1 Processor -- start 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.051+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbc6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.051+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbc3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.051+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03ca800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03ca800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34278/0 (socket says 172.21.15.79:34278) 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03ca800700 1 -- 172.21.15.79:0/1032681647 learned_addr learned my addr 172.21.15.79:0/1032681647 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5561cbaaaf60 con 0x5561cbbc7300 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5561cbb0aac0 con 0x5561cbbbc5b0 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5561cbbdafd0 con 0x5561cbb38e40 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03c9e00700 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbc6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03c9400700 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbc3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03d71c7b80 10 client.0 _ll_register_callbacks cb 0x5561cbaa8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03ca800700 1 -- 172.21.15.79:0/1032681647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 msgr2=0x5561cbbc6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:36.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03ca800700 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbc6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03ca800700 1 -- 172.21.15.79:0/1032681647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 msgr2=0x5561cbbc3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.052+0000 7f03ca800700 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbc3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.053+0000 7f03ca800700 1 -- 172.21.15.79:0/1032681647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cba5d8d0 con 0x5561cbbc7300 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.053+0000 7f03ca800700 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc4330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f03c00045e0 tx=0x7f03c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b242b0aba7c2435 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.053+0000 7f03a7e00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.056+0000 7f03a7e00700 1 -- 172.21.15.79:0/1032681647 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0003e50 con 0x5561cbbc7300 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.056+0000 7f03a7e00700 1 -- 172.21.15.79:0/1032681647 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f03c001b8f0 con 0x5561cbbc7300 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.056+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b4004d10 con 0x5561cbbc7300 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.057+0000 7f03a7e00700 1 -- 172.21.15.79:0/1032681647 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f03c001baf0 con 0x5561cbbc7300 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.057+0000 7f03d71c7b80 10 client.15822 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.057+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5561cbbe5670 con 0x5561cbbc7300 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.059+0000 7f03a7e00700 1 -- 172.21.15.79:0/1032681647 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b4004d10 con 0x5561cbbc7300 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.059+0000 7f03a7e00700 1 -- 172.21.15.79:0/1032681647 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f03c0022070 con 0x5561cbbc7300 2024-06-03T02:27:36.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.061+0000 7f03d71c7b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.061+0000 7f03d71c7b80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.061+0000 7f03d71c7b80 20 client.15822 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.061+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5561cbbe7c00 0x5561cbbea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.061+0000 7f03d71c7b80 4 client.15822 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.061+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561cbb1c000 con 0x5561cbbe7c00 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.061+0000 7f03c9e00700 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5561cbbe7c00 0x5561cbbea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.062+0000 7f03c9e00700 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5561cbbe7c00 0x5561cbbea010 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe27263b0010ad2 server_cookie=39643c6061df9390 in_seq=0 out_seq=0 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.062+0000 7f03a7e00700 10 client.15822 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.066+0000 7f03a7e00700 1 -- 172.21.15.79:0/1032681647 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561cbb1c000 con 0x5561cbbe7c00 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.066+0000 7f03a7e00700 10 client.15822 handle_command_reply: tid=0 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.067+0000 7f03d71c7b80 1 client.15822 shutdown 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.067+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 msgr2=0x5561cbbc4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.067+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc4330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f03c00045e0 tx=0x7f03c0004ad0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.067+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 shutdown_connections 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.067+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5561cbbe7c00 0x5561cbbea010 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.067+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc4330 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.067+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbc6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.067+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1032681647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbc3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.067+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 >> 172.21.15.79:0/1032681647 conn(0x5561cbae9170 msgr2=0x5561cbbb7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.069+0000 7f03a7e00700 0 client.15822 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.070+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 shutdown_connections 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.070+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1032681647 wait complete. 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.070+0000 7f03d71c7b80 20 client.15822 trim_cache size 0 max 16384 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.128+0000 7f03d71c7b80 1 Processor -- start 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.128+0000 7f03d71c7b80 1 -- start start 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.128+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbb75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.128+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbc86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.128+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.128+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41ac0b0 con 0x5561cbbc7300 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.128+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41adbf0 con 0x5561cbbbc5b0 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.128+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41ac780 con 0x5561cbb38e40 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.129+0000 7f03c9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbb75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.129+0000 7f03c9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbb75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:35680/0 (socket says 172.21.15.79:35680) 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.129+0000 7f03c9e00700 1 -- 172.21.15.79:0/1730624162 learned_addr learned my addr 172.21.15.79:0/1730624162 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.129+0000 7f03c9400700 1 --2- 172.21.15.79:0/1730624162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbc86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.129+0000 7f03c9400700 1 -- 172.21.15.79:0/1730624162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 msgr2=0x5561cbbb75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.129+0000 7f03c9400700 1 --2- 172.21.15.79:0/1730624162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbb75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.129+0000 7f03c9400700 1 -- 172.21.15.79:0/1730624162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 msgr2=0x5561cbbc8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.130+0000 7f03c9400700 1 --2- 172.21.15.79:0/1730624162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.130+0000 7f03c9400700 1 -- 172.21.15.79:0/1730624162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cbbc23a0 con 0x5561cbbbc5b0 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.130+0000 7f03c9400700 1 --2- 172.21.15.79:0/1730624162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbc86f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f03b4005e90 tx=0x7f03b400fbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1185a6a67520941 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.132+0000 7f03b2a00700 1 -- 172.21.15.79:0/1730624162 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4015070 con 0x5561cbbbc5b0 2024-06-03T02:27:36.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.132+0000 7f03b2a00700 1 -- 172.21.15.79:0/1730624162 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b4004510 con 0x5561cbbbc5b0 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.132+0000 7f03b2a00700 1 -- 172.21.15.79:0/1730624162 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4004970 con 0x5561cbbbc5b0 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.133+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1730624162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 msgr2=0x5561cbbc86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.133+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1730624162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbc86f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f03b4005e90 tx=0x7f03b400fbb0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.133+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1730624162 shutdown_connections 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.133+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1730624162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbbc7300 0x5561cbbc8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.133+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1730624162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbb38e40 0x5561cbbb75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.133+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1730624162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbbc5b0 0x5561cbbc86f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.134+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1730624162 >> 172.21.15.79:0/1730624162 conn(0x5561cbae9170 msgr2=0x5561cbbb6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.134+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1730624162 shutdown_connections 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.134+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1730624162 wait complete. 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.135+0000 7f03d71c7b80 1 -- start start 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.135+0000 7f03d71c7b80 1 Processor -- start 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.137+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbb38e40 0x5561cbbe0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.137+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbbbc5b0 0x5561cbbe0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.137+0000 7f03d71c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbc7300 0x5561cbbc00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.137+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41ad520 con 0x5561cbb38e40 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.137+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41ace50 con 0x5561cbbc7300 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.137+0000 7f03d71c7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4081560 con 0x5561cbbbc5b0 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.138+0000 7f03d71c7b80 10 client.0 _ll_register_callbacks cb 0x5561cbaa8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.138+0000 7f03ca800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbc7300 0x5561cbbc00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.138+0000 7f03ca800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbc7300 0x5561cbbc00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47762/0 (socket says 172.21.15.79:47762) 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.138+0000 7f03ca800700 1 -- 172.21.15.79:0/1709358005 learned_addr learned my addr 172.21.15.79:0/1709358005 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.139+0000 7f03c9400700 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbbbc5b0 0x5561cbbe0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.139+0000 7f03c9e00700 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbb38e40 0x5561cbbe0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.139+0000 7f03ca800700 1 -- 172.21.15.79:0/1709358005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbbbc5b0 msgr2=0x5561cbbe0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.139+0000 7f03ca800700 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbbbc5b0 0x5561cbbe0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.139+0000 7f03ca800700 1 -- 172.21.15.79:0/1709358005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbb38e40 msgr2=0x5561cbbe0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.139+0000 7f03ca800700 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbb38e40 0x5561cbbe0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.139+0000 7f03ca800700 1 -- 172.21.15.79:0/1709358005 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cbbc0600 con 0x5561cbbc7300 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.140+0000 7f03c9e00700 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbb38e40 0x5561cbbe0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.140+0000 7f03ca800700 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbc7300 0x5561cbbc00c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f03c0024600 tx=0x7f03c0024440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb976860b4309ca7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.140+0000 7f03a7e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.140+0000 7f03a7e00700 1 -- 172.21.15.79:0/1709358005 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000eb50 con 0x5561cbbc7300 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.141+0000 7f03a7e00700 1 -- 172.21.15.79:0/1709358005 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f03c0010040 con 0x5561cbbc7300 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.141+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1709358005 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b4004510 con 0x5561cbbc7300 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.141+0000 7f03a7e00700 1 -- 172.21.15.79:0/1709358005 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f03c0004ad0 con 0x5561cbbc7300 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.142+0000 7f03d71c7b80 10 client.26198 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.142+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1709358005 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5561cbbc1880 con 0x5561cbbc7300 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.142+0000 7f03a7e00700 1 -- 172.21.15.79:0/1709358005 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b4004510 con 0x5561cbbc7300 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.142+0000 7f03a7e00700 1 -- 172.21.15.79:0/1709358005 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f03c0022070 con 0x5561cbbc7300 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.144+0000 7f03d71c7b80 10 client.26198 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:36.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.145+0000 7f03d71c7b80 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.145+0000 7f03d71c7b80 20 client.26198 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.145+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5561cbc538c0 0x5561cbc55cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.145+0000 7f03d71c7b80 4 client.26198 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.145+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1709358005 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561cbb16580 con 0x5561cbc538c0 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.145+0000 7f03c9e00700 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5561cbc538c0 0x5561cbc55cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.148+0000 7f03a7e00700 10 client.26198 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.148+0000 7f03c9e00700 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5561cbc538c0 0x5561cbc55cb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5e98a00d723d333 server_cookie=6398b8ba70ae422b in_seq=0 out_seq=0 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.149+0000 7f03a7e00700 1 -- 172.21.15.79:0/1709358005 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5561cbb16580 con 0x5561cbc538c0 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.149+0000 7f03a7e00700 10 client.26198 handle_command_reply: tid=0 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.149+0000 7f03d71c7b80 1 client.26198 shutdown 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.149+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1709358005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbc7300 msgr2=0x5561cbbc00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.149+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbc7300 0x5561cbbc00c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f03c0024600 tx=0x7f03c0024440 comp rx=0 tx=0).stop 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.150+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1709358005 shutdown_connections 2024-06-03T02:27:36.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.150+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561cbbc7300 0x5561cbbc00c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.150+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5561cbc538c0 0x5561cbc55cb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.150+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5561cbb38e40 0x5561cbbe0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.150+0000 7f03d71c7b80 1 --2- 172.21.15.79:0/1709358005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561cbbbc5b0 0x5561cbbe0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.150+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1709358005 >> 172.21.15.79:0/1709358005 conn(0x5561cbae9170 msgr2=0x5561cbbb7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:36.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.150+0000 7f03a7e00700 0 client.26198 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:27:36.665 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:27:36.665 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:27:36.665 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:27:36.665 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:27:36.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.151+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1709358005 shutdown_connections 2024-06-03T02:27:36.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.151+0000 7f03d71c7b80 1 -- 172.21.15.79:0/1709358005 wait complete. 2024-06-03T02:27:36.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.152+0000 7f03d71c7b80 20 client.26198 trim_cache size 0 max 16384 2024-06-03T02:27:36.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.156+0000 7f03cc800700 1 -- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03ac0a2df0 msgr2=0x7f03ac0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:36.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.156+0000 7f03cc800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03ac0a2df0 0x7f03ac0a52b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f03c000e400 tx=0x7f03c0007ac0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.156+0000 7f03cc800700 1 -- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c4108ac0 msgr2=0x7f03c419f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:36.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.156+0000 7f03cc800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c4108ac0 0x7f03c419f290 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f03b8006010 tx=0x7f03b8005ec0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.156+0000 7f03cc800700 1 -- 172.21.15.79:0/2881784811 shutdown_connections 2024-06-03T02:27:36.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.156+0000 7f03cc800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03c410eac0 0x7f03c41a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.156+0000 7f03cc800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03ac0a2df0 0x7f03ac0a52b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.156+0000 7f03cc800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c4108ac0 0x7f03c419f290 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f03b8006010 tx=0x7f03b8005ec0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.156+0000 7f03cc800700 1 --2- 172.21.15.79:0/2881784811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c4109480 0x7f03c419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:36.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.157+0000 7f03cc800700 1 -- 172.21.15.79:0/2881784811 >> 172.21.15.79:0/2881784811 conn(0x7f03c4075c60 msgr2=0x7f03c4076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:36.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.157+0000 7f03cc800700 1 -- 172.21.15.79:0/2881784811 shutdown_connections 2024-06-03T02:27:36.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:36.157+0000 7f03cc800700 1 -- 172.21.15.79:0/2881784811 wait complete. 2024-06-03T02:27:37.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:37 smithi123 ceph-mon[36640]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 575 op/s 2024-06-03T02:27:38.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:37 smithi079 ceph-mon[31819]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 575 op/s 2024-06-03T02:27:38.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:37 smithi135 ceph-mon[36653]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 575 op/s 2024-06-03T02:27:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 450.0 s, 416.6 tps, lat 235.606 ms stddev 283.199 2024-06-03T02:27:38.778 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:27:38.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:27:39.459 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:27:39.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:39 smithi123 ceph-mon[36640]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.4 MiB/s wr, 572 op/s 2024-06-03T02:27:40.005 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:39 smithi079 ceph-mon[31819]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.4 MiB/s wr, 572 op/s 2024-06-03T02:27:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:39 smithi135 ceph-mon[36653]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.4 MiB/s wr, 572 op/s 2024-06-03T02:27:41.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:41 smithi123 ceph-mon[36640]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-06-03T02:27:42.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:41 smithi079 ceph-mon[31819]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-06-03T02:27:42.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:41 smithi135 ceph-mon[36653]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-06-03T02:27:43.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:42 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:27:43.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:42 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:27:43.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:42 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:27:43.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 455.0 s, 431.4 tps, lat 224.323 ms stddev 300.750 2024-06-03T02:27:44.262 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:44 smithi079 ceph-mon[31819]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 537 op/s 2024-06-03T02:27:44.262 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.262 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.262 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.262 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:44 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:44 smithi135 ceph-mon[36653]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 537 op/s 2024-06-03T02:27:44.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:44 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:44 smithi123 ceph-mon[36640]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 537 op/s 2024-06-03T02:27:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:44 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.149+0000 7fac8c000700 1 -- 172.21.15.79:0/2815451575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac8410e220 msgr2=0x7fac84115df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.149+0000 7fac8c000700 1 --2- 172.21.15.79:0/2815451575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac8410e220 0x7fac84115df0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00b0a0 tx=0x7fac7c00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.151+0000 7fac8c000700 1 -- 172.21.15.79:0/2815451575 shutdown_connections 2024-06-03T02:27:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.151+0000 7fac8c000700 1 --2- 172.21.15.79:0/2815451575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac8410e220 0x7fac84115df0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.151+0000 7fac8c000700 1 --2- 172.21.15.79:0/2815451575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac841096f0 0x7fac84109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.151+0000 7fac8c000700 1 --2- 172.21.15.79:0/2815451575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac8410a0a0 0x7fac8410db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.152+0000 7fac8c000700 1 -- 172.21.15.79:0/2815451575 >> 172.21.15.79:0/2815451575 conn(0x7fac8406d600 msgr2=0x7fac8406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.152+0000 7fac8c000700 1 -- 172.21.15.79:0/2815451575 shutdown_connections 2024-06-03T02:27:45.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.152+0000 7fac8c000700 1 -- 172.21.15.79:0/2815451575 wait complete. 2024-06-03T02:27:45.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.154+0000 7fac8c000700 1 Processor -- start 2024-06-03T02:27:45.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.154+0000 7fac8c000700 1 -- start start 2024-06-03T02:27:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.156+0000 7fac8c000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac841096f0 0x7fac84115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.156+0000 7fac8c000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8410a0a0 0x7fac84112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.156+0000 7fac8c000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac8410e220 0x7fac84113490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.158+0000 7fac88c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8410a0a0 0x7fac84112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.158+0000 7fac88c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8410a0a0 0x7fac84112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36066/0 (socket says 172.21.15.79:36066) 2024-06-03T02:27:45.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.159+0000 7fac89600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac841096f0 0x7fac84115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:45.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.159+0000 7fac8a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac8410e220 0x7fac84113490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:45.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.159+0000 7fac88c00700 1 -- 172.21.15.79:0/302215840 learned_addr learned my addr 172.21.15.79:0/302215840 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:45.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.160+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac84118c60 con 0x7fac841096f0 2024-06-03T02:27:45.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.160+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac84118ae0 con 0x7fac8410a0a0 2024-06-03T02:27:45.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.160+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fac84118de0 con 0x7fac8410e220 2024-06-03T02:27:45.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.162+0000 7fac8a000700 1 -- 172.21.15.79:0/302215840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8410a0a0 msgr2=0x7fac84112dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:45.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.162+0000 7fac8a000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8410a0a0 0x7fac84112dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.162+0000 7fac8a000700 1 -- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac841096f0 msgr2=0x7fac84115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:45.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.164+0000 7fac8a000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac841096f0 0x7fac84115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.164+0000 7fac8a000700 1 -- 172.21.15.79:0/302215840 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac841b3e90 con 0x7fac8410e220 2024-06-03T02:27:45.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.167+0000 7fac89600700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac841096f0 0x7fac84115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:45.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.167+0000 7fac8a000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac8410e220 0x7fac84113490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00b0a0 tx=0x7fac7c004c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:45.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.171+0000 7fac82000700 1 -- 172.21.15.79:0/302215840 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c020070 con 0x7fac8410e220 2024-06-03T02:27:45.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.172+0000 7fac82000700 1 -- 172.21.15.79:0/302215840 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac7c0075c0 con 0x7fac8410e220 2024-06-03T02:27:45.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.172+0000 7fac82000700 1 -- 172.21.15.79:0/302215840 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c01c6a0 con 0x7fac8410e220 2024-06-03T02:27:45.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.172+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac841b40c0 con 0x7fac8410e220 2024-06-03T02:27:45.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.172+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac841b4580 con 0x7fac8410e220 2024-06-03T02:27:45.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.173+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8410a6b0 con 0x7fac8410e220 2024-06-03T02:27:45.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.179+0000 7fac82000700 1 -- 172.21.15.79:0/302215840 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fac7c023070 con 0x7fac8410e220 2024-06-03T02:27:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.183+0000 7fac82000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac6c0a2db0 0x7fac6c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.185+0000 7fac82000700 1 -- 172.21.15.79:0/302215840 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fac7c0d07c0 con 0x7fac8410e220 2024-06-03T02:27:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.185+0000 7fac82000700 1 -- 172.21.15.79:0/302215840 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fac7c0fe8d0 con 0x7fac8410e220 2024-06-03T02:27:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.186+0000 7fac89600700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac6c0a2db0 0x7fac6c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.187+0000 7fac89600700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac6c0a2db0 0x7fac6c0a5270 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac7800afd0 tx=0x7fac7800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:45.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.652+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac841142f0 con 0x7fac8410e220 2024-06-03T02:27:45.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.653+0000 7fac82000700 1 -- 172.21.15.79:0/302215840 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7fac7c09d480 con 0x7fac8410e220 2024-06-03T02:27:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:27:45.655 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:27:45.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac6c0a2db0 msgr2=0x7fac6c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:45.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac6c0a2db0 0x7fac6c0a5270 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac7800afd0 tx=0x7fac7800c040 comp rx=0 tx=0).stop 2024-06-03T02:27:45.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac8410e220 msgr2=0x7fac84113490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:45.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac8410e220 0x7fac84113490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00b0a0 tx=0x7fac7c004c40 comp rx=0 tx=0).stop 2024-06-03T02:27:45.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 shutdown_connections 2024-06-03T02:27:45.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac8410e220 0x7fac84113490 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac6c0a2db0 0x7fac6c0a5270 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac841096f0 0x7fac84115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 --2- 172.21.15.79:0/302215840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8410a0a0 0x7fac84112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:45.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.659+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 >> 172.21.15.79:0/302215840 conn(0x7fac8406d600 msgr2=0x7fac84072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:45.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.661+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 shutdown_connections 2024-06-03T02:27:45.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:45.661+0000 7fac8c000700 1 -- 172.21.15.79:0/302215840 wait complete. 2024-06-03T02:27:45.667 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:27:46.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:46 smithi079 ceph-mon[31819]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 431 op/s 2024-06-03T02:27:46.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:46 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:27:46.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:46 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:27:46.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:46 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:46.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:46 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/302215840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:27:46.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:46 smithi135 ceph-mon[36653]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 431 op/s 2024-06-03T02:27:46.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:46 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:27:46.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:46 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:27:46.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:46 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:46.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:46 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/302215840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:27:46.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:46 smithi123 ceph-mon[36640]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 431 op/s 2024-06-03T02:27:46.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:46 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:27:46.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:46 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:27:46.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:46 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:27:46.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:46 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/302215840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:27:46.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:27:47.348 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:27:48.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 460.0 s, 491.2 tps, lat 203.717 ms stddev 287.034 2024-06-03T02:27:48.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:48 smithi079 ceph-mon[31819]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.1 MiB/s wr, 423 op/s 2024-06-03T02:27:48.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:48 smithi135 ceph-mon[36653]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.1 MiB/s wr, 423 op/s 2024-06-03T02:27:48.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:48 smithi123 ceph-mon[36640]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.1 MiB/s wr, 423 op/s 2024-06-03T02:27:50.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:50 smithi079 ceph-mon[31819]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-06-03T02:27:50.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:50 smithi135 ceph-mon[36653]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-06-03T02:27:50.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:50 smithi123 ceph-mon[36640]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-06-03T02:27:50.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.645+0000 7f1334000700 1 -- 172.21.15.79:0/2809288465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132c06a170 msgr2=0x7f132c06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:50.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.645+0000 7f1334000700 1 --2- 172.21.15.79:0/2809288465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132c06a170 0x7f132c06a5f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f131c00faf0 tx=0x7f131c00fe00 comp rx=0 tx=0).stop 2024-06-03T02:27:50.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.645+0000 7f1334000700 1 -- 172.21.15.79:0/2809288465 shutdown_connections 2024-06-03T02:27:50.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.645+0000 7f1334000700 1 --2- 172.21.15.79:0/2809288465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f132c06ab30 0x7f132c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.645+0000 7f1334000700 1 --2- 172.21.15.79:0/2809288465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f132c108500 0x7f132c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.645+0000 7f1334000700 1 --2- 172.21.15.79:0/2809288465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132c06a170 0x7f132c06a5f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.645+0000 7f1334000700 1 -- 172.21.15.79:0/2809288465 >> 172.21.15.79:0/2809288465 conn(0x7f132c078340 msgr2=0x7f132c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:50.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.646+0000 7f1334000700 1 -- 172.21.15.79:0/2809288465 shutdown_connections 2024-06-03T02:27:50.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.646+0000 7f1334000700 1 -- 172.21.15.79:0/2809288465 wait complete. 2024-06-03T02:27:50.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.648+0000 7f1334000700 1 Processor -- start 2024-06-03T02:27:50.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.652+0000 7f1334000700 1 -- start start 2024-06-03T02:27:50.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.654+0000 7f1334000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f132c06a170 0x7f132c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.654+0000 7f1334000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f132c06ab30 0x7f132c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.655+0000 7f1334000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132c108500 0x7f132c196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.655+0000 7f1334000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f132c110690 con 0x7f132c06a170 2024-06-03T02:27:50.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.655+0000 7f1334000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f132c110510 con 0x7f132c108500 2024-06-03T02:27:50.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.655+0000 7f1334000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f132c110810 con 0x7f132c06ab30 2024-06-03T02:27:50.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.656+0000 7f1331600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f132c06a170 0x7f132c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.655+0000 7f1330c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f132c06ab30 0x7f132c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.656+0000 7f1330c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f132c06ab30 0x7f132c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:45146/0 (socket says 172.21.15.79:45146) 2024-06-03T02:27:50.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.656+0000 7f1330c00700 1 -- 172.21.15.79:0/2628315294 learned_addr learned my addr 172.21.15.79:0/2628315294 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:50.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.656+0000 7f1332000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132c108500 0x7f132c196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.658+0000 7f1330c00700 1 -- 172.21.15.79:0/2628315294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132c108500 msgr2=0x7f132c196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:50.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.658+0000 7f1330c00700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132c108500 0x7f132c196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.658+0000 7f1330c00700 1 -- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f132c06a170 msgr2=0x7f132c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:50.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.661+0000 7f1330c00700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f132c06a170 0x7f132c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.662+0000 7f1330c00700 1 -- 172.21.15.79:0/2628315294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c197030 con 0x7f132c06ab30 2024-06-03T02:27:50.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.662+0000 7f1331600700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f132c06a170 0x7f132c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:27:50.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.665+0000 7f1330c00700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f132c06ab30 0x7f132c19cc10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f131c009870 tx=0x7f131c0036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:50.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.665+0000 7f132a000700 1 -- 172.21.15.79:0/2628315294 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c00f5a0 con 0x7f132c06ab30 2024-06-03T02:27:50.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.665+0000 7f1334000700 1 -- 172.21.15.79:0/2628315294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c1972c0 con 0x7f132c06ab30 2024-06-03T02:27:50.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.666+0000 7f1334000700 1 -- 172.21.15.79:0/2628315294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c1a3530 con 0x7f132c06ab30 2024-06-03T02:27:50.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.666+0000 7f132a000700 1 -- 172.21.15.79:0/2628315294 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c0042e0 con 0x7f132c06ab30 2024-06-03T02:27:50.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.666+0000 7f132a000700 1 -- 172.21.15.79:0/2628315294 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c005290 con 0x7f132c06ab30 2024-06-03T02:27:50.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.674+0000 7f132a000700 1 -- 172.21.15.79:0/2628315294 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f131c005570 con 0x7f132c06ab30 2024-06-03T02:27:50.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.679+0000 7f133e88fb80 1 Processor -- start 2024-06-03T02:27:50.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.679+0000 7f132a000700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f13140a2e90 0x7f13140a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.680+0000 7f132a000700 1 -- 172.21.15.79:0/2628315294 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f131c0d4510 con 0x7f132c06ab30 2024-06-03T02:27:50.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.680+0000 7f133e88fb80 1 -- start start 2024-06-03T02:27:50.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.680+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eebb90 0x562e90eebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.680+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90ee5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.680+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90ee6240 0x562e90ee6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.680+0000 7f133e88fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562e90df4a10 con 0x562e90eebb90 2024-06-03T02:27:50.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.680+0000 7f133e88fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562e90e37b00 con 0x562e90eec4b0 2024-06-03T02:27:50.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.680+0000 7f133e88fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562e90df28f0 con 0x562e90ee6240 2024-06-03T02:27:50.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.680+0000 7f1332000700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f13140a2e90 0x7f13140a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.681+0000 7f1330c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90ee5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.681+0000 7f1330c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90ee5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36098/0 (socket says 172.21.15.79:36098) 2024-06-03T02:27:50.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.681+0000 7f1330c00700 1 -- 172.21.15.79:0/2595338379 learned_addr learned my addr 172.21.15.79:0/2595338379 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:50.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.681+0000 7f1330c00700 1 -- 172.21.15.79:0/2595338379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90ee6240 msgr2=0x562e90ee6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:50.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.681+0000 7f1330c00700 1 --2- 172.21.15.79:0/2595338379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90ee6240 0x562e90ee6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.681+0000 7f1330c00700 1 -- 172.21.15.79:0/2595338379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eebb90 msgr2=0x562e90eebf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:50.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.681+0000 7f1330c00700 1 --2- 172.21.15.79:0/2595338379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eebb90 0x562e90eebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.681+0000 7f1330c00700 1 -- 172.21.15.79:0/2595338379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e90ddff90 con 0x562e90eec4b0 2024-06-03T02:27:50.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.682+0000 7f1332000700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f13140a2e90 0x7f13140a5350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1324003f70 tx=0x7f1324009c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:50.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.683+0000 7f1330c00700 1 --2- 172.21.15.79:0/2595338379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90ee5d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f131c0a9100 tx=0x7f131c0a0430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=339a11734177de1e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:50.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.683+0000 7f131aa00700 1 -- 172.21.15.79:0/2595338379 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c02e040 con 0x562e90eec4b0 2024-06-03T02:27:50.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.683+0000 7f131aa00700 1 -- 172.21.15.79:0/2595338379 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c0a4e00 con 0x562e90eec4b0 2024-06-03T02:27:50.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.683+0000 7f131aa00700 1 -- 172.21.15.79:0/2595338379 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c0a70d0 con 0x562e90eec4b0 2024-06-03T02:27:50.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.685+0000 7f133e88fb80 1 -- 172.21.15.79:0/2595338379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 msgr2=0x562e90ee5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:50.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.685+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2595338379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90ee5d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f131c0a9100 tx=0x7f131c0a0430 comp rx=0 tx=0).stop 2024-06-03T02:27:50.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.685+0000 7f133e88fb80 1 -- 172.21.15.79:0/2595338379 shutdown_connections 2024-06-03T02:27:50.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.685+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2595338379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90ee6240 0x562e90ee6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.685+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2595338379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eebb90 0x562e90eebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.685+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2595338379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90ee5d00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.687+0000 7f133e88fb80 1 -- 172.21.15.79:0/2595338379 >> 172.21.15.79:0/2595338379 conn(0x562e90e12170 msgr2=0x562e90ee0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:50.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.687+0000 7f133e88fb80 1 -- 172.21.15.79:0/2595338379 shutdown_connections 2024-06-03T02:27:50.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.687+0000 7f133e88fb80 1 -- 172.21.15.79:0/2595338379 wait complete. 2024-06-03T02:27:50.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.688+0000 7f133e88fb80 1 -- start start 2024-06-03T02:27:50.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.688+0000 7f133e88fb80 1 Processor -- start 2024-06-03T02:27:50.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.689+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 0x562e90ef22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.689+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eebb90 0x562e90eef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.690+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eec4b0 0x562e90eefb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.690+0000 7f133e88fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562e90dd3f60 con 0x562e90eec4b0 2024-06-03T02:27:50.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.690+0000 7f133e88fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562e90e33ac0 con 0x562e90ee6240 2024-06-03T02:27:50.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.690+0000 7f133e88fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562e90e37b00 con 0x562e90eebb90 2024-06-03T02:27:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.690+0000 7f133e88fb80 10 client.0 _ll_register_callbacks cb 0x562e90dcd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.690+0000 7f1332000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eec4b0 0x562e90eefb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.690+0000 7f1332000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eec4b0 0x562e90eefb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42752/0 (socket says 172.21.15.79:42752) 2024-06-03T02:27:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.690+0000 7f1332000700 1 -- 172.21.15.79:0/1723601068 learned_addr learned my addr 172.21.15.79:0/1723601068 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.691+0000 7f1332000700 1 -- 172.21.15.79:0/1723601068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eebb90 msgr2=0x562e90eef5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:27:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.691+0000 7f1332000700 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eebb90 0x562e90eef5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.691+0000 7f1332000700 1 -- 172.21.15.79:0/1723601068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 msgr2=0x562e90ef22d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:27:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.691+0000 7f1332000700 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 0x562e90ef22d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.691+0000 7f1332000700 1 -- 172.21.15.79:0/1723601068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e90d868d0 con 0x562e90eec4b0 2024-06-03T02:27:50.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.692+0000 7f1332000700 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eec4b0 0x562e90eefb10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1324009140 tx=0x7f1324004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6337aea82c7a2294 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:50.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.693+0000 7f130f400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:27:50.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.693+0000 7f130f400700 1 -- 172.21.15.79:0/1723601068 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132401c050 con 0x562e90eec4b0 2024-06-03T02:27:50.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.693+0000 7f133e88fb80 1 -- 172.21.15.79:0/1723601068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f131c0a4e00 con 0x562e90eec4b0 2024-06-03T02:27:50.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.693+0000 7f130f400700 1 -- 172.21.15.79:0/1723601068 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1324015ce0 con 0x562e90eec4b0 2024-06-03T02:27:50.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.694+0000 7f130f400700 1 -- 172.21.15.79:0/1723601068 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1324009e20 con 0x562e90eec4b0 2024-06-03T02:27:50.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.694+0000 7f133e88fb80 10 client.15831 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:50.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.694+0000 7f133e88fb80 1 -- 172.21.15.79:0/1723601068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e90f05be0 con 0x562e90eec4b0 2024-06-03T02:27:50.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.695+0000 7f130f400700 1 -- 172.21.15.79:0/1723601068 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c0a4e00 con 0x562e90eec4b0 2024-06-03T02:27:50.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.696+0000 7f130f400700 1 -- 172.21.15.79:0/1723601068 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1324016500 con 0x562e90eec4b0 2024-06-03T02:27:50.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.696+0000 7f133e88fb80 10 client.15831 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:50.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.696+0000 7f133e88fb80 10 client.15831 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:27:50.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.696+0000 7f133e88fb80 20 client.15831 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:50.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.696+0000 7f133e88fb80 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x562e90f08270 0x562e90f0a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.696+0000 7f133e88fb80 4 client.15831 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:27:50.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.696+0000 7f133e88fb80 1 -- 172.21.15.79:0/1723601068 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e90e45000 con 0x562e90f08270 2024-06-03T02:27:50.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.696+0000 7f1331600700 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x562e90f08270 0x562e90f0a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.697+0000 7f1331600700 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x562e90f08270 0x562e90f0a6f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34c06a3035f63cee server_cookie=5888a32f8305312 in_seq=0 out_seq=0 2024-06-03T02:27:50.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.697+0000 7f130f400700 10 client.15831 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:27:50.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.698+0000 7f130f400700 1 -- 172.21.15.79:0/1723601068 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562e90e45000 con 0x562e90f08270 2024-06-03T02:27:50.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.698+0000 7f130f400700 10 client.15831 handle_command_reply: tid=0 2024-06-03T02:27:50.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.698+0000 7f133e88fb80 1 client.15831 shutdown 2024-06-03T02:27:50.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.702+0000 7f133e88fb80 1 -- 172.21.15.79:0/1723601068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eec4b0 msgr2=0x562e90eefb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:50.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.702+0000 7f133e88fb80 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eec4b0 0x562e90eefb10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1324009140 tx=0x7f1324004b40 comp rx=0 tx=0).stop 2024-06-03T02:27:50.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.702+0000 7f133e88fb80 1 -- 172.21.15.79:0/1723601068 shutdown_connections 2024-06-03T02:27:50.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.702+0000 7f133e88fb80 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x562e90f08270 0x562e90f0a6f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.702+0000 7f133e88fb80 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eec4b0 0x562e90eefb10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.702+0000 7f133e88fb80 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 0x562e90ef22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.702+0000 7f133e88fb80 1 --2- 172.21.15.79:0/1723601068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eebb90 0x562e90eef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.702+0000 7f133e88fb80 1 -- 172.21.15.79:0/1723601068 >> 172.21.15.79:0/1723601068 conn(0x562e90e12170 msgr2=0x562e90eda790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:50.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.702+0000 7f130f400700 0 client.15831 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:27:50.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.703+0000 7f133e88fb80 1 -- 172.21.15.79:0/1723601068 shutdown_connections 2024-06-03T02:27:50.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.703+0000 7f133e88fb80 1 -- 172.21.15.79:0/1723601068 wait complete. 2024-06-03T02:27:50.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.703+0000 7f133e88fb80 20 client.15831 trim_cache size 0 max 16384 2024-06-03T02:27:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.744+0000 7f133e88fb80 1 Processor -- start 2024-06-03T02:27:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.744+0000 7f133e88fb80 1 -- start start 2024-06-03T02:27:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.744+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 0x562e90ee5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.744+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eebb90 0x562e90ee11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.744+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eec4b0 0x562e90edb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.744+0000 7f133e88fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f131000eb40 con 0x562e90eebb90 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.744+0000 7f133e88fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1310010680 con 0x562e90ee6240 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.744+0000 7f133e88fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f131000f210 con 0x562e90eec4b0 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1331600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 0x562e90ee5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1331600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 0x562e90ee5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36112/0 (socket says 172.21.15.79:36112) 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1331600700 1 -- 172.21.15.79:0/118170464 learned_addr learned my addr 172.21.15.79:0/118170464 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1332000700 1 --2- 172.21.15.79:0/118170464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eec4b0 0x562e90edb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1330c00700 1 --2- 172.21.15.79:0/118170464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eebb90 0x562e90ee11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1331600700 1 -- 172.21.15.79:0/118170464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eec4b0 msgr2=0x562e90edb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1331600700 1 --2- 172.21.15.79:0/118170464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eec4b0 0x562e90edb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1331600700 1 -- 172.21.15.79:0/118170464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eebb90 msgr2=0x562e90ee11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1331600700 1 --2- 172.21.15.79:0/118170464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eebb90 0x562e90ee11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.745+0000 7f1331600700 1 -- 172.21.15.79:0/118170464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e90ef05e0 con 0x562e90ee6240 2024-06-03T02:27:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.746+0000 7f1331600700 1 --2- 172.21.15.79:0/118170464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 0x562e90ee5d00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x562e90f0af30 tx=0x7f1320004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c377080fe98b9bf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.746+0000 7f131a000700 1 -- 172.21.15.79:0/118170464 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320004020 con 0x562e90ee6240 2024-06-03T02:27:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.746+0000 7f131a000700 1 -- 172.21.15.79:0/118170464 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f132001e030 con 0x562e90ee6240 2024-06-03T02:27:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.746+0000 7f131a000700 1 -- 172.21.15.79:0/118170464 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132001a610 con 0x562e90ee6240 2024-06-03T02:27:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.747+0000 7f133e88fb80 1 -- 172.21.15.79:0/118170464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 msgr2=0x562e90ee5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.747+0000 7f133e88fb80 1 --2- 172.21.15.79:0/118170464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 0x562e90ee5d00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x562e90f0af30 tx=0x7f1320004750 comp rx=0 tx=0).stop 2024-06-03T02:27:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.748+0000 7f133e88fb80 1 -- 172.21.15.79:0/118170464 shutdown_connections 2024-06-03T02:27:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.748+0000 7f133e88fb80 1 --2- 172.21.15.79:0/118170464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eec4b0 0x562e90edb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.748+0000 7f133e88fb80 1 --2- 172.21.15.79:0/118170464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90ee6240 0x562e90ee5d00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.748+0000 7f133e88fb80 1 --2- 172.21.15.79:0/118170464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90eebb90 0x562e90ee11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.748+0000 7f133e88fb80 1 -- 172.21.15.79:0/118170464 >> 172.21.15.79:0/118170464 conn(0x562e90e12170 msgr2=0x562e90ee0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.749+0000 7f133e88fb80 1 -- 172.21.15.79:0/118170464 shutdown_connections 2024-06-03T02:27:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.749+0000 7f133e88fb80 1 -- 172.21.15.79:0/118170464 wait complete. 2024-06-03T02:27:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.750+0000 7f133e88fb80 1 -- start start 2024-06-03T02:27:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.750+0000 7f133e88fb80 1 Processor -- start 2024-06-03T02:27:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.753+0000 7f133e88fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90ee6240 0x562e90f005e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.754+0000 7f1331600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90ee6240 0x562e90f005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.754+0000 7f1331600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90ee6240 0x562e90f005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42768/0 (socket says 172.21.15.79:42768) 2024-06-03T02:27:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.754+0000 7f1331600700 1 -- 172.21.15.79:0/2115610728 learned_addr learned my addr 172.21.15.79:0/2115610728 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.754+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eebb90 0x562e90f00b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.754+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90f0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.755+0000 7f1330c00700 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eebb90 0x562e90f00b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.755+0000 7f1332000700 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90f0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.755+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f131000ffb0 con 0x562e90ee6240 2024-06-03T02:27:50.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.755+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f131000f8e0 con 0x562e90eec4b0 2024-06-03T02:27:50.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.755+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f131000cae0 con 0x562e90eebb90 2024-06-03T02:27:50.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.756+0000 7f1332000700 1 -- 172.21.15.79:0/2115610728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eebb90 msgr2=0x562e90f00b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.756+0000 7f1332000700 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eebb90 0x562e90f00b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.756+0000 7f1332000700 1 -- 172.21.15.79:0/2115610728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90ee6240 msgr2=0x562e90f005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.756+0000 7f1332000700 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90ee6240 0x562e90f005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.756+0000 7f1332000700 1 -- 172.21.15.79:0/2115610728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e90f0f590 con 0x562e90eec4b0 2024-06-03T02:27:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.756+0000 7f1331600700 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90ee6240 0x562e90f005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:27:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.756+0000 7f1332000700 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90f0f050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1324009310 tx=0x7f1324005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e43b87d674aea server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.756+0000 7f130f400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:27:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.757+0000 7f133e88fb80 10 client.0 _ll_register_callbacks cb 0x562e90dcd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.757+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f132001e030 con 0x562e90eec4b0 2024-06-03T02:27:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.757+0000 7f130f400700 1 -- 172.21.15.79:0/2115610728 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132401c040 con 0x562e90eec4b0 2024-06-03T02:27:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.757+0000 7f130f400700 1 -- 172.21.15.79:0/2115610728 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1324039020 con 0x562e90eec4b0 2024-06-03T02:27:50.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.758+0000 7f130f400700 1 -- 172.21.15.79:0/2115610728 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1324041050 con 0x562e90eec4b0 2024-06-03T02:27:50.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.758+0000 7f133e88fb80 10 client.26219 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:50.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.758+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x562e90f72570 con 0x562e90eec4b0 2024-06-03T02:27:50.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.759+0000 7f130f400700 1 -- 172.21.15.79:0/2115610728 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1324033070 con 0x562e90eec4b0 2024-06-03T02:27:50.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.759+0000 7f133e88fb80 10 client.26219 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:50.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.759+0000 7f133e88fb80 10 client.26219 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:27:50.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.759+0000 7f133e88fb80 20 client.26219 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:50.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.759+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x562e90f72bc0 0x562e90f75060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:50.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.759+0000 7f133e88fb80 4 client.26219 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:27:50.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.759+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e90e3f580 con 0x562e90f72bc0 2024-06-03T02:27:50.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.760+0000 7f1331600700 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x562e90f72bc0 0x562e90f75060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:50.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.760+0000 7f1331600700 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x562e90f72bc0 0x562e90f75060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3ace18c6766dd9b server_cookie=8c9697f8334e3170 in_seq=0 out_seq=0 2024-06-03T02:27:50.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.760+0000 7f130f400700 10 client.26219 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:27:50.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.760+0000 7f130f400700 1 -- 172.21.15.79:0/2115610728 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e90e3f580 con 0x562e90f72bc0 2024-06-03T02:27:50.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.760+0000 7f130f400700 10 client.26219 handle_command_reply: tid=0 2024-06-03T02:27:50.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 client.26219 shutdown 2024-06-03T02:27:50.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 msgr2=0x562e90f0f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:50.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90f0f050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1324009310 tx=0x7f1324005c60 comp rx=0 tx=0).stop 2024-06-03T02:27:50.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 shutdown_connections 2024-06-03T02:27:50.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x562e90f72bc0 0x562e90f75060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e90eec4b0 0x562e90f0f050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e90ee6240 0x562e90f005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 --2- 172.21.15.79:0/2115610728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e90eebb90 0x562e90f00b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 >> 172.21.15.79:0/2115610728 conn(0x562e90e12170 msgr2=0x562e90eec930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:50.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f130f400700 0 client.26219 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:27:50.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.761+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 shutdown_connections 2024-06-03T02:27:50.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.762+0000 7f133e88fb80 1 -- 172.21.15.79:0/2115610728 wait complete. 2024-06-03T02:27:50.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.762+0000 7f133e88fb80 20 client.26219 trim_cache size 0 max 16384 2024-06-03T02:27:50.764 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:27:50.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 -- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f13140a2e90 msgr2=0x7f13140a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f13140a2e90 0x7f13140a5350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1324003f70 tx=0x7f1324009c50 comp rx=0 tx=0).stop 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 -- 172.21.15.79:0/2628315294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f132c06ab30 msgr2=0x7f132c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f132c06ab30 0x7f132c19cc10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f131c009870 tx=0x7f131c0036f0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 -- 172.21.15.79:0/2628315294 shutdown_connections 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f132c108500 0x7f132c196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f13140a2e90 0x7f13140a5350 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f132c06a170 0x7f132c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 --2- 172.21.15.79:0/2628315294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f132c06ab30 0x7f132c19cc10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.763+0000 7f1334000700 1 -- 172.21.15.79:0/2628315294 >> 172.21.15.79:0/2628315294 conn(0x7f132c078340 msgr2=0x7f132c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.764+0000 7f1334000700 1 -- 172.21.15.79:0/2628315294 shutdown_connections 2024-06-03T02:27:50.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:50.764+0000 7f1334000700 1 -- 172.21.15.79:0/2628315294 wait complete. 2024-06-03T02:27:51.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:27:52.285 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:27:52.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:52 smithi135 ceph-mon[36653]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-06-03T02:27:52.349 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:52 smithi079 ceph-mon[31819]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-06-03T02:27:52.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:52 smithi123 ceph-mon[36640]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-06-03T02:27:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 465.0 s, 561.4 tps, lat 185.864 ms stddev 224.963 2024-06-03T02:27:54.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:54 smithi079 ceph-mon[31819]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.9 MiB/s wr, 504 op/s 2024-06-03T02:27:54.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:54 smithi135 ceph-mon[36653]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.9 MiB/s wr, 504 op/s 2024-06-03T02:27:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:54 smithi123 ceph-mon[36640]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.9 MiB/s wr, 504 op/s 2024-06-03T02:27:56.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:56 smithi079 ceph-mon[31819]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-06-03T02:27:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:56 smithi135 ceph-mon[36653]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-06-03T02:27:56.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:56 smithi123 ceph-mon[36640]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-06-03T02:27:57.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.410+0000 7f2ae2c00700 1 -- 172.21.15.79:0/635289399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2adc102540 msgr2=0x7f2adc1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:57.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.410+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/635289399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2adc102540 0x7f2adc1029a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009990 tx=0x7f2ac8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.411+0000 7f2ae2c00700 1 -- 172.21.15.79:0/635289399 shutdown_connections 2024-06-03T02:27:57.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.411+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/635289399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc102ee0 0x7f2adc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.411+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/635289399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2adc107450 0x7f2adc107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.411+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/635289399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2adc102540 0x7f2adc1029a0 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009990 tx=0x7f2ac8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.411+0000 7f2ae2c00700 1 -- 172.21.15.79:0/635289399 >> 172.21.15.79:0/635289399 conn(0x7f2adc0795f0 msgr2=0x7f2adc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:57.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.411+0000 7f2ae2c00700 1 -- 172.21.15.79:0/635289399 shutdown_connections 2024-06-03T02:27:57.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.411+0000 7f2ae2c00700 1 -- 172.21.15.79:0/635289399 wait complete. 2024-06-03T02:27:57.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.413+0000 7f2ae2c00700 1 Processor -- start 2024-06-03T02:27:57.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.413+0000 7f2ae2c00700 1 -- start start 2024-06-03T02:27:57.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.413+0000 7f2ae2c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc102ee0 0x7f2adc19b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.413+0000 7f2ae2c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2adc107450 0x7f2adc19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.413+0000 7f2ae2c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2adc19fa10 0x7f2adc19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.413+0000 7f2ae2c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1128b0 con 0x7f2adc19fa10 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.413+0000 7f2ae2c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc112730 con 0x7f2adc107450 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.413+0000 7f2ae2c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc112a30 con 0x7f2adc102ee0 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc102ee0 0x7f2adc19b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc102ee0 0x7f2adc19b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42918/0 (socket says 172.21.15.79:42918) 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adbe00700 1 -- 172.21.15.79:0/1089361292 learned_addr learned my addr 172.21.15.79:0/1089361292 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adb400700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2adc107450 0x7f2adc19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adbe00700 1 -- 172.21.15.79:0/1089361292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2adc107450 msgr2=0x7f2adc19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adbe00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2adc107450 0x7f2adc19b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adbe00700 1 -- 172.21.15.79:0/1089361292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2adc19fa10 msgr2=0x7f2adc19fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adbe00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2adc19fa10 0x7f2adc19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adbe00700 1 -- 172.21.15.79:0/1089361292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc1a0530 con 0x7f2adc102ee0 2024-06-03T02:27:57.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.414+0000 7f2adbe00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc102ee0 0x7f2adc19b140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00b550 tx=0x7f2acc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:57.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.415+0000 7f2ad8c00700 1 -- 172.21.15.79:0/1089361292 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc015020 con 0x7f2adc102ee0 2024-06-03T02:27:57.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.415+0000 7f2ad8c00700 1 -- 172.21.15.79:0/1089361292 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2acc011d10 con 0x7f2adc102ee0 2024-06-03T02:27:57.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.415+0000 7f2ad8c00700 1 -- 172.21.15.79:0/1089361292 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc00f2e0 con 0x7f2adc102ee0 2024-06-03T02:27:57.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.415+0000 7f2ae2c00700 1 -- 172.21.15.79:0/1089361292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc1a0820 con 0x7f2adc102ee0 2024-06-03T02:27:57.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.415+0000 7f2ae2c00700 1 -- 172.21.15.79:0/1089361292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc1a8000 con 0x7f2adc102ee0 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.419+0000 7f2ad8c00700 1 -- 172.21.15.79:0/1089361292 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2acc018020 con 0x7f2adc102ee0 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.421+0000 7f2ad8c00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ac00a2d70 0x7f2ac00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.422+0000 7f2ad8c00700 1 -- 172.21.15.79:0/1089361292 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2acc0ceb60 con 0x7f2adc102ee0 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.423+0000 7f2adb400700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ac00a2d70 0x7f2ac00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.424+0000 7f2adb400700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ac00a2d70 0x7f2ac00a5230 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b5c0 tx=0x7f2ac8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.425+0000 7f2aed564b80 1 Processor -- start 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.425+0000 7f2aed564b80 1 -- start start 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.426+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b800433f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.426+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8004344b0 0x55b80042dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.426+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b80042e240 0x55b80042e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.426+0000 7f2aed564b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b80033ca10 con 0x55b8004344b0 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.426+0000 7f2aed564b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b80037fb00 con 0x55b800433b90 2024-06-03T02:27:57.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.426+0000 7f2aed564b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b80033a8f0 con 0x55b80042e240 2024-06-03T02:27:57.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2ae0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b80042e240 0x55b80042e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2ae0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b80042e240 0x55b80042e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42930/0 (socket says 172.21.15.79:42930) 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2ae0c00700 1 -- 172.21.15.79:0/2787346869 learned_addr learned my addr 172.21.15.79:0/2787346869 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2adbe00700 1 --2- 172.21.15.79:0/2787346869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b800433f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2adbe00700 1 -- 172.21.15.79:0/2787346869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b80042e240 msgr2=0x55b80042e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2adbe00700 1 --2- 172.21.15.79:0/2787346869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b80042e240 0x55b80042e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2adbe00700 1 -- 172.21.15.79:0/2787346869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8004344b0 msgr2=0x55b80042dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2adbe00700 1 --2- 172.21.15.79:0/2787346869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8004344b0 0x55b80042dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2adbe00700 1 -- 172.21.15.79:0/2787346869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b800327f90 con 0x55b800433b90 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.427+0000 7f2adbe00700 1 --2- 172.21.15.79:0/2787346869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b800433f70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2acc09ad10 tx=0x7f2acc09ac20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b430bf030a2d01 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.428+0000 7f2ad1600700 1 -- 172.21.15.79:0/2787346869 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc09b010 con 0x55b800433b90 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.428+0000 7f2ad1600700 1 -- 172.21.15.79:0/2787346869 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2acc09f3c0 con 0x55b800433b90 2024-06-03T02:27:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.428+0000 7f2ad1600700 1 -- 172.21.15.79:0/2787346869 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc09b010 con 0x55b800433b90 2024-06-03T02:27:57.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.429+0000 7f2aed564b80 1 -- 172.21.15.79:0/2787346869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 msgr2=0x55b800433f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:57.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.429+0000 7f2aed564b80 1 --2- 172.21.15.79:0/2787346869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b800433f70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2acc09ad10 tx=0x7f2acc09ac20 comp rx=0 tx=0).stop 2024-06-03T02:27:57.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.429+0000 7f2aed564b80 1 -- 172.21.15.79:0/2787346869 shutdown_connections 2024-06-03T02:27:57.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.429+0000 7f2aed564b80 1 --2- 172.21.15.79:0/2787346869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b80042e240 0x55b80042e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.429+0000 7f2aed564b80 1 --2- 172.21.15.79:0/2787346869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b800433f70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.429+0000 7f2aed564b80 1 --2- 172.21.15.79:0/2787346869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8004344b0 0x55b80042dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.429+0000 7f2aed564b80 1 -- 172.21.15.79:0/2787346869 >> 172.21.15.79:0/2787346869 conn(0x55b80035a170 msgr2=0x55b800428ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:57.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.431+0000 7f2aed564b80 1 -- 172.21.15.79:0/2787346869 shutdown_connections 2024-06-03T02:27:57.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.434+0000 7f2aed564b80 1 -- 172.21.15.79:0/2787346869 wait complete. 2024-06-03T02:27:57.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.434+0000 7f2aed564b80 1 -- start start 2024-06-03T02:27:57.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.435+0000 7f2aed564b80 1 Processor -- start 2024-06-03T02:27:57.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.435+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b80042e240 0x55b80043a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.435+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b8004375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.435+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 0x55b800437b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.436+0000 7f2adb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b8004375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.436+0000 7f2adb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b8004375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:44296/0 (socket says 172.21.15.79:44296) 2024-06-03T02:27:57.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.436+0000 7f2adb400700 1 -- 172.21.15.79:0/3192084433 learned_addr learned my addr 172.21.15.79:0/3192084433 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:57.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.436+0000 7f2adbe00700 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b80042e240 0x55b80043a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.436+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b80031bf60 con 0x55b80042e240 2024-06-03T02:27:57.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.436+0000 7f2ae0c00700 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 0x55b800437b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.436+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b80037bac0 con 0x55b800433b90 2024-06-03T02:27:57.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.436+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b80043ae90 con 0x55b8004344b0 2024-06-03T02:27:57.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.436+0000 7f2aed564b80 10 client.0 _ll_register_callbacks cb 0x55b800315380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.437+0000 7f2adbe00700 1 -- 172.21.15.79:0/3192084433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 msgr2=0x55b800437b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.437+0000 7f2adbe00700 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 0x55b800437b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.437+0000 7f2adbe00700 1 -- 172.21.15.79:0/3192084433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 msgr2=0x55b8004375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.437+0000 7f2adbe00700 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b8004375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.437+0000 7f2adbe00700 1 -- 172.21.15.79:0/3192084433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8002ce8d0 con 0x55b80042e240 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.437+0000 7f2adb400700 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b8004375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.437+0000 7f2adbe00700 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b80042e240 0x55b80043a2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0107b0 tx=0x7f2acc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01836e9cb1da7c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.437+0000 7f2ac6000700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.438+0000 7f2ac6000700 1 -- 172.21.15.79:0/3192084433 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc07e060 con 0x55b80042e240 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.438+0000 7f2ac6000700 1 -- 172.21.15.79:0/3192084433 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2acc09b270 con 0x55b80042e240 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.438+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2acc09f3c0 con 0x55b80042e240 2024-06-03T02:27:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.439+0000 7f2ac6000700 1 -- 172.21.15.79:0/3192084433 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2acc0990b0 con 0x55b80042e240 2024-06-03T02:27:57.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.439+0000 7f2aed564b80 10 client.15840 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:57.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.440+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b8004454b0 con 0x55b80042e240 2024-06-03T02:27:57.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.440+0000 7f2ac6000700 1 -- 172.21.15.79:0/3192084433 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2acc09f3c0 con 0x55b80042e240 2024-06-03T02:27:57.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.440+0000 7f2ac6000700 1 -- 172.21.15.79:0/3192084433 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2acc09a460 con 0x55b80042e240 2024-06-03T02:27:57.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.440+0000 7f2aed564b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:57.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.441+0000 7f2aed564b80 10 client.15840 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:27:57.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.441+0000 7f2aed564b80 20 client.15840 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:57.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.441+0000 7f2aed564b80 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b800447980 0x55b800449d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.441+0000 7f2aed564b80 4 client.15840 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:27:57.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.441+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b80038d000 con 0x55b800447980 2024-06-03T02:27:57.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.443+0000 7f2adb400700 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b800447980 0x55b800449d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.444+0000 7f2ac6000700 10 client.15840 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:27:57.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.444+0000 7f2adb400700 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b800447980 0x55b800449d70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4040a47626d9349d server_cookie=bc1af9a6ad8f6d6a in_seq=0 out_seq=0 2024-06-03T02:27:57.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.445+0000 7f2ac6000700 1 -- 172.21.15.79:0/3192084433 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b80038d000 con 0x55b800447980 2024-06-03T02:27:57.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.445+0000 7f2ac6000700 10 client.15840 handle_command_reply: tid=0 2024-06-03T02:27:57.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.445+0000 7f2aed564b80 1 client.15840 shutdown 2024-06-03T02:27:57.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.445+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b80042e240 msgr2=0x55b80043a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:57.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.445+0000 7f2aed564b80 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b80042e240 0x55b80043a2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0107b0 tx=0x7f2acc010740 comp rx=0 tx=0).stop 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 shutdown_connections 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2aed564b80 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 0x55b800437b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2aed564b80 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b800447980 0x55b800449d70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2aed564b80 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b80042e240 0x55b80043a2f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2aed564b80 1 --2- 172.21.15.79:0/3192084433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b800433b90 0x55b8004375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2ac6000700 0 client.15840 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 >> 172.21.15.79:0/3192084433 conn(0x55b80035a170 msgr2=0x55b800422790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 shutdown_connections 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2aed564b80 1 -- 172.21.15.79:0/3192084433 wait complete. 2024-06-03T02:27:57.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.446+0000 7f2aed564b80 20 client.15840 trim_cache size 0 max 16384 2024-06-03T02:27:57.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.464+0000 7f2aed564b80 1 Processor -- start 2024-06-03T02:27:57.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.464+0000 7f2aed564b80 1 -- start start 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 0x55b80042dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b800433b90 0x55b8004351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 0x55b800431c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2aed564b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1ac7a0 con 0x55b800433b90 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2aed564b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1ae2e0 con 0x55b80042e240 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2aed564b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1ace70 con 0x55b8004344b0 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2adb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b800433b90 0x55b8004351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2adb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b800433b90 0x55b8004351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41868/0 (socket says 172.21.15.79:41868) 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2adb400700 1 -- 172.21.15.79:0/293421692 learned_addr learned my addr 172.21.15.79:0/293421692 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:57.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.465+0000 7f2ae0c00700 1 --2- 172.21.15.79:0/293421692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 0x55b800431c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.466+0000 7f2adb400700 1 -- 172.21.15.79:0/293421692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 msgr2=0x55b800431c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:57.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.466+0000 7f2adb400700 1 --2- 172.21.15.79:0/293421692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 0x55b800431c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.466+0000 7f2adb400700 1 -- 172.21.15.79:0/293421692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 msgr2=0x55b80042dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:57.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.466+0000 7f2adb400700 1 --2- 172.21.15.79:0/293421692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 0x55b80042dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.466+0000 7f2adb400700 1 -- 172.21.15.79:0/293421692 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b800435b60 con 0x55b800433b90 2024-06-03T02:27:57.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.466+0000 7f2adb400700 1 --2- 172.21.15.79:0/293421692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b800433b90 0x55b8004351c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009990 tx=0x7f2ac801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4770429218854414 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:57.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.468+0000 7f2ad0c00700 1 -- 172.21.15.79:0/293421692 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8033070 con 0x55b800433b90 2024-06-03T02:27:57.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.468+0000 7f2ad0c00700 1 -- 172.21.15.79:0/293421692 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac8005690 con 0x55b800433b90 2024-06-03T02:27:57.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.468+0000 7f2ad0c00700 1 -- 172.21.15.79:0/293421692 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac802e970 con 0x55b800433b90 2024-06-03T02:27:57.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.469+0000 7f2aed564b80 1 -- 172.21.15.79:0/293421692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b800433b90 msgr2=0x55b8004351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:57.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.469+0000 7f2aed564b80 1 --2- 172.21.15.79:0/293421692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b800433b90 0x55b8004351c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009990 tx=0x7f2ac801e000 comp rx=0 tx=0).stop 2024-06-03T02:27:57.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.469+0000 7f2aed564b80 1 -- 172.21.15.79:0/293421692 shutdown_connections 2024-06-03T02:27:57.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.469+0000 7f2aed564b80 1 --2- 172.21.15.79:0/293421692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8004344b0 0x55b800431c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.469+0000 7f2aed564b80 1 --2- 172.21.15.79:0/293421692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 0x55b80042dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.469+0000 7f2aed564b80 1 --2- 172.21.15.79:0/293421692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b800433b90 0x55b8004351c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.469+0000 7f2aed564b80 1 -- 172.21.15.79:0/293421692 >> 172.21.15.79:0/293421692 conn(0x55b80035a170 msgr2=0x55b800432790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:57.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.469+0000 7f2aed564b80 1 -- 172.21.15.79:0/293421692 shutdown_connections 2024-06-03T02:27:57.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.470+0000 7f2aed564b80 1 -- 172.21.15.79:0/293421692 wait complete. 2024-06-03T02:27:57.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.470+0000 7f2aed564b80 1 -- start start 2024-06-03T02:27:57.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.470+0000 7f2aed564b80 1 Processor -- start 2024-06-03T02:27:57.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.473+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 0x55b80044d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.474+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b800433b90 0x55b80044a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.474+0000 7f2adbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 0x55b80044d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.474+0000 7f2aed564b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8004344b0 0x55b80044ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.474+0000 7f2adbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 0x55b80044d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:44312/0 (socket says 172.21.15.79:44312) 2024-06-03T02:27:57.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.474+0000 7f2adb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b800433b90 0x55b80044a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.474+0000 7f2adb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b800433b90 0x55b80044a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42952/0 (socket says 172.21.15.79:42952) 2024-06-03T02:27:57.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.474+0000 7f2ae0c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8004344b0 0x55b80044ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.474+0000 7f2adbe00700 1 -- 172.21.15.79:0/1837936428 learned_addr learned my addr 172.21.15.79:0/1837936428 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:27:57.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.475+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1adc10 con 0x55b8004344b0 2024-06-03T02:27:57.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.475+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1ad540 con 0x55b80042e240 2024-06-03T02:27:57.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.475+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1aa740 con 0x55b800433b90 2024-06-03T02:27:57.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.475+0000 7f2aed564b80 10 client.0 _ll_register_callbacks cb 0x55b800315380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:27:57.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.475+0000 7f2adbe00700 1 -- 172.21.15.79:0/1837936428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b800433b90 msgr2=0x55b80044a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:57.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.475+0000 7f2adbe00700 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b800433b90 0x55b80044a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.475+0000 7f2adbe00700 1 -- 172.21.15.79:0/1837936428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8004344b0 msgr2=0x55b80044ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:27:57.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.475+0000 7f2adbe00700 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8004344b0 0x55b80044ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.475+0000 7f2adbe00700 1 -- 172.21.15.79:0/1837936428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b80044b3f0 con 0x55b80042e240 2024-06-03T02:27:57.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.476+0000 7f2adbe00700 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 0x55b80044d400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2acc004840 tx=0x7f2acc0a3a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=befe572adc230488 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:27:57.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.476+0000 7f2ac6000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:27:57.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.476+0000 7f2ac6000700 1 -- 172.21.15.79:0/1837936428 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc089030 con 0x55b80042e240 2024-06-03T02:27:57.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.476+0000 7f2ac6000700 1 -- 172.21.15.79:0/1837936428 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2acc0bb0b0 con 0x55b80042e240 2024-06-03T02:27:57.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.476+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac8005690 con 0x55b80042e240 2024-06-03T02:27:57.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.476+0000 7f2ac6000700 1 -- 172.21.15.79:0/1837936428 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2acc09f3c0 con 0x55b80042e240 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.476+0000 7f2aed564b80 10 client.26228 fetch_fsmap learned FSMap version 30 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.476+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b8004b21c0 con 0x55b80042e240 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.477+0000 7f2ac6000700 1 -- 172.21.15.79:0/1837936428 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac8005690 con 0x55b80042e240 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.477+0000 7f2ac6000700 1 -- 172.21.15.79:0/1837936428 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2acc095b80 con 0x55b80042e240 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.477+0000 7f2aed564b80 10 client.26228 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.477+0000 7f2aed564b80 10 client.26228 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.477+0000 7f2aed564b80 20 client.26228 populate_metadata read hostname 'smithi079' 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.477+0000 7f2aed564b80 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b8004b2680 0x55b8004b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.478+0000 7f2aed564b80 4 client.26228 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:27:57.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.478+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b800387580 con 0x55b8004b2680 2024-06-03T02:27:57.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.478+0000 7f2adb400700 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b8004b2680 0x55b8004b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2adb400700 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b8004b2680 0x55b8004b4a70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a9c83fc28c96df9 server_cookie=f9fbadc10cadcf58 in_seq=0 out_seq=0 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2ac6000700 10 client.26228 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2ac6000700 1 -- 172.21.15.79:0/1837936428 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b800387580 con 0x55b8004b2680 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2ac6000700 10 client.26228 handle_command_reply: tid=0 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2aed564b80 1 client.26228 shutdown 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 msgr2=0x55b80044d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2aed564b80 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 0x55b80044d400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2acc004840 tx=0x7f2acc0a3a20 comp rx=0 tx=0).stop 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 shutdown_connections 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2aed564b80 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b8004b2680 0x55b8004b4a70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2aed564b80 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b8004344b0 0x55b80044ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2aed564b80 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80042e240 0x55b80044d400 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2aed564b80 1 --2- 172.21.15.79:0/1837936428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b800433b90 0x55b80044a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 >> 172.21.15.79:0/1837936428 conn(0x55b80035a170 msgr2=0x55b800433020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:57.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.479+0000 7f2ac6000700 0 client.26228 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:27:57.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.480+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 shutdown_connections 2024-06-03T02:27:57.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.480+0000 7f2aed564b80 1 -- 172.21.15.79:0/1837936428 wait complete. 2024-06-03T02:27:57.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.480+0000 7f2aed564b80 20 client.26228 trim_cache size 0 max 16384 2024-06-03T02:27:57.483 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:27:57.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 -- 172.21.15.79:0/1089361292 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ac00a2d70 msgr2=0x7f2ac00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:57.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ac00a2d70 0x7f2ac00a5230 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b5c0 tx=0x7f2ac8005fb0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 -- 172.21.15.79:0/1089361292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc102ee0 msgr2=0x7f2adc19b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:27:57.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc102ee0 0x7f2adc19b140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00b550 tx=0x7f2acc00b910 comp rx=0 tx=0).stop 2024-06-03T02:27:57.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 -- 172.21.15.79:0/1089361292 shutdown_connections 2024-06-03T02:27:57.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ac00a2d70 0x7f2ac00a5230 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2adc19fa10 0x7f2adc19fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc102ee0 0x7f2adc19b140 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 --2- 172.21.15.79:0/1089361292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2adc107450 0x7f2adc19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:27:57.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 -- 172.21.15.79:0/1089361292 >> 172.21.15.79:0/1089361292 conn(0x7f2adc0795f0 msgr2=0x7f2adc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:27:57.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.482+0000 7f2ae2c00700 1 -- 172.21.15.79:0/1089361292 shutdown_connections 2024-06-03T02:27:57.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:27:57.483+0000 7f2ae2c00700 1 -- 172.21.15.79:0/1089361292 wait complete. 2024-06-03T02:27:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 470.0 s, 535.0 tps, lat 184.656 ms stddev 226.212 2024-06-03T02:27:58.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:27:58 smithi079 ceph-mon[31819]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 500 op/s 2024-06-03T02:27:58.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:27:58 smithi135 ceph-mon[36653]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 500 op/s 2024-06-03T02:27:58.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:27:58 smithi123 ceph-mon[36640]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 500 op/s 2024-06-03T02:27:58.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:27:59.324 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:28:00.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:00 smithi079 ceph-mon[31819]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 533 op/s 2024-06-03T02:28:00.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:00 smithi135 ceph-mon[36653]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 533 op/s 2024-06-03T02:28:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:00 smithi123 ceph-mon[36640]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 533 op/s 2024-06-03T02:28:02.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:02 smithi079 ceph-mon[31819]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 457 op/s 2024-06-03T02:28:02.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:02 smithi135 ceph-mon[36653]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 457 op/s 2024-06-03T02:28:02.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:02 smithi123 ceph-mon[36640]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 457 op/s 2024-06-03T02:28:02.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.731+0000 7ffa0c400700 1 -- 172.21.15.79:0/1916348975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04102540 msgr2=0x7ffa041029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:02.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.731+0000 7ffa0c400700 1 --2- 172.21.15.79:0/1916348975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04102540 0x7ffa041029a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009990 tx=0x7ff9f0009ca0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.732+0000 7ffa0c400700 1 -- 172.21.15.79:0/1916348975 shutdown_connections 2024-06-03T02:28:02.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.733+0000 7ffa0c400700 1 --2- 172.21.15.79:0/1916348975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa04102ee0 0x7ffa0410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.733+0000 7ffa0c400700 1 --2- 172.21.15.79:0/1916348975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa04107450 0x7ffa04107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.733+0000 7ffa0c400700 1 --2- 172.21.15.79:0/1916348975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04102540 0x7ffa041029a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.733+0000 7ffa0c400700 1 -- 172.21.15.79:0/1916348975 >> 172.21.15.79:0/1916348975 conn(0x7ffa040795f0 msgr2=0x7ffa04079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:02.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.733+0000 7ffa0c400700 1 -- 172.21.15.79:0/1916348975 shutdown_connections 2024-06-03T02:28:02.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.733+0000 7ffa0c400700 1 -- 172.21.15.79:0/1916348975 wait complete. 2024-06-03T02:28:02.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.734+0000 7ffa0c400700 1 Processor -- start 2024-06-03T02:28:02.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.736+0000 7ffa0c400700 1 -- start start 2024-06-03T02:28:02.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.736+0000 7ffa0c400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa04102540 0x7ffa0419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.736+0000 7ffa0c400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa04102ee0 0x7ffa0419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.736+0000 7ffa0c400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04107450 0x7ffa0419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.736+0000 7ffa0c400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04112840 con 0x7ffa04102ee0 2024-06-03T02:28:02.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.736+0000 7ffa0c400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041126c0 con 0x7ffa04107450 2024-06-03T02:28:02.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.736+0000 7ffa0c400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041129c0 con 0x7ffa04102540 2024-06-03T02:28:02.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.737+0000 7ffa09a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa04102540 0x7ffa0419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.737+0000 7ffa0a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04107450 0x7ffa0419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.737+0000 7ffa0a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04107450 0x7ffa0419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:44336/0 (socket says 172.21.15.79:44336) 2024-06-03T02:28:02.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.737+0000 7ffa0a400700 1 -- 172.21.15.79:0/2966401062 learned_addr learned my addr 172.21.15.79:0/2966401062 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:02.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.737+0000 7ffa09a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa04102540 0x7ffa0419aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42978/0 (socket says 172.21.15.79:42978) 2024-06-03T02:28:02.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.738+0000 7ffa0a400700 1 -- 172.21.15.79:0/2966401062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa04102540 msgr2=0x7ffa0419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:02.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.738+0000 7ffa0a400700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa04102540 0x7ffa0419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.738+0000 7ffa0a400700 1 -- 172.21.15.79:0/2966401062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa04102ee0 msgr2=0x7ffa0419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:02.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.738+0000 7ffa0a400700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa04102ee0 0x7ffa0419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.738+0000 7ffa0a400700 1 -- 172.21.15.79:0/2966401062 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0419fed0 con 0x7ffa04107450 2024-06-03T02:28:02.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.738+0000 7ffa0a400700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04107450 0x7ffa0419f790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00b9e0 tx=0x7ff9fc00bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:02.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.738+0000 7ffa02000700 1 -- 172.21.15.79:0/2966401062 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc019070 con 0x7ffa04107450 2024-06-03T02:28:02.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.738+0000 7ffa02000700 1 -- 172.21.15.79:0/2966401062 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc00ce30 con 0x7ffa04107450 2024-06-03T02:28:02.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.739+0000 7ffa02000700 1 -- 172.21.15.79:0/2966401062 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0075d0 con 0x7ffa04107450 2024-06-03T02:28:02.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.739+0000 7ffa0c400700 1 -- 172.21.15.79:0/2966401062 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa041a01c0 con 0x7ffa04107450 2024-06-03T02:28:02.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.739+0000 7ffa0c400700 1 -- 172.21.15.79:0/2966401062 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa041a7a40 con 0x7ffa04107450 2024-06-03T02:28:02.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.746+0000 7ffa02000700 1 -- 172.21.15.79:0/2966401062 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff9fc01c070 con 0x7ffa04107450 2024-06-03T02:28:02.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.753+0000 7ffa02000700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff9ec0a2ca0 0x7ff9ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.753+0000 7ffa02000700 1 -- 172.21.15.79:0/2966401062 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff9fc0ce8b0 con 0x7ffa04107450 2024-06-03T02:28:02.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.754+0000 7ffa09a00700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff9ec0a2ca0 0x7ff9ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.756+0000 7ffa09a00700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff9ec0a2ca0 0x7ff9ec0a5160 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffa04103720 tx=0x7ff9f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:02.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.757+0000 7ffa16debb80 1 Processor -- start 2024-06-03T02:28:02.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.758+0000 7ffa16debb80 1 -- start start 2024-06-03T02:28:02.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be106e40 0x5596be18a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be18a5b0 0x5596be18aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be19ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa16debb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5596be099a10 con 0x5596be19db10 2024-06-03T02:28:02.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa16debb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5596be0dcb00 con 0x5596be106e40 2024-06-03T02:28:02.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa16debb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596be0978f0 con 0x5596be18a5b0 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa09000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be18a5b0 0x5596be18aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa09000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be18a5b0 0x5596be18aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42980/0 (socket says 172.21.15.79:42980) 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa09000700 1 -- 172.21.15.79:0/582584524 learned_addr learned my addr 172.21.15.79:0/582584524 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.762+0000 7ffa09a00700 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be106e40 0x5596be18a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.763+0000 7ffa0a400700 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be19ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.763+0000 7ffa09000700 1 -- 172.21.15.79:0/582584524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be106e40 msgr2=0x5596be18a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.763+0000 7ffa09000700 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be106e40 0x5596be18a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.763+0000 7ffa09000700 1 -- 172.21.15.79:0/582584524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 msgr2=0x5596be19ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.763+0000 7ffa09000700 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be19ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.763+0000 7ffa09000700 1 -- 172.21.15.79:0/582584524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596be084f90 con 0x5596be18a5b0 2024-06-03T02:28:02.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.763+0000 7ffa09a00700 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be106e40 0x5596be18a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:28:02.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.763+0000 7ffa0a400700 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be19ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:02.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.764+0000 7ffa09000700 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be18a5b0 0x5596be18aa30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000b7a0 tx=0x7ff9f0005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb6edd22e938a929 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:02.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.764+0000 7ff9f6a00700 1 -- 172.21.15.79:0/582584524 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0027070 con 0x5596be18a5b0 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.765+0000 7ff9f6a00700 1 -- 172.21.15.79:0/582584524 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f0009510 con 0x5596be18a5b0 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.765+0000 7ff9f6a00700 1 -- 172.21.15.79:0/582584524 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0004230 con 0x5596be18a5b0 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.765+0000 7ffa16debb80 1 -- 172.21.15.79:0/582584524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be18a5b0 msgr2=0x5596be18aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.765+0000 7ffa16debb80 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be18a5b0 0x5596be18aa30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000b7a0 tx=0x7ff9f0005c60 comp rx=0 tx=0).stop 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.766+0000 7ffa16debb80 1 -- 172.21.15.79:0/582584524 shutdown_connections 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.766+0000 7ffa16debb80 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be19ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.766+0000 7ffa16debb80 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be106e40 0x5596be18a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.766+0000 7ffa16debb80 1 --2- 172.21.15.79:0/582584524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be18a5b0 0x5596be18aa30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.766+0000 7ffa16debb80 1 -- 172.21.15.79:0/582584524 >> 172.21.15.79:0/582584524 conn(0x5596be0b7170 msgr2=0x5596be1864b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:02.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.766+0000 7ffa16debb80 1 -- 172.21.15.79:0/582584524 shutdown_connections 2024-06-03T02:28:02.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.766+0000 7ffa16debb80 1 -- 172.21.15.79:0/582584524 wait complete. 2024-06-03T02:28:02.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.767+0000 7ffa16debb80 1 -- start start 2024-06-03T02:28:02.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.767+0000 7ffa16debb80 1 Processor -- start 2024-06-03T02:28:02.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be192ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be18fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be1902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa16debb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5596be078f60 con 0x5596be19db10 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa16debb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5596be0d8ac0 con 0x5596be18a5b0 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa16debb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596be1a0770 con 0x5596be106e40 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa16debb80 10 client.0 _ll_register_callbacks cb 0x5596be076720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa09a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be192ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa09000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be18fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa09000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be18fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:44362/0 (socket says 172.21.15.79:44362) 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa09a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be192ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:42990/0 (socket says 172.21.15.79:42990) 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.768+0000 7ffa09000700 1 -- 172.21.15.79:0/348777488 learned_addr learned my addr 172.21.15.79:0/348777488 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.769+0000 7ffa09000700 1 -- 172.21.15.79:0/348777488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 msgr2=0x5596be192ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.769+0000 7ffa09000700 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be192ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.769+0000 7ffa09000700 1 -- 172.21.15.79:0/348777488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 msgr2=0x5596be1902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.769+0000 7ffa09000700 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be1902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.769+0000 7ffa09000700 1 -- 172.21.15.79:0/348777488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596be02b8d0 con 0x5596be18a5b0 2024-06-03T02:28:02.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.769+0000 7ffa09000700 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be18fdb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0005c30 tx=0x7ff9f000bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af1fbf83de36d09 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:02.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.769+0000 7ff9eb400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:28:02.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.770+0000 7ff9eb400700 1 -- 172.21.15.79:0/348777488 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0030060 con 0x5596be18a5b0 2024-06-03T02:28:02.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.770+0000 7ff9eb400700 1 -- 172.21.15.79:0/348777488 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff9f0003680 con 0x5596be18a5b0 2024-06-03T02:28:02.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.770+0000 7ffa16debb80 1 -- 172.21.15.79:0/348777488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f0009510 con 0x5596be18a5b0 2024-06-03T02:28:02.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.770+0000 7ff9eb400700 1 -- 172.21.15.79:0/348777488 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9f0027040 con 0x5596be18a5b0 2024-06-03T02:28:02.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.770+0000 7ffa16debb80 10 client.26240 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:02.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.770+0000 7ffa16debb80 1 -- 172.21.15.79:0/348777488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596be1aae60 con 0x5596be18a5b0 2024-06-03T02:28:02.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.770+0000 7ff9eb400700 1 -- 172.21.15.79:0/348777488 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f0009510 con 0x5596be18a5b0 2024-06-03T02:28:02.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.772+0000 7ff9eb400700 1 -- 172.21.15.79:0/348777488 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff9f002a030 con 0x5596be18a5b0 2024-06-03T02:28:02.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.772+0000 7ffa16debb80 10 client.26240 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:02.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.772+0000 7ffa16debb80 10 client.26240 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:28:02.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.772+0000 7ffa16debb80 20 client.26240 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:02.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.772+0000 7ffa16debb80 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5596be1ad330 0x5596be1af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.772+0000 7ffa16debb80 4 client.26240 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:28:02.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.772+0000 7ffa16debb80 1 -- 172.21.15.79:0/348777488 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596be0ea000 con 0x5596be1ad330 2024-06-03T02:28:02.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.772+0000 7ffa09a00700 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5596be1ad330 0x5596be1af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.773+0000 7ffa09a00700 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5596be1ad330 0x5596be1af720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e1e4469b1c26530 server_cookie=ef2d2ef9421d8b7 in_seq=0 out_seq=0 2024-06-03T02:28:02.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.773+0000 7ff9eb400700 10 client.26240 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:28:02.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.773+0000 7ff9eb400700 1 -- 172.21.15.79:0/348777488 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5596be0ea000 con 0x5596be1ad330 2024-06-03T02:28:02.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.773+0000 7ff9eb400700 10 client.26240 handle_command_reply: tid=0 2024-06-03T02:28:02.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.774+0000 7ffa16debb80 1 client.26240 shutdown 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.774+0000 7ffa16debb80 1 -- 172.21.15.79:0/348777488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 msgr2=0x5596be18fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.774+0000 7ffa16debb80 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be18fdb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0005c30 tx=0x7ff9f000bee0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.774+0000 7ffa16debb80 1 -- 172.21.15.79:0/348777488 shutdown_connections 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.774+0000 7ffa16debb80 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be1902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.774+0000 7ffa16debb80 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be192ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.774+0000 7ffa16debb80 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5596be1ad330 0x5596be1af720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.775+0000 7ffa16debb80 1 --2- 172.21.15.79:0/348777488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be18fdb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.775+0000 7ffa16debb80 1 -- 172.21.15.79:0/348777488 >> 172.21.15.79:0/348777488 conn(0x5596be0b7170 msgr2=0x5596be185740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.775+0000 7ff9eb400700 0 client.26240 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.775+0000 7ffa16debb80 1 -- 172.21.15.79:0/348777488 shutdown_connections 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.775+0000 7ffa16debb80 1 -- 172.21.15.79:0/348777488 wait complete. 2024-06-03T02:28:02.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.775+0000 7ffa16debb80 20 client.26240 trim_cache size 0 max 16384 2024-06-03T02:28:02.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.806+0000 7ffa16debb80 1 Processor -- start 2024-06-03T02:28:02.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.806+0000 7ffa16debb80 1 -- start start 2024-06-03T02:28:02.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.806+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be19e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.806+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.806+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be18d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.806+0000 7ffa16debb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041ac140 con 0x5596be19db10 2024-06-03T02:28:02.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.806+0000 7ffa16debb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041adc80 con 0x5596be18a5b0 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.806+0000 7ffa16debb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041ac810 con 0x5596be106e40 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.807+0000 7ffa09000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.807+0000 7ffa09000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1925f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:44374/0 (socket says 172.21.15.79:44374) 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.807+0000 7ffa09000700 1 -- 172.21.15.79:0/798316628 learned_addr learned my addr 172.21.15.79:0/798316628 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.807+0000 7ffa09a00700 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.807+0000 7ffa0a400700 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be18d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ffa09000700 1 -- 172.21.15.79:0/798316628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 msgr2=0x5596be19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ffa09000700 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ffa09000700 1 -- 172.21.15.79:0/798316628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 msgr2=0x5596be18d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ffa09000700 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be18d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ffa09000700 1 -- 172.21.15.79:0/798316628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596be19eb40 con 0x5596be18a5b0 2024-06-03T02:28:02.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ffa09a00700 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ffa09000700 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1925f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000fb20 tx=0x7ff9f0009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8560ad9e86cc9fac server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ffa0a400700 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be18d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ff9f6000700 1 -- 172.21.15.79:0/798316628 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0030040 con 0x5596be18a5b0 2024-06-03T02:28:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ff9f6000700 1 -- 172.21.15.79:0/798316628 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f00058e0 con 0x5596be18a5b0 2024-06-03T02:28:02.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.808+0000 7ff9f6000700 1 -- 172.21.15.79:0/798316628 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0003af0 con 0x5596be18a5b0 2024-06-03T02:28:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.809+0000 7ffa16debb80 1 -- 172.21.15.79:0/798316628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 msgr2=0x5596be1925f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.809+0000 7ffa16debb80 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1925f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000fb20 tx=0x7ff9f0009890 comp rx=0 tx=0).stop 2024-06-03T02:28:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.809+0000 7ffa16debb80 1 -- 172.21.15.79:0/798316628 shutdown_connections 2024-06-03T02:28:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.809+0000 7ffa16debb80 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be18d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.809+0000 7ffa16debb80 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.809+0000 7ffa16debb80 1 --2- 172.21.15.79:0/798316628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1925f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.809+0000 7ffa16debb80 1 -- 172.21.15.79:0/798316628 >> 172.21.15.79:0/798316628 conn(0x5596be0b7170 msgr2=0x5596be19def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.810+0000 7ffa16debb80 1 -- 172.21.15.79:0/798316628 shutdown_connections 2024-06-03T02:28:02.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.810+0000 7ffa16debb80 1 -- 172.21.15.79:0/798316628 wait complete. 2024-06-03T02:28:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.811+0000 7ffa16debb80 1 -- start start 2024-06-03T02:28:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.811+0000 7ffa16debb80 1 Processor -- start 2024-06-03T02:28:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.812+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be1b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.812+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.812+0000 7ffa16debb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be18c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.812+0000 7ffa16debb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041ad5b0 con 0x5596be19db10 2024-06-03T02:28:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.812+0000 7ffa16debb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041acee0 con 0x5596be18a5b0 2024-06-03T02:28:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.812+0000 7ffa16debb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041aa0e0 con 0x5596be106e40 2024-06-03T02:28:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.812+0000 7ffa16debb80 10 client.0 _ll_register_callbacks cb 0x5596be076720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.812+0000 7ffa09a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be1b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.813+0000 7ffa09a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be1b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:43006/0 (socket says 172.21.15.79:43006) 2024-06-03T02:28:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.813+0000 7ffa09000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.813+0000 7ffa09000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:44388/0 (socket says 172.21.15.79:44388) 2024-06-03T02:28:02.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.813+0000 7ffa09000700 1 -- 172.21.15.79:0/2438609149 learned_addr learned my addr 172.21.15.79:0/2438609149 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:02.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.813+0000 7ffa0a400700 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be18c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.814+0000 7ffa09a00700 1 -- 172.21.15.79:0/2438609149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 msgr2=0x5596be1b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:02.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.814+0000 7ffa09a00700 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.814+0000 7ffa09a00700 1 -- 172.21.15.79:0/2438609149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 msgr2=0x5596be18c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:02.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.814+0000 7ffa09a00700 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be18c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.814+0000 7ffa09a00700 1 -- 172.21.15.79:0/2438609149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596be18dcc0 con 0x5596be106e40 2024-06-03T02:28:02.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.814+0000 7ffa09a00700 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be1b2750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80097d0 tx=0x7ff9f80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ff600c3ffa8d7e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:02.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.814+0000 7ff9eb400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:28:02.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.815+0000 7ff9eb400700 1 -- 172.21.15.79:0/2438609149 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8012db0 con 0x5596be106e40 2024-06-03T02:28:02.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.815+0000 7ff9eb400700 1 -- 172.21.15.79:0/2438609149 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff9f802b420 con 0x5596be106e40 2024-06-03T02:28:02.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.815+0000 7ffa16debb80 1 -- 172.21.15.79:0/2438609149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f00058e0 con 0x5596be106e40 2024-06-03T02:28:02.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.815+0000 7ff9eb400700 1 -- 172.21.15.79:0/2438609149 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9f802b620 con 0x5596be106e40 2024-06-03T02:28:02.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.815+0000 7ffa16debb80 10 client.26248 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:02.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.815+0000 7ffa16debb80 1 -- 172.21.15.79:0/2438609149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596be228120 con 0x5596be106e40 2024-06-03T02:28:02.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.816+0000 7ff9eb400700 1 -- 172.21.15.79:0/2438609149 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f00058e0 con 0x5596be106e40 2024-06-03T02:28:02.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.816+0000 7ff9eb400700 1 -- 172.21.15.79:0/2438609149 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff9f8033060 con 0x5596be106e40 2024-06-03T02:28:02.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.816+0000 7ffa16debb80 10 client.26248 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:02.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.816+0000 7ffa16debb80 10 client.26248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:28:02.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.816+0000 7ffa16debb80 20 client.26248 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:02.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.816+0000 7ffa16debb80 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5596be228860 0x5596be22ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:02.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.816+0000 7ffa16debb80 4 client.26248 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:28:02.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.816+0000 7ffa16debb80 1 -- 172.21.15.79:0/2438609149 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596be0e4580 con 0x5596be228860 2024-06-03T02:28:02.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.817+0000 7ffa09000700 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5596be228860 0x5596be22ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:02.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ff9eb400700 10 client.26248 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa09000700 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5596be228860 0x5596be22ac80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae63e5078715cabd server_cookie=e6699ef626e39397 in_seq=0 out_seq=0 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ff9eb400700 1 -- 172.21.15.79:0/2438609149 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596be0e4580 con 0x5596be228860 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ff9eb400700 10 client.26248 handle_command_reply: tid=0 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa16debb80 1 client.26248 shutdown 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa16debb80 1 -- 172.21.15.79:0/2438609149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 msgr2=0x5596be1b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa16debb80 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be1b2750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80097d0 tx=0x7ff9f80047c0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa16debb80 1 -- 172.21.15.79:0/2438609149 shutdown_connections 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa16debb80 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596be19db10 0x5596be18c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa16debb80 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596be106e40 0x5596be1b2750 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa16debb80 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5596be228860 0x5596be22ac80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa16debb80 1 --2- 172.21.15.79:0/2438609149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596be18a5b0 0x5596be1b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.818+0000 7ffa16debb80 1 -- 172.21.15.79:0/2438609149 >> 172.21.15.79:0/2438609149 conn(0x5596be0b7170 msgr2=0x5596be18e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.819+0000 7ff9eb400700 0 client.26248 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.819+0000 7ffa16debb80 1 -- 172.21.15.79:0/2438609149 shutdown_connections 2024-06-03T02:28:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.819+0000 7ffa16debb80 1 -- 172.21.15.79:0/2438609149 wait complete. 2024-06-03T02:28:02.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.819+0000 7ffa16debb80 20 client.26248 trim_cache size 0 max 16384 2024-06-03T02:28:02.820 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:28:02.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 -- 172.21.15.79:0/2966401062 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff9ec0a2ca0 msgr2=0x7ff9ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff9ec0a2ca0 0x7ff9ec0a5160 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffa04103720 tx=0x7ff9f8009260 comp rx=0 tx=0).stop 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 -- 172.21.15.79:0/2966401062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04107450 msgr2=0x7ffa0419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04107450 0x7ffa0419f790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00b9e0 tx=0x7ff9fc00bda0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 -- 172.21.15.79:0/2966401062 shutdown_connections 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04107450 0x7ffa0419f790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa04102540 0x7ffa0419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff9ec0a2ca0 0x7ff9ec0a5160 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 --2- 172.21.15.79:0/2966401062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa04102ee0 0x7ffa0419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 -- 172.21.15.79:0/2966401062 >> 172.21.15.79:0/2966401062 conn(0x7ffa040795f0 msgr2=0x7ffa041003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 -- 172.21.15.79:0/2966401062 shutdown_connections 2024-06-03T02:28:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:02.820+0000 7ffa0c400700 1 -- 172.21.15.79:0/2966401062 wait complete. 2024-06-03T02:28:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 475.0 s, 650.2 tps, lat 154.255 ms stddev 183.562 2024-06-03T02:28:03.772 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:28:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:04 smithi135 ceph-mon[36653]: pgmap v756: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-06-03T02:28:04.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:04 smithi123 ceph-mon[36640]: pgmap v756: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-06-03T02:28:04.502 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:28:04.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:04 smithi079 ceph-mon[31819]: pgmap v756: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-06-03T02:28:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:06 smithi123 ceph-mon[36640]: pgmap v757: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-06-03T02:28:06.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:06 smithi079 ceph-mon[31819]: pgmap v757: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-06-03T02:28:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:06 smithi135 ceph-mon[36653]: pgmap v757: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-06-03T02:28:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 480.0 s, 796.4 tps, lat 126.975 ms stddev 177.771 2024-06-03T02:28:08.311 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:08 smithi079 ceph-mon[31819]: pgmap v758: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.5 MiB/s wr, 594 op/s 2024-06-03T02:28:08.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:08 smithi123 ceph-mon[36640]: pgmap v758: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.5 MiB/s wr, 594 op/s 2024-06-03T02:28:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:08 smithi135 ceph-mon[36653]: pgmap v758: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.5 MiB/s wr, 594 op/s 2024-06-03T02:28:10.337 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:10 smithi079 ceph-mon[31819]: pgmap v759: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.8 MiB/s wr, 717 op/s 2024-06-03T02:28:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:10 smithi123 ceph-mon[36640]: pgmap v759: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.8 MiB/s wr, 717 op/s 2024-06-03T02:28:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:10 smithi135 ceph-mon[36653]: pgmap v759: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.8 MiB/s wr, 717 op/s 2024-06-03T02:28:11.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.520+0000 7fd9fc000700 1 -- 172.21.15.79:0/781967251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9f406a170 msgr2=0x7fd9f406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:11.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.520+0000 7fd9fc000700 1 --2- 172.21.15.79:0/781967251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9f406a170 0x7fd9f406a5d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009920 tx=0x7fd9dc009c30 comp rx=0 tx=0).stop 2024-06-03T02:28:11.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.521+0000 7fd9fc000700 1 -- 172.21.15.79:0/781967251 shutdown_connections 2024-06-03T02:28:11.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.521+0000 7fd9fc000700 1 --2- 172.21.15.79:0/781967251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9f406ab10 0x7fd9f410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.521+0000 7fd9fc000700 1 --2- 172.21.15.79:0/781967251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f41084e0 0x7fd9f41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.521+0000 7fd9fc000700 1 --2- 172.21.15.79:0/781967251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9f406a170 0x7fd9f406a5d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.521+0000 7fd9fc000700 1 -- 172.21.15.79:0/781967251 >> 172.21.15.79:0/781967251 conn(0x7fd9f40782c0 msgr2=0x7fd9f40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:11.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.528+0000 7fd9fc000700 1 -- 172.21.15.79:0/781967251 shutdown_connections 2024-06-03T02:28:11.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.529+0000 7fd9fc000700 1 -- 172.21.15.79:0/781967251 wait complete. 2024-06-03T02:28:11.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.531+0000 7fd9fc000700 1 Processor -- start 2024-06-03T02:28:11.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.531+0000 7fd9fc000700 1 -- start start 2024-06-03T02:28:11.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9fc000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9f406a170 0x7fd9f419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9fc000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f406ab10 0x7fd9f419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9fc000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9f41084e0 0x7fd9f41a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9fc000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4101960 con 0x7fd9f41084e0 2024-06-03T02:28:11.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9fc000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41017e0 con 0x7fd9f406ab10 2024-06-03T02:28:11.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9fc000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4101ae0 con 0x7fd9f406a170 2024-06-03T02:28:11.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9f8c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f406ab10 0x7fd9f419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9f8c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f406ab10 0x7fd9f419bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:45298/0 (socket says 172.21.15.79:45298) 2024-06-03T02:28:11.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9f8c00700 1 -- 172.21.15.79:0/2847908798 learned_addr learned my addr 172.21.15.79:0/2847908798 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:11.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.532+0000 7fd9f9600700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9f406a170 0x7fd9f419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.533+0000 7fd9fa000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9f41084e0 0x7fd9f41a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.535+0000 7fd9f8c00700 1 -- 172.21.15.79:0/2847908798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9f406a170 msgr2=0x7fd9f419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:11.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.535+0000 7fd9f8c00700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9f406a170 0x7fd9f419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.535+0000 7fd9f8c00700 1 -- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9f41084e0 msgr2=0x7fd9f41a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:11.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.535+0000 7fd9f8c00700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9f41084e0 0x7fd9f41a3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.535+0000 7fd9f8c00700 1 -- 172.21.15.79:0/2847908798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f410b040 con 0x7fd9f406ab10 2024-06-03T02:28:11.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.535+0000 7fd9f9600700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9f406a170 0x7fd9f419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:28:11.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.535+0000 7fd9fa000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9f41084e0 0x7fd9f41a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:11.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.537+0000 7fd9f8c00700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f406ab10 0x7fd9f419bd20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc010fc0 tx=0x7fd9dc010ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:11.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.537+0000 7fd9f2000700 1 -- 172.21.15.79:0/2847908798 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00f460 con 0x7fd9f406ab10 2024-06-03T02:28:11.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.537+0000 7fd9f2000700 1 -- 172.21.15.79:0/2847908798 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9dc00fae0 con 0x7fd9f406ab10 2024-06-03T02:28:11.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.537+0000 7fd9f2000700 1 -- 172.21.15.79:0/2847908798 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc017400 con 0x7fd9f406ab10 2024-06-03T02:28:11.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.537+0000 7fd9fc000700 1 -- 172.21.15.79:0/2847908798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f410b2d0 con 0x7fd9f406ab10 2024-06-03T02:28:11.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.538+0000 7fd9fc000700 1 -- 172.21.15.79:0/2847908798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f410b760 con 0x7fd9f406ab10 2024-06-03T02:28:11.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.551+0000 7fda0695eb80 1 Processor -- start 2024-06-03T02:28:11.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.551+0000 7fda0695eb80 1 -- start start 2024-06-03T02:28:11.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.552+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.552+0000 7fd9f2000700 1 -- 172.21.15.79:0/2847908798 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd9dc0175a0 con 0x7fd9f406ab10 2024-06-03T02:28:11.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.552+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dcfe5b0 0x56403dcfea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.552+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 0x56403dd0b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.552+0000 7fda0695eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56403dc0da10 con 0x56403dc7ae40 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.552+0000 7fda0695eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56403dc50b00 con 0x56403dd09300 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.552+0000 7fda0695eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56403dc0b8f0 con 0x56403dcfe5b0 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9f9600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dcfe5b0 0x56403dcfea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9f9600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dcfe5b0 0x56403dcfea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36272/0 (socket says 172.21.15.79:36272) 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9f9600700 1 -- 172.21.15.79:0/3609607935 learned_addr learned my addr 172.21.15.79:0/3609607935 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9f8c00700 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 0x56403dd0b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9fa000700 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9f9600700 1 -- 172.21.15.79:0/3609607935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 msgr2=0x56403dd0b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9f9600700 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 0x56403dd0b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9f9600700 1 -- 172.21.15.79:0/3609607935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dc7ae40 msgr2=0x56403dcfe070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:11.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9f9600700 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9f9600700 1 -- 172.21.15.79:0/3609607935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56403dbf8f90 con 0x56403dcfe5b0 2024-06-03T02:28:11.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.553+0000 7fd9fa000700 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:11.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.554+0000 7fd9f9600700 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dcfe5b0 0x56403dcfea30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00f3f0 tx=0x7fd9ec00f700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff9bf85ab33033a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:11.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.554+0000 7fd9eaa00700 1 -- 172.21.15.79:0/3609607935 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec01d070 con 0x56403dcfe5b0 2024-06-03T02:28:11.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.556+0000 7fd9eaa00700 1 -- 172.21.15.79:0/3609607935 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ec018b20 con 0x56403dcfe5b0 2024-06-03T02:28:11.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.556+0000 7fd9eaa00700 1 -- 172.21.15.79:0/3609607935 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec010c80 con 0x56403dcfe5b0 2024-06-03T02:28:11.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.557+0000 7fd9f2000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd9d80a2d20 0x7fd9d80a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.558+0000 7fda0695eb80 1 -- 172.21.15.79:0/3609607935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dcfe5b0 msgr2=0x56403dcfea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:11.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.558+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dcfe5b0 0x56403dcfea30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00f3f0 tx=0x7fd9ec00f700 comp rx=0 tx=0).stop 2024-06-03T02:28:11.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.558+0000 7fd9fa000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd9d80a2d20 0x7fd9d80a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.558+0000 7fda0695eb80 1 -- 172.21.15.79:0/3609607935 shutdown_connections 2024-06-03T02:28:11.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.558+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 0x56403dd0b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.558+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.559+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3609607935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dcfe5b0 0x56403dcfea30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.559+0000 7fda0695eb80 1 -- 172.21.15.79:0/3609607935 >> 172.21.15.79:0/3609607935 conn(0x56403dc2b170 msgr2=0x56403dcfa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:11.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.559+0000 7fd9f2000700 1 -- 172.21.15.79:0/2847908798 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd9dc0d04a0 con 0x7fd9f406ab10 2024-06-03T02:28:11.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.560+0000 7fda0695eb80 1 -- 172.21.15.79:0/3609607935 shutdown_connections 2024-06-03T02:28:11.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.560+0000 7fda0695eb80 1 -- 172.21.15.79:0/3609607935 wait complete. 2024-06-03T02:28:11.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.561+0000 7fda0695eb80 1 -- start start 2024-06-03T02:28:11.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.561+0000 7fd9fa000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd9d80a2d20 0x7fd9d80a51e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40062a0 tx=0x7fd9e400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:11.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.561+0000 7fda0695eb80 1 Processor -- start 2024-06-03T02:28:11.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.562+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 0x56403dd08900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.562+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd05ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.562+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.562+0000 7fda0695eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56403dbecf60 con 0x56403dcfe5b0 2024-06-03T02:28:11.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.562+0000 7fda0695eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56403dc4cac0 con 0x56403dc7ae40 2024-06-03T02:28:11.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.562+0000 7fda0695eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56403dd0bfc0 con 0x56403dd09300 2024-06-03T02:28:11.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.563+0000 7fda0695eb80 10 client.0 _ll_register_callbacks cb 0x56403dbea720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:11.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.563+0000 7fd9fa000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.563+0000 7fd9fa000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36282/0 (socket says 172.21.15.79:36282) 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.563+0000 7fd9fa000700 1 -- 172.21.15.79:0/3031742281 learned_addr learned my addr 172.21.15.79:0/3031742281 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.563+0000 7fd9f8c00700 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd05ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.564+0000 7fd9fa000700 1 -- 172.21.15.79:0/3031742281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 msgr2=0x56403dd08900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.564+0000 7fd9fa000700 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 0x56403dd08900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.564+0000 7fd9fa000700 1 -- 172.21.15.79:0/3031742281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 msgr2=0x56403dd05ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.564+0000 7fd9fa000700 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd05ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.564+0000 7fd9fa000700 1 -- 172.21.15.79:0/3031742281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56403db9f8d0 con 0x56403dd09300 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.564+0000 7fd9f8c00700 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd05ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:28:11.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.566+0000 7fd9fa000700 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd061e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40091e0 tx=0x7fd9e4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2875d8967fe64bce server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:11.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.566+0000 7fd9e3400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:28:11.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.567+0000 7fd9e3400700 1 -- 172.21.15.79:0/3031742281 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4004db0 con 0x56403dd09300 2024-06-03T02:28:11.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.567+0000 7fd9e3400700 1 -- 172.21.15.79:0/3031742281 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd9e4014860 con 0x56403dd09300 2024-06-03T02:28:11.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.567+0000 7fda0695eb80 1 -- 172.21.15.79:0/3031742281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ec018b20 con 0x56403dd09300 2024-06-03T02:28:11.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.567+0000 7fd9e3400700 1 -- 172.21.15.79:0/3031742281 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd9e4014a60 con 0x56403dd09300 2024-06-03T02:28:11.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.568+0000 7fda0695eb80 10 client.26260 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:11.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.568+0000 7fda0695eb80 1 -- 172.21.15.79:0/3031742281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56403dd16670 con 0x56403dd09300 2024-06-03T02:28:11.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.570+0000 7fd9e3400700 1 -- 172.21.15.79:0/3031742281 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ec018b20 con 0x56403dd09300 2024-06-03T02:28:11.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.570+0000 7fd9e3400700 1 -- 172.21.15.79:0/3031742281 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd9e4005020 con 0x56403dd09300 2024-06-03T02:28:11.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.570+0000 7fda0695eb80 10 client.26260 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:11.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.570+0000 7fda0695eb80 10 client.26260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:28:11.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.570+0000 7fda0695eb80 20 client.26260 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:11.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.570+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x56403dd18d00 0x56403dd1b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.570+0000 7fda0695eb80 4 client.26260 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:28:11.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.570+0000 7fda0695eb80 1 -- 172.21.15.79:0/3031742281 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56403dc5e000 con 0x56403dd18d00 2024-06-03T02:28:11.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.573+0000 7fd9f9600700 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x56403dd18d00 0x56403dd1b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.574+0000 7fd9f9600700 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x56403dd18d00 0x56403dd1b1c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3435a43d0a8d697c server_cookie=5561840d94c47294 in_seq=0 out_seq=0 2024-06-03T02:28:11.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.574+0000 7fd9e3400700 10 client.26260 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:28:11.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.576+0000 7fd9e3400700 1 -- 172.21.15.79:0/3031742281 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56403dc5e000 con 0x56403dd18d00 2024-06-03T02:28:11.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.576+0000 7fd9e3400700 10 client.26260 handle_command_reply: tid=0 2024-06-03T02:28:11.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.576+0000 7fda0695eb80 1 client.26260 shutdown 2024-06-03T02:28:11.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.577+0000 7fda0695eb80 1 -- 172.21.15.79:0/3031742281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 msgr2=0x56403dd061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:11.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.577+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd061e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40091e0 tx=0x7fd9e4004b40 comp rx=0 tx=0).stop 2024-06-03T02:28:11.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.578+0000 7fda0695eb80 1 -- 172.21.15.79:0/3031742281 shutdown_connections 2024-06-03T02:28:11.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.578+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x56403dd18d00 0x56403dd1b1c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.578+0000 7fd9e3400700 0 client.26260 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:28:11.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.578+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd061e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.578+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 0x56403dd08900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.578+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3031742281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd05ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.578+0000 7fda0695eb80 1 -- 172.21.15.79:0/3031742281 >> 172.21.15.79:0/3031742281 conn(0x56403dc2b170 msgr2=0x56403dcf9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:11.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.579+0000 7fda0695eb80 1 -- 172.21.15.79:0/3031742281 shutdown_connections 2024-06-03T02:28:11.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.579+0000 7fda0695eb80 1 -- 172.21.15.79:0/3031742281 wait complete. 2024-06-03T02:28:11.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.579+0000 7fda0695eb80 20 client.26260 trim_cache size 0 max 16384 2024-06-03T02:28:11.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.621+0000 7fda0695eb80 1 Processor -- start 2024-06-03T02:28:11.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.621+0000 7fda0695eb80 1 -- start start 2024-06-03T02:28:11.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.622+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.622+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd0a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.622+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 0x56403dd082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.622+0000 7fda0695eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d000ec70 con 0x56403dcfe5b0 2024-06-03T02:28:11.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.622+0000 7fda0695eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d00107b0 con 0x56403dd09300 2024-06-03T02:28:11.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.622+0000 7fda0695eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d000f340 con 0x56403dc7ae40 2024-06-03T02:28:11.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.623+0000 7fd9fa000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 0x56403dd082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.623+0000 7fd9fa000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 0x56403dd082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:45336/0 (socket says 172.21.15.79:45336) 2024-06-03T02:28:11.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.623+0000 7fd9fa000700 1 -- 172.21.15.79:0/5298683 learned_addr learned my addr 172.21.15.79:0/5298683 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:11.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.623+0000 7fd9f9600700 1 --2- 172.21.15.79:0/5298683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.624+0000 7fd9f8c00700 1 --2- 172.21.15.79:0/5298683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd0a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.624+0000 7fd9f9600700 1 -- 172.21.15.79:0/5298683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 msgr2=0x56403dd082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:11.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.624+0000 7fd9f9600700 1 --2- 172.21.15.79:0/5298683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 0x56403dd082b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.624+0000 7fd9f9600700 1 -- 172.21.15.79:0/5298683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 msgr2=0x56403dd0a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:11.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.624+0000 7fd9f9600700 1 --2- 172.21.15.79:0/5298683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd0a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.624+0000 7fd9f9600700 1 -- 172.21.15.79:0/5298683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56403dd064f0 con 0x56403dc7ae40 2024-06-03T02:28:11.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.625+0000 7fd9f9600700 1 --2- 172.21.15.79:0/5298683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00fa50 tx=0x7fd9ec00aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4a7a4196170f04f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:11.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.625+0000 7fd9ea000700 1 -- 172.21.15.79:0/5298683 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec01d070 con 0x56403dc7ae40 2024-06-03T02:28:11.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.625+0000 7fd9ea000700 1 -- 172.21.15.79:0/5298683 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ec028070 con 0x56403dc7ae40 2024-06-03T02:28:11.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.625+0000 7fd9ea000700 1 -- 172.21.15.79:0/5298683 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0046f0 con 0x56403dc7ae40 2024-06-03T02:28:11.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.628+0000 7fda0695eb80 1 -- 172.21.15.79:0/5298683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dc7ae40 msgr2=0x56403dcfe070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:11.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.628+0000 7fda0695eb80 1 --2- 172.21.15.79:0/5298683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00fa50 tx=0x7fd9ec00aba0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.629+0000 7fda0695eb80 1 -- 172.21.15.79:0/5298683 shutdown_connections 2024-06-03T02:28:11.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.629+0000 7fda0695eb80 1 --2- 172.21.15.79:0/5298683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dd09300 0x56403dd082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.629+0000 7fda0695eb80 1 --2- 172.21.15.79:0/5298683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dc7ae40 0x56403dcfe070 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.629+0000 7fda0695eb80 1 --2- 172.21.15.79:0/5298683 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd0a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.629+0000 7fda0695eb80 1 -- 172.21.15.79:0/5298683 >> 172.21.15.79:0/5298683 conn(0x56403dc2b170 msgr2=0x56403dcf9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:11.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.629+0000 7fda0695eb80 1 -- 172.21.15.79:0/5298683 shutdown_connections 2024-06-03T02:28:11.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.630+0000 7fda0695eb80 1 -- 172.21.15.79:0/5298683 wait complete. 2024-06-03T02:28:11.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.630+0000 7fda0695eb80 1 -- start start 2024-06-03T02:28:11.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.631+0000 7fda0695eb80 1 Processor -- start 2024-06-03T02:28:11.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.636+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 0x56403dd09cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.636+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd0a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.637+0000 7fda0695eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd0a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.637+0000 7fda0695eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d00100e0 con 0x56403dcfe5b0 2024-06-03T02:28:11.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.636+0000 7fd9f9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 0x56403dd09cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.637+0000 7fd9f9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 0x56403dd09cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:45338/0 (socket says 172.21.15.79:45338) 2024-06-03T02:28:11.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.637+0000 7fd9f9600700 1 -- 172.21.15.79:0/3817922580 learned_addr learned my addr 172.21.15.79:0/3817922580 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.637+0000 7fd9fa000700 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd0a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.638+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d000fa10 con 0x56403dc7ae40 2024-06-03T02:28:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.638+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d000cc10 con 0x56403dd09300 2024-06-03T02:28:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.638+0000 7fda0695eb80 10 client.0 _ll_register_callbacks cb 0x56403dbea720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.638+0000 7fd9fa000700 1 -- 172.21.15.79:0/3817922580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 msgr2=0x56403dd09cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.638+0000 7fd9fa000700 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 0x56403dd09cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.638+0000 7fd9fa000700 1 -- 172.21.15.79:0/3817922580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 msgr2=0x56403dd0a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.638+0000 7fd9fa000700 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd0a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.638+0000 7fd9fa000700 1 -- 172.21.15.79:0/3817922580 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56403dd102e0 con 0x56403dd09300 2024-06-03T02:28:11.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.640+0000 7fd9fa000700 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd0a740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ad0 tx=0x7fd9e4004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11e60c5564cfa53a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:11.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.640+0000 7fd9e3400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:28:11.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.640+0000 7fd9e3400700 1 -- 172.21.15.79:0/3817922580 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4038060 con 0x56403dd09300 2024-06-03T02:28:11.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.641+0000 7fd9e3400700 1 -- 172.21.15.79:0/3817922580 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd9e4005840 con 0x56403dd09300 2024-06-03T02:28:11.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.641+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ec028070 con 0x56403dd09300 2024-06-03T02:28:11.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.641+0000 7fd9e3400700 1 -- 172.21.15.79:0/3817922580 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd9e403f050 con 0x56403dd09300 2024-06-03T02:28:11.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.641+0000 7fda0695eb80 10 client.26266 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:11.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.641+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56403dd11540 con 0x56403dd09300 2024-06-03T02:28:11.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.642+0000 7fd9e3400700 1 -- 172.21.15.79:0/3817922580 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ec028070 con 0x56403dd09300 2024-06-03T02:28:11.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.642+0000 7fd9e3400700 1 -- 172.21.15.79:0/3817922580 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd9e402b640 con 0x56403dd09300 2024-06-03T02:28:11.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.642+0000 7fda0695eb80 10 client.26266 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:11.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.642+0000 7fda0695eb80 10 client.26266 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:28:11.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.642+0000 7fda0695eb80 20 client.26266 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:11.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.642+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x56403dd18d00 0x56403dd85ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:11.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.643+0000 7fda0695eb80 4 client.26266 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:28:11.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.643+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56403dc58580 con 0x56403dd18d00 2024-06-03T02:28:11.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.643+0000 7fd9f9600700 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x56403dd18d00 0x56403dd85ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:11.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.644+0000 7fd9f9600700 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x56403dd18d00 0x56403dd85ab0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd7465d273c38995 server_cookie=9bfb82e14aafa53a in_seq=0 out_seq=0 2024-06-03T02:28:11.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.644+0000 7fd9e3400700 10 client.26266 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:28:11.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.645+0000 7fd9e3400700 1 -- 172.21.15.79:0/3817922580 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56403dc58580 con 0x56403dd18d00 2024-06-03T02:28:11.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.645+0000 7fd9e3400700 10 client.26266 handle_command_reply: tid=0 2024-06-03T02:28:11.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.645+0000 7fda0695eb80 1 client.26266 shutdown 2024-06-03T02:28:11.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.645+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 msgr2=0x56403dd0a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:11.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.645+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd0a740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ad0 tx=0x7fd9e4004a50 comp rx=0 tx=0).stop 2024-06-03T02:28:11.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.645+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 shutdown_connections 2024-06-03T02:28:11.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.645+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x56403dd18d00 0x56403dd85ab0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.646+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56403dd09300 0x56403dd0a740 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.646+0000 7fd9e3400700 0 client.26266 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:28:11.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.646+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56403dc7ae40 0x56403dd09cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.646+0000 7fda0695eb80 1 --2- 172.21.15.79:0/3817922580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56403dcfe5b0 0x56403dd0a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.646+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 >> 172.21.15.79:0/3817922580 conn(0x56403dc2b170 msgr2=0x56403dd04530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:11.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.646+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 shutdown_connections 2024-06-03T02:28:11.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.647+0000 7fda0695eb80 1 -- 172.21.15.79:0/3817922580 wait complete. 2024-06-03T02:28:11.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.647+0000 7fda0695eb80 20 client.26266 trim_cache size 0 max 16384 2024-06-03T02:28:11.651 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:28:11.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.651+0000 7fd9fc000700 1 -- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd9d80a2d20 msgr2=0x7fd9d80a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:11.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.651+0000 7fd9fc000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd9d80a2d20 0x7fd9d80a51e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40062a0 tx=0x7fd9e400b040 comp rx=0 tx=0).stop 2024-06-03T02:28:11.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 -- 172.21.15.79:0/2847908798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f406ab10 msgr2=0x7fd9f419bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:11.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f406ab10 0x7fd9f419bd20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc010fc0 tx=0x7fd9dc010ff0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 -- 172.21.15.79:0/2847908798 shutdown_connections 2024-06-03T02:28:11.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd9d80a2d20 0x7fd9d80a51e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9f41084e0 0x7fd9f41a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9f406a170 0x7fd9f419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 --2- 172.21.15.79:0/2847908798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9f406ab10 0x7fd9f419bd20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:11.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 -- 172.21.15.79:0/2847908798 >> 172.21.15.79:0/2847908798 conn(0x7fd9f40782c0 msgr2=0x7fd9f4103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:11.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 -- 172.21.15.79:0/2847908798 shutdown_connections 2024-06-03T02:28:11.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:11.652+0000 7fd9fc000700 1 -- 172.21.15.79:0/2847908798 wait complete. 2024-06-03T02:28:12.397 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:12 smithi135 ceph-mon[36653]: pgmap v760: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-06-03T02:28:12.402 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:12 smithi079 ceph-mon[31819]: pgmap v760: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-06-03T02:28:12.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:12 smithi123 ceph-mon[36640]: pgmap v760: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-06-03T02:28:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 485.0 s, 546.8 tps, lat 173.127 ms stddev 207.984 2024-06-03T02:28:14.396 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:14 smithi079 ceph-mon[31819]: pgmap v761: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 764 op/s 2024-06-03T02:28:14.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:14 smithi123 ceph-mon[36640]: pgmap v761: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 764 op/s 2024-06-03T02:28:14.475 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:28:14.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:14 smithi135 ceph-mon[36653]: pgmap v761: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 764 op/s 2024-06-03T02:28:15.152 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:28:16.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:16 smithi123 ceph-mon[36640]: pgmap v762: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 678 op/s 2024-06-03T02:28:16.525 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:16 smithi079 ceph-mon[31819]: pgmap v762: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 678 op/s 2024-06-03T02:28:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:16 smithi135 ceph-mon[36653]: pgmap v762: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 678 op/s 2024-06-03T02:28:18.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 490.0 s, 436.6 tps, lat 235.715 ms stddev 385.540 2024-06-03T02:28:18.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:18 smithi079 ceph-mon[31819]: pgmap v763: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 596 op/s 2024-06-03T02:28:18.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:18 smithi123 ceph-mon[36640]: pgmap v763: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 596 op/s 2024-06-03T02:28:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:18 smithi135 ceph-mon[36653]: pgmap v763: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 596 op/s 2024-06-03T02:28:19.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.596+0000 7f7ca2400700 1 -- 172.21.15.79:0/1453895482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c9c108ac0 msgr2=0x7f7c9c108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.596+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1453895482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c9c108ac0 0x7f7c9c108f40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009990 tx=0x7f7c84009ca0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.597+0000 7f7ca2400700 1 -- 172.21.15.79:0/1453895482 shutdown_connections 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.597+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1453895482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c109480 0x7f7c9c115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.597+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1453895482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c10eac0 0x7f7c9c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.597+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1453895482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c9c108ac0 0x7f7c9c108f40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.597+0000 7f7ca2400700 1 -- 172.21.15.79:0/1453895482 >> 172.21.15.79:0/1453895482 conn(0x7f7c9c075c60 msgr2=0x7f7c9c078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.597+0000 7f7ca2400700 1 -- 172.21.15.79:0/1453895482 shutdown_connections 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.597+0000 7f7ca2400700 1 -- 172.21.15.79:0/1453895482 wait complete. 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.599+0000 7f7ca2400700 1 Processor -- start 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.599+0000 7f7ca2400700 1 -- start start 2024-06-03T02:28:19.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.599+0000 7f7ca2400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c108ac0 0x7f7c9c073b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.599+0000 7f7ca2400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c9c109480 0x7f7c9c074090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.599+0000 7f7ca2400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c10eac0 0x7f7c9c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.599+0000 7f7ca2400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c118ef0 con 0x7f7c9c109480 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7ca2400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c118d70 con 0x7f7c9c10eac0 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c9be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c10eac0 0x7f7c9c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c9be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c10eac0 0x7f7c9c06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53724/0 (socket says 172.21.15.79:53724) 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c9be00700 1 -- 172.21.15.79:0/1295582232 learned_addr learned my addr 172.21.15.79:0/1295582232 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7ca2400700 1 -- 172.21.15.79:0/1295582232 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c119070 con 0x7f7c9c108ac0 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c9b400700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c108ac0 0x7f7c9c073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c92a00700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c9c109480 0x7f7c9c074090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c9be00700 1 -- 172.21.15.79:0/1295582232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c108ac0 msgr2=0x7f7c9c073b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c9be00700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c108ac0 0x7f7c9c073b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c9be00700 1 -- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c9c109480 msgr2=0x7f7c9c074090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c9be00700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c9c109480 0x7f7c9c074090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c9be00700 1 -- 172.21.15.79:0/1295582232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c06d5f0 con 0x7f7c9c10eac0 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.600+0000 7f7c92a00700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c9c109480 0x7f7c9c074090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.601+0000 7f7c9be00700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c10eac0 0x7f7c9c06cd20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7c9400e940 tx=0x7f7c9400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.601+0000 7f7c98c00700 1 -- 172.21.15.79:0/1295582232 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400cbd0 con 0x7f7c9c10eac0 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.601+0000 7f7c98c00700 1 -- 172.21.15.79:0/1295582232 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c9400ee90 con 0x7f7c9c10eac0 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.601+0000 7f7c98c00700 1 -- 172.21.15.79:0/1295582232 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c940056c0 con 0x7f7c9c10eac0 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.601+0000 7f7ca2400700 1 -- 172.21.15.79:0/1295582232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c06d8e0 con 0x7f7c9c10eac0 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.601+0000 7f7ca2400700 1 -- 172.21.15.79:0/1295582232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c0718b0 con 0x7f7c9c10eac0 2024-06-03T02:28:19.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.611+0000 7f7cacd2eb80 1 Processor -- start 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.612+0000 7f7c98c00700 1 -- 172.21.15.79:0/1295582232 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7c94004750 con 0x7f7c9c10eac0 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.612+0000 7f7cacd2eb80 1 -- start start 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.613+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.613+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 0x55f67f3b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.613+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b3240 0x55f67f3b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.613+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f67f2c1a10 con 0x55f67f3b94b0 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.613+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f67f304b00 con 0x55f67f3b8b90 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.613+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f67f2bf8f0 con 0x55f67f3b3240 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.613+0000 7f7c98c00700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c800a2d70 0x7f7c800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.613+0000 7f7c9b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 0x55f67f3b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.613+0000 7f7c92a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b3240 0x55f67f3b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.614+0000 7f7c98c00700 1 -- 172.21.15.79:0/1295582232 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7c940cf650 con 0x7f7c9c10eac0 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.614+0000 7f7c9be00700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c800a2d70 0x7f7c800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.614+0000 7f7c9b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 0x55f67f3b2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38158/0 (socket says 172.21.15.79:38158) 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.614+0000 7f7c9b400700 1 -- 172.21.15.79:0/457909581 learned_addr learned my addr 172.21.15.79:0/457909581 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.614+0000 7f7c92a00700 1 --2- 172.21.15.79:0/457909581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.615+0000 7f7c92a00700 1 -- 172.21.15.79:0/457909581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 msgr2=0x55f67f3b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.615+0000 7f7c92a00700 1 --2- 172.21.15.79:0/457909581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.615+0000 7f7c92a00700 1 -- 172.21.15.79:0/457909581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 msgr2=0x55f67f3b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.615+0000 7f7c92a00700 1 --2- 172.21.15.79:0/457909581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 0x55f67f3b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.615+0000 7f7c92a00700 1 -- 172.21.15.79:0/457909581 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f67f2acf90 con 0x55f67f3b3240 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.615+0000 7f7c92a00700 1 --2- 172.21.15.79:0/457909581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b3240 0x55f67f3b3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7c84005b40 tx=0x7f7c84005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f71650bddb025f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.615+0000 7f7c90c00700 1 -- 172.21.15.79:0/457909581 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84020040 con 0x55f67f3b3240 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.615+0000 7f7c9be00700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c800a2d70 0x7f7c800a5230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c94000c00 tx=0x7f7c9400cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.615+0000 7f7c90c00700 1 -- 172.21.15.79:0/457909581 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c8400bd60 con 0x55f67f3b3240 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.616+0000 7f7c90c00700 1 -- 172.21.15.79:0/457909581 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84003e30 con 0x55f67f3b3240 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.616+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/457909581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b3240 msgr2=0x55f67f3b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.616+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/457909581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b3240 0x55f67f3b3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7c84005b40 tx=0x7f7c84005ab0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.617+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/457909581 shutdown_connections 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.617+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/457909581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b3240 0x55f67f3b3680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.617+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/457909581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.617+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/457909581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 0x55f67f3b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.617+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/457909581 >> 172.21.15.79:0/457909581 conn(0x55f67f2df170 msgr2=0x55f67f3adff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.617+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/457909581 shutdown_connections 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.617+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/457909581 wait complete. 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.618+0000 7f7cacd2eb80 1 -- start start 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.618+0000 7f7cacd2eb80 1 Processor -- start 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 0x55f67f3bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3bc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b94b0 0x55f67f3bca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f67f2a0f60 con 0x55f67f3b3240 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f67f300ac0 con 0x55f67f3b8b90 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f67f3bfe30 con 0x55f67f3b94b0 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7cacd2eb80 10 client.0 _ll_register_callbacks cb 0x55f67f29a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:19.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7c9b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3bc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7c9b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3bc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53734/0 (socket says 172.21.15.79:53734) 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7c9b400700 1 -- 172.21.15.79:0/4244473564 learned_addr learned my addr 172.21.15.79:0/4244473564 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.619+0000 7f7c92a00700 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 0x55f67f3bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7c9b400700 1 -- 172.21.15.79:0/4244473564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b94b0 msgr2=0x55f67f3bca60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7c9b400700 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b94b0 0x55f67f3bca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7c9b400700 1 -- 172.21.15.79:0/4244473564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 msgr2=0x55f67f3bf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7c9b400700 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 0x55f67f3bf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7c9b400700 1 -- 172.21.15.79:0/4244473564 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f67f2538d0 con 0x55f67f3b8b90 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7c9b400700 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3bc520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c007c30 tx=0x7f7c8c007c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd94f6eb876cb23a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:19.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7c89600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7c89600700 1 -- 172.21.15.79:0/4244473564 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c01a030 con 0x55f67f3b8b90 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/4244473564 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c8400bd60 con 0x55f67f3b8b90 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.620+0000 7f7c89600700 1 -- 172.21.15.79:0/4244473564 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7c8c0138d0 con 0x55f67f3b8b90 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7c89600700 1 -- 172.21.15.79:0/4244473564 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c8c027050 con 0x55f67f3b8b90 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7cacd2eb80 10 client.26270 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/4244473564 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f67f3ca610 con 0x55f67f3b8b90 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7c89600700 1 -- 172.21.15.79:0/4244473564 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c8400bd60 con 0x55f67f3b8b90 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7c89600700 1 -- 172.21.15.79:0/4244473564 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7c8c016bb0 con 0x55f67f3b8b90 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7cacd2eb80 10 client.26270 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7cacd2eb80 10 client.26270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7cacd2eb80 20 client.26270 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f67f3ccc50 0x55f67f3cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7cacd2eb80 4 client.26270 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/4244473564 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f67f312000 con 0x55f67f3ccc50 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.621+0000 7f7c92a00700 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f67f3ccc50 0x55f67f3cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.622+0000 7f7c89600700 10 client.26270 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.622+0000 7f7c92a00700 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f67f3ccc50 0x55f67f3cf040 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd80409ca8401cf server_cookie=929433283506a375 in_seq=0 out_seq=0 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7c89600700 1 -- 172.21.15.79:0/4244473564 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f67f312000 con 0x55f67f3ccc50 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7c89600700 10 client.26270 handle_command_reply: tid=0 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 client.26270 shutdown 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/4244473564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 msgr2=0x55f67f3bc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3bc520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c007c30 tx=0x7f7c8c007c60 comp rx=0 tx=0).stop 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/4244473564 shutdown_connections 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b94b0 0x55f67f3bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f67f3ccc50 0x55f67f3cf040 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 0x55f67f3bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/4244473564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b8b90 0x55f67f3bc520 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/4244473564 >> 172.21.15.79:0/4244473564 conn(0x55f67f2df170 msgr2=0x55f67f3a7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7c89600700 0 client.26270 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.623+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/4244473564 shutdown_connections 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.624+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/4244473564 wait complete. 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.624+0000 7f7cacd2eb80 20 client.26270 trim_cache size 0 max 16384 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.648+0000 7f7cacd2eb80 1 Processor -- start 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.648+0000 7f7cacd2eb80 1 -- start start 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.649+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 0x55f67f3b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.649+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3ade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.649+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b94b0 0x55f67f3bcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.649+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1bd240 con 0x55f67f3b3240 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.649+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1bed80 con 0x55f67f3b94b0 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.649+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1bd910 con 0x55f67f3b8b90 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c92a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b94b0 0x55f67f3bcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c92a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b94b0 0x55f67f3bcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53742/0 (socket says 172.21.15.79:53742) 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c92a00700 1 -- 172.21.15.79:0/3929204462 learned_addr learned my addr 172.21.15.79:0/3929204462 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c92a00700 1 --2- 172.21.15.79:0/3929204462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 0x55f67f3b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c9b400700 1 --2- 172.21.15.79:0/3929204462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c9b400700 1 -- 172.21.15.79:0/3929204462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b94b0 msgr2=0x55f67f3bcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c9b400700 1 --2- 172.21.15.79:0/3929204462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b94b0 0x55f67f3bcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c9b400700 1 -- 172.21.15.79:0/3929204462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 msgr2=0x55f67f3b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c9b400700 1 --2- 172.21.15.79:0/3929204462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 0x55f67f3b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.650+0000 7f7c9b400700 1 -- 172.21.15.79:0/3929204462 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f67f3bd420 con 0x55f67f3b8b90 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.651+0000 7f7c9b400700 1 --2- 172.21.15.79:0/3929204462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3ade90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c000f80 tx=0x7f7c8c009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=706c3e55e331c26c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.651+0000 7f7c8be00700 1 -- 172.21.15.79:0/3929204462 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c02f070 con 0x55f67f3b8b90 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.651+0000 7f7c8be00700 1 -- 172.21.15.79:0/3929204462 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c005e60 con 0x55f67f3b8b90 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.651+0000 7f7c8be00700 1 -- 172.21.15.79:0/3929204462 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c01a040 con 0x55f67f3b8b90 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.652+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3929204462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 msgr2=0x55f67f3ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.652+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3929204462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3ade90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c000f80 tx=0x7f7c8c009ac0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.652+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3929204462 shutdown_connections 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.652+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3929204462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b94b0 0x55f67f3bcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.653+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3929204462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b3240 0x55f67f3b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.653+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3929204462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3ade90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.653+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3929204462 >> 172.21.15.79:0/3929204462 conn(0x55f67f2df170 msgr2=0x55f67f3b9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.653+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3929204462 shutdown_connections 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.653+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3929204462 wait complete. 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.653+0000 7f7cacd2eb80 1 -- start start 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.653+0000 7f7cacd2eb80 1 Processor -- start 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.654+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b3240 0x55f67f3c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.654+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.654+0000 7f7cacd2eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 0x55f67f3d3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.654+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1be6b0 con 0x55f67f3b94b0 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.654+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1bdfe0 con 0x55f67f3b3240 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.654+0000 7f7cacd2eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1bb1e0 con 0x55f67f3b8b90 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.654+0000 7f7cacd2eb80 10 client.0 _ll_register_callbacks cb 0x55f67f29a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c92a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b3240 0x55f67f3c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c92a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 0x55f67f3d3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c9b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c9b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56288/0 (socket says 172.21.15.79:56288) 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c9b400700 1 -- 172.21.15.79:0/3970712135 learned_addr learned my addr 172.21.15.79:0/3970712135 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c9b400700 1 -- 172.21.15.79:0/3970712135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b3240 msgr2=0x55f67f3c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c9b400700 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b3240 0x55f67f3c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c9b400700 1 -- 172.21.15.79:0/3970712135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 msgr2=0x55f67f3d3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c9b400700 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 0x55f67f3d3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c9b400700 1 -- 172.21.15.79:0/3970712135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f67f3d3fd0 con 0x55f67f3b8b90 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c9b400700 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3c5630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c007980 tx=0x7f7c8c0097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be58c4fcf2793892 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.655+0000 7f7c89600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.656+0000 7f7c89600700 1 -- 172.21.15.79:0/3970712135 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c02f070 con 0x55f67f3b8b90 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.656+0000 7f7c89600700 1 -- 172.21.15.79:0/3970712135 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7c8c013ae0 con 0x55f67f3b8b90 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.656+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3970712135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c8c005e60 con 0x55f67f3b8b90 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.656+0000 7f7c89600700 1 -- 172.21.15.79:0/3970712135 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c8c01a040 con 0x55f67f3b8b90 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.656+0000 7f7cacd2eb80 10 client.26281 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.656+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3970712135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f67f437160 con 0x55f67f3b8b90 2024-06-03T02:28:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.656+0000 7f7c89600700 1 -- 172.21.15.79:0/3970712135 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c005e60 con 0x55f67f3b8b90 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.657+0000 7f7c89600700 1 -- 172.21.15.79:0/3970712135 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7c8c01fc30 con 0x55f67f3b8b90 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.657+0000 7f7cacd2eb80 10 client.26281 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.657+0000 7f7cacd2eb80 10 client.26281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.657+0000 7f7cacd2eb80 20 client.26281 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.657+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f67f437760 0x55f67f439b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.657+0000 7f7cacd2eb80 4 client.26281 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.657+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3970712135 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f67f30c580 con 0x55f67f437760 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.657+0000 7f7c92a00700 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f67f437760 0x55f67f439b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.658+0000 7f7c89600700 10 client.26281 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.658+0000 7f7c92a00700 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f67f437760 0x55f67f439b80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c84542ff579f09 server_cookie=a4da960c81fa7b67 in_seq=0 out_seq=0 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.658+0000 7f7c89600700 1 -- 172.21.15.79:0/3970712135 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f67f30c580 con 0x55f67f437760 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.658+0000 7f7c89600700 10 client.26281 handle_command_reply: tid=0 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.658+0000 7f7cacd2eb80 1 client.26281 shutdown 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.659+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3970712135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 msgr2=0x55f67f3c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:19.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.659+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3c5630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c007980 tx=0x7f7c8c0097a0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.659+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3970712135 shutdown_connections 2024-06-03T02:28:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.659+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f67f437760 0x55f67f439b80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.659+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f67f3b94b0 0x55f67f3d3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.659+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f67f3b3240 0x55f67f3c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.659+0000 7f7c89600700 0 client.26281 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.659+0000 7f7cacd2eb80 1 --2- 172.21.15.79:0/3970712135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f67f3b8b90 0x55f67f3c5630 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.659+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3970712135 >> 172.21.15.79:0/3970712135 conn(0x55f67f2df170 msgr2=0x55f67f3b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.660+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3970712135 shutdown_connections 2024-06-03T02:28:19.881 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.660+0000 7f7cacd2eb80 1 -- 172.21.15.79:0/3970712135 wait complete. 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.660+0000 7f7cacd2eb80 20 client.26281 trim_cache size 0 max 16384 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 -- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c800a2d70 msgr2=0x7f7c800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c800a2d70 0x7f7c800a5230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c94000c00 tx=0x7f7c9400cf90 comp rx=0 tx=0).stop 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 -- 172.21.15.79:0/1295582232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c10eac0 msgr2=0x7f7c9c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c10eac0 0x7f7c9c06cd20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7c9400e940 tx=0x7f7c9400ec50 comp rx=0 tx=0).stop 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 -- 172.21.15.79:0/1295582232 shutdown_connections 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c10eac0 0x7f7c9c06cd20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c108ac0 0x7f7c9c073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c800a2d70 0x7f7c800a5230 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 --2- 172.21.15.79:0/1295582232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c9c109480 0x7f7c9c074090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:19.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.661+0000 7f7ca2400700 1 -- 172.21.15.79:0/1295582232 >> 172.21.15.79:0/1295582232 conn(0x7f7c9c075c60 msgr2=0x7f7c9c076250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:19.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.662+0000 7f7ca2400700 1 -- 172.21.15.79:0/1295582232 shutdown_connections 2024-06-03T02:28:19.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:19.662+0000 7f7ca2400700 1 -- 172.21.15.79:0/1295582232 wait complete. 2024-06-03T02:28:20.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:20 smithi123 ceph-mon[36640]: pgmap v764: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 628 op/s 2024-06-03T02:28:20.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:20 smithi079 ceph-mon[31819]: pgmap v764: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 628 op/s 2024-06-03T02:28:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:20 smithi135 ceph-mon[36653]: pgmap v764: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 628 op/s 2024-06-03T02:28:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:22 smithi123 ceph-mon[36640]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-06-03T02:28:22.578 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:28:22.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:28:22.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:22 smithi079 ceph-mon[31819]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-06-03T02:28:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:22 smithi135 ceph-mon[36653]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-06-03T02:28:23.243 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:28:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 495.0 s, 517.0 tps, lat 193.129 ms stddev 269.334 2024-06-03T02:28:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:24 smithi123 ceph-mon[36640]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.1 MiB/s wr, 535 op/s 2024-06-03T02:28:24.591 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:24 smithi079 ceph-mon[31819]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.1 MiB/s wr, 535 op/s 2024-06-03T02:28:24.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:24 smithi135 ceph-mon[36653]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.1 MiB/s wr, 535 op/s 2024-06-03T02:28:26.195 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:26 smithi079 ceph-mon[31819]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.2 MiB/s wr, 467 op/s 2024-06-03T02:28:26.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:26 smithi123 ceph-mon[36640]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.2 MiB/s wr, 467 op/s 2024-06-03T02:28:26.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:26 smithi135 ceph-mon[36653]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.2 MiB/s wr, 467 op/s 2024-06-03T02:28:27.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.361+0000 7f08e9200700 1 -- 172.21.15.79:0/4089441305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e4107840 msgr2=0x7f08e410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:27.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.361+0000 7f08e9200700 1 --2- 172.21.15.79:0/4089441305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e4107840 0x7f08e410f3c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b0d0 tx=0x7f08dc00b3e0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.362+0000 7f08e9200700 1 -- 172.21.15.79:0/4089441305 shutdown_connections 2024-06-03T02:28:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.362+0000 7f08e9200700 1 --2- 172.21.15.79:0/4089441305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e4107840 0x7f08e410f3c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.362+0000 7f08e9200700 1 --2- 172.21.15.79:0/4089441305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e4102740 0x7f08e4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.362+0000 7f08e9200700 1 --2- 172.21.15.79:0/4089441305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e41030f0 0x7f08e4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.362+0000 7f08e9200700 1 -- 172.21.15.79:0/4089441305 >> 172.21.15.79:0/4089441305 conn(0x7f08e40795f0 msgr2=0x7f08e4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.362+0000 7f08e9200700 1 -- 172.21.15.79:0/4089441305 shutdown_connections 2024-06-03T02:28:27.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.362+0000 7f08e9200700 1 -- 172.21.15.79:0/4089441305 wait complete. 2024-06-03T02:28:27.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.363+0000 7f08e9200700 1 Processor -- start 2024-06-03T02:28:27.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.364+0000 7f08e9200700 1 -- start start 2024-06-03T02:28:27.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.364+0000 7f08e9200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e4102740 0x7f08e419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.364+0000 7f08e9200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e41030f0 0x7f08e419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.364+0000 7f08e9200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e4107840 0x7f08e419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.364+0000 7f08e9200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4114ac0 con 0x7f08e4107840 2024-06-03T02:28:27.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.364+0000 7f08e9200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4114940 con 0x7f08e4102740 2024-06-03T02:28:27.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.364+0000 7f08e9200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4114c40 con 0x7f08e41030f0 2024-06-03T02:28:27.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.365+0000 7f08e1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e41030f0 0x7f08e419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.365+0000 7f08e1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e41030f0 0x7f08e419b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36762/0 (socket says 172.21.15.79:36762) 2024-06-03T02:28:27.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.365+0000 7f08e1400700 1 -- 172.21.15.79:0/1921343880 learned_addr learned my addr 172.21.15.79:0/1921343880 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:27.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.365+0000 7f08e2800700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e4107840 0x7f08e419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.366+0000 7f08e1400700 1 -- 172.21.15.79:0/1921343880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e4102740 msgr2=0x7f08e419af30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:28:27.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.366+0000 7f08e1400700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e4102740 0x7f08e419af30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.366+0000 7f08e1400700 1 -- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e4107840 msgr2=0x7f08e419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:27.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.369+0000 7f08e1400700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e4107840 0x7f08e419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.369+0000 7f08e1400700 1 -- 172.21.15.79:0/1921343880 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e419ff40 con 0x7f08e41030f0 2024-06-03T02:28:27.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.369+0000 7f08e2800700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e4107840 0x7f08e419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:28:27.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.370+0000 7f08e1400700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e41030f0 0x7f08e419b470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f08d000b530 tx=0x7f08d000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:27.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.370+0000 7f08daa00700 1 -- 172.21.15.79:0/1921343880 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0015020 con 0x7f08e41030f0 2024-06-03T02:28:27.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.370+0000 7f08daa00700 1 -- 172.21.15.79:0/1921343880 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d0011d10 con 0x7f08e41030f0 2024-06-03T02:28:27.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.370+0000 7f08daa00700 1 -- 172.21.15.79:0/1921343880 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d000f2e0 con 0x7f08e41030f0 2024-06-03T02:28:27.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.370+0000 7f08e9200700 1 -- 172.21.15.79:0/1921343880 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e41a0230 con 0x7f08e41030f0 2024-06-03T02:28:27.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.370+0000 7f08e9200700 1 -- 172.21.15.79:0/1921343880 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e410ff60 con 0x7f08e41030f0 2024-06-03T02:28:27.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.384+0000 7f08daa00700 1 -- 172.21.15.79:0/1921343880 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f08d0018020 con 0x7f08e41030f0 2024-06-03T02:28:27.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.386+0000 7f08daa00700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08c80a2d20 0x7f08c80a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.386+0000 7f08daa00700 1 -- 172.21.15.79:0/1921343880 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f08d00ceb80 con 0x7f08e41030f0 2024-06-03T02:28:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.387+0000 7f08e1e00700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08c80a2d20 0x7f08c80a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.388+0000 7f08e1e00700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08c80a2d20 0x7f08c80a51e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009710 tx=0x7f08d4006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.389+0000 7f08f3b41b80 1 Processor -- start 2024-06-03T02:28:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.389+0000 7f08f3b41b80 1 -- start start 2024-06-03T02:28:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.390+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.390+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.390+0000 7f08e1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.390+0000 7f08e1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89a6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36776/0 (socket says 172.21.15.79:36776) 2024-06-03T02:28:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.392+0000 7f08e1e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.392+0000 7f08e1e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33546/0 (socket says 172.21.15.79:33546) 2024-06-03T02:28:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08e1400700 1 -- 172.21.15.79:0/1234766986 learned_addr learned my addr 172.21.15.79:0/1234766986 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89a7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1234766986 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5583f88b5a90 con 0x5583f89a72c0 2024-06-03T02:28:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1234766986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583f88f8b80 con 0x5583f89acc10 2024-06-03T02:28:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1234766986 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5583f88b3970 con 0x5583f89ad530 2024-06-03T02:28:27.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08e2800700 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08e1400700 1 -- 172.21.15.79:0/1234766986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 msgr2=0x5583f89acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:27.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08e1400700 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08e1400700 1 -- 172.21.15.79:0/1234766986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 msgr2=0x5583f89a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:27.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08e1400700 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89a7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.393+0000 7f08e1400700 1 -- 172.21.15.79:0/1234766986 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f88a1010 con 0x5583f89ad530 2024-06-03T02:28:27.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.394+0000 7f08e1e00700 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:28:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.394+0000 7f08e1400700 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89a6d80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f08d00979a0 tx=0x7f08d009aeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a08e57a7413ce3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.394+0000 7f08cf400700 1 -- 172.21.15.79:0/1234766986 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d009afd0 con 0x5583f89ad530 2024-06-03T02:28:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.394+0000 7f08cf400700 1 -- 172.21.15.79:0/1234766986 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d009f380 con 0x5583f89ad530 2024-06-03T02:28:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.396+0000 7f08cf400700 1 -- 172.21.15.79:0/1234766986 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d00a37b0 con 0x5583f89ad530 2024-06-03T02:28:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.397+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1234766986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 msgr2=0x5583f89a6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.397+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89a6d80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f08d00979a0 tx=0x7f08d009aeb0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.399+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1234766986 shutdown_connections 2024-06-03T02:28:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.399+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.399+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.399+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1234766986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89a6d80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.399+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1234766986 >> 172.21.15.79:0/1234766986 conn(0x5583f88d31f0 msgr2=0x5583f89a2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:27.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.399+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1234766986 shutdown_connections 2024-06-03T02:28:27.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.399+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1234766986 wait complete. 2024-06-03T02:28:27.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.400+0000 7f08f3b41b80 1 -- start start 2024-06-03T02:28:27.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.401+0000 7f08f3b41b80 1 Processor -- start 2024-06-03T02:28:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.401+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89a72c0 0x5583f89b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.401+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.401+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89ad530 0x5583f89b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.401+0000 7f08f3b41b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5583f8894fe0 con 0x5583f89ad530 2024-06-03T02:28:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.401+0000 7f08f3b41b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583f88f4b40 con 0x5583f89acc10 2024-06-03T02:28:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.402+0000 7f08f3b41b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5583f89b3f10 con 0x5583f89a72c0 2024-06-03T02:28:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.402+0000 7f08f3b41b80 10 client.0 _ll_register_callbacks cb 0x5583f888e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:27.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.402+0000 7f08e1400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.402+0000 7f08e1400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33556/0 (socket says 172.21.15.79:33556) 2024-06-03T02:28:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.402+0000 7f08e1400700 1 -- 172.21.15.79:0/1581121056 learned_addr learned my addr 172.21.15.79:0/1581121056 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.402+0000 7f08e2800700 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89ad530 0x5583f89b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.402+0000 7f08e1e00700 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89a72c0 0x5583f89b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.402+0000 7f08e1400700 1 -- 172.21.15.79:0/1581121056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89a72c0 msgr2=0x5583f89b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:27.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.403+0000 7f08e1400700 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89a72c0 0x5583f89b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.403+0000 7f08e1400700 1 -- 172.21.15.79:0/1581121056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89ad530 msgr2=0x5583f89b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:27.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.403+0000 7f08e1400700 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89ad530 0x5583f89b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.403+0000 7f08e1400700 1 -- 172.21.15.79:0/1581121056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f8847950 con 0x5583f89acc10 2024-06-03T02:28:27.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.403+0000 7f08e1400700 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89b05a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f08d00cc2c0 tx=0x7f08d0010fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66a76fb65a921e26 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:27.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.403+0000 7f08c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:28:27.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.404+0000 7f08c3e00700 1 -- 172.21.15.79:0/1581121056 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d007e060 con 0x5583f89acc10 2024-06-03T02:28:27.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.404+0000 7f08c3e00700 1 -- 172.21.15.79:0/1581121056 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f08d00cc380 con 0x5583f89acc10 2024-06-03T02:28:27.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.404+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1581121056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d009f380 con 0x5583f89acc10 2024-06-03T02:28:27.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.404+0000 7f08c3e00700 1 -- 172.21.15.79:0/1581121056 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f08d009afd0 con 0x5583f89acc10 2024-06-03T02:28:27.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.404+0000 7f08f3b41b80 10 client.26285 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:27.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.404+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1581121056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5583f89cf6d0 con 0x5583f89acc10 2024-06-03T02:28:27.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.405+0000 7f08c3e00700 1 -- 172.21.15.79:0/1581121056 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d009f380 con 0x5583f89acc10 2024-06-03T02:28:27.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.405+0000 7f08c3e00700 1 -- 172.21.15.79:0/1581121056 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f08d009a500 con 0x5583f89acc10 2024-06-03T02:28:27.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.405+0000 7f08f3b41b80 10 client.26285 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:27.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.405+0000 7f08f3b41b80 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:28:27.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.405+0000 7f08f3b41b80 20 client.26285 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:27.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.405+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583f89d1ba0 0x5583f89d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.406+0000 7f08f3b41b80 4 client.26285 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:28:27.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.406+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1581121056 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583f8906080 con 0x5583f89d1ba0 2024-06-03T02:28:27.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.406+0000 7f08e1e00700 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583f89d1ba0 0x5583f89d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.411+0000 7f08e1e00700 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583f89d1ba0 0x5583f89d3f90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32786aba356f4f6 server_cookie=c7849a92877144b in_seq=0 out_seq=0 2024-06-03T02:28:27.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.411+0000 7f08c3e00700 10 client.26285 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:27.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.412+0000 7f08c3e00700 1 -- 172.21.15.79:0/1581121056 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5583f8906080 con 0x5583f89d1ba0 2024-06-03T02:28:27.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.412+0000 7f08c3e00700 10 client.26285 handle_command_reply: tid=0 2024-06-03T02:28:27.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.412+0000 7f08f3b41b80 1 client.26285 shutdown 2024-06-03T02:28:27.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.412+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1581121056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 msgr2=0x5583f89b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:27.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.412+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89b05a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f08d00cc2c0 tx=0x7f08d0010fe0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.414+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1581121056 shutdown_connections 2024-06-03T02:28:27.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.414+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583f89d1ba0 0x5583f89d3f90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.414+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89ad530 0x5583f89b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.414+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89a72c0 0x5583f89b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.414+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1581121056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f89b05a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.414+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1581121056 >> 172.21.15.79:0/1581121056 conn(0x5583f88d31f0 msgr2=0x5583f899b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:27.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.414+0000 7f08c3e00700 0 client.26285 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:27.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.415+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1581121056 shutdown_connections 2024-06-03T02:28:27.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.415+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1581121056 wait complete. 2024-06-03T02:28:27.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.416+0000 7f08f3b41b80 20 client.26285 trim_cache size 0 max 16384 2024-06-03T02:28:27.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.461+0000 7f08f3b41b80 1 Processor -- start 2024-06-03T02:28:27.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.461+0000 7f08f3b41b80 1 -- start start 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f899c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08f3b41b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400ec30 con 0x5583f89a72c0 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08f3b41b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4010770 con 0x5583f89acc10 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08f3b41b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400f300 con 0x5583f89ad530 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08e2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08e2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89bca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36794/0 (socket says 172.21.15.79:36794) 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08e2800700 1 -- 172.21.15.79:0/794046393 learned_addr learned my addr 172.21.15.79:0/794046393 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:27.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.462+0000 7f08e1400700 1 --2- 172.21.15.79:0/794046393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f899c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08e2800700 1 -- 172.21.15.79:0/794046393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 msgr2=0x5583f899c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08e2800700 1 --2- 172.21.15.79:0/794046393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f899c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08e2800700 1 -- 172.21.15.79:0/794046393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 msgr2=0x5583f89a6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08e2800700 1 --2- 172.21.15.79:0/794046393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08e2800700 1 -- 172.21.15.79:0/794046393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f89aeb30 con 0x5583f89ad530 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08e1400700 1 --2- 172.21.15.79:0/794046393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f899c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08e2800700 1 --2- 172.21.15.79:0/794046393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89bca90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b010 tx=0x7f08dc003c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b9182296dd64f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08cea00700 1 -- 172.21.15.79:0/794046393 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0076e0 con 0x5583f89ad530 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08cea00700 1 -- 172.21.15.79:0/794046393 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08dc003e10 con 0x5583f89ad530 2024-06-03T02:28:27.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.463+0000 7f08cea00700 1 -- 172.21.15.79:0/794046393 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0058e0 con 0x5583f89ad530 2024-06-03T02:28:27.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.464+0000 7f08f3b41b80 1 -- 172.21.15.79:0/794046393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 msgr2=0x5583f89bca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:27.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.464+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/794046393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89bca90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b010 tx=0x7f08dc003c40 comp rx=0 tx=0).stop 2024-06-03T02:28:27.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.465+0000 7f08f3b41b80 1 -- 172.21.15.79:0/794046393 shutdown_connections 2024-06-03T02:28:27.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.465+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/794046393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89ad530 0x5583f89bca90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.465+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/794046393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.465+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/794046393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89acc10 0x5583f899c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.465+0000 7f08f3b41b80 1 -- 172.21.15.79:0/794046393 >> 172.21.15.79:0/794046393 conn(0x5583f88d31f0 msgr2=0x5583f89ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:27.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.465+0000 7f08f3b41b80 1 -- 172.21.15.79:0/794046393 shutdown_connections 2024-06-03T02:28:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.465+0000 7f08f3b41b80 1 -- 172.21.15.79:0/794046393 wait complete. 2024-06-03T02:28:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.466+0000 7f08f3b41b80 1 -- start start 2024-06-03T02:28:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.466+0000 7f08f3b41b80 1 Processor -- start 2024-06-03T02:28:27.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.468+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.468+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89acc10 0x5583f89d5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.468+0000 7f08f3b41b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89ad530 0x5583f89d6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.468+0000 7f08f3b41b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08c40100a0 con 0x5583f89a72c0 2024-06-03T02:28:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.468+0000 7f08f3b41b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400f9d0 con 0x5583f89ad530 2024-06-03T02:28:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.468+0000 7f08f3b41b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400cb70 con 0x5583f89acc10 2024-06-03T02:28:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.468+0000 7f08f3b41b80 10 client.0 _ll_register_callbacks cb 0x5583f888e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:27.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.468+0000 7f08e1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89acc10 0x5583f89d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.469+0000 7f08e2800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89ad530 0x5583f89d6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.469+0000 7f08e2800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89ad530 0x5583f89d6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33574/0 (socket says 172.21.15.79:33574) 2024-06-03T02:28:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.469+0000 7f08e2800700 1 -- 172.21.15.79:0/1809345962 learned_addr learned my addr 172.21.15.79:0/1809345962 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.469+0000 7f08e1e00700 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.469+0000 7f08e1400700 1 -- 172.21.15.79:0/1809345962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89ad530 msgr2=0x5583f89d6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.469+0000 7f08e1400700 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89ad530 0x5583f89d6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.469+0000 7f08e1400700 1 -- 172.21.15.79:0/1809345962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 msgr2=0x5583f89d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:27.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.469+0000 7f08e1400700 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.469+0000 7f08e1400700 1 -- 172.21.15.79:0/1809345962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f8a3c4f0 con 0x5583f89acc10 2024-06-03T02:28:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.470+0000 7f08e1400700 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89acc10 0x5583f89d5c10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5583f89a2bb0 tx=0x7f08d0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eabf84dbdc5cbe86 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.470+0000 7f08c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:28:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.471+0000 7f08c3e00700 1 -- 172.21.15.79:0/1809345962 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d00a7030 con 0x5583f89acc10 2024-06-03T02:28:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.471+0000 7f08c3e00700 1 -- 172.21.15.79:0/1809345962 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f08d00b5070 con 0x5583f89acc10 2024-06-03T02:28:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.471+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1809345962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08dc003e10 con 0x5583f89acc10 2024-06-03T02:28:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.471+0000 7f08c3e00700 1 -- 172.21.15.79:0/1809345962 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f08d009afd0 con 0x5583f89acc10 2024-06-03T02:28:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.471+0000 7f08f3b41b80 10 client.26296 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.471+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1809345962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5583f8a3d600 con 0x5583f89acc10 2024-06-03T02:28:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.471+0000 7f08c3e00700 1 -- 172.21.15.79:0/1809345962 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08dc003e10 con 0x5583f89acc10 2024-06-03T02:28:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.472+0000 7f08c3e00700 1 -- 172.21.15.79:0/1809345962 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f08d0085ce0 con 0x5583f89acc10 2024-06-03T02:28:27.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.472+0000 7f08f3b41b80 10 client.26296 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:27.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.472+0000 7f08f3b41b80 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:28:27.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.472+0000 7f08f3b41b80 20 client.26296 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:27.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.472+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583f8a3dad0 0x5583f8a3fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:27.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.472+0000 7f08f3b41b80 4 client.26296 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:28:27.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.472+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1809345962 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5583f8900600 con 0x5583f8a3dad0 2024-06-03T02:28:27.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.472+0000 7f08e1e00700 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583f8a3dad0 0x5583f8a3fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:27.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.473+0000 7f08c3e00700 10 client.26296 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:27.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.473+0000 7f08e1e00700 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583f8a3dad0 0x5583f8a3fef0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252e39196698098 server_cookie=e50aeeb5cd539e0a in_seq=0 out_seq=0 2024-06-03T02:28:27.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.474+0000 7f08c3e00700 1 -- 172.21.15.79:0/1809345962 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5583f8900600 con 0x5583f8a3dad0 2024-06-03T02:28:27.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.474+0000 7f08c3e00700 10 client.26296 handle_command_reply: tid=0 2024-06-03T02:28:27.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.475+0000 7f08f3b41b80 1 client.26296 shutdown 2024-06-03T02:28:27.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.476+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1809345962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89acc10 msgr2=0x5583f89d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:27.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.476+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89acc10 0x5583f89d5c10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5583f89a2bb0 tx=0x7f08d0000bc0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.476+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1809345962 shutdown_connections 2024-06-03T02:28:27.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.476+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f89ad530 0x5583f89d6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.476+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5583f8a3dad0 0x5583f8a3fef0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.476+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5583f89a72c0 0x5583f89d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.476+0000 7f08f3b41b80 1 --2- 172.21.15.79:0/1809345962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583f89acc10 0x5583f89d5c10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.476+0000 7f08c3e00700 0 client.26296 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:27.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.476+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1809345962 >> 172.21.15.79:0/1809345962 conn(0x5583f88d31f0 msgr2=0x5583f89b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:27.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.477+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1809345962 shutdown_connections 2024-06-03T02:28:27.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.477+0000 7f08f3b41b80 1 -- 172.21.15.79:0/1809345962 wait complete. 2024-06-03T02:28:27.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.478+0000 7f08f3b41b80 20 client.26296 trim_cache size 0 max 16384 2024-06-03T02:28:27.478 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:28:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:28:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "5a4f5275-1842-457e-845f-097c73eb9a69", 2024-06-03T02:28:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:28:27.479 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:28:27.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 -- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08c80a2d20 msgr2=0x7f08c80a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:27.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08c80a2d20 0x7f08c80a51e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009710 tx=0x7f08d4006c60 comp rx=0 tx=0).stop 2024-06-03T02:28:27.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 -- 172.21.15.79:0/1921343880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e41030f0 msgr2=0x7f08e419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:27.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e41030f0 0x7f08e419b470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f08d000b530 tx=0x7f08d000b840 comp rx=0 tx=0).stop 2024-06-03T02:28:27.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 -- 172.21.15.79:0/1921343880 shutdown_connections 2024-06-03T02:28:27.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f08c80a2d20 0x7f08c80a51e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08e4107840 0x7f08e419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e4102740 0x7f08e419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:27.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 --2- 172.21.15.79:0/1921343880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e41030f0 0x7f08e419b470 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f08d000b530 tx=0x7f08d000b840 comp rx=0 tx=0).stop 2024-06-03T02:28:27.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 -- 172.21.15.79:0/1921343880 >> 172.21.15.79:0/1921343880 conn(0x7f08e40795f0 msgr2=0x7f08e4108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:27.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.479+0000 7f08e9200700 1 -- 172.21.15.79:0/1921343880 shutdown_connections 2024-06-03T02:28:27.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:27.480+0000 7f08e9200700 1 -- 172.21.15.79:0/1921343880 wait complete. 2024-06-03T02:28:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 500.0 s, 509.2 tps, lat 199.020 ms stddev 289.125 2024-06-03T02:28:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:28 smithi079 ceph-mon[31819]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-06-03T02:28:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:28 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:28:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:28 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:28:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:28 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:28:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:28 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:28:28.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:28 smithi123 ceph-mon[36640]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-06-03T02:28:28.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:28 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:28:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:28 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:28:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:28 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:28:28.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:28 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:28:28.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:28:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:28 smithi135 ceph-mon[36653]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-06-03T02:28:28.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:28 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:28:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:28 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:28:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:28 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:28:28.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:28 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:28:29.233 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:28:29.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:29 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:28:29.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:29 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:28:29.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:29 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:28:30.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:30 smithi079 ceph-mon[31819]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 541 op/s 2024-06-03T02:28:30.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:30 smithi135 ceph-mon[36653]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 541 op/s 2024-06-03T02:28:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:30 smithi123 ceph-mon[36640]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 541 op/s 2024-06-03T02:28:32.571 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:32 smithi079 ceph-mon[31819]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 425 op/s 2024-06-03T02:28:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:32 smithi135 ceph-mon[36653]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 425 op/s 2024-06-03T02:28:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:32 smithi123 ceph-mon[36640]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 425 op/s 2024-06-03T02:28:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 505.0 s, 490.2 tps, lat 203.814 ms stddev 241.714 2024-06-03T02:28:34.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.147+0000 7f3b9e200700 1 -- 172.21.15.79:0/1784625822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b900a51e0 msgr2=0x7f3b900a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:34.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.147+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1784625822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b900a51e0 0x7f3b900a55c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009990 tx=0x7f3b8c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.149+0000 7f3b9e200700 1 -- 172.21.15.79:0/1784625822 shutdown_connections 2024-06-03T02:28:34.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.149+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1784625822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b900aa2c0 0x7f3b900b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.149+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1784625822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b900a51e0 0x7f3b900a55c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.149+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1784625822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b900a5b90 0x7f3b900a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.149+0000 7f3b9e200700 1 -- 172.21.15.79:0/1784625822 >> 172.21.15.79:0/1784625822 conn(0x7f3b9001aac0 msgr2=0x7f3b9001aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:34.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.150+0000 7f3b9e200700 1 -- 172.21.15.79:0/1784625822 shutdown_connections 2024-06-03T02:28:34.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.150+0000 7f3b9e200700 1 -- 172.21.15.79:0/1784625822 wait complete. 2024-06-03T02:28:34.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.154+0000 7f3b9e200700 1 Processor -- start 2024-06-03T02:28:34.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.155+0000 7f3b9e200700 1 -- start start 2024-06-03T02:28:34.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.158+0000 7f3b9e200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b900a51e0 0x7f3b900bb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.158+0000 7f3b9e200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b900a5b90 0x7f3b900b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.158+0000 7f3b9e200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b900aa2c0 0x7f3b900b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.158+0000 7f3b9e200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3b900b3cd0 con 0x7f3b900a5b90 2024-06-03T02:28:34.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.158+0000 7f3b9e200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b900b3b50 con 0x7f3b900aa2c0 2024-06-03T02:28:34.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.158+0000 7f3b9e200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b900b3e50 con 0x7f3b900a51e0 2024-06-03T02:28:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.158+0000 7f3b9d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b900aa2c0 0x7f3b900b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.158+0000 7f3b9d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b900aa2c0 0x7f3b900b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33598/0 (socket says 172.21.15.79:33598) 2024-06-03T02:28:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.158+0000 7f3b9d800700 1 -- 172.21.15.79:0/1495194265 learned_addr learned my addr 172.21.15.79:0/1495194265 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.159+0000 7f3b9d800700 1 -- 172.21.15.79:0/1495194265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b900a51e0 msgr2=0x7f3b900bb160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:28:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.159+0000 7f3b9d800700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b900a51e0 0x7f3b900bb160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.159+0000 7f3b9d800700 1 -- 172.21.15.79:0/1495194265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b900a5b90 msgr2=0x7f3b900b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.159+0000 7f3b9d800700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b900a5b90 0x7f3b900b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.159+0000 7f3b9d800700 1 -- 172.21.15.79:0/1495194265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b900b5e50 con 0x7f3b900aa2c0 2024-06-03T02:28:34.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.161+0000 7f3b9d800700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b900aa2c0 0x7f3b900b5720 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3b980517a0 tx=0x7f3b9806cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:34.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.161+0000 7f3b95600700 1 -- 172.21.15.79:0/1495194265 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b980717f0 con 0x7f3b900aa2c0 2024-06-03T02:28:34.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.162+0000 7f3b95600700 1 -- 172.21.15.79:0/1495194265 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b98077420 con 0x7f3b900aa2c0 2024-06-03T02:28:34.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.162+0000 7f3b95600700 1 -- 172.21.15.79:0/1495194265 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b980765e0 con 0x7f3b900aa2c0 2024-06-03T02:28:34.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.162+0000 7f3b9e200700 1 -- 172.21.15.79:0/1495194265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b90158a20 con 0x7f3b900aa2c0 2024-06-03T02:28:34.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.162+0000 7f3b9e200700 1 -- 172.21.15.79:0/1495194265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b90158e80 con 0x7f3b900aa2c0 2024-06-03T02:28:34.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.170+0000 7f3b95600700 1 -- 172.21.15.79:0/1495194265 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3b9807e030 con 0x7f3b900aa2c0 2024-06-03T02:28:34.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.173+0000 7f3baa18db80 1 Processor -- start 2024-06-03T02:28:34.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.174+0000 7f3b95600700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3b800a2c90 0x7f3b800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.174+0000 7f3baa18db80 1 -- start start 2024-06-03T02:28:34.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.174+0000 7f3b95600700 1 -- 172.21.15.79:0/1495194265 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3b981323e0 con 0x7f3b900aa2c0 2024-06-03T02:28:34.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.174+0000 7f3b97e00700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3b800a2c90 0x7f3b800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.175+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0721420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.175+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f0721de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.175+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f072c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.175+0000 7f3b9ce00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0721420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.175+0000 7f3b9ce00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0721420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36818/0 (socket says 172.21.15.79:36818) 2024-06-03T02:28:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3baa18db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5563f062ea10 con 0x5563f0721960 2024-06-03T02:28:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3baa18db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5563f0671b00 con 0x5563f072a570 2024-06-03T02:28:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3baa18db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5563f062c8f0 con 0x5563f069be40 2024-06-03T02:28:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3b9ce00700 1 -- 172.21.15.79:0/3740032016 learned_addr learned my addr 172.21.15.79:0/3740032016 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:34.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3b97e00700 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f0721de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3b97e00700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3b800a2c90 0x7f3b800a5150 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005fd0 tx=0x7f3b84005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3b9d800700 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f072c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3b9ce00700 1 -- 172.21.15.79:0/3740032016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 msgr2=0x5563f072c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3b9ce00700 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f072c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3b9ce00700 1 -- 172.21.15.79:0/3740032016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 msgr2=0x5563f0721de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3b9ce00700 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f0721de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.176+0000 7f3b9ce00700 1 -- 172.21.15.79:0/3740032016 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f0619f90 con 0x5563f069be40 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.177+0000 7f3b97e00700 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f0721de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.177+0000 7f3b9d800700 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f072c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:34.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.177+0000 7f3b9ce00700 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0721420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c000c00 tx=0x7f3b8c005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ee6a73cd67eee8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.177+0000 7f3b8a000700 1 -- 172.21.15.79:0/3740032016 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c003780 con 0x5563f069be40 2024-06-03T02:28:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.177+0000 7f3b8a000700 1 -- 172.21.15.79:0/3740032016 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b8c0050c0 con 0x5563f069be40 2024-06-03T02:28:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.177+0000 7f3b8a000700 1 -- 172.21.15.79:0/3740032016 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00bbd0 con 0x5563f069be40 2024-06-03T02:28:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.178+0000 7f3baa18db80 1 -- 172.21.15.79:0/3740032016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 msgr2=0x5563f0721420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:34.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.178+0000 7f3baa18db80 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0721420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c000c00 tx=0x7f3b8c005da0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.179+0000 7f3baa18db80 1 -- 172.21.15.79:0/3740032016 shutdown_connections 2024-06-03T02:28:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.179+0000 7f3baa18db80 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f072c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.179+0000 7f3baa18db80 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0721420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.179+0000 7f3baa18db80 1 --2- 172.21.15.79:0/3740032016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f0721de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.179+0000 7f3baa18db80 1 -- 172.21.15.79:0/3740032016 >> 172.21.15.79:0/3740032016 conn(0x5563f064c170 msgr2=0x5563f071b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.179+0000 7f3baa18db80 1 -- 172.21.15.79:0/3740032016 shutdown_connections 2024-06-03T02:28:34.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.179+0000 7f3baa18db80 1 -- 172.21.15.79:0/3740032016 wait complete. 2024-06-03T02:28:34.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.180+0000 7f3baa18db80 1 -- start start 2024-06-03T02:28:34.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.180+0000 7f3baa18db80 1 Processor -- start 2024-06-03T02:28:34.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.181+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f069be40 0x5563f0729d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.181+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f0721960 0x5563f0727000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.181+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f072a570 0x5563f0727540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.181+0000 7f3baa18db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5563f060df60 con 0x5563f072a570 2024-06-03T02:28:34.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.181+0000 7f3baa18db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5563f066dac0 con 0x5563f069be40 2024-06-03T02:28:34.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.181+0000 7f3baa18db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5563f072d120 con 0x5563f0721960 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.181+0000 7f3baa18db80 10 client.0 _ll_register_callbacks cb 0x5563f060b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.182+0000 7f3b97e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f0721960 0x5563f0727000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.182+0000 7f3b97e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f0721960 0x5563f0727000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36822/0 (socket says 172.21.15.79:36822) 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.182+0000 7f3b97e00700 1 -- 172.21.15.79:0/386938781 learned_addr learned my addr 172.21.15.79:0/386938781 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.182+0000 7f3b9d800700 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f072a570 0x5563f0727540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.183+0000 7f3b97e00700 1 -- 172.21.15.79:0/386938781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f069be40 msgr2=0x5563f0729d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.183+0000 7f3b97e00700 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f069be40 0x5563f0729d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.183+0000 7f3b97e00700 1 -- 172.21.15.79:0/386938781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f072a570 msgr2=0x5563f0727540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.183+0000 7f3b97e00700 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f072a570 0x5563f0727540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.183+0000 7f3b97e00700 1 -- 172.21.15.79:0/386938781 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f05c08d0 con 0x5563f0721960 2024-06-03T02:28:34.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.183+0000 7f3b97e00700 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f0721960 0x5563f0727000 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400f780 tx=0x7f3b84004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d09d3876c4279f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:34.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.183+0000 7f3b7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:28:34.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.184+0000 7f3b7ea00700 1 -- 172.21.15.79:0/386938781 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84022020 con 0x5563f0721960 2024-06-03T02:28:34.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.185+0000 7f3b7ea00700 1 -- 172.21.15.79:0/386938781 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3b8401e920 con 0x5563f0721960 2024-06-03T02:28:34.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.185+0000 7f3baa18db80 1 -- 172.21.15.79:0/386938781 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8c0050c0 con 0x5563f0721960 2024-06-03T02:28:34.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.185+0000 7f3b7ea00700 1 -- 172.21.15.79:0/386938781 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b8402d050 con 0x5563f0721960 2024-06-03T02:28:34.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.185+0000 7f3baa18db80 10 client.26305 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:34.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.185+0000 7f3baa18db80 1 -- 172.21.15.79:0/386938781 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5563f0737920 con 0x5563f0721960 2024-06-03T02:28:34.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.185+0000 7f3b7ea00700 1 -- 172.21.15.79:0/386938781 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b8c0050c0 con 0x5563f0721960 2024-06-03T02:28:34.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.187+0000 7f3b7ea00700 1 -- 172.21.15.79:0/386938781 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3b8401d3b0 con 0x5563f0721960 2024-06-03T02:28:34.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.187+0000 7f3baa18db80 10 client.26305 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:34.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.187+0000 7f3baa18db80 10 client.26305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:28:34.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.187+0000 7f3baa18db80 20 client.26305 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:34.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.187+0000 7f3baa18db80 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5563f073a000 0x5563f073c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.187+0000 7f3baa18db80 4 client.26305 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:28:34.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.187+0000 7f3baa18db80 1 -- 172.21.15.79:0/386938781 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563f067f000 con 0x5563f073a000 2024-06-03T02:28:34.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.188+0000 7f3b9ce00700 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5563f073a000 0x5563f073c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.189+0000 7f3b9ce00700 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5563f073a000 0x5563f073c470 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738eaa7268779b60 server_cookie=f806ad509ca21b51 in_seq=0 out_seq=0 2024-06-03T02:28:34.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.191+0000 7f3b7ea00700 10 client.26305 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:34.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.191+0000 7f3b7ea00700 1 -- 172.21.15.79:0/386938781 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5563f067f000 con 0x5563f073a000 2024-06-03T02:28:34.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.191+0000 7f3b7ea00700 10 client.26305 handle_command_reply: tid=0 2024-06-03T02:28:34.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.191+0000 7f3baa18db80 1 client.26305 shutdown 2024-06-03T02:28:34.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.192+0000 7f3baa18db80 1 -- 172.21.15.79:0/386938781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f0721960 msgr2=0x5563f0727000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:34.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.192+0000 7f3baa18db80 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f0721960 0x5563f0727000 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400f780 tx=0x7f3b84004750 comp rx=0 tx=0).stop 2024-06-03T02:28:34.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.192+0000 7f3baa18db80 1 -- 172.21.15.79:0/386938781 shutdown_connections 2024-06-03T02:28:34.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.192+0000 7f3baa18db80 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5563f073a000 0x5563f073c470 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.192+0000 7f3baa18db80 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f072a570 0x5563f0727540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.192+0000 7f3baa18db80 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f069be40 0x5563f0729d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.192+0000 7f3b7ea00700 0 client.26305 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:34.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.192+0000 7f3baa18db80 1 --2- 172.21.15.79:0/386938781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f0721960 0x5563f0727000 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.192+0000 7f3baa18db80 1 -- 172.21.15.79:0/386938781 >> 172.21.15.79:0/386938781 conn(0x5563f064c170 msgr2=0x5563f071b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:34.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.193+0000 7f3baa18db80 1 -- 172.21.15.79:0/386938781 shutdown_connections 2024-06-03T02:28:34.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.193+0000 7f3baa18db80 1 -- 172.21.15.79:0/386938781 wait complete. 2024-06-03T02:28:34.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.193+0000 7f3baa18db80 20 client.26305 trim_cache size 0 max 16384 2024-06-03T02:28:34.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.237+0000 7f3baa18db80 1 Processor -- start 2024-06-03T02:28:34.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.237+0000 7f3baa18db80 1 -- start start 2024-06-03T02:28:34.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.238+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f069be40 0x5563f071b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.238+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f0721960 0x5563f0725910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.238+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f072a570 0x5563f0725e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.238+0000 7f3baa18db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901619a0 con 0x5563f069be40 2024-06-03T02:28:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.238+0000 7f3baa18db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901634e0 con 0x5563f0721960 2024-06-03T02:28:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.238+0000 7f3baa18db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90162070 con 0x5563f072a570 2024-06-03T02:28:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.239+0000 7f3b97e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f0721960 0x5563f0725910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.239+0000 7f3b97e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f0721960 0x5563f0725910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33638/0 (socket says 172.21.15.79:33638) 2024-06-03T02:28:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.239+0000 7f3b97e00700 1 -- 172.21.15.79:0/1144187016 learned_addr learned my addr 172.21.15.79:0/1144187016 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:34.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.239+0000 7f3b9d800700 1 --2- 172.21.15.79:0/1144187016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f072a570 0x5563f0725e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.239+0000 7f3b97e00700 1 -- 172.21.15.79:0/1144187016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f072a570 msgr2=0x5563f0725e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.239+0000 7f3b97e00700 1 --2- 172.21.15.79:0/1144187016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f072a570 0x5563f0725e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.239+0000 7f3b97e00700 1 -- 172.21.15.79:0/1144187016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f069be40 msgr2=0x5563f071b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.239+0000 7f3b97e00700 1 --2- 172.21.15.79:0/1144187016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f069be40 0x5563f071b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.239+0000 7f3b97e00700 1 -- 172.21.15.79:0/1144187016 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f0721760 con 0x5563f0721960 2024-06-03T02:28:34.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.240+0000 7f3b97e00700 1 --2- 172.21.15.79:0/1144187016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f0721960 0x5563f0725910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3b84011040 tx=0x7f3b8401de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6faf209ee655db4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:34.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.240+0000 7f3b89600700 1 -- 172.21.15.79:0/1144187016 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84022040 con 0x5563f0721960 2024-06-03T02:28:34.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.241+0000 7f3b89600700 1 -- 172.21.15.79:0/1144187016 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b84016070 con 0x5563f0721960 2024-06-03T02:28:34.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.241+0000 7f3b89600700 1 -- 172.21.15.79:0/1144187016 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84031690 con 0x5563f0721960 2024-06-03T02:28:34.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.242+0000 7f3baa18db80 1 -- 172.21.15.79:0/1144187016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f0721960 msgr2=0x5563f0725910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:34.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.242+0000 7f3baa18db80 1 --2- 172.21.15.79:0/1144187016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f0721960 0x5563f0725910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3b84011040 tx=0x7f3b8401de10 comp rx=0 tx=0).stop 2024-06-03T02:28:34.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.242+0000 7f3baa18db80 1 -- 172.21.15.79:0/1144187016 shutdown_connections 2024-06-03T02:28:34.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.242+0000 7f3baa18db80 1 --2- 172.21.15.79:0/1144187016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f072a570 0x5563f0725e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.242+0000 7f3baa18db80 1 --2- 172.21.15.79:0/1144187016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f069be40 0x5563f071b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.242+0000 7f3baa18db80 1 --2- 172.21.15.79:0/1144187016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f0721960 0x5563f0725910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.242+0000 7f3baa18db80 1 -- 172.21.15.79:0/1144187016 >> 172.21.15.79:0/1144187016 conn(0x5563f064c170 msgr2=0x5563f07223d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:34.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.243+0000 7f3baa18db80 1 -- 172.21.15.79:0/1144187016 shutdown_connections 2024-06-03T02:28:34.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.244+0000 7f3baa18db80 1 -- 172.21.15.79:0/1144187016 wait complete. 2024-06-03T02:28:34.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.245+0000 7f3baa18db80 1 -- start start 2024-06-03T02:28:34.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.246+0000 7f3baa18db80 1 Processor -- start 2024-06-03T02:28:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.247+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0723eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.248+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f07243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.248+0000 7f3baa18db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f0724930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.248+0000 7f3baa18db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90162e10 con 0x5563f0721960 2024-06-03T02:28:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.248+0000 7f3baa18db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90162740 con 0x5563f072a570 2024-06-03T02:28:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.248+0000 7f3baa18db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9015f940 con 0x5563f069be40 2024-06-03T02:28:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.248+0000 7f3baa18db80 10 client.0 _ll_register_callbacks cb 0x5563f060b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:34.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.248+0000 7f3b9d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f0724930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.249+0000 7f3b9d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f0724930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33644/0 (socket says 172.21.15.79:33644) 2024-06-03T02:28:34.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.249+0000 7f3b9d800700 1 -- 172.21.15.79:0/2726275784 learned_addr learned my addr 172.21.15.79:0/2726275784 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:34.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.249+0000 7f3b9ce00700 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0723eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.249+0000 7f3b9d800700 1 -- 172.21.15.79:0/2726275784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 msgr2=0x5563f0723eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:34.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.249+0000 7f3b97e00700 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f07243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.249+0000 7f3b9d800700 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0723eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.249+0000 7f3b9d800700 1 -- 172.21.15.79:0/2726275784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 msgr2=0x5563f07243f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:34.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.249+0000 7f3b9d800700 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f07243f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.249+0000 7f3b9d800700 1 -- 172.21.15.79:0/2726275784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f07a3d10 con 0x5563f072a570 2024-06-03T02:28:34.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.250+0000 7f3b97e00700 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f07243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:34.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.251+0000 7f3b7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:28:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.251+0000 7f3b9d800700 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f0724930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3b980fb2f0 tx=0x7f3b980fe280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c88cc78c02e6b7e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.251+0000 7f3b7ea00700 1 -- 172.21.15.79:0/2726275784 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98116020 con 0x5563f072a570 2024-06-03T02:28:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.251+0000 7f3b7ea00700 1 -- 172.21.15.79:0/2726275784 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3b981320d0 con 0x5563f072a570 2024-06-03T02:28:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.251+0000 7f3baa18db80 1 -- 172.21.15.79:0/2726275784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b84016070 con 0x5563f072a570 2024-06-03T02:28:34.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.258+0000 7f3b7ea00700 1 -- 172.21.15.79:0/2726275784 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b98116020 con 0x5563f072a570 2024-06-03T02:28:34.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.258+0000 7f3baa18db80 10 client.26297 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:34.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.258+0000 7f3baa18db80 1 -- 172.21.15.79:0/2726275784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5563f07a4ec0 con 0x5563f072a570 2024-06-03T02:28:34.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.260+0000 7f3b7ea00700 1 -- 172.21.15.79:0/2726275784 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b98102cd0 con 0x5563f072a570 2024-06-03T02:28:34.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.260+0000 7f3b7ea00700 1 -- 172.21.15.79:0/2726275784 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3b98077910 con 0x5563f072a570 2024-06-03T02:28:34.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.260+0000 7f3baa18db80 10 client.26297 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:34.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.261+0000 7f3baa18db80 10 client.26297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:28:34.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.261+0000 7f3baa18db80 20 client.26297 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:34.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.261+0000 7f3baa18db80 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5563f07a5570 0x5563f07a7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:34.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.261+0000 7f3baa18db80 4 client.26297 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:28:34.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.261+0000 7f3baa18db80 1 -- 172.21.15.79:0/2726275784 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3b84016070 con 0x5563f07a5570 2024-06-03T02:28:34.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.261+0000 7f3b9ce00700 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5563f07a5570 0x5563f07a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:34.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.262+0000 7f3b7ea00700 10 client.26297 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:34.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.262+0000 7f3b9ce00700 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5563f07a5570 0x5563f07a7990 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e89ce53186c28ae server_cookie=de01a05a818a1ddf in_seq=0 out_seq=0 2024-06-03T02:28:34.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.262+0000 7f3b7ea00700 1 -- 172.21.15.79:0/2726275784 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f3b84016070 con 0x5563f07a5570 2024-06-03T02:28:34.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.262+0000 7f3b7ea00700 10 client.26297 handle_command_reply: tid=0 2024-06-03T02:28:34.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 client.26297 shutdown 2024-06-03T02:28:34.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 -- 172.21.15.79:0/2726275784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 msgr2=0x5563f0724930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:34.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f0724930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3b980fb2f0 tx=0x7f3b980fe280 comp rx=0 tx=0).stop 2024-06-03T02:28:34.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 -- 172.21.15.79:0/2726275784 shutdown_connections 2024-06-03T02:28:34.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563f072a570 0x5563f0724930 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563f069be40 0x5563f0723eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5563f07a5570 0x5563f07a7990 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 --2- 172.21.15.79:0/2726275784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563f0721960 0x5563f07243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 -- 172.21.15.79:0/2726275784 >> 172.21.15.79:0/2726275784 conn(0x5563f064c170 msgr2=0x5563f0725890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3b7ea00700 0 client.26297 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:28:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 -- 172.21.15.79:0/2726275784 shutdown_connections 2024-06-03T02:28:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.263+0000 7f3baa18db80 1 -- 172.21.15.79:0/2726275784 wait complete. 2024-06-03T02:28:34.265 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:28:34.265 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:28:34.265 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:28:34.265 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:28:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.264+0000 7f3baa18db80 20 client.26297 trim_cache size 0 max 16384 2024-06-03T02:28:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 -- 172.21.15.79:0/1495194265 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3b800a2c90 msgr2=0x7f3b800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:34.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3b800a2c90 0x7f3b800a5150 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005fd0 tx=0x7f3b84005d00 comp rx=0 tx=0).stop 2024-06-03T02:28:34.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 -- 172.21.15.79:0/1495194265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b900aa2c0 msgr2=0x7f3b900b5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:34.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b900aa2c0 0x7f3b900b5720 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3b980517a0 tx=0x7f3b9806cc70 comp rx=0 tx=0).stop 2024-06-03T02:28:34.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 -- 172.21.15.79:0/1495194265 shutdown_connections 2024-06-03T02:28:34.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b900aa2c0 0x7f3b900b5720 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b900a51e0 0x7f3b900bb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3b800a2c90 0x7f3b800a5150 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 --2- 172.21.15.79:0/1495194265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b900a5b90 0x7f3b900b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:34.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.267+0000 7f3b9e200700 1 -- 172.21.15.79:0/1495194265 >> 172.21.15.79:0/1495194265 conn(0x7f3b9001aac0 msgr2=0x7f3b900a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:34.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.268+0000 7f3b9e200700 1 -- 172.21.15.79:0/1495194265 shutdown_connections 2024-06-03T02:28:34.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:34.268+0000 7f3b9e200700 1 -- 172.21.15.79:0/1495194265 wait complete. 2024-06-03T02:28:34.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:34 smithi079 ceph-mon[31819]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 538 op/s 2024-06-03T02:28:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:34 smithi135 ceph-mon[36653]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 538 op/s 2024-06-03T02:28:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:34 smithi123 ceph-mon[36640]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 538 op/s 2024-06-03T02:28:35.885 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:28:35.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:28:36.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:36 smithi079 ceph-mon[31819]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 489 op/s 2024-06-03T02:28:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:36 smithi135 ceph-mon[36653]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 489 op/s 2024-06-03T02:28:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:36 smithi123 ceph-mon[36640]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 489 op/s 2024-06-03T02:28:36.804 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:28:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 510.0 s, 492.4 tps, lat 201.065 ms stddev 224.923 2024-06-03T02:28:38.429 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:38 smithi079 ceph-mon[31819]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-06-03T02:28:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:38 smithi135 ceph-mon[36653]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-06-03T02:28:38.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:38 smithi123 ceph-mon[36640]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-06-03T02:28:40.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:40 smithi079 ceph-mon[31819]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 566 op/s 2024-06-03T02:28:40.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:40 smithi135 ceph-mon[36653]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 566 op/s 2024-06-03T02:28:40.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:40 smithi123 ceph-mon[36640]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 566 op/s 2024-06-03T02:28:40.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.710+0000 7f0c5f200700 1 -- 172.21.15.79:0/2815841244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58102540 msgr2=0x7f0c581029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:40.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.710+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2815841244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58102540 0x7f0c581029a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0c480098f0 tx=0x7f0c48009c00 comp rx=0 tx=0).stop 2024-06-03T02:28:40.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.710+0000 7f0c5f200700 1 -- 172.21.15.79:0/2815841244 shutdown_connections 2024-06-03T02:28:40.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.710+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2815841244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c58102ee0 0x7f0c5810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:40.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.710+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2815841244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c58107450 0x7f0c58107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:40.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.710+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2815841244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58102540 0x7f0c581029a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.710+0000 7f0c5f200700 1 -- 172.21.15.79:0/2815841244 >> 172.21.15.79:0/2815841244 conn(0x7f0c580795f0 msgr2=0x7f0c58079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.711+0000 7f0c5f200700 1 -- 172.21.15.79:0/2815841244 shutdown_connections 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.711+0000 7f0c5f200700 1 -- 172.21.15.79:0/2815841244 wait complete. 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.712+0000 7f0c5f200700 1 Processor -- start 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.712+0000 7f0c5f200700 1 -- start start 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.713+0000 7f0c5f200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c58102540 0x7f0c5819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.713+0000 7f0c5f200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c58102ee0 0x7f0c5819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.713+0000 7f0c5f200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58107450 0x7f0c5819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.713+0000 7f0c5f200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581127d0 con 0x7f0c58107450 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.713+0000 7f0c5f200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c58112650 con 0x7f0c58102ee0 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.713+0000 7f0c5f200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c58112950 con 0x7f0c58102540 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.715+0000 7f0c5d200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58107450 0x7f0c5819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:40.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.715+0000 7f0c5d200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58107450 0x7f0c5819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53320/0 (socket says 172.21.15.79:53320) 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.715+0000 7f0c57e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c58102540 0x7f0c5819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.715+0000 7f0c5d200700 1 -- 172.21.15.79:0/2337757886 learned_addr learned my addr 172.21.15.79:0/2337757886 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.715+0000 7f0c57400700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c58102ee0 0x7f0c5819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.716+0000 7f0c57e00700 1 -- 172.21.15.79:0/2337757886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c58102ee0 msgr2=0x7f0c5819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.716+0000 7f0c57e00700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c58102ee0 0x7f0c5819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.716+0000 7f0c57e00700 1 -- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58107450 msgr2=0x7f0c5819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.716+0000 7f0c57e00700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58107450 0x7f0c5819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.716+0000 7f0c57e00700 1 -- 172.21.15.79:0/2337757886 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5819fe40 con 0x7f0c58102540 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.716+0000 7f0c5d200700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58107450 0x7f0c5819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.717+0000 7f0c57e00700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c58102540 0x7f0c5819ae30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400d740 tx=0x7f0c4400da50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.719+0000 7f0c54c00700 1 -- 172.21.15.79:0/2337757886 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44017020 con 0x7f0c58102540 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.719+0000 7f0c54c00700 1 -- 172.21.15.79:0/2337757886 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c44004440 con 0x7f0c58102540 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.719+0000 7f0c54c00700 1 -- 172.21.15.79:0/2337757886 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400fc30 con 0x7f0c58102540 2024-06-03T02:28:41.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.719+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c581a0130 con 0x7f0c58102540 2024-06-03T02:28:41.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.719+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c581a79b0 con 0x7f0c58102540 2024-06-03T02:28:41.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.720+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c58069180 con 0x7f0c58102540 2024-06-03T02:28:41.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.729+0000 7f0c54c00700 1 -- 172.21.15.79:0/2337757886 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0c4401a020 con 0x7f0c58102540 2024-06-03T02:28:41.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.731+0000 7f0c54c00700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c3c0a2d70 0x7f0c3c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:41.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.732+0000 7f0c57400700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c3c0a2d70 0x7f0c3c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:41.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.734+0000 7f0c54c00700 1 -- 172.21.15.79:0/2337757886 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0c44003df0 con 0x7f0c58102540 2024-06-03T02:28:41.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.734+0000 7f0c57400700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c3c0a2d70 0x7f0c3c0a5230 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800b2e0 tx=0x7f0c4801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:41.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:40.734+0000 7f0c54c00700 1 -- 172.21.15.79:0/2337757886 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0c44021030 con 0x7f0c58102540 2024-06-03T02:28:41.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.172+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c5819c190 con 0x7f0c58102540 2024-06-03T02:28:41.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.173+0000 7f0c54c00700 1 -- 172.21.15.79:0/2337757886 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f0c4409baf0 con 0x7f0c58102540 2024-06-03T02:28:41.176 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:28:41.176 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:28:41.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c3c0a2d70 msgr2=0x7f0c3c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:41.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c3c0a2d70 0x7f0c3c0a5230 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800b2e0 tx=0x7f0c4801a040 comp rx=0 tx=0).stop 2024-06-03T02:28:41.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c58102540 msgr2=0x7f0c5819ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:41.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c58102540 0x7f0c5819ae30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400d740 tx=0x7f0c4400da50 comp rx=0 tx=0).stop 2024-06-03T02:28:41.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 shutdown_connections 2024-06-03T02:28:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0c3c0a2d70 0x7f0c3c0a5230 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c58107450 0x7f0c5819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c58102540 0x7f0c5819ae30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 --2- 172.21.15.79:0/2337757886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c58102ee0 0x7f0c5819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.177+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 >> 172.21.15.79:0/2337757886 conn(0x7f0c580795f0 msgr2=0x7f0c581003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:41.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.180+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 shutdown_connections 2024-06-03T02:28:41.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:41.180+0000 7f0c5f200700 1 -- 172.21.15.79:0/2337757886 wait complete. 2024-06-03T02:28:41.186 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:28:41.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:41 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2337757886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:28:41.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:41 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2337757886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:28:41.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:41 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2337757886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:28:42.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:42 smithi079 ceph-mon[31819]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2024-06-03T02:28:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:42 smithi135 ceph-mon[36653]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2024-06-03T02:28:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:42 smithi123 ceph-mon[36640]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2024-06-03T02:28:43.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 515.0 s, 506.7 tps, lat 201.515 ms stddev 239.706 2024-06-03T02:28:43.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:28:44.203 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:28:44.392 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:44 smithi079 ceph-mon[31819]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 518 op/s 2024-06-03T02:28:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:44 smithi135 ceph-mon[36653]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 518 op/s 2024-06-03T02:28:44.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:44 smithi123 ceph-mon[36640]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 518 op/s 2024-06-03T02:28:46.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:46 smithi079 ceph-mon[31819]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-06-03T02:28:46.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:46 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:28:46.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:46 smithi135 ceph-mon[36653]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-06-03T02:28:46.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:46 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:28:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:46 smithi123 ceph-mon[36640]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-06-03T02:28:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:46 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:28:48.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 520.0 s, 468.8 tps, lat 215.048 ms stddev 236.226 2024-06-03T02:28:48.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:48 smithi079 ceph-mon[31819]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-06-03T02:28:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:48 smithi135 ceph-mon[36653]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-06-03T02:28:48.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.628+0000 7f2e68e00700 1 -- 172.21.15.79:0/179681274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e64109db0 msgr2=0x7f2e641150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:48.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.628+0000 7f2e68e00700 1 --2- 172.21.15.79:0/179681274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e64109db0 0x7f2e641150a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009990 tx=0x7f2e4c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.629+0000 7f2e68e00700 1 -- 172.21.15.79:0/179681274 shutdown_connections 2024-06-03T02:28:48.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.629+0000 7f2e68e00700 1 --2- 172.21.15.79:0/179681274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e641155e0 0x7f2e641179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.629+0000 7f2e68e00700 1 --2- 172.21.15.79:0/179681274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e64109490 0x7f2e64109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.629+0000 7f2e68e00700 1 --2- 172.21.15.79:0/179681274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e64109db0 0x7f2e641150a0 secure :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009990 tx=0x7f2e4c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.629+0000 7f2e68e00700 1 -- 172.21.15.79:0/179681274 >> 172.21.15.79:0/179681274 conn(0x7f2e6406d600 msgr2=0x7f2e6406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:48.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.632+0000 7f2e68e00700 1 -- 172.21.15.79:0/179681274 shutdown_connections 2024-06-03T02:28:48.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.632+0000 7f2e68e00700 1 -- 172.21.15.79:0/179681274 wait complete. 2024-06-03T02:28:48.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.633+0000 7f2e68e00700 1 Processor -- start 2024-06-03T02:28:48.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.634+0000 7f2e68e00700 1 -- start start 2024-06-03T02:28:48.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.635+0000 7f2e68e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e64109490 0x7f2e64114c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.637+0000 7f2e68e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e641155e0 0x7f2e6410bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.637+0000 7f2e68e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e6410c100 0x7f2e6410c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.637+0000 7f2e68e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6411b0a0 con 0x7f2e64109490 2024-06-03T02:28:48.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.637+0000 7f2e68e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6411af20 con 0x7f2e6410c100 2024-06-03T02:28:48.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.637+0000 7f2e68e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6411b220 con 0x7f2e641155e0 2024-06-03T02:28:48.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.637+0000 7f2e62800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e6410c100 0x7f2e6410c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.637+0000 7f2e62800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e6410c100 0x7f2e6410c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60474/0 (socket says 172.21.15.79:60474) 2024-06-03T02:28:48.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.638+0000 7f2e62800700 1 -- 172.21.15.79:0/1035453690 learned_addr learned my addr 172.21.15.79:0/1035453690 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:48.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.638+0000 7f2e61e00700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e64109490 0x7f2e64114c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.638+0000 7f2e62800700 1 -- 172.21.15.79:0/1035453690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e641155e0 msgr2=0x7f2e6410bbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:28:48.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.638+0000 7f2e62800700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e641155e0 0x7f2e6410bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.638+0000 7f2e62800700 1 -- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e64109490 msgr2=0x7f2e64114c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:48.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.638+0000 7f2e62800700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e64109490 0x7f2e64114c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.638+0000 7f2e62800700 1 -- 172.21.15.79:0/1035453690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e64110650 con 0x7f2e6410c100 2024-06-03T02:28:48.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.638+0000 7f2e62800700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e6410c100 0x7f2e6410c5b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007ee0 tx=0x7f2e5400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:48.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.639+0000 7f2e5aa00700 1 -- 172.21.15.79:0/1035453690 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400dbd0 con 0x7f2e6410c100 2024-06-03T02:28:48.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.639+0000 7f2e68e00700 1 -- 172.21.15.79:0/1035453690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e64110940 con 0x7f2e6410c100 2024-06-03T02:28:48.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.639+0000 7f2e5aa00700 1 -- 172.21.15.79:0/1035453690 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e5400dd70 con 0x7f2e6410c100 2024-06-03T02:28:48.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.639+0000 7f2e68e00700 1 -- 172.21.15.79:0/1035453690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e64110e80 con 0x7f2e6410c100 2024-06-03T02:28:48.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.639+0000 7f2e5aa00700 1 -- 172.21.15.79:0/1035453690 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540133f0 con 0x7f2e6410c100 2024-06-03T02:28:48.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.644+0000 7f2e5aa00700 1 -- 172.21.15.79:0/1035453690 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2e5401a020 con 0x7f2e6410c100 2024-06-03T02:28:48.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.650+0000 7f2e736a2b80 1 Processor -- start 2024-06-03T02:28:48.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.650+0000 7f2e5aa00700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e480a2cf0 0x7f2e480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.651+0000 7f2e5aa00700 1 -- 172.21.15.79:0/1035453690 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2e540cfc30 con 0x7f2e6410c100 2024-06-03T02:28:48.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.651+0000 7f2e736a2b80 1 -- start start 2024-06-03T02:28:48.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.651+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ab90 0x556a5de6af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.651+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de6b4b0 0x556a5de64d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.651+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de65240 0x556a5de65680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.651+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556a5dd73a10 con 0x556a5de65240 2024-06-03T02:28:48.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.651+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a5ddb6b00 con 0x556a5de6ab90 2024-06-03T02:28:48.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.651+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556a5dd718f0 con 0x556a5de6b4b0 2024-06-03T02:28:48.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.652+0000 7f2e61e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ab90 0x556a5de6af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.652+0000 7f2e61e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ab90 0x556a5de6af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:60476/0 (socket says 172.21.15.79:60476) 2024-06-03T02:28:48.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.652+0000 7f2e61e00700 1 -- 172.21.15.79:0/1752287754 learned_addr learned my addr 172.21.15.79:0/1752287754 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:48.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.652+0000 7f2e62800700 1 --2- 172.21.15.79:0/1752287754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de65240 0x556a5de65680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.653+0000 7f2e61400700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e480a2cf0 0x7f2e480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.653+0000 7f2e61e00700 1 -- 172.21.15.79:0/1752287754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de6b4b0 msgr2=0x556a5de64d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:28:48.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.653+0000 7f2e61e00700 1 --2- 172.21.15.79:0/1752287754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de6b4b0 0x556a5de64d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.653+0000 7f2e61e00700 1 -- 172.21.15.79:0/1752287754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de65240 msgr2=0x556a5de65680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:48.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.653+0000 7f2e61e00700 1 --2- 172.21.15.79:0/1752287754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de65240 0x556a5de65680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.653+0000 7f2e61e00700 1 -- 172.21.15.79:0/1752287754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a5dd5ef90 con 0x556a5de6ab90 2024-06-03T02:28:48.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.656+0000 7f2e61400700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e480a2cf0 0x7f2e480a51b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e6410d380 tx=0x7f2e4c009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:48.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.657+0000 7f2e61e00700 1 --2- 172.21.15.79:0/1752287754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ab90 0x556a5de6af70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009c40 tx=0x7f2e5c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f01c715df7b029 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:48.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.658+0000 7f2e53400700 1 -- 172.21.15.79:0/1752287754 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c016070 con 0x556a5de6ab90 2024-06-03T02:28:48.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.659+0000 7f2e53400700 1 -- 172.21.15.79:0/1752287754 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e5c004830 con 0x556a5de6ab90 2024-06-03T02:28:48.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.659+0000 7f2e53400700 1 -- 172.21.15.79:0/1752287754 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c005610 con 0x556a5de6ab90 2024-06-03T02:28:48.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.661+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1752287754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ab90 msgr2=0x556a5de6af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:48.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.661+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1752287754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ab90 0x556a5de6af70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009c40 tx=0x7f2e5c009f50 comp rx=0 tx=0).stop 2024-06-03T02:28:48.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.661+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1752287754 shutdown_connections 2024-06-03T02:28:48.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.661+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1752287754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de65240 0x556a5de65680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.661+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1752287754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ab90 0x556a5de6af70 secure :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009c40 tx=0x7f2e5c009f50 comp rx=0 tx=0).stop 2024-06-03T02:28:48.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.661+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1752287754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de6b4b0 0x556a5de64d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.661+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1752287754 >> 172.21.15.79:0/1752287754 conn(0x556a5dd91170 msgr2=0x556a5de5fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:48.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.662+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1752287754 shutdown_connections 2024-06-03T02:28:48.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.662+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1752287754 wait complete. 2024-06-03T02:28:48.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.663+0000 7f2e736a2b80 1 -- start start 2024-06-03T02:28:48.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.663+0000 7f2e736a2b80 1 Processor -- start 2024-06-03T02:28:48.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.666+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de71450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.667+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de6e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.667+0000 7f2e61e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de71450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.667+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5de6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.667+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556a5dd52f60 con 0x556a5de6b4b0 2024-06-03T02:28:48.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.667+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a5ddb2ac0 con 0x556a5de6ece0 2024-06-03T02:28:48.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.667+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556a5de71e90 con 0x556a5de65240 2024-06-03T02:28:48.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.667+0000 7f2e61e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de71450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56080/0 (socket says 172.21.15.79:56080) 2024-06-03T02:28:48.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.667+0000 7f2e61e00700 1 -- 172.21.15.79:0/1444752175 learned_addr learned my addr 172.21.15.79:0/1444752175 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:48.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.668+0000 7f2e736a2b80 10 client.0 _ll_register_callbacks cb 0x556a5dd4c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:48.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.668+0000 7f2e62800700 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5de6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.668+0000 7f2e61e00700 1 -- 172.21.15.79:0/1444752175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 msgr2=0x556a5de6f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:48.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.668+0000 7f2e61e00700 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5de6f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.668+0000 7f2e61e00700 1 -- 172.21.15.79:0/1444752175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 msgr2=0x556a5de6e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:48.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.668+0000 7f2e61400700 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de6e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.669+0000 7f2e61e00700 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de6e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.669+0000 7f2e61e00700 1 -- 172.21.15.79:0/1444752175 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a5dd058d0 con 0x556a5de65240 2024-06-03T02:28:48.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.669+0000 7f2e62800700 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5de6f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:28:48.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.669+0000 7f2e61400700 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de6e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:48.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.670+0000 7f2e61e00700 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de71450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c000c00 tx=0x7f2e5c005570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c36ec6b42ddc55b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:48.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.670+0000 7f2e47e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:28:48.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.670+0000 7f2e47e00700 1 -- 172.21.15.79:0/1444752175 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c028060 con 0x556a5de65240 2024-06-03T02:28:48.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.670+0000 7f2e47e00700 1 -- 172.21.15.79:0/1444752175 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2e5c00eb10 con 0x556a5de65240 2024-06-03T02:28:48.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.670+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1444752175 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e5c004830 con 0x556a5de65240 2024-06-03T02:28:48.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.671+0000 7f2e47e00700 1 -- 172.21.15.79:0/1444752175 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e5c016040 con 0x556a5de65240 2024-06-03T02:28:48.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.672+0000 7f2e736a2b80 10 client.26320 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:48.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.672+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1444752175 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a5de7cc80 con 0x556a5de65240 2024-06-03T02:28:48.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.672+0000 7f2e47e00700 1 -- 172.21.15.79:0/1444752175 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e5c004830 con 0x556a5de65240 2024-06-03T02:28:48.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.673+0000 7f2e47e00700 1 -- 172.21.15.79:0/1444752175 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2e5c00f040 con 0x556a5de65240 2024-06-03T02:28:48.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.677+0000 7f2e736a2b80 10 client.26320 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:48.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.677+0000 7f2e736a2b80 10 client.26320 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:28:48.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.677+0000 7f2e736a2b80 20 client.26320 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:48.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.677+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x556a5de7f150 0x556a5de81540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.677+0000 7f2e736a2b80 4 client.26320 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:28:48.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.677+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1444752175 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a5ddc4000 con 0x556a5de7f150 2024-06-03T02:28:48.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.677+0000 7f2e61400700 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x556a5de7f150 0x556a5de81540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.678+0000 7f2e61400700 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x556a5de7f150 0x556a5de81540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4bb40faa0611d964 server_cookie=78c6fd4f3d26ab9d in_seq=0 out_seq=0 2024-06-03T02:28:48.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.678+0000 7f2e47e00700 10 client.26320 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:28:48.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.680+0000 7f2e47e00700 1 -- 172.21.15.79:0/1444752175 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556a5ddc4000 con 0x556a5de7f150 2024-06-03T02:28:48.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.680+0000 7f2e47e00700 10 client.26320 handle_command_reply: tid=0 2024-06-03T02:28:48.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.680+0000 7f2e736a2b80 1 client.26320 shutdown 2024-06-03T02:28:48.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.681+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1444752175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 msgr2=0x556a5de71450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:48.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.681+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de71450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c000c00 tx=0x7f2e5c005570 comp rx=0 tx=0).stop 2024-06-03T02:28:48.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.681+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1444752175 shutdown_connections 2024-06-03T02:28:48.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.681+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x556a5de7f150 0x556a5de81540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.682+0000 7f2e47e00700 0 client.26320 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:28:48.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.682+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5de6f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.682+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de71450 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.682+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/1444752175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de6e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.682+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1444752175 >> 172.21.15.79:0/1444752175 conn(0x556a5dd91170 msgr2=0x556a5de59790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:48.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.682+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1444752175 shutdown_connections 2024-06-03T02:28:48.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.682+0000 7f2e736a2b80 1 -- 172.21.15.79:0/1444752175 wait complete. 2024-06-03T02:28:48.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.682+0000 7f2e736a2b80 20 client.26320 trim_cache size 0 max 16384 2024-06-03T02:28:48.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.715+0000 7f2e736a2b80 1 Processor -- start 2024-06-03T02:28:48.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.715+0000 7f2e736a2b80 1 -- start start 2024-06-03T02:28:48.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de5a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de6bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5de70f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4000eb40 con 0x556a5de6b4b0 2024-06-03T02:28:48.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40010680 con 0x556a5de6ece0 2024-06-03T02:28:48.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4000f210 con 0x556a5de65240 2024-06-03T02:28:48.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e61e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de5a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e61e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de5a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56094/0 (socket says 172.21.15.79:56094) 2024-06-03T02:28:48.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e61e00700 1 -- 172.21.15.79:0/368592781 learned_addr learned my addr 172.21.15.79:0/368592781 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:48.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.716+0000 7f2e62800700 1 --2- 172.21.15.79:0/368592781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5de70f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.717+0000 7f2e61e00700 1 -- 172.21.15.79:0/368592781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 msgr2=0x556a5de70f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:48.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.717+0000 7f2e61e00700 1 --2- 172.21.15.79:0/368592781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5de70f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.717+0000 7f2e61e00700 1 -- 172.21.15.79:0/368592781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 msgr2=0x556a5de6bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:48.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.717+0000 7f2e61400700 1 --2- 172.21.15.79:0/368592781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de6bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.717+0000 7f2e61e00700 1 --2- 172.21.15.79:0/368592781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de6bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.717+0000 7f2e61e00700 1 -- 172.21.15.79:0/368592781 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a5de6b1c0 con 0x556a5de65240 2024-06-03T02:28:48.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.717+0000 7f2e61e00700 1 --2- 172.21.15.79:0/368592781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de5a130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c004f30 tx=0x7f2e5c011bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10d9dfe05cc5e68 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:48.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.718+0000 7f2e52a00700 1 -- 172.21.15.79:0/368592781 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c028070 con 0x556a5de65240 2024-06-03T02:28:48.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.718+0000 7f2e52a00700 1 -- 172.21.15.79:0/368592781 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e5c00e5d0 con 0x556a5de65240 2024-06-03T02:28:48.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.718+0000 7f2e52a00700 1 -- 172.21.15.79:0/368592781 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c032080 con 0x556a5de65240 2024-06-03T02:28:48.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.719+0000 7f2e736a2b80 1 -- 172.21.15.79:0/368592781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 msgr2=0x556a5de5a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:48.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.719+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/368592781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de5a130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c004f30 tx=0x7f2e5c011bc0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.719+0000 7f2e736a2b80 1 -- 172.21.15.79:0/368592781 shutdown_connections 2024-06-03T02:28:48.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.719+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/368592781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5de70f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.719+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/368592781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de5a130 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.719+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/368592781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de6bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.719+0000 7f2e736a2b80 1 -- 172.21.15.79:0/368592781 >> 172.21.15.79:0/368592781 conn(0x556a5dd91170 msgr2=0x556a5de6b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:48.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.719+0000 7f2e736a2b80 1 -- 172.21.15.79:0/368592781 shutdown_connections 2024-06-03T02:28:48.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.720+0000 7f2e736a2b80 1 -- 172.21.15.79:0/368592781 wait complete. 2024-06-03T02:28:48.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.720+0000 7f2e736a2b80 1 -- start start 2024-06-03T02:28:48.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.720+0000 7f2e736a2b80 1 Processor -- start 2024-06-03T02:28:48.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de81720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de81c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e736a2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5dee7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4000ffb0 con 0x556a5de6b4b0 2024-06-03T02:28:48.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e736a2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4000f8e0 con 0x556a5de6ece0 2024-06-03T02:28:48.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e61e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de81720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e61e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de81720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56104/0 (socket says 172.21.15.79:56104) 2024-06-03T02:28:48.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e61e00700 1 -- 172.21.15.79:0/527895342 learned_addr learned my addr 172.21.15.79:0/527895342 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:48.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e736a2b80 1 -- 172.21.15.79:0/527895342 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4000cae0 con 0x556a5de65240 2024-06-03T02:28:48.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e62800700 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5dee7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:48 smithi123 ceph-mon[36640]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-06-03T02:28:48.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.721+0000 7f2e736a2b80 10 client.0 _ll_register_callbacks cb 0x556a5dd4c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:48.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e62800700 1 -- 172.21.15.79:0/527895342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 msgr2=0x556a5de81720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:48.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e62800700 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de81720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e62800700 1 -- 172.21.15.79:0/527895342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 msgr2=0x556a5de81c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:48.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e62800700 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de81c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e62800700 1 -- 172.21.15.79:0/527895342 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a5dee7fc0 con 0x556a5de6ece0 2024-06-03T02:28:48.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e62800700 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5dee7a80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2e5409bd80 tx=0x7f2e540094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a87b86a3ead774 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:48.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e47e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:28:48.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e61e00700 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de81720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:28:48.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e47e00700 1 -- 172.21.15.79:0/527895342 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540a4cf0 con 0x556a5de6ece0 2024-06-03T02:28:48.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.722+0000 7f2e47e00700 1 -- 172.21.15.79:0/527895342 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2e540cd430 con 0x556a5de6ece0 2024-06-03T02:28:48.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.723+0000 7f2e736a2b80 1 -- 172.21.15.79:0/527895342 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e5c00e5d0 con 0x556a5de6ece0 2024-06-03T02:28:48.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.723+0000 7f2e47e00700 1 -- 172.21.15.79:0/527895342 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e540a00d0 con 0x556a5de6ece0 2024-06-03T02:28:48.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.723+0000 7f2e736a2b80 10 client.26321 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:48.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.723+0000 7f2e736a2b80 1 -- 172.21.15.79:0/527895342 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a5dee9180 con 0x556a5de6ece0 2024-06-03T02:28:48.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.724+0000 7f2e47e00700 1 -- 172.21.15.79:0/527895342 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e5c00e5d0 con 0x556a5de6ece0 2024-06-03T02:28:48.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.724+0000 7f2e47e00700 1 -- 172.21.15.79:0/527895342 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2e5409b4d0 con 0x556a5de6ece0 2024-06-03T02:28:48.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.724+0000 7f2e736a2b80 10 client.26321 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:48.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.724+0000 7f2e736a2b80 10 client.26321 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:28:48.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.724+0000 7f2e736a2b80 20 client.26321 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:48.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.724+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x556a5de7f150 0x556a5deeb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:48.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.724+0000 7f2e736a2b80 4 client.26321 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:28:48.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.724+0000 7f2e736a2b80 1 -- 172.21.15.79:0/527895342 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a5ddbe580 con 0x556a5de7f150 2024-06-03T02:28:48.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.725+0000 7f2e61e00700 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x556a5de7f150 0x556a5deeb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:48.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.725+0000 7f2e61e00700 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x556a5de7f150 0x556a5deeb8b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9942f2d68fa05e41 server_cookie=192ce18b3e63f04d in_seq=0 out_seq=0 2024-06-03T02:28:48.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.725+0000 7f2e47e00700 10 client.26321 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:28:48.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.726+0000 7f2e47e00700 1 -- 172.21.15.79:0/527895342 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a5ddbe580 con 0x556a5de7f150 2024-06-03T02:28:48.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.726+0000 7f2e47e00700 10 client.26321 handle_command_reply: tid=0 2024-06-03T02:28:48.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.726+0000 7f2e736a2b80 1 client.26321 shutdown 2024-06-03T02:28:48.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.726+0000 7f2e736a2b80 1 -- 172.21.15.79:0/527895342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 msgr2=0x556a5dee7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:48.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.726+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5dee7a80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2e5409bd80 tx=0x7f2e540094a0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.727+0000 7f2e736a2b80 1 -- 172.21.15.79:0/527895342 shutdown_connections 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.727+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x556a5de7f150 0x556a5deeb8b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.727+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a5de6ece0 0x556a5dee7a80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.727+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5de65240 0x556a5de81720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.727+0000 7f2e736a2b80 1 --2- 172.21.15.79:0/527895342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556a5de6b4b0 0x556a5de81c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.727+0000 7f2e47e00700 0 client.26321 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.727+0000 7f2e736a2b80 1 -- 172.21.15.79:0/527895342 >> 172.21.15.79:0/527895342 conn(0x556a5dd91170 msgr2=0x556a5de6c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.727+0000 7f2e736a2b80 1 -- 172.21.15.79:0/527895342 shutdown_connections 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.727+0000 7f2e736a2b80 1 -- 172.21.15.79:0/527895342 wait complete. 2024-06-03T02:28:48.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.728+0000 7f2e736a2b80 20 client.26321 trim_cache size 0 max 16384 2024-06-03T02:28:48.729 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:28:48.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 -- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e480a2cf0 msgr2=0x7f2e480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:48.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e480a2cf0 0x7f2e480a51b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e6410d380 tx=0x7f2e4c009ee0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 -- 172.21.15.79:0/1035453690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e6410c100 msgr2=0x7f2e6410c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:48.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e6410c100 0x7f2e6410c5b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007ee0 tx=0x7f2e5400d3b0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 -- 172.21.15.79:0/1035453690 shutdown_connections 2024-06-03T02:28:48.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e6410c100 0x7f2e6410c5b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e480a2cf0 0x7f2e480a51b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e64109490 0x7f2e64114c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 --2- 172.21.15.79:0/1035453690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e641155e0 0x7f2e6410bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:48.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.729+0000 7f2e68e00700 1 -- 172.21.15.79:0/1035453690 >> 172.21.15.79:0/1035453690 conn(0x7f2e6406d600 msgr2=0x7f2e64071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:48.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.731+0000 7f2e68e00700 1 -- 172.21.15.79:0/1035453690 shutdown_connections 2024-06-03T02:28:48.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:48.731+0000 7f2e68e00700 1 -- 172.21.15.79:0/1035453690 wait complete. 2024-06-03T02:28:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:49 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:28:49.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:49 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:28:49.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:49 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:28:49.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:49 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:28:49.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:49 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:28:49.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:49 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:28:49.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:49 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:28:49.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:49 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:28:49.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:49 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:28:50.714 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:50 smithi079 ceph-mon[31819]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 534 op/s 2024-06-03T02:28:50.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:50 smithi123 ceph-mon[36640]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 534 op/s 2024-06-03T02:28:50.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:28:50.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:50 smithi135 ceph-mon[36653]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 534 op/s 2024-06-03T02:28:51.467 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:28:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:52 smithi123 ceph-mon[36640]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.5 MiB/s wr, 403 op/s 2024-06-03T02:28:52.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:52 smithi079 ceph-mon[31819]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.5 MiB/s wr, 403 op/s 2024-06-03T02:28:52.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:52 smithi135 ceph-mon[36653]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.5 MiB/s wr, 403 op/s 2024-06-03T02:28:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 525.0 s, 460.9 tps, lat 213.158 ms stddev 245.781 2024-06-03T02:28:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:54 smithi123 ceph-mon[36640]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 480 op/s 2024-06-03T02:28:54.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:54 smithi079 ceph-mon[31819]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 480 op/s 2024-06-03T02:28:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:54 smithi135 ceph-mon[36653]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 480 op/s 2024-06-03T02:28:55.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.984+0000 7f26fde00700 1 -- 172.21.15.79:0/1388858528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f0094750 msgr2=0x7f26f0094bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:55.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.984+0000 7f26fde00700 1 --2- 172.21.15.79:0/1388858528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f0094750 0x7f26f0094bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009990 tx=0x7f26ec009ca0 comp rx=0 tx=0).stop 2024-06-03T02:28:55.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.984+0000 7f26fde00700 1 -- 172.21.15.79:0/1388858528 shutdown_connections 2024-06-03T02:28:55.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.984+0000 7f26fde00700 1 --2- 172.21.15.79:0/1388858528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26f00a6a00 0x7f26f00a8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:55.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.984+0000 7f26fde00700 1 --2- 172.21.15.79:0/1388858528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f0097ca0 0x7f26f0094210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:55.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.984+0000 7f26fde00700 1 --2- 172.21.15.79:0/1388858528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f0094750 0x7f26f0094bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:55.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.984+0000 7f26fde00700 1 -- 172.21.15.79:0/1388858528 >> 172.21.15.79:0/1388858528 conn(0x7f26f0009ad0 msgr2=0x7f26f0009ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:55.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.984+0000 7f26fde00700 1 -- 172.21.15.79:0/1388858528 shutdown_connections 2024-06-03T02:28:55.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.984+0000 7f26fde00700 1 -- 172.21.15.79:0/1388858528 wait complete. 2024-06-03T02:28:55.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.986+0000 7f26fde00700 1 Processor -- start 2024-06-03T02:28:55.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.986+0000 7f26fde00700 1 -- start start 2024-06-03T02:28:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.987+0000 7f26fde00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26f0094750 0x7f26f0149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.987+0000 7f26fde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f0097ca0 0x7f26f0149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.988+0000 7f26fca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26f0094750 0x7f26f0149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.988+0000 7f26fca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26f0094750 0x7f26f0149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:37618/0 (socket says 172.21.15.79:37618) 2024-06-03T02:28:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.988+0000 7f26fca00700 1 -- 172.21.15.79:0/1668926059 learned_addr learned my addr 172.21.15.79:0/1668926059 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:55.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.988+0000 7f26fde00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f00a6a00 0x7f26f0143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.988+0000 7f26f7e00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f0097ca0 0x7f26f0149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.989+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f26f00ac730 con 0x7f26f0097ca0 2024-06-03T02:28:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.989+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26f00ac5b0 con 0x7f26f0094750 2024-06-03T02:28:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.990+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26f00ac8b0 con 0x7f26f00a6a00 2024-06-03T02:28:55.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.990+0000 7f26fd400700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f00a6a00 0x7f26f0143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.991+0000 7f26f7e00700 1 -- 172.21.15.79:0/1668926059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f00a6a00 msgr2=0x7f26f0143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.991+0000 7f26f7e00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f00a6a00 0x7f26f0143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.991+0000 7f26f7e00700 1 -- 172.21.15.79:0/1668926059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26f0094750 msgr2=0x7f26f0149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.991+0000 7f26f7e00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26f0094750 0x7f26f0149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:55.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.991+0000 7f26f7e00700 1 -- 172.21.15.79:0/1668926059 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f01441e0 con 0x7f26f0097ca0 2024-06-03T02:28:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.994+0000 7f26f7e00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f0097ca0 0x7f26f0149db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f26ec000c00 tx=0x7f26ec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.995+0000 7f26f5600700 1 -- 172.21.15.79:0/1668926059 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec004830 con 0x7f26f0097ca0 2024-06-03T02:28:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.997+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f0144470 con 0x7f26f0097ca0 2024-06-03T02:28:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.997+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f0150740 con 0x7f26f0097ca0 2024-06-03T02:28:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.997+0000 7f26f5600700 1 -- 172.21.15.79:0/1668926059 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ec005020 con 0x7f26f0097ca0 2024-06-03T02:28:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:55.997+0000 7f26f5600700 1 -- 172.21.15.79:0/1668926059 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec003e20 con 0x7f26f0097ca0 2024-06-03T02:28:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.003+0000 7f26f5600700 1 -- 172.21.15.79:0/1668926059 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f26ec0051d0 con 0x7f26f0097ca0 2024-06-03T02:28:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.006+0000 7f26f5600700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26e00a2d20 0x7f26e00a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.007+0000 7f26f5600700 1 -- 172.21.15.79:0/1668926059 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f26ec0d0300 con 0x7f26f0097ca0 2024-06-03T02:28:56.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.008+0000 7f26fca00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26e00a2d20 0x7f26e00a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.013+0000 7f26fca00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26e00a2d20 0x7f26e00a51e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f26f009d490 tx=0x7f26e4009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:56.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.014+0000 7f2709cf5b80 1 Processor -- start 2024-06-03T02:28:56.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.017+0000 7f2709cf5b80 1 -- start start 2024-06-03T02:28:56.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff94fb3e40 0x55ff95039420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff95039de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff95044960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ff94f46a10 con 0x55ff94fb3e40 2024-06-03T02:28:56.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ff94f89b00 con 0x55ff95039960 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ff94f448f0 con 0x55ff95042570 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f26f7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff95039de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f26f7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff95039de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:37628/0 (socket says 172.21.15.79:37628) 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f26f7e00700 1 -- 172.21.15.79:0/1291810214 learned_addr learned my addr 172.21.15.79:0/1291810214 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f26fd400700 1 --2- 172.21.15.79:0/1291810214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff95044960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f26fd400700 1 -- 172.21.15.79:0/1291810214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 msgr2=0x55ff95039de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.018+0000 7f26fd400700 1 --2- 172.21.15.79:0/1291810214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff95039de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.019+0000 7f26fd400700 1 -- 172.21.15.79:0/1291810214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff94fb3e40 msgr2=0x55ff95039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.019+0000 7f26fd400700 1 --2- 172.21.15.79:0/1291810214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff94fb3e40 0x55ff95039420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.019+0000 7f26fd400700 1 -- 172.21.15.79:0/1291810214 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff94f31f90 con 0x55ff95042570 2024-06-03T02:28:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.019+0000 7f26fd400700 1 --2- 172.21.15.79:0/1291810214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff95044960 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f26f8067c70 tx=0x7f26f8068aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3db1a5efbdfeb7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.019+0000 7f26ea000700 1 -- 172.21.15.79:0/1291810214 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f806ba30 con 0x55ff95042570 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.019+0000 7f26ea000700 1 -- 172.21.15.79:0/1291810214 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f806bbd0 con 0x55ff95042570 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.019+0000 7f26ea000700 1 -- 172.21.15.79:0/1291810214 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f807b5c0 con 0x55ff95042570 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.020+0000 7f2709cf5b80 1 -- 172.21.15.79:0/1291810214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 msgr2=0x55ff95044960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.020+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/1291810214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff95044960 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f26f8067c70 tx=0x7f26f8068aa0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.021+0000 7f2709cf5b80 1 -- 172.21.15.79:0/1291810214 shutdown_connections 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.021+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/1291810214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff95044960 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.021+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/1291810214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff94fb3e40 0x55ff95039420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.021+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/1291810214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff95039de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.021+0000 7f2709cf5b80 1 -- 172.21.15.79:0/1291810214 >> 172.21.15.79:0/1291810214 conn(0x55ff94f64170 msgr2=0x55ff95033820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:56.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.021+0000 7f2709cf5b80 1 -- 172.21.15.79:0/1291810214 shutdown_connections 2024-06-03T02:28:56.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.021+0000 7f2709cf5b80 1 -- 172.21.15.79:0/1291810214 wait complete. 2024-06-03T02:28:56.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.022+0000 7f2709cf5b80 1 -- start start 2024-06-03T02:28:56.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.023+0000 7f2709cf5b80 1 Processor -- start 2024-06-03T02:28:56.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.023+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff94fb3e40 0x55ff95041c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.023+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff95039960 0x55ff9503efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.023+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff9503f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.023+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ff94f25f60 con 0x55ff95039960 2024-06-03T02:28:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.023+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ff94f85ac0 con 0x55ff94fb3e40 2024-06-03T02:28:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.023+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ff950451e0 con 0x55ff95042570 2024-06-03T02:28:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.023+0000 7f2709cf5b80 10 client.0 _ll_register_callbacks cb 0x55ff94f23720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.024+0000 7f26fd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff9503f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.024+0000 7f26fd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff9503f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36258/0 (socket says 172.21.15.79:36258) 2024-06-03T02:28:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.024+0000 7f26fd400700 1 -- 172.21.15.79:0/973012345 learned_addr learned my addr 172.21.15.79:0/973012345 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.024+0000 7f26fd400700 1 -- 172.21.15.79:0/973012345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff94fb3e40 msgr2=0x55ff95041c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:28:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.024+0000 7f26fd400700 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff94fb3e40 0x55ff95041c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.024+0000 7f26fd400700 1 -- 172.21.15.79:0/973012345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff95039960 msgr2=0x55ff9503efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.024+0000 7f26fd400700 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff95039960 0x55ff9503efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.025+0000 7f26fd400700 1 -- 172.21.15.79:0/973012345 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff94ed88d0 con 0x55ff95042570 2024-06-03T02:28:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.025+0000 7f26fd400700 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff9503f520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f26f8068bb0 tx=0x7f26f8068eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b03c5404d0719a69 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:56.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.025+0000 7f26dea00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:28:56.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.025+0000 7f26dea00700 1 -- 172.21.15.79:0/973012345 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f808b060 con 0x55ff95042570 2024-06-03T02:28:56.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.026+0000 7f26dea00700 1 -- 172.21.15.79:0/973012345 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f26f8075040 con 0x55ff95042570 2024-06-03T02:28:56.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.026+0000 7f2709cf5b80 1 -- 172.21.15.79:0/973012345 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f806bbd0 con 0x55ff95042570 2024-06-03T02:28:56.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.027+0000 7f26dea00700 1 -- 172.21.15.79:0/973012345 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f26f807a2f0 con 0x55ff95042570 2024-06-03T02:28:56.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.027+0000 7f2709cf5b80 10 client.26335 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:56.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.027+0000 7f2709cf5b80 1 -- 172.21.15.79:0/973012345 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff9504f970 con 0x55ff95042570 2024-06-03T02:28:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.030+0000 7f26dea00700 1 -- 172.21.15.79:0/973012345 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f806bbd0 con 0x55ff95042570 2024-06-03T02:28:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.031+0000 7f26dea00700 1 -- 172.21.15.79:0/973012345 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f26f807b020 con 0x55ff95042570 2024-06-03T02:28:56.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.031+0000 7f2709cf5b80 10 client.26335 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:56.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.031+0000 7f2709cf5b80 10 client.26335 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:28:56.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.032+0000 7f2709cf5b80 20 client.26335 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.032+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff95051e40 0x55ff95054230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.032+0000 7f2709cf5b80 4 client.26335 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:28:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.033+0000 7f2709cf5b80 1 -- 172.21.15.79:0/973012345 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff94f97000 con 0x55ff95051e40 2024-06-03T02:28:56.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.033+0000 7f26fca00700 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff95051e40 0x55ff95054230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.034+0000 7f26fca00700 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff95051e40 0x55ff95054230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e6c90785951da61 server_cookie=e4e0a4f61d5d4e04 in_seq=0 out_seq=0 2024-06-03T02:28:56.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.034+0000 7f26dea00700 10 client.26335 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:28:56.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f26dea00700 1 -- 172.21.15.79:0/973012345 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ff94f97000 con 0x55ff95051e40 2024-06-03T02:28:56.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f26dea00700 10 client.26335 handle_command_reply: tid=0 2024-06-03T02:28:56.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f2709cf5b80 1 client.26335 shutdown 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f2709cf5b80 1 -- 172.21.15.79:0/973012345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 msgr2=0x55ff9503f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff9503f520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f26f8068bb0 tx=0x7f26f8068eb0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f2709cf5b80 1 -- 172.21.15.79:0/973012345 shutdown_connections 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95042570 0x55ff9503f520 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff94fb3e40 0x55ff95041c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff95051e40 0x55ff95054230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/973012345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff95039960 0x55ff9503efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.035+0000 7f2709cf5b80 1 -- 172.21.15.79:0/973012345 >> 172.21.15.79:0/973012345 conn(0x55ff94f64170 msgr2=0x55ff95033410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.036+0000 7f26dea00700 0 client.26335 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.036+0000 7f2709cf5b80 1 -- 172.21.15.79:0/973012345 shutdown_connections 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.036+0000 7f2709cf5b80 1 -- 172.21.15.79:0/973012345 wait complete. 2024-06-03T02:28:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.036+0000 7f2709cf5b80 20 client.26335 trim_cache size 0 max 16384 2024-06-03T02:28:56.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.067+0000 7f2709cf5b80 1 Processor -- start 2024-06-03T02:28:56.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f2709cf5b80 1 -- start start 2024-06-03T02:28:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff94fb3e40 0x55ff9503f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff9503d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff95042570 0x55ff9503de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f26f01593a0 con 0x55ff95042570 2024-06-03T02:28:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26f015aee0 con 0x55ff95039960 2024-06-03T02:28:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26f0159a70 con 0x55ff94fb3e40 2024-06-03T02:28:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f26f7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff9503d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f26f7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff9503d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:37634/0 (socket says 172.21.15.79:37634) 2024-06-03T02:28:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f26fd400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff95042570 0x55ff9503de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.068+0000 7f26f7e00700 1 -- 172.21.15.79:0/4101345655 learned_addr learned my addr 172.21.15.79:0/4101345655 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.069+0000 7f26fca00700 1 --2- 172.21.15.79:0/4101345655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff94fb3e40 0x55ff9503f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.069+0000 7f26f7e00700 1 -- 172.21.15.79:0/4101345655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff94fb3e40 msgr2=0x55ff9503f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.069+0000 7f26f7e00700 1 --2- 172.21.15.79:0/4101345655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff94fb3e40 0x55ff9503f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.069+0000 7f26f7e00700 1 -- 172.21.15.79:0/4101345655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff95042570 msgr2=0x55ff9503de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.070+0000 7f26f7e00700 1 --2- 172.21.15.79:0/4101345655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff95042570 0x55ff9503de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.070+0000 7f26f7e00700 1 -- 172.21.15.79:0/4101345655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff95034a20 con 0x55ff95039960 2024-06-03T02:28:56.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.072+0000 7f26f7e00700 1 --2- 172.21.15.79:0/4101345655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff9503d8d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0098e0 tx=0x7f26ec087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b749af0c2fe2c850 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:56.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.072+0000 7f26e9600700 1 -- 172.21.15.79:0/4101345655 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0a0b90 con 0x55ff95039960 2024-06-03T02:28:56.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.072+0000 7f26e9600700 1 -- 172.21.15.79:0/4101345655 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ec0a53c0 con 0x55ff95039960 2024-06-03T02:28:56.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.072+0000 7f26e9600700 1 -- 172.21.15.79:0/4101345655 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec005660 con 0x55ff95039960 2024-06-03T02:28:56.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.074+0000 7f2709cf5b80 1 -- 172.21.15.79:0/4101345655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 msgr2=0x55ff9503d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:56.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.074+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/4101345655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff9503d8d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0098e0 tx=0x7f26ec087040 comp rx=0 tx=0).stop 2024-06-03T02:28:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.075+0000 7f2709cf5b80 1 -- 172.21.15.79:0/4101345655 shutdown_connections 2024-06-03T02:28:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.075+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/4101345655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff95042570 0x55ff9503de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.075+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/4101345655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff94fb3e40 0x55ff9503f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.075+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/4101345655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95039960 0x55ff9503d8d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.075+0000 7f2709cf5b80 1 -- 172.21.15.79:0/4101345655 >> 172.21.15.79:0/4101345655 conn(0x55ff94f64170 msgr2=0x55ff95033350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.075+0000 7f2709cf5b80 1 -- 172.21.15.79:0/4101345655 shutdown_connections 2024-06-03T02:28:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.075+0000 7f2709cf5b80 1 -- 172.21.15.79:0/4101345655 wait complete. 2024-06-03T02:28:56.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.076+0000 7f2709cf5b80 1 -- start start 2024-06-03T02:28:56.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.077+0000 7f2709cf5b80 1 Processor -- start 2024-06-03T02:28:56.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.077+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff94fb3e40 0x55ff9503cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.077+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95039960 0x55ff9504a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.078+0000 7f2709cf5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95042570 0x55ff9504ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.078+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f26f015a810 con 0x55ff94fb3e40 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.078+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26f015a140 con 0x55ff95042570 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.078+0000 7f2709cf5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26f0157340 con 0x55ff95039960 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.078+0000 7f2709cf5b80 10 client.0 _ll_register_callbacks cb 0x55ff94f23720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.078+0000 7f26fd400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95042570 0x55ff9504ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.078+0000 7f26fd400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95042570 0x55ff9504ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:37638/0 (socket says 172.21.15.79:37638) 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.078+0000 7f26fd400700 1 -- 172.21.15.79:0/2221855328 learned_addr learned my addr 172.21.15.79:0/2221855328 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:28:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.078+0000 7f26f7e00700 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95039960 0x55ff9504a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26fca00700 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff94fb3e40 0x55ff9503cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26fd400700 1 -- 172.21.15.79:0/2221855328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95039960 msgr2=0x55ff9504a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26fd400700 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95039960 0x55ff9504a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26fd400700 1 -- 172.21.15.79:0/2221855328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff94fb3e40 msgr2=0x55ff9503cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26fd400700 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff94fb3e40 0x55ff9503cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26fd400700 1 -- 172.21.15.79:0/2221855328 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9503b2a0 con 0x55ff95042570 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26f7e00700 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95039960 0x55ff9504a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26fd400700 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95042570 0x55ff9504ac10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f26f80680c0 tx=0x7f26f807de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab1371f5e6f28fc8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26dea00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.079+0000 7f26dea00700 1 -- 172.21.15.79:0/2221855328 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f808b070 con 0x55ff95042570 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.080+0000 7f26dea00700 1 -- 172.21.15.79:0/2221855328 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f26f8075070 con 0x55ff95042570 2024-06-03T02:28:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.080+0000 7f2709cf5b80 1 -- 172.21.15.79:0/2221855328 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ec0a53c0 con 0x55ff95042570 2024-06-03T02:28:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.083+0000 7f26dea00700 1 -- 172.21.15.79:0/2221855328 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f26f8051140 con 0x55ff95042570 2024-06-03T02:28:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.083+0000 7f2709cf5b80 10 client.26333 fetch_fsmap learned FSMap version 30 2024-06-03T02:28:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.083+0000 7f2709cf5b80 1 -- 172.21.15.79:0/2221855328 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff950bbd60 con 0x55ff95042570 2024-06-03T02:28:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.084+0000 7f26dea00700 1 -- 172.21.15.79:0/2221855328 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ec0a53c0 con 0x55ff95042570 2024-06-03T02:28:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.085+0000 7f26dea00700 1 -- 172.21.15.79:0/2221855328 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f26f807a490 con 0x55ff95042570 2024-06-03T02:28:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.086+0000 7f2709cf5b80 10 client.26333 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:28:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.086+0000 7f2709cf5b80 10 client.26333 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:28:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.086+0000 7f2709cf5b80 20 client.26333 populate_metadata read hostname 'smithi079' 2024-06-03T02:28:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.086+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff950bc420 0x55ff950be840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:28:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.086+0000 7f2709cf5b80 4 client.26333 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:28:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.086+0000 7f2709cf5b80 1 -- 172.21.15.79:0/2221855328 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff94f91580 con 0x55ff950bc420 2024-06-03T02:28:56.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.086+0000 7f26fca00700 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff950bc420 0x55ff950be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:28:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.087+0000 7f26fca00700 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff950bc420 0x55ff950be840 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=934422afef74797a server_cookie=9dfe3d2644cace3c in_seq=0 out_seq=0 2024-06-03T02:28:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.087+0000 7f26dea00700 10 client.26333 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:28:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f26dea00700 1 -- 172.21.15.79:0/2221855328 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff94f91580 con 0x55ff950bc420 2024-06-03T02:28:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f26dea00700 10 client.26333 handle_command_reply: tid=0 2024-06-03T02:28:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f2709cf5b80 1 client.26333 shutdown 2024-06-03T02:28:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f2709cf5b80 1 -- 172.21.15.79:0/2221855328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95042570 msgr2=0x55ff9504ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95042570 0x55ff9504ac10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f26f80680c0 tx=0x7f26f807de10 comp rx=0 tx=0).stop 2024-06-03T02:28:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f2709cf5b80 1 -- 172.21.15.79:0/2221855328 shutdown_connections 2024-06-03T02:28:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff95042570 0x55ff9504ac10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55ff950bc420 0x55ff950be840 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff94fb3e40 0x55ff9503cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f2709cf5b80 1 --2- 172.21.15.79:0/2221855328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff95039960 0x55ff9504a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f2709cf5b80 1 -- 172.21.15.79:0/2221855328 >> 172.21.15.79:0/2221855328 conn(0x55ff94f64170 msgr2=0x55ff95032fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.088+0000 7f26dea00700 0 client.26333 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:28:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.090+0000 7f2709cf5b80 1 -- 172.21.15.79:0/2221855328 shutdown_connections 2024-06-03T02:28:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.090+0000 7f2709cf5b80 1 -- 172.21.15.79:0/2221855328 wait complete. 2024-06-03T02:28:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.090+0000 7f2709cf5b80 20 client.26333 trim_cache size 0 max 16384 2024-06-03T02:28:56.091 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:28:56.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26e00a2d20 msgr2=0x7f26e00a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:56.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26e00a2d20 0x7f26e00a51e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f26f009d490 tx=0x7f26e4009290 comp rx=0 tx=0).stop 2024-06-03T02:28:56.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f0097ca0 msgr2=0x7f26f0149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:28:56.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f0097ca0 0x7f26f0149db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f26ec000c00 tx=0x7f26ec0047b0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 shutdown_connections 2024-06-03T02:28:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f00a6a00 0x7f26f0143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26f0094750 0x7f26f0149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26e00a2d20 0x7f26e00a51e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 --2- 172.21.15.79:0/1668926059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26f0097ca0 0x7f26f0149db0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:28:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.091+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 >> 172.21.15.79:0/1668926059 conn(0x7f26f0009ad0 msgr2=0x7f26f0095b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:28:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.092+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 shutdown_connections 2024-06-03T02:28:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:28:56.092+0000 7f26fde00700 1 -- 172.21.15.79:0/1668926059 wait complete. 2024-06-03T02:28:56.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:56 smithi123 ceph-mon[36640]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 474 op/s 2024-06-03T02:28:56.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:56 smithi079 ceph-mon[31819]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 474 op/s 2024-06-03T02:28:56.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:56 smithi135 ceph-mon[36653]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 474 op/s 2024-06-03T02:28:58.038 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:28:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 530.0 s, 533.4 tps, lat 188.163 ms stddev 211.071 2024-06-03T02:28:58.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:28:58 smithi123 ceph-mon[36640]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.8 MiB/s wr, 433 op/s 2024-06-03T02:28:58.801 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:28:58.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:28:58 smithi079 ceph-mon[31819]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.8 MiB/s wr, 433 op/s 2024-06-03T02:28:58.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:28:58 smithi135 ceph-mon[36653]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.8 MiB/s wr, 433 op/s 2024-06-03T02:29:00.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:00 smithi123 ceph-mon[36640]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.7 MiB/s wr, 530 op/s 2024-06-03T02:29:00.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:00 smithi079 ceph-mon[31819]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.7 MiB/s wr, 530 op/s 2024-06-03T02:29:00.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:00 smithi135 ceph-mon[36653]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.7 MiB/s wr, 530 op/s 2024-06-03T02:29:02.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:02 smithi123 ceph-mon[36640]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-06-03T02:29:02.777 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:02 smithi079 ceph-mon[31819]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-06-03T02:29:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:02 smithi135 ceph-mon[36653]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-06-03T02:29:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 535.0 s, 673.8 tps, lat 139.337 ms stddev 165.884 2024-06-03T02:29:03.568 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:29:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.223+0000 7f46ca600700 1 -- 172.21.15.79:0/957662516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46c406ab30 msgr2=0x7f46c410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.223+0000 7f46ca600700 1 --2- 172.21.15.79:0/957662516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46c406ab30 0x7f46c410f320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009f90 tx=0x7f46b400b330 comp rx=0 tx=0).stop 2024-06-03T02:29:03.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.225+0000 7f46ca600700 1 -- 172.21.15.79:0/957662516 shutdown_connections 2024-06-03T02:29:03.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.225+0000 7f46ca600700 1 --2- 172.21.15.79:0/957662516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46c406ab30 0x7f46c410f320 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.225+0000 7f46ca600700 1 --2- 172.21.15.79:0/957662516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c4108500 0x7f46c41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.225+0000 7f46ca600700 1 --2- 172.21.15.79:0/957662516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c406a170 0x7f46c406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.225+0000 7f46ca600700 1 -- 172.21.15.79:0/957662516 >> 172.21.15.79:0/957662516 conn(0x7f46c4078340 msgr2=0x7f46c4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:03.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.226+0000 7f46ca600700 1 -- 172.21.15.79:0/957662516 shutdown_connections 2024-06-03T02:29:03.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.226+0000 7f46ca600700 1 -- 172.21.15.79:0/957662516 wait complete. 2024-06-03T02:29:03.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.229+0000 7f46ca600700 1 Processor -- start 2024-06-03T02:29:03.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.229+0000 7f46ca600700 1 -- start start 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.229+0000 7f46ca600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c406a170 0x7f46c4116a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.229+0000 7f46ca600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c406ab30 0x7f46c4116f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.229+0000 7f46ca600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46c4108500 0x7f46c411c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.230+0000 7f46ca600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46c41033a0 con 0x7f46c406a170 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.230+0000 7f46ca600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4103220 con 0x7f46c406ab30 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.230+0000 7f46c2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c406ab30 0x7f46c4116f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.230+0000 7f46ca600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46c41107a0 con 0x7f46c4108500 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.230+0000 7f46c2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c406ab30 0x7f46c4116f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:37656/0 (socket says 172.21.15.79:37656) 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.230+0000 7f46c2a00700 1 -- 172.21.15.79:0/2433994433 learned_addr learned my addr 172.21.15.79:0/2433994433 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.230+0000 7f46c3e00700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46c4108500 0x7f46c411c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.230+0000 7f46c3400700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c406a170 0x7f46c4116a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.231+0000 7f46c3e00700 1 -- 172.21.15.79:0/2433994433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c406ab30 msgr2=0x7f46c4116f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.231+0000 7f46c3e00700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c406ab30 0x7f46c4116f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.231+0000 7f46c3e00700 1 -- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c406a170 msgr2=0x7f46c4116a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.231+0000 7f46c3e00700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c406a170 0x7f46c4116a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.231+0000 7f46c3e00700 1 -- 172.21.15.79:0/2433994433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c411d070 con 0x7f46c4108500 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.232+0000 7f46c3e00700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46c4108500 0x7f46c411c770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009f90 tx=0x7f46b400cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.233+0000 7f46bbe00700 1 -- 172.21.15.79:0/2433994433 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4021070 con 0x7f46c4108500 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.233+0000 7f46ca600700 1 -- 172.21.15.79:0/2433994433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c4111b70 con 0x7f46c4108500 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.233+0000 7f46ca600700 1 -- 172.21.15.79:0/2433994433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c41120b0 con 0x7f46c4108500 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.234+0000 7f46bbe00700 1 -- 172.21.15.79:0/2433994433 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b4009c20 con 0x7f46c4108500 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.234+0000 7f46bbe00700 1 -- 172.21.15.79:0/2433994433 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4021070 con 0x7f46c4108500 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.244+0000 7f46bbe00700 1 -- 172.21.15.79:0/2433994433 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f46b4024070 con 0x7f46c4108500 2024-06-03T02:29:03.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.247+0000 7f46d504fb80 1 Processor -- start 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.248+0000 7f46d504fb80 1 -- start start 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.248+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29a57e40 0x562b29adb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.248+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29adb5b0 0x562b29adba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.248+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae6300 0x562b29ae86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.248+0000 7f46d504fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562b299eaa10 con 0x562b29adb5b0 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.248+0000 7f46d504fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b29a2db00 con 0x562b29ae6300 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.248+0000 7f46d504fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562b299e88f0 con 0x562b29a57e40 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.249+0000 7f46c3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29adb5b0 0x562b29adba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.249+0000 7f46c2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29a57e40 0x562b29adb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.249+0000 7f46c2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29a57e40 0x562b29adb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36310/0 (socket says 172.21.15.79:36310) 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.249+0000 7f46c2a00700 1 -- 172.21.15.79:0/2577326560 learned_addr learned my addr 172.21.15.79:0/2577326560 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.249+0000 7f46c2a00700 1 --2- 172.21.15.79:0/2577326560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae6300 0x562b29ae86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.250+0000 7f46c3400700 1 -- 172.21.15.79:0/2577326560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29a57e40 msgr2=0x562b29adb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.250+0000 7f46c3400700 1 --2- 172.21.15.79:0/2577326560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29a57e40 0x562b29adb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.250+0000 7f46c3400700 1 -- 172.21.15.79:0/2577326560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae6300 msgr2=0x562b29ae86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.250+0000 7f46c3400700 1 --2- 172.21.15.79:0/2577326560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae6300 0x562b29ae86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.250+0000 7f46c3400700 1 -- 172.21.15.79:0/2577326560 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b299d5f90 con 0x562b29adb5b0 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.251+0000 7f46c3400700 1 --2- 172.21.15.79:0/2577326560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29adb5b0 0x562b29adba30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009c00 tx=0x7f46bc009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9acf141ab754eba server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.251+0000 7f46b8c00700 1 -- 172.21.15.79:0/2577326560 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc004220 con 0x562b29adb5b0 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.251+0000 7f46b8c00700 1 -- 172.21.15.79:0/2577326560 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46bc00fdd0 con 0x562b29adb5b0 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.252+0000 7f46b8c00700 1 -- 172.21.15.79:0/2577326560 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc033450 con 0x562b29adb5b0 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.252+0000 7f46bbe00700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f46a80a3090 0x7f46a80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.252+0000 7f46bbe00700 1 -- 172.21.15.79:0/2433994433 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f46b40078c0 con 0x7f46c4108500 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.253+0000 7f46d504fb80 1 -- 172.21.15.79:0/2577326560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29adb5b0 msgr2=0x562b29adba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.253+0000 7f46d504fb80 1 --2- 172.21.15.79:0/2577326560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29adb5b0 0x562b29adba30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009c00 tx=0x7f46bc009f10 comp rx=0 tx=0).stop 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.254+0000 7f46d504fb80 1 -- 172.21.15.79:0/2577326560 shutdown_connections 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.254+0000 7f46d504fb80 1 --2- 172.21.15.79:0/2577326560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae6300 0x562b29ae86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.254+0000 7f46d504fb80 1 --2- 172.21.15.79:0/2577326560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29a57e40 0x562b29adb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.254+0000 7f46d504fb80 1 --2- 172.21.15.79:0/2577326560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29adb5b0 0x562b29adba30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.254+0000 7f46d504fb80 1 -- 172.21.15.79:0/2577326560 >> 172.21.15.79:0/2577326560 conn(0x562b29a08170 msgr2=0x562b29ad74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.255+0000 7f46d504fb80 1 -- 172.21.15.79:0/2577326560 shutdown_connections 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.257+0000 7f46c2a00700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f46a80a3090 0x7f46a80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.257+0000 7f46d504fb80 1 -- 172.21.15.79:0/2577326560 wait complete. 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.257+0000 7f46d504fb80 1 -- start start 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.258+0000 7f46c2a00700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f46a80a3090 0x7f46a80a5550 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f46ac005d10 tx=0x7f46ac013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.258+0000 7f46d504fb80 1 Processor -- start 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.262+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29adb5b0 0x562b29ae5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.262+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae2f50 0x562b29ae33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.262+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 0x562b29af2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.262+0000 7f46d504fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562b299c9f60 con 0x562b29ae2f50 2024-06-03T02:29:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.262+0000 7f46d504fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b29a29ac0 con 0x562b29adb5b0 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.262+0000 7f46d504fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562b29ae8fc0 con 0x562b29ae3910 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.263+0000 7f46d504fb80 10 client.0 _ll_register_callbacks cb 0x562b299c7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.263+0000 7f46c2a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae2f50 0x562b29ae33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.263+0000 7f46c2a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae2f50 0x562b29ae33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39122/0 (socket says 172.21.15.79:39122) 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.263+0000 7f46c2a00700 1 -- 172.21.15.79:0/1081903703 learned_addr learned my addr 172.21.15.79:0/1081903703 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.264+0000 7f46c2a00700 1 -- 172.21.15.79:0/1081903703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 msgr2=0x562b29af2380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.264+0000 7f46c2a00700 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 0x562b29af2380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.264+0000 7f46c2a00700 1 -- 172.21.15.79:0/1081903703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29adb5b0 msgr2=0x562b29ae5c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.264+0000 7f46c2a00700 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29adb5b0 0x562b29ae5c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.264+0000 7f46c2a00700 1 -- 172.21.15.79:0/1081903703 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b2997c8d0 con 0x562b29ae2f50 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.268+0000 7f46b1600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.268+0000 7f46c2a00700 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae2f50 0x562b29ae33d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009750 tx=0x7f46ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cb070173463046 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.268+0000 7f46b1600700 1 -- 172.21.15.79:0/1081903703 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac02d060 con 0x562b29ae2f50 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.269+0000 7f46d504fb80 1 -- 172.21.15.79:0/1081903703 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46bc00fdd0 con 0x562b29ae2f50 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.269+0000 7f46b1600700 1 -- 172.21.15.79:0/1081903703 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f46ac01a070 con 0x562b29ae2f50 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.269+0000 7f46b1600700 1 -- 172.21.15.79:0/1081903703 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46ac01d620 con 0x562b29ae2f50 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.269+0000 7f46d504fb80 10 client.15915 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.269+0000 7f46d504fb80 1 -- 172.21.15.79:0/1081903703 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b29af4180 con 0x562b29ae2f50 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.270+0000 7f46b1600700 1 -- 172.21.15.79:0/1081903703 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46bc00fdd0 con 0x562b29ae2f50 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.270+0000 7f46b1600700 1 -- 172.21.15.79:0/1081903703 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f46ac028af0 con 0x562b29ae2f50 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.270+0000 7f46d504fb80 10 client.15915 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.270+0000 7f46d504fb80 10 client.15915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.270+0000 7f46d504fb80 20 client.15915 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.270+0000 7f46d504fb80 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562b29af6760 0x562b29af8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.270+0000 7f46d504fb80 4 client.15915 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.270+0000 7f46d504fb80 1 -- 172.21.15.79:0/1081903703 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b29a3b000 con 0x562b29af6760 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.271+0000 7f46c3400700 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562b29af6760 0x562b29af8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.273+0000 7f46b1600700 10 client.15915 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.273+0000 7f46c3400700 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562b29af6760 0x562b29af8c00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b09b142c684f0f1 server_cookie=6a3b168ff1a1ead6 in_seq=0 out_seq=0 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.274+0000 7f46b1600700 1 -- 172.21.15.79:0/1081903703 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b29a3b000 con 0x562b29af6760 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.274+0000 7f46b1600700 10 client.15915 handle_command_reply: tid=0 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.274+0000 7f46d504fb80 1 client.15915 shutdown 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 -- 172.21.15.79:0/1081903703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae2f50 msgr2=0x562b29ae33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae2f50 0x562b29ae33d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009750 tx=0x7f46ac004750 comp rx=0 tx=0).stop 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 -- 172.21.15.79:0/1081903703 shutdown_connections 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 0x562b29af2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562b29af6760 0x562b29af8c00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29adb5b0 0x562b29ae5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 --2- 172.21.15.79:0/1081903703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae2f50 0x562b29ae33d0 secure :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009750 tx=0x7f46ac004750 comp rx=0 tx=0).stop 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 -- 172.21.15.79:0/1081903703 >> 172.21.15.79:0/1081903703 conn(0x562b29a08170 msgr2=0x562b29ad6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46b1600700 0 client.15915 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:29:03.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 -- 172.21.15.79:0/1081903703 shutdown_connections 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.275+0000 7f46d504fb80 1 -- 172.21.15.79:0/1081903703 wait complete. 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.276+0000 7f46d504fb80 20 client.15915 trim_cache size 0 max 16384 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.319+0000 7f46d504fb80 1 Processor -- start 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.321+0000 7f46d504fb80 1 -- start start 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.322+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29adb5b0 0x562b29af2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.322+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 0x562b29ae76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.322+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae7c30 0x562b29ad6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.322+0000 7f46d504fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46a000ec70 con 0x562b29ae7c30 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.322+0000 7f46d504fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46a00107b0 con 0x562b29adb5b0 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.322+0000 7f46d504fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46a000f340 con 0x562b29ae3910 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.322+0000 7f46c2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 0x562b29ae76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.322+0000 7f46c2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 0x562b29ae76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36326/0 (socket says 172.21.15.79:36326) 2024-06-03T02:29:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.322+0000 7f46c2a00700 1 -- 172.21.15.79:0/848265299 learned_addr learned my addr 172.21.15.79:0/848265299 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.323+0000 7f46c3400700 1 --2- 172.21.15.79:0/848265299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29adb5b0 0x562b29af2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.323+0000 7f46c2a00700 1 -- 172.21.15.79:0/848265299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29adb5b0 msgr2=0x562b29af2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.323+0000 7f46c2a00700 1 --2- 172.21.15.79:0/848265299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29adb5b0 0x562b29af2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.323+0000 7f46c2a00700 1 -- 172.21.15.79:0/848265299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae7c30 msgr2=0x562b29ad6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.323+0000 7f46c2a00700 1 --2- 172.21.15.79:0/848265299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae7c30 0x562b29ad6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.323+0000 7f46c2a00700 1 -- 172.21.15.79:0/848265299 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b29ae5fc0 con 0x562b29ae3910 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.324+0000 7f46c2a00700 1 --2- 172.21.15.79:0/848265299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 0x562b29ae76f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00f4c0 tx=0x7f46ac00f6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f118b09bd386aa89 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.325+0000 7f46b3e00700 1 -- 172.21.15.79:0/848265299 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac037080 con 0x562b29ae3910 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.325+0000 7f46b3e00700 1 -- 172.21.15.79:0/848265299 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac00fd40 con 0x562b29ae3910 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.325+0000 7f46b3e00700 1 -- 172.21.15.79:0/848265299 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac01ddb0 con 0x562b29ae3910 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.326+0000 7f46d504fb80 1 -- 172.21.15.79:0/848265299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 msgr2=0x562b29ae76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:03.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.326+0000 7f46d504fb80 1 --2- 172.21.15.79:0/848265299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 0x562b29ae76f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00f4c0 tx=0x7f46ac00f6e0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.330+0000 7f46d504fb80 1 -- 172.21.15.79:0/848265299 shutdown_connections 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.330+0000 7f46d504fb80 1 --2- 172.21.15.79:0/848265299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae7c30 0x562b29ad6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.330+0000 7f46d504fb80 1 --2- 172.21.15.79:0/848265299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29adb5b0 0x562b29af2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.330+0000 7f46d504fb80 1 --2- 172.21.15.79:0/848265299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29ae3910 0x562b29ae76f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.332+0000 7f46d504fb80 1 -- 172.21.15.79:0/848265299 >> 172.21.15.79:0/848265299 conn(0x562b29a08170 msgr2=0x562b29ade250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.332+0000 7f46d504fb80 1 -- 172.21.15.79:0/848265299 shutdown_connections 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.332+0000 7f46d504fb80 1 -- 172.21.15.79:0/848265299 wait complete. 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.333+0000 7f46d504fb80 1 -- start start 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.333+0000 7f46d504fb80 1 Processor -- start 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.335+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29adb5b0 0x562b29aee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.335+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae3910 0x562b29aeeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.335+0000 7f46d504fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae7c30 0x562b29afdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.335+0000 7f46c3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29adb5b0 0x562b29aee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.335+0000 7f46c3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29adb5b0 0x562b29aee5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36334/0 (socket says 172.21.15.79:36334) 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.335+0000 7f46c2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae3910 0x562b29aeeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.335+0000 7f46c3400700 1 -- 172.21.15.79:0/3182777760 learned_addr learned my addr 172.21.15.79:0/3182777760 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46a00100e0 con 0x562b29ae7c30 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46a000fa10 con 0x562b29ae3910 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46a000cc10 con 0x562b29adb5b0 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46c3e00700 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae7c30 0x562b29afdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46d504fb80 10 client.0 _ll_register_callbacks cb 0x562b299c7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46c3400700 1 -- 172.21.15.79:0/3182777760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae3910 msgr2=0x562b29aeeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46c3400700 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae3910 0x562b29aeeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46c3400700 1 -- 172.21.15.79:0/3182777760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae7c30 msgr2=0x562b29afdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46c3400700 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae7c30 0x562b29afdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46c3400700 1 -- 172.21.15.79:0/3182777760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b29afe440 con 0x562b29adb5b0 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.336+0000 7f46c2a00700 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae3910 0x562b29aeeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.337+0000 7f46c3400700 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29adb5b0 0x562b29aee5a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00f640 tx=0x7f46bc003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a021f449a9d4eb31 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.337+0000 7f46b1600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.338+0000 7f46b1600700 1 -- 172.21.15.79:0/3182777760 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc04d060 con 0x562b29adb5b0 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.338+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ac00fd40 con 0x562b29adb5b0 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.338+0000 7f46b1600700 1 -- 172.21.15.79:0/3182777760 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f46bc00faf0 con 0x562b29adb5b0 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.338+0000 7f46b1600700 1 -- 172.21.15.79:0/3182777760 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46bc054050 con 0x562b29adb5b0 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.339+0000 7f46d504fb80 10 client.26356 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.339+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b29aedf40 con 0x562b29adb5b0 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.340+0000 7f46b1600700 1 -- 172.21.15.79:0/3182777760 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac00fd40 con 0x562b29adb5b0 2024-06-03T02:29:03.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.340+0000 7f46b1600700 1 -- 172.21.15.79:0/3182777760 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f46bc004800 con 0x562b29adb5b0 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.340+0000 7f46d504fb80 10 client.26356 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.340+0000 7f46d504fb80 10 client.26356 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.340+0000 7f46d504fb80 20 client.26356 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.340+0000 7f46d504fb80 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562b29b615d0 0x562b29b639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.340+0000 7f46d504fb80 4 client.26356 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.340+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b29a35580 con 0x562b29b615d0 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.340+0000 7f46c2a00700 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562b29b615d0 0x562b29b639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.341+0000 7f46b1600700 10 client.26356 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.341+0000 7f46c2a00700 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562b29b615d0 0x562b29b639c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe393636c34ba7ce server_cookie=23c756cd590aa641 in_seq=0 out_seq=0 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.341+0000 7f46b1600700 1 -- 172.21.15.79:0/3182777760 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b29a35580 con 0x562b29b615d0 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.341+0000 7f46b1600700 10 client.26356 handle_command_reply: tid=0 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.341+0000 7f46d504fb80 1 client.26356 shutdown 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.341+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29adb5b0 msgr2=0x562b29aee5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.341+0000 7f46d504fb80 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29adb5b0 0x562b29aee5a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00f640 tx=0x7f46bc003c30 comp rx=0 tx=0).stop 2024-06-03T02:29:03.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 shutdown_connections 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46d504fb80 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562b29ae7c30 0x562b29afdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46d504fb80 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b29adb5b0 0x562b29aee5a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46d504fb80 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x562b29b615d0 0x562b29b639c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46d504fb80 1 --2- 172.21.15.79:0/3182777760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b29ae3910 0x562b29aeeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46b1600700 0 client.26356 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 >> 172.21.15.79:0/3182777760 conn(0x562b29a08170 msgr2=0x562b29ad6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 shutdown_connections 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46d504fb80 1 -- 172.21.15.79:0/3182777760 wait complete. 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.342+0000 7f46d504fb80 20 client.26356 trim_cache size 0 max 16384 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 -- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f46a80a3090 msgr2=0x7f46a80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f46a80a3090 0x7f46a80a5550 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f46ac005d10 tx=0x7f46ac013040 comp rx=0 tx=0).stop 2024-06-03T02:29:03.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 -- 172.21.15.79:0/2433994433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46c4108500 msgr2=0x7f46c411c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:03.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46c4108500 0x7f46c411c770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009f90 tx=0x7f46b400cfa0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 -- 172.21.15.79:0/2433994433 shutdown_connections 2024-06-03T02:29:03.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46c4108500 0x7f46c411c770 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f46a80a3090 0x7f46a80a5550 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c406a170 0x7f46c4116a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 --2- 172.21.15.79:0/2433994433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c406ab30 0x7f46c4116f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:03.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.345+0000 7f46ca600700 1 -- 172.21.15.79:0/2433994433 >> 172.21.15.79:0/2433994433 conn(0x7f46c4078340 msgr2=0x7f46c40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:03.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.346+0000 7f46ca600700 1 -- 172.21.15.79:0/2433994433 shutdown_connections 2024-06-03T02:29:03.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:03.346+0000 7f46ca600700 1 -- 172.21.15.79:0/2433994433 wait complete. 2024-06-03T02:29:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:04 smithi123 ceph-mon[36640]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.6 MiB/s wr, 529 op/s 2024-06-03T02:29:04.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:04 smithi079 ceph-mon[31819]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.6 MiB/s wr, 529 op/s 2024-06-03T02:29:04.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:04 smithi135 ceph-mon[36653]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.6 MiB/s wr, 529 op/s 2024-06-03T02:29:05.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:29:05.901 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:29:06.818 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:06 smithi079 ceph-mon[31819]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 595 op/s 2024-06-03T02:29:06.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:06 smithi135 ceph-mon[36653]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 595 op/s 2024-06-03T02:29:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:06 smithi123 ceph-mon[36640]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 595 op/s 2024-06-03T02:29:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 540.0 s, 448.4 tps, lat 230.497 ms stddev 300.004 2024-06-03T02:29:08.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:08 smithi079 ceph-mon[31819]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.7 MiB/s wr, 531 op/s 2024-06-03T02:29:08.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:08 smithi135 ceph-mon[36653]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.7 MiB/s wr, 531 op/s 2024-06-03T02:29:08.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:08 smithi123 ceph-mon[36640]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.7 MiB/s wr, 531 op/s 2024-06-03T02:29:09.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.984+0000 7f3b09e00700 1 -- 172.21.15.79:0/3560507398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b04102740 msgr2=0x7f3b04102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:09.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.984+0000 7f3b09e00700 1 --2- 172.21.15.79:0/3560507398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b04102740 0x7f3b04102b20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009990 tx=0x7f3af4009ca0 comp rx=0 tx=0).stop 2024-06-03T02:29:09.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.985+0000 7f3b09e00700 1 -- 172.21.15.79:0/3560507398 shutdown_connections 2024-06-03T02:29:09.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.985+0000 7f3b09e00700 1 --2- 172.21.15.79:0/3560507398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b04107840 0x7f3b0410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:09.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.985+0000 7f3b09e00700 1 --2- 172.21.15.79:0/3560507398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b04102740 0x7f3b04102b20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:09.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.985+0000 7f3b09e00700 1 --2- 172.21.15.79:0/3560507398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b041030f0 0x7f3b04107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:09.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.985+0000 7f3b09e00700 1 -- 172.21.15.79:0/3560507398 >> 172.21.15.79:0/3560507398 conn(0x7f3b040795f0 msgr2=0x7f3b04079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:09.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.985+0000 7f3b09e00700 1 -- 172.21.15.79:0/3560507398 shutdown_connections 2024-06-03T02:29:09.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.986+0000 7f3b09e00700 1 -- 172.21.15.79:0/3560507398 wait complete. 2024-06-03T02:29:09.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.987+0000 7f3b09e00700 1 Processor -- start 2024-06-03T02:29:09.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.987+0000 7f3b09e00700 1 -- start start 2024-06-03T02:29:09.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.987+0000 7f3b09e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b04102740 0x7f3b0419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:09.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b09e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b041030f0 0x7f3b0419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:09.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b09e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b04107840 0x7f3b0419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:09.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b09e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3b04112830 con 0x7f3b041030f0 2024-06-03T02:29:09.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b09e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041126b0 con 0x7f3b04102740 2024-06-03T02:29:09.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b09e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041129b0 con 0x7f3b04107840 2024-06-03T02:29:09.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b04102740 0x7f3b0419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:09.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b04102740 0x7f3b0419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58138/0 (socket says 172.21.15.79:58138) 2024-06-03T02:29:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b02a00700 1 -- 172.21.15.79:0/2812150404 learned_addr learned my addr 172.21.15.79:0/2812150404 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b03400700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b04107840 0x7f3b0419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b02a00700 1 -- 172.21.15.79:0/2812150404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b04107840 msgr2=0x7f3b0419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b02a00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b04107840 0x7f3b0419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b02000700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b041030f0 0x7f3b0419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.988+0000 7f3b02a00700 1 -- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b041030f0 msgr2=0x7f3b0419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.989+0000 7f3b02a00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b041030f0 0x7f3b0419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.989+0000 7f3b02a00700 1 -- 172.21.15.79:0/2812150404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0419fe90 con 0x7f3b04102740 2024-06-03T02:29:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.989+0000 7f3b02000700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b041030f0 0x7f3b0419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:29:09.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.989+0000 7f3b02a00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b04102740 0x7f3b0419ae80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3af4000c00 tx=0x7f3af40038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:09.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.990+0000 7f3b08a00700 1 -- 172.21.15.79:0/2812150404 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4027070 con 0x7f3b04102740 2024-06-03T02:29:09.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.990+0000 7f3b08a00700 1 -- 172.21.15.79:0/2812150404 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af4004050 con 0x7f3b04102740 2024-06-03T02:29:09.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.990+0000 7f3b09e00700 1 -- 172.21.15.79:0/2812150404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b041a0120 con 0x7f3b04102740 2024-06-03T02:29:09.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.990+0000 7f3b08a00700 1 -- 172.21.15.79:0/2812150404 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af400f460 con 0x7f3b04102740 2024-06-03T02:29:09.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.990+0000 7f3b09e00700 1 -- 172.21.15.79:0/2812150404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b041a7a00 con 0x7f3b04102740 2024-06-03T02:29:10.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:09.999+0000 7f3b08a00700 1 -- 172.21.15.79:0/2812150404 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3af400f600 con 0x7f3b04102740 2024-06-03T02:29:10.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.001+0000 7f3b08a00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3ae80a2d70 0x7f3ae80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.001+0000 7f3b08a00700 1 -- 172.21.15.79:0/2812150404 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3af40d0600 con 0x7f3b04102740 2024-06-03T02:29:10.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.003+0000 7f3b14789b80 1 Processor -- start 2024-06-03T02:29:10.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.003+0000 7f3b02000700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3ae80a2d70 0x7f3ae80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.003+0000 7f3b14789b80 1 -- start start 2024-06-03T02:29:10.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.003+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x556035012780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.003+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603500c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.003+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603500ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.004+0000 7f3b14789b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556034f12a10 con 0x5560350123a0 2024-06-03T02:29:10.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.004+0000 7f3b14789b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556034f55b00 con 0x556035012cc0 2024-06-03T02:29:10.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.004+0000 7f3b14789b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556034f108f0 con 0x55603500ca50 2024-06-03T02:29:10.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.004+0000 7f3b03400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603500ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.004+0000 7f3b03400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603500ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54496/0 (socket says 172.21.15.79:54496) 2024-06-03T02:29:10.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.004+0000 7f3b03400700 1 -- 172.21.15.79:0/2381557623 learned_addr learned my addr 172.21.15.79:0/2381557623 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:10.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.004+0000 7f3b03400700 1 -- 172.21.15.79:0/2381557623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 msgr2=0x55603500c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:10.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.005+0000 7f3b02000700 1 --2- 172.21.15.79:0/2381557623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603500c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.005+0000 7f3b03400700 1 --2- 172.21.15.79:0/2381557623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603500c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.005+0000 7f3b03400700 1 -- 172.21.15.79:0/2381557623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 msgr2=0x556035012780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.005+0000 7f3b03400700 1 --2- 172.21.15.79:0/2381557623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x556035012780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.005+0000 7f3b03400700 1 -- 172.21.15.79:0/2381557623 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556034efdf90 con 0x55603500ca50 2024-06-03T02:29:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.005+0000 7f3b02000700 1 --2- 172.21.15.79:0/2381557623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603500c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:29:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.005+0000 7f3b03400700 1 --2- 172.21.15.79:0/2381557623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603500ce90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009f20 tx=0x7f3afc009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf872d68e97b211 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.006+0000 7f3b02000700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3ae80a2d70 0x7f3ae80a5230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3af0005fd0 tx=0x7f3af00116c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.006+0000 7f3af8c00700 1 -- 172.21.15.79:0/2381557623 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc01a070 con 0x55603500ca50 2024-06-03T02:29:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.006+0000 7f3af8c00700 1 -- 172.21.15.79:0/2381557623 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3afc015410 con 0x55603500ca50 2024-06-03T02:29:10.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.006+0000 7f3af8c00700 1 -- 172.21.15.79:0/2381557623 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00a640 con 0x55603500ca50 2024-06-03T02:29:10.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.007+0000 7f3b14789b80 1 -- 172.21.15.79:0/2381557623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 msgr2=0x55603500ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:10.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.007+0000 7f3b14789b80 1 --2- 172.21.15.79:0/2381557623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603500ce90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009f20 tx=0x7f3afc009f50 comp rx=0 tx=0).stop 2024-06-03T02:29:10.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.007+0000 7f3b14789b80 1 -- 172.21.15.79:0/2381557623 shutdown_connections 2024-06-03T02:29:10.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.007+0000 7f3b14789b80 1 --2- 172.21.15.79:0/2381557623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603500ce90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.007+0000 7f3b14789b80 1 --2- 172.21.15.79:0/2381557623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x556035012780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.007+0000 7f3b14789b80 1 --2- 172.21.15.79:0/2381557623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603500c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.007+0000 7f3b14789b80 1 -- 172.21.15.79:0/2381557623 >> 172.21.15.79:0/2381557623 conn(0x556034f30170 msgr2=0x556034ffeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:10.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.007+0000 7f3b14789b80 1 -- 172.21.15.79:0/2381557623 shutdown_connections 2024-06-03T02:29:10.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.007+0000 7f3b14789b80 1 -- 172.21.15.79:0/2381557623 wait complete. 2024-06-03T02:29:10.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.008+0000 7f3b14789b80 1 -- start start 2024-06-03T02:29:10.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.008+0000 7f3b14789b80 1 Processor -- start 2024-06-03T02:29:10.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.008+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x556035018ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.008+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x556035015de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.008+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556035016320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.008+0000 7f3b14789b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x556034ef1f60 con 0x5560350123a0 2024-06-03T02:29:10.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.008+0000 7f3b14789b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556034f51ac0 con 0x556035012cc0 2024-06-03T02:29:10.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.008+0000 7f3b14789b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5560350196a0 con 0x55603500ca50 2024-06-03T02:29:10.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b14789b80 10 client.0 _ll_register_callbacks cb 0x556034eeb380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b02000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x556035015de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b02000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x556035015de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58792/0 (socket says 172.21.15.79:58792) 2024-06-03T02:29:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b02000700 1 -- 172.21.15.79:0/1115232373 learned_addr learned my addr 172.21.15.79:0/1115232373 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b03400700 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556035016320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b02000700 1 -- 172.21.15.79:0/1115232373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 msgr2=0x556035018ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:29:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b02000700 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x556035018ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b02000700 1 -- 172.21.15.79:0/1115232373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 msgr2=0x556035016320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b02000700 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556035016320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.009+0000 7f3b02000700 1 -- 172.21.15.79:0/1115232373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556034ea48d0 con 0x5560350123a0 2024-06-03T02:29:10.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.010+0000 7f3b02000700 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x556035015de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3af0012010 tx=0x7f3af00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d8dcf075c2bb48 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:10.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.010+0000 7f3aed600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:29:10.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.010+0000 7f3aed600700 1 -- 172.21.15.79:0/1115232373 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0022020 con 0x5560350123a0 2024-06-03T02:29:10.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.010+0000 7f3aed600700 1 -- 172.21.15.79:0/1115232373 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3af0012650 con 0x5560350123a0 2024-06-03T02:29:10.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.010+0000 7f3b14789b80 1 -- 172.21.15.79:0/1115232373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3afc015410 con 0x5560350123a0 2024-06-03T02:29:10.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.013+0000 7f3aed600700 1 -- 172.21.15.79:0/1115232373 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3af0012850 con 0x5560350123a0 2024-06-03T02:29:10.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.013+0000 7f3b14789b80 10 client.15921 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:10.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.013+0000 7f3b14789b80 1 -- 172.21.15.79:0/1115232373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556035023ed0 con 0x5560350123a0 2024-06-03T02:29:10.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.013+0000 7f3aed600700 1 -- 172.21.15.79:0/1115232373 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3afc015410 con 0x5560350123a0 2024-06-03T02:29:10.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.014+0000 7f3aed600700 1 -- 172.21.15.79:0/1115232373 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3af001ec80 con 0x5560350123a0 2024-06-03T02:29:10.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.014+0000 7f3b14789b80 10 client.15921 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:10.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.014+0000 7f3b14789b80 10 client.15921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:29:10.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.014+0000 7f3b14789b80 20 client.15921 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:10.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.015+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x556035026460 0x5560350288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.015+0000 7f3b14789b80 4 client.15921 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:29:10.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.015+0000 7f3b14789b80 1 -- 172.21.15.79:0/1115232373 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556034f63000 con 0x556035026460 2024-06-03T02:29:10.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.015+0000 7f3b02a00700 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x556035026460 0x5560350288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.015+0000 7f3aed600700 10 client.15921 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:29:10.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.015+0000 7f3b02a00700 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x556035026460 0x5560350288b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=786516f107f7b867 server_cookie=9bb3467c3ee680be in_seq=0 out_seq=0 2024-06-03T02:29:10.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.016+0000 7f3aed600700 1 -- 172.21.15.79:0/1115232373 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556034f63000 con 0x556035026460 2024-06-03T02:29:10.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.016+0000 7f3aed600700 10 client.15921 handle_command_reply: tid=0 2024-06-03T02:29:10.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.016+0000 7f3b14789b80 1 client.15921 shutdown 2024-06-03T02:29:10.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 -- 172.21.15.79:0/1115232373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 msgr2=0x556035015de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:10.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x556035015de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3af0012010 tx=0x7f3af00047c0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 -- 172.21.15.79:0/1115232373 shutdown_connections 2024-06-03T02:29:10.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556035016320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x556035026460 0x5560350288b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x556035018ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1115232373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x556035015de0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 -- 172.21.15.79:0/1115232373 >> 172.21.15.79:0/1115232373 conn(0x556034f30170 msgr2=0x556034ff8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:10.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3aed600700 0 client.15921 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:29:10.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 -- 172.21.15.79:0/1115232373 shutdown_connections 2024-06-03T02:29:10.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 1 -- 172.21.15.79:0/1115232373 wait complete. 2024-06-03T02:29:10.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.017+0000 7f3b14789b80 20 client.15921 trim_cache size 0 max 16384 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.042+0000 7f3b14789b80 1 Processor -- start 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.042+0000 7f3b14789b80 1 -- start start 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.042+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55603500ca50 0x55603500c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.042+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560350123a0 0x556034ff9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.042+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556034fff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.042+0000 7f3b14789b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041ac130 con 0x55603500ca50 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.042+0000 7f3b14789b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041adc70 con 0x556035012cc0 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.042+0000 7f3b14789b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041ac800 con 0x5560350123a0 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.042+0000 7f3b03400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556034fff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.043+0000 7f3b03400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556034fff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:58166/0 (socket says 172.21.15.79:58166) 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.043+0000 7f3b03400700 1 -- 172.21.15.79:0/2172793527 learned_addr learned my addr 172.21.15.79:0/2172793527 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:10.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.043+0000 7f3b02a00700 1 --2- 172.21.15.79:0/2172793527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55603500ca50 0x55603500c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.043+0000 7f3b03400700 1 -- 172.21.15.79:0/2172793527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560350123a0 msgr2=0x556034ff9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:10.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.043+0000 7f3b03400700 1 --2- 172.21.15.79:0/2172793527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560350123a0 0x556034ff9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.043+0000 7f3b03400700 1 -- 172.21.15.79:0/2172793527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55603500ca50 msgr2=0x55603500c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:10.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.043+0000 7f3b03400700 1 --2- 172.21.15.79:0/2172793527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55603500ca50 0x55603500c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.043+0000 7f3b03400700 1 -- 172.21.15.79:0/2172793527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556035010850 con 0x556035012cc0 2024-06-03T02:29:10.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.043+0000 7f3b03400700 1 --2- 172.21.15.79:0/2172793527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556034fff100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3afc018040 tx=0x7f3afc015db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d45a35598b6988a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:10.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.044+0000 7f3aefe00700 1 -- 172.21.15.79:0/2172793527 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc025070 con 0x556035012cc0 2024-06-03T02:29:10.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.044+0000 7f3aefe00700 1 -- 172.21.15.79:0/2172793527 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3afc015410 con 0x556035012cc0 2024-06-03T02:29:10.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.044+0000 7f3aefe00700 1 -- 172.21.15.79:0/2172793527 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc01a040 con 0x556035012cc0 2024-06-03T02:29:10.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 -- 172.21.15.79:0/2172793527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 msgr2=0x556034fff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:10.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 --2- 172.21.15.79:0/2172793527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556034fff100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3afc018040 tx=0x7f3afc015db0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 -- 172.21.15.79:0/2172793527 shutdown_connections 2024-06-03T02:29:10.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 --2- 172.21.15.79:0/2172793527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x556034fff100 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 --2- 172.21.15.79:0/2172793527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55603500ca50 0x55603500c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 --2- 172.21.15.79:0/2172793527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560350123a0 0x556034ff9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 -- 172.21.15.79:0/2172793527 >> 172.21.15.79:0/2172793527 conn(0x556034f30170 msgr2=0x5560350146d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:10.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 -- 172.21.15.79:0/2172793527 shutdown_connections 2024-06-03T02:29:10.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 -- 172.21.15.79:0/2172793527 wait complete. 2024-06-03T02:29:10.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.045+0000 7f3b14789b80 1 -- start start 2024-06-03T02:29:10.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b14789b80 1 Processor -- start 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603501ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x55603501f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b14789b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603502d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b14789b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041ad5a0 con 0x5560350123a0 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b14789b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041aced0 con 0x556035012cc0 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b14789b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041aa100 con 0x55603500ca50 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b14789b80 10 client.0 _ll_register_callbacks cb 0x556034eeb380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603501ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603501ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54518/0 (socket says 172.21.15.79:54518) 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.046+0000 7f3b02a00700 1 -- 172.21.15.79:0/1118639238 learned_addr learned my addr 172.21.15.79:0/1118639238 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3b03400700 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603502d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3b02a00700 1 -- 172.21.15.79:0/1118639238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 msgr2=0x55603502d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3b02a00700 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603502d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3b02a00700 1 -- 172.21.15.79:0/1118639238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 msgr2=0x55603501f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3b02a00700 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x55603501f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3b02a00700 1 -- 172.21.15.79:0/1118639238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55603502df90 con 0x55603500ca50 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3b03400700 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603502d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3b02a00700 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603501ebb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3af40cdda0 tx=0x7f3af4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=843f28400bdcb5b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3aed600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3aed600700 1 -- 172.21.15.79:0/1118639238 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40aa070 con 0x55603500ca50 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.047+0000 7f3aed600700 1 -- 172.21.15.79:0/1118639238 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3af409dd70 con 0x55603500ca50 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3b14789b80 1 -- 172.21.15.79:0/1118639238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3afc015410 con 0x55603500ca50 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3aed600700 1 -- 172.21.15.79:0/1118639238 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3af409db10 con 0x55603500ca50 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3b14789b80 10 client.26368 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:10.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3b14789b80 1 -- 172.21.15.79:0/1118639238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556035090cf0 con 0x55603500ca50 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3aed600700 1 -- 172.21.15.79:0/1118639238 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3afc015410 con 0x55603500ca50 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3aed600700 1 -- 172.21.15.79:0/1118639238 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3af409cf60 con 0x55603500ca50 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3b14789b80 10 client.26368 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3b14789b80 10 client.26368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3b14789b80 20 client.26368 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x556035091350 0x5560350937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3b14789b80 4 client.26368 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.048+0000 7f3b14789b80 1 -- 172.21.15.79:0/1118639238 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556034f5d580 con 0x556035091350 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.049+0000 7f3b02000700 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x556035091350 0x5560350937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:10.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.049+0000 7f3aed600700 10 client.26368 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.049+0000 7f3b02000700 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x556035091350 0x5560350937b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d363238dd5dd5a5e server_cookie=49e3044b86d332b6 in_seq=0 out_seq=0 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3aed600700 1 -- 172.21.15.79:0/1118639238 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556034f5d580 con 0x556035091350 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3aed600700 10 client.26368 handle_command_reply: tid=0 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 client.26368 shutdown 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 -- 172.21.15.79:0/1118639238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 msgr2=0x55603501ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603501ebb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3af40cdda0 tx=0x7f3af4009ca0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 -- 172.21.15.79:0/1118639238 shutdown_connections 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556035012cc0 0x55603502d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x556035091350 0x5560350937b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55603500ca50 0x55603501ebb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 --2- 172.21.15.79:0/1118639238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5560350123a0 0x55603501f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 -- 172.21.15.79:0/1118639238 >> 172.21.15.79:0/1118639238 conn(0x556034f30170 msgr2=0x556035013900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:10.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3aed600700 0 client.26368 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:29:10.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 -- 172.21.15.79:0/1118639238 shutdown_connections 2024-06-03T02:29:10.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.050+0000 7f3b14789b80 1 -- 172.21.15.79:0/1118639238 wait complete. 2024-06-03T02:29:10.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b14789b80 20 client.26368 trim_cache size 0 max 16384 2024-06-03T02:29:10.055 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:29:10.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 -- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3ae80a2d70 msgr2=0x7f3ae80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:10.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3ae80a2d70 0x7f3ae80a5230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3af0005fd0 tx=0x7f3af00116c0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 -- 172.21.15.79:0/2812150404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b04102740 msgr2=0x7f3b0419ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:10.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b04102740 0x7f3b0419ae80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3af4000c00 tx=0x7f3af40038b0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 -- 172.21.15.79:0/2812150404 shutdown_connections 2024-06-03T02:29:10.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b04107840 0x7f3b0419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b04102740 0x7f3b0419ae80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3ae80a2d70 0x7f3ae80a5230 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 --2- 172.21.15.79:0/2812150404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b041030f0 0x7f3b0419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:10.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.051+0000 7f3b09e00700 1 -- 172.21.15.79:0/2812150404 >> 172.21.15.79:0/2812150404 conn(0x7f3b040795f0 msgr2=0x7f3b04100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:10.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.052+0000 7f3b09e00700 1 -- 172.21.15.79:0/2812150404 shutdown_connections 2024-06-03T02:29:10.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:10.052+0000 7f3b09e00700 1 -- 172.21.15.79:0/2812150404 wait complete. 2024-06-03T02:29:10.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:10 smithi079 ceph-mon[31819]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.8 MiB/s wr, 631 op/s 2024-06-03T02:29:10.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:10 smithi135 ceph-mon[36653]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.8 MiB/s wr, 631 op/s 2024-06-03T02:29:10.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:10 smithi123 ceph-mon[36640]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.8 MiB/s wr, 631 op/s 2024-06-03T02:29:12.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:12 smithi079 ceph-mon[31819]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.8 MiB/s wr, 508 op/s 2024-06-03T02:29:12.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:12 smithi135 ceph-mon[36653]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.8 MiB/s wr, 508 op/s 2024-06-03T02:29:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:12 smithi123 ceph-mon[36640]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.8 MiB/s wr, 508 op/s 2024-06-03T02:29:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 545.0 s, 442.4 tps, lat 226.487 ms stddev 279.822 2024-06-03T02:29:13.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:29:14.265 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:29:14.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:14 smithi079 ceph-mon[31819]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-06-03T02:29:14.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:14 smithi135 ceph-mon[36653]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-06-03T02:29:14.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:14 smithi123 ceph-mon[36640]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-06-03T02:29:16.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:16 smithi079 ceph-mon[31819]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-06-03T02:29:16.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:16 smithi135 ceph-mon[36653]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-06-03T02:29:16.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:16 smithi123 ceph-mon[36640]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-06-03T02:29:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 550.0 s, 418.0 tps, lat 244.446 ms stddev 277.971 2024-06-03T02:29:18.677 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:18 smithi079 ceph-mon[31819]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-06-03T02:29:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:18 smithi135 ceph-mon[36653]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-06-03T02:29:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:18 smithi123 ceph-mon[36640]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-06-03T02:29:19.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.598+0000 7f2660600700 1 -- 172.21.15.79:0/3943477521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f265810ef40 msgr2=0x7f2658111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:19.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.598+0000 7f2660600700 1 --2- 172.21.15.79:0/3943477521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f265810ef40 0x7f2658111330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f265000b0a0 tx=0x7f265000b3b0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.601+0000 7f2660600700 1 -- 172.21.15.79:0/3943477521 shutdown_connections 2024-06-03T02:29:19.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.601+0000 7f2660600700 1 --2- 172.21.15.79:0/3943477521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f265810ef40 0x7f2658111330 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.601+0000 7f2660600700 1 --2- 172.21.15.79:0/3943477521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f265806a1a0 0x7f265806a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.601+0000 7f2660600700 1 --2- 172.21.15.79:0/3943477521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265806aac0 0x7f265810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.601+0000 7f2660600700 1 -- 172.21.15.79:0/3943477521 >> 172.21.15.79:0/3943477521 conn(0x7f2658078340 msgr2=0x7f2658078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:19.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.602+0000 7f2660600700 1 -- 172.21.15.79:0/3943477521 shutdown_connections 2024-06-03T02:29:19.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.602+0000 7f2660600700 1 -- 172.21.15.79:0/3943477521 wait complete. 2024-06-03T02:29:19.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.603+0000 7f2660600700 1 Processor -- start 2024-06-03T02:29:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.603+0000 7f2660600700 1 -- start start 2024-06-03T02:29:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.604+0000 7f2660600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f265806a1a0 0x7f2658103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.604+0000 7f2660600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f265806aac0 0x7f2658073d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.604+0000 7f2660600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265810ef40 0x7f2658103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.604+0000 7f2660600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f265806c8c0 con 0x7f265810ef40 2024-06-03T02:29:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.604+0000 7f2660600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f265806c740 con 0x7f265806a1a0 2024-06-03T02:29:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.604+0000 7f2660600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f265806ca40 con 0x7f265806aac0 2024-06-03T02:29:19.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.605+0000 7f265dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f265806a1a0 0x7f2658103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.605+0000 7f265dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f265806a1a0 0x7f2658103670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40376/0 (socket says 172.21.15.79:40376) 2024-06-03T02:29:19.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.605+0000 7f265dc00700 1 -- 172.21.15.79:0/1148557276 learned_addr learned my addr 172.21.15.79:0/1148557276 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:19.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.605+0000 7f265e600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265810ef40 0x7f2658103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.605+0000 7f265dc00700 1 -- 172.21.15.79:0/1148557276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f265806aac0 msgr2=0x7f2658073d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:29:19.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.605+0000 7f265dc00700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f265806aac0 0x7f2658073d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.605+0000 7f265dc00700 1 -- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265810ef40 msgr2=0x7f2658103bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:19.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.605+0000 7f265dc00700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265810ef40 0x7f2658103bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.605+0000 7f265dc00700 1 -- 172.21.15.79:0/1148557276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2658074410 con 0x7f265806a1a0 2024-06-03T02:29:19.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.606+0000 7f265e600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265810ef40 0x7f2658103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:29:19.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.606+0000 7f265dc00700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f265806a1a0 0x7f2658103670 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f264c00eb30 tx=0x7f264c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:19.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.606+0000 7f2656000700 1 -- 172.21.15.79:0/1148557276 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c00cd40 con 0x7f265806a1a0 2024-06-03T02:29:19.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.606+0000 7f2656000700 1 -- 172.21.15.79:0/1148557276 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264c004500 con 0x7f265806a1a0 2024-06-03T02:29:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.606+0000 7f2656000700 1 -- 172.21.15.79:0/1148557276 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c010430 con 0x7f265806a1a0 2024-06-03T02:29:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.606+0000 7f2660600700 1 -- 172.21.15.79:0/1148557276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2658104150 con 0x7f265806a1a0 2024-06-03T02:29:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.607+0000 7f2660600700 1 -- 172.21.15.79:0/1148557276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581a9d10 con 0x7f265806a1a0 2024-06-03T02:29:19.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.612+0000 7f2656000700 1 -- 172.21.15.79:0/1148557276 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f264c004000 con 0x7f265806a1a0 2024-06-03T02:29:19.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.613+0000 7f266afb1b80 1 Processor -- start 2024-06-03T02:29:19.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.613+0000 7f266afb1b80 1 -- start start 2024-06-03T02:29:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.614+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 0x5607cde65f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.614+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 0x5607cde5fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde60680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f266afb1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5607cdd6ea10 con 0x5607cde664b0 2024-06-03T02:29:19.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f266afb1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607cddb1b00 con 0x5607cde60240 2024-06-03T02:29:19.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f266afb1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5607cdd6c8f0 con 0x5607cde65b90 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f265d200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f265d200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde60680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40392/0 (socket says 172.21.15.79:40392) 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f265d200700 1 -- 172.21.15.79:0/1454534982 learned_addr learned my addr 172.21.15.79:0/1454534982 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f265d200700 1 -- 172.21.15.79:0/1454534982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 msgr2=0x5607cde65f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f265d200700 1 --2- 172.21.15.79:0/1454534982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 0x5607cde65f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f265d200700 1 -- 172.21.15.79:0/1454534982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 msgr2=0x5607cde5fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f265d200700 1 --2- 172.21.15.79:0/1454534982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 0x5607cde5fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.615+0000 7f265d200700 1 -- 172.21.15.79:0/1454534982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607cdd59f90 con 0x5607cde60240 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.616+0000 7f265d200700 1 --2- 172.21.15.79:0/1454534982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde60680 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2644009b50 tx=0x7f2644009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f07f81e6c45a563 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:19.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.616+0000 7f264aa00700 1 -- 172.21.15.79:0/1454534982 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264400eae0 con 0x5607cde60240 2024-06-03T02:29:19.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.616+0000 7f2656000700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26400a2d20 0x7f26400a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.616+0000 7f264aa00700 1 -- 172.21.15.79:0/1454534982 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2644004d10 con 0x5607cde60240 2024-06-03T02:29:19.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.616+0000 7f264aa00700 1 -- 172.21.15.79:0/1454534982 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644010430 con 0x5607cde60240 2024-06-03T02:29:19.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.616+0000 7f2656000700 1 -- 172.21.15.79:0/1148557276 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f264c0cf7c0 con 0x7f265806a1a0 2024-06-03T02:29:19.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.617+0000 7f265e600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26400a2d20 0x7f26400a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.617+0000 7f266afb1b80 1 -- 172.21.15.79:0/1454534982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 msgr2=0x5607cde60680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.617+0000 7f266afb1b80 1 --2- 172.21.15.79:0/1454534982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde60680 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2644009b50 tx=0x7f2644009e60 comp rx=0 tx=0).stop 2024-06-03T02:29:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.617+0000 7f266afb1b80 1 -- 172.21.15.79:0/1454534982 shutdown_connections 2024-06-03T02:29:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.617+0000 7f266afb1b80 1 --2- 172.21.15.79:0/1454534982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde60680 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.617+0000 7f266afb1b80 1 --2- 172.21.15.79:0/1454534982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 0x5607cde65f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.617+0000 7f266afb1b80 1 --2- 172.21.15.79:0/1454534982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 0x5607cde5fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.617+0000 7f266afb1b80 1 -- 172.21.15.79:0/1454534982 >> 172.21.15.79:0/1454534982 conn(0x5607cdd8c170 msgr2=0x5607cde5aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:19.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.618+0000 7f266afb1b80 1 -- 172.21.15.79:0/1454534982 shutdown_connections 2024-06-03T02:29:19.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.618+0000 7f266afb1b80 1 -- 172.21.15.79:0/1454534982 wait complete. 2024-06-03T02:29:19.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.618+0000 7f266afb1b80 1 -- start start 2024-06-03T02:29:19.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.619+0000 7f265e600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26400a2d20 0x7f26400a51e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2650009e60 tx=0x7f265000b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:19.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.619+0000 7f266afb1b80 1 Processor -- start 2024-06-03T02:29:19.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.619+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 0x5607cde6c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.619+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 0x5607cde695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.619+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde69ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.619+0000 7f266afb1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5607cdd4df60 con 0x5607cde65b90 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.619+0000 7f266afb1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607cddadac0 con 0x5607cde664b0 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.619+0000 7f266afb1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5607cde6ce90 con 0x5607cde60240 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.619+0000 7f266afb1b80 10 client.0 _ll_register_callbacks cb 0x5607cdd47380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265e600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde69ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265dc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 0x5607cde6c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265e600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde69ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40404/0 (socket says 172.21.15.79:40404) 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265e600700 1 -- 172.21.15.79:0/2543533158 learned_addr learned my addr 172.21.15.79:0/2543533158 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265d200700 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 0x5607cde695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265dc00700 1 -- 172.21.15.79:0/2543533158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 msgr2=0x5607cde69ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265dc00700 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde69ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265dc00700 1 -- 172.21.15.79:0/2543533158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 msgr2=0x5607cde695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265dc00700 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 0x5607cde695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265dc00700 1 -- 172.21.15.79:0/2543533158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607cdd008d0 con 0x5607cde60240 2024-06-03T02:29:19.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.620+0000 7f265d200700 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 0x5607cde695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:29:19.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.621+0000 7f265dc00700 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 0x5607cde6c250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f264c09bba0 tx=0x7f264c09c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fcaf387cb4ac8a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:19.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.621+0000 7f263f400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.621+0000 7f263f400700 1 -- 172.21.15.79:0/2543533158 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c09ffc0 con 0x5607cde60240 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.621+0000 7f263f400700 1 -- 172.21.15.79:0/2543533158 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f264c01fde0 con 0x5607cde60240 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.621+0000 7f266afb1b80 1 -- 172.21.15.79:0/2543533158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2644004d10 con 0x5607cde60240 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.621+0000 7f263f400700 1 -- 172.21.15.79:0/2543533158 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f264c0a4880 con 0x5607cde60240 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.621+0000 7f266afb1b80 10 client.26374 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.621+0000 7f266afb1b80 1 -- 172.21.15.79:0/2543533158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607cde776d0 con 0x5607cde60240 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.622+0000 7f263f400700 1 -- 172.21.15.79:0/2543533158 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2644004d10 con 0x5607cde60240 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.622+0000 7f263f400700 1 -- 172.21.15.79:0/2543533158 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f264c0a4480 con 0x5607cde60240 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.623+0000 7f266afb1b80 10 client.26374 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.623+0000 7f266afb1b80 10 client.26374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.623+0000 7f266afb1b80 20 client.26374 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.623+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5607cde79ef0 0x5607cde7c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.623+0000 7f266afb1b80 4 client.26374 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:29:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.623+0000 7f266afb1b80 1 -- 172.21.15.79:0/2543533158 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607cddbf000 con 0x5607cde79ef0 2024-06-03T02:29:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.623+0000 7f265d200700 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5607cde79ef0 0x5607cde7c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.624+0000 7f265d200700 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5607cde79ef0 0x5607cde7c2e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61fab20c1b954837 server_cookie=232565594bac7e13 in_seq=0 out_seq=0 2024-06-03T02:29:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.624+0000 7f263f400700 10 client.26374 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:19.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.625+0000 7f263f400700 1 -- 172.21.15.79:0/2543533158 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607cddbf000 con 0x5607cde79ef0 2024-06-03T02:29:19.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.625+0000 7f263f400700 10 client.26374 handle_command_reply: tid=0 2024-06-03T02:29:19.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.627+0000 7f266afb1b80 1 client.26374 shutdown 2024-06-03T02:29:19.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.629+0000 7f266afb1b80 1 -- 172.21.15.79:0/2543533158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 msgr2=0x5607cde6c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:19.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.629+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 0x5607cde6c250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f264c09bba0 tx=0x7f264c09c310 comp rx=0 tx=0).stop 2024-06-03T02:29:19.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.629+0000 7f266afb1b80 1 -- 172.21.15.79:0/2543533158 shutdown_connections 2024-06-03T02:29:19.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.629+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde69ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.629+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 0x5607cde6c250 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.629+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5607cde79ef0 0x5607cde7c2e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.629+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2543533158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 0x5607cde695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.629+0000 7f266afb1b80 1 -- 172.21.15.79:0/2543533158 >> 172.21.15.79:0/2543533158 conn(0x5607cdd8c170 msgr2=0x5607cde54790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:19.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.632+0000 7f263f400700 0 client.26374 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:19.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.632+0000 7f266afb1b80 1 -- 172.21.15.79:0/2543533158 shutdown_connections 2024-06-03T02:29:19.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.633+0000 7f266afb1b80 1 -- 172.21.15.79:0/2543533158 wait complete. 2024-06-03T02:29:19.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.633+0000 7f266afb1b80 20 client.26374 trim_cache size 0 max 16384 2024-06-03T02:29:19.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.662+0000 7f266afb1b80 1 Processor -- start 2024-06-03T02:29:19.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.662+0000 7f266afb1b80 1 -- start start 2024-06-03T02:29:19.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.663+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 0x5607cde5fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.663+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 0x5607cde55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.663+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde68720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.663+0000 7f266afb1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f26581c5bc0 con 0x5607cde65b90 2024-06-03T02:29:19.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.663+0000 7f266afb1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26581c7700 con 0x5607cde664b0 2024-06-03T02:29:19.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.663+0000 7f266afb1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26581c6290 con 0x5607cde60240 2024-06-03T02:29:19.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.664+0000 7f265e600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde68720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.664+0000 7f265e600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde68720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40414/0 (socket says 172.21.15.79:40414) 2024-06-03T02:29:19.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.664+0000 7f265e600700 1 -- 172.21.15.79:0/3931416408 learned_addr learned my addr 172.21.15.79:0/3931416408 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:19.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.664+0000 7f265e600700 1 -- 172.21.15.79:0/3931416408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 msgr2=0x5607cde5fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:19.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.664+0000 7f265e600700 1 --2- 172.21.15.79:0/3931416408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 0x5607cde5fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.664+0000 7f265e600700 1 -- 172.21.15.79:0/3931416408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 msgr2=0x5607cde55130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:19.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.664+0000 7f265e600700 1 --2- 172.21.15.79:0/3931416408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 0x5607cde55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.664+0000 7f265e600700 1 -- 172.21.15.79:0/3931416408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607cde6a4d0 con 0x5607cde664b0 2024-06-03T02:29:19.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.665+0000 7f265e600700 1 --2- 172.21.15.79:0/3931416408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde68720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f265000bbe0 tx=0x7f265001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150b94078218f7e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:19.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.665+0000 7f264a000700 1 -- 172.21.15.79:0/3931416408 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265002e030 con 0x5607cde664b0 2024-06-03T02:29:19.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.665+0000 7f264a000700 1 -- 172.21.15.79:0/3931416408 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2650022070 con 0x5607cde664b0 2024-06-03T02:29:19.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.666+0000 7f264a000700 1 -- 172.21.15.79:0/3931416408 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265002a6e0 con 0x5607cde664b0 2024-06-03T02:29:19.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.667+0000 7f266afb1b80 1 -- 172.21.15.79:0/3931416408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 msgr2=0x5607cde68720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:19.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.667+0000 7f266afb1b80 1 --2- 172.21.15.79:0/3931416408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde68720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f265000bbe0 tx=0x7f265001b000 comp rx=0 tx=0).stop 2024-06-03T02:29:19.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.667+0000 7f266afb1b80 1 -- 172.21.15.79:0/3931416408 shutdown_connections 2024-06-03T02:29:19.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.667+0000 7f266afb1b80 1 --2- 172.21.15.79:0/3931416408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde664b0 0x5607cde68720 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.667+0000 7f266afb1b80 1 --2- 172.21.15.79:0/3931416408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde60240 0x5607cde5fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.667+0000 7f266afb1b80 1 --2- 172.21.15.79:0/3931416408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde65b90 0x5607cde55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.667+0000 7f266afb1b80 1 -- 172.21.15.79:0/3931416408 >> 172.21.15.79:0/3931416408 conn(0x5607cdd8c170 msgr2=0x5607cde66fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:19.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.667+0000 7f266afb1b80 1 -- 172.21.15.79:0/3931416408 shutdown_connections 2024-06-03T02:29:19.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.667+0000 7f266afb1b80 1 -- 172.21.15.79:0/3931416408 wait complete. 2024-06-03T02:29:19.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.668+0000 7f266afb1b80 1 -- start start 2024-06-03T02:29:19.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.668+0000 7f266afb1b80 1 Processor -- start 2024-06-03T02:29:19.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.669+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.669+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 0x5607cde7cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.669+0000 7f266afb1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 0x5607cdee2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.669+0000 7f266afb1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f26581c7030 con 0x5607cde664b0 2024-06-03T02:29:19.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.669+0000 7f266afb1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26581c6960 con 0x5607cde60240 2024-06-03T02:29:19.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.669+0000 7f266afb1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26581b0b00 con 0x5607cde65b90 2024-06-03T02:29:19.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.669+0000 7f266afb1b80 10 client.0 _ll_register_callbacks cb 0x5607cdd47380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:19.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.669+0000 7f265dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265e600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 0x5607cdee2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265d200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 0x5607cde7cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265e600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 0x5607cdee2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52310/0 (socket says 172.21.15.79:52310) 2024-06-03T02:29:19.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265d200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 0x5607cde7cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:48276/0 (socket says 172.21.15.79:48276) 2024-06-03T02:29:19.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265e600700 1 -- 172.21.15.79:0/2921528999 learned_addr learned my addr 172.21.15.79:0/2921528999 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:19.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265e600700 1 -- 172.21.15.79:0/2921528999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 msgr2=0x5607cde7cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:19.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265e600700 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 0x5607cde7cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265e600700 1 -- 172.21.15.79:0/2921528999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 msgr2=0x5607cde7c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:19.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265e600700 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde7c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.670+0000 7f265e600700 1 -- 172.21.15.79:0/2921528999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607cdee2d60 con 0x5607cde664b0 2024-06-03T02:29:19.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.671+0000 7f265e600700 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 0x5607cdee2820 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2650007b40 tx=0x7f2650007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332297bf7ba7939b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:19.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.671+0000 7f263f400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:29:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.673+0000 7f263f400700 1 -- 172.21.15.79:0/2921528999 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650037070 con 0x5607cde664b0 2024-06-03T02:29:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.673+0000 7f263f400700 1 -- 172.21.15.79:0/2921528999 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2650007560 con 0x5607cde664b0 2024-06-03T02:29:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.673+0000 7f266afb1b80 1 -- 172.21.15.79:0/2921528999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2650022070 con 0x5607cde664b0 2024-06-03T02:29:19.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.674+0000 7f263f400700 1 -- 172.21.15.79:0/2921528999 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f265002e040 con 0x5607cde664b0 2024-06-03T02:29:19.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.674+0000 7f266afb1b80 10 client.15933 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:19.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.674+0000 7f266afb1b80 1 -- 172.21.15.79:0/2921528999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607cdee4030 con 0x5607cde664b0 2024-06-03T02:29:19.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.675+0000 7f263f400700 1 -- 172.21.15.79:0/2921528999 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2650022070 con 0x5607cde664b0 2024-06-03T02:29:19.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.675+0000 7f263f400700 1 -- 172.21.15.79:0/2921528999 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2650027050 con 0x5607cde664b0 2024-06-03T02:29:19.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.676+0000 7f266afb1b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:19.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.676+0000 7f266afb1b80 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:29:19.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.676+0000 7f266afb1b80 20 client.15933 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:19.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.676+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5607cdee4610 0x5607cdee6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:19.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.676+0000 7f266afb1b80 4 client.15933 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:29:19.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.676+0000 7f266afb1b80 1 -- 172.21.15.79:0/2921528999 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607cddb9580 con 0x5607cdee4610 2024-06-03T02:29:19.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.676+0000 7f265dc00700 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5607cdee4610 0x5607cdee6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:19.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.677+0000 7f265dc00700 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5607cdee4610 0x5607cdee6a30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=573a95c5b194fd4f server_cookie=f8afdc32e08790f8 in_seq=0 out_seq=0 2024-06-03T02:29:19.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.678+0000 7f263f400700 10 client.15933 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:19.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.678+0000 7f263f400700 1 -- 172.21.15.79:0/2921528999 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607cddb9580 con 0x5607cdee4610 2024-06-03T02:29:19.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.678+0000 7f263f400700 10 client.15933 handle_command_reply: tid=0 2024-06-03T02:29:19.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.678+0000 7f266afb1b80 1 client.15933 shutdown 2024-06-03T02:29:19.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.682+0000 7f266afb1b80 1 -- 172.21.15.79:0/2921528999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 msgr2=0x5607cdee2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:19.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.682+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 0x5607cdee2820 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2650007b40 tx=0x7f2650007b70 comp rx=0 tx=0).stop 2024-06-03T02:29:19.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.682+0000 7f266afb1b80 1 -- 172.21.15.79:0/2921528999 shutdown_connections 2024-06-03T02:29:19.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.682+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5607cdee4610 0x5607cdee6a30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.682+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607cde664b0 0x5607cdee2820 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.682+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607cde60240 0x5607cde7c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.682+0000 7f266afb1b80 1 --2- 172.21.15.79:0/2921528999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607cde65b90 0x5607cde7cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.682+0000 7f266afb1b80 1 -- 172.21.15.79:0/2921528999 >> 172.21.15.79:0/2921528999 conn(0x5607cdd8c170 msgr2=0x5607cde64ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:19.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.682+0000 7f263f400700 0 client.15933 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:19.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.683+0000 7f266afb1b80 1 -- 172.21.15.79:0/2921528999 shutdown_connections 2024-06-03T02:29:19.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.684+0000 7f266afb1b80 1 -- 172.21.15.79:0/2921528999 wait complete. 2024-06-03T02:29:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.685+0000 7f266afb1b80 20 client.15933 trim_cache size 0 max 16384 2024-06-03T02:29:19.687 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:29:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 -- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26400a2d20 msgr2=0x7f26400a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26400a2d20 0x7f26400a51e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2650009e60 tx=0x7f265000b740 comp rx=0 tx=0).stop 2024-06-03T02:29:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 -- 172.21.15.79:0/1148557276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f265806a1a0 msgr2=0x7f2658103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f265806a1a0 0x7f2658103670 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f264c00eb30 tx=0x7f264c00eef0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 -- 172.21.15.79:0/1148557276 shutdown_connections 2024-06-03T02:29:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f26400a2d20 0x7f26400a51e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265810ef40 0x7f2658103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f265806a1a0 0x7f2658103670 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 --2- 172.21.15.79:0/1148557276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f265806aac0 0x7f2658073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:19.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.687+0000 7f2660600700 1 -- 172.21.15.79:0/1148557276 >> 172.21.15.79:0/1148557276 conn(0x7f2658078340 msgr2=0x7f26581102b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:19.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.688+0000 7f2660600700 1 -- 172.21.15.79:0/1148557276 shutdown_connections 2024-06-03T02:29:19.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:19.688+0000 7f2660600700 1 -- 172.21.15.79:0/1148557276 wait complete. 2024-06-03T02:29:20.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:20 smithi079 ceph-mon[31819]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-03T02:29:20.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:20 smithi135 ceph-mon[36653]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-03T02:29:20.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:20 smithi123 ceph-mon[36640]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-03T02:29:21.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:21 smithi079 ceph-mon[31819]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-06-03T02:29:21.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:21 smithi135 ceph-mon[36653]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-06-03T02:29:21.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:21 smithi123 ceph-mon[36640]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-06-03T02:29:21.982 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:29:21.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:29:22.845 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:29:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 555.0 s, 489.0 tps, lat 208.289 ms stddev 233.803 2024-06-03T02:29:24.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:23 smithi079 ceph-mon[31819]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 443 op/s 2024-06-03T02:29:24.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:23 smithi135 ceph-mon[36653]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 443 op/s 2024-06-03T02:29:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:23 smithi123 ceph-mon[36640]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 443 op/s 2024-06-03T02:29:26.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:25 smithi079 ceph-mon[31819]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-06-03T02:29:26.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:25 smithi135 ceph-mon[36653]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-06-03T02:29:26.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:25 smithi123 ceph-mon[36640]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-06-03T02:29:27.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:27 smithi079 ceph-mon[31819]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.6 MiB/s wr, 404 op/s 2024-06-03T02:29:28.025 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:27 smithi123 ceph-mon[36640]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.6 MiB/s wr, 404 op/s 2024-06-03T02:29:28.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:27 smithi135 ceph-mon[36653]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.6 MiB/s wr, 404 op/s 2024-06-03T02:29:28.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 560.0 s, 480.8 tps, lat 189.684 ms stddev 211.804 2024-06-03T02:29:28.771 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:29:28.771 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:29:28.771 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "21ac4df9-c4da-46e6-b8f4-9d147259f5e9", 2024-06-03T02:29:28.771 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:29:28.771 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.221+0000 7f2ed4e00700 1 -- 172.21.15.79:0/1787238735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 msgr2=0x7f2ed0108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.221+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/1787238735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 0x7f2ed0108a30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009990 tx=0x7f2eb8009ca0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.222+0000 7f2ed4e00700 1 -- 172.21.15.79:0/1787238735 shutdown_connections 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.222+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/1787238735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed0103170 0x7f2ed010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.222+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/1787238735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 0x7f2ed0108a30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.222+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/1787238735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ed01027b0 0x7f2ed0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.222+0000 7f2ed4e00700 1 -- 172.21.15.79:0/1787238735 >> 172.21.15.79:0/1787238735 conn(0x7f2ed007a420 msgr2=0x7f2ed007a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.222+0000 7f2ed4e00700 1 -- 172.21.15.79:0/1787238735 shutdown_connections 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.222+0000 7f2ed4e00700 1 -- 172.21.15.79:0/1787238735 wait complete. 2024-06-03T02:29:28.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.224+0000 7f2ed4e00700 1 Processor -- start 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.225+0000 7f2ed4e00700 1 -- start start 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.226+0000 7f2ed4e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ed01027b0 0x7f2ed0073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.226+0000 7f2ed4e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed0103170 0x7f2ed0074030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.226+0000 7f2ed4e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 0x7f2ed006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.226+0000 7f2ed4e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0076290 con 0x7f2ed01027b0 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.226+0000 7f2ed4e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0076110 con 0x7f2ed0108650 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.226+0000 7f2ed4e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0076410 con 0x7f2ed0103170 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.226+0000 7f2ecde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ed01027b0 0x7f2ed0073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.226+0000 7f2ece800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 0x7f2ed006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.226+0000 7f2ecd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed0103170 0x7f2ed0074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.227+0000 7f2ece800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 0x7f2ed006cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41058/0 (socket says 172.21.15.79:41058) 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.227+0000 7f2ecd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed0103170 0x7f2ed0074030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33636/0 (socket says 172.21.15.79:33636) 2024-06-03T02:29:28.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.227+0000 7f2ece800700 1 -- 172.21.15.79:0/3099921188 learned_addr learned my addr 172.21.15.79:0/3099921188 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.227+0000 7f2ece800700 1 -- 172.21.15.79:0/3099921188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed0103170 msgr2=0x7f2ed0074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.227+0000 7f2ece800700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed0103170 0x7f2ed0074030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.227+0000 7f2ece800700 1 -- 172.21.15.79:0/3099921188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ed01027b0 msgr2=0x7f2ed0073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.227+0000 7f2ece800700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ed01027b0 0x7f2ed0073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.227+0000 7f2ece800700 1 -- 172.21.15.79:0/3099921188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed006d5d0 con 0x7f2ed0108650 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.228+0000 7f2ece800700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 0x7f2ed006cd10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000e970 tx=0x7f2ec000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.228+0000 7f2ec6a00700 1 -- 172.21.15.79:0/3099921188 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000cbd0 con 0x7f2ed0108650 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.228+0000 7f2ec6a00700 1 -- 172.21.15.79:0/3099921188 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec000cd70 con 0x7f2ed0108650 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.228+0000 7f2ed4e00700 1 -- 172.21.15.79:0/3099921188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed006d8c0 con 0x7f2ed0108650 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.228+0000 7f2ed4e00700 1 -- 172.21.15.79:0/3099921188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed00717f0 con 0x7f2ed0108650 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.228+0000 7f2ec6a00700 1 -- 172.21.15.79:0/3099921188 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0005660 con 0x7f2ed0108650 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.244+0000 7f2ec6a00700 1 -- 172.21.15.79:0/3099921188 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2ec0004830 con 0x7f2ed0108650 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.245+0000 7f2edf7c7b80 1 Processor -- start 2024-06-03T02:29:28.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.247+0000 7f2edf7c7b80 1 -- start start 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.248+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca664c10 0x5649ca664ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.248+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca65ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.248+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca65f2c0 0x5649ca65f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.248+0000 7f2edf7c7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5649ca56da90 con 0x5649ca664c10 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.248+0000 7f2edf7c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649ca5b0b80 con 0x5649ca665530 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.248+0000 7f2edf7c7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5649ca56b970 con 0x5649ca65f2c0 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.248+0000 7f2ecde00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca65ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.248+0000 7f2ecde00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca65ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41074/0 (socket says 172.21.15.79:41074) 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.248+0000 7f2ecde00700 1 -- 172.21.15.79:0/3269742139 learned_addr learned my addr 172.21.15.79:0/3269742139 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.249+0000 7f2ecde00700 1 -- 172.21.15.79:0/3269742139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca65f2c0 msgr2=0x5649ca65f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.249+0000 7f2ecd400700 1 --2- 172.21.15.79:0/3269742139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca65f2c0 0x5649ca65f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.249+0000 7f2ecde00700 1 --2- 172.21.15.79:0/3269742139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca65f2c0 0x5649ca65f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.249+0000 7f2ecde00700 1 -- 172.21.15.79:0/3269742139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca664c10 msgr2=0x5649ca664ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.249+0000 7f2ecde00700 1 --2- 172.21.15.79:0/3269742139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca664c10 0x5649ca664ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.249+0000 7f2ecde00700 1 -- 172.21.15.79:0/3269742139 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ca559010 con 0x5649ca665530 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.249+0000 7f2ecde00700 1 --2- 172.21.15.79:0/3269742139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca65ed80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8005830 tx=0x7f2eb8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74fd66377c8fb2ca server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.250+0000 7f2ecd400700 1 --2- 172.21.15.79:0/3269742139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca65f2c0 0x5649ca65f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.250+0000 7f2ebf400700 1 -- 172.21.15.79:0/3269742139 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800bd10 con 0x5649ca665530 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.250+0000 7f2ebf400700 1 -- 172.21.15.79:0/3269742139 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb8003680 con 0x5649ca665530 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.250+0000 7f2ebf400700 1 -- 172.21.15.79:0/3269742139 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800f6c0 con 0x5649ca665530 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.250+0000 7f2ec6a00700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2eb40a2d70 0x7f2eb40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.251+0000 7f2ec6a00700 1 -- 172.21.15.79:0/3099921188 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2ec00cf470 con 0x7f2ed0108650 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.251+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/3269742139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 msgr2=0x5649ca65ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.251+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/3269742139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca65ed80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8005830 tx=0x7f2eb8005c00 comp rx=0 tx=0).stop 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.251+0000 7f2ecd400700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2eb40a2d70 0x7f2eb40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.252+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/3269742139 shutdown_connections 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.252+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/3269742139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca65f2c0 0x5649ca65f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.252+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/3269742139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca664c10 0x5649ca664ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.252+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/3269742139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca65ed80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.252+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/3269742139 >> 172.21.15.79:0/3269742139 conn(0x5649ca58b1f0 msgr2=0x5649ca65a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.253+0000 7f2ecd400700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2eb40a2d70 0x7f2eb40a5230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8004b10 tx=0x7f2ec8009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.254+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/3269742139 shutdown_connections 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.254+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/3269742139 wait complete. 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.255+0000 7f2edf7c7b80 1 -- start start 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.255+0000 7f2edf7c7b80 1 Processor -- start 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.256+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 0x5649ca66b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.256+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca664c10 0x5649ca668500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.257+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca668a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.257+0000 7f2edf7c7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5649ca54cfe0 con 0x5649ca65f2c0 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.257+0000 7f2edf7c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649ca5acb40 con 0x5649ca664c10 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.257+0000 7f2edf7c7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5649ca66be50 con 0x5649ca665530 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.257+0000 7f2edf7c7b80 10 client.0 _ll_register_callbacks cb 0x5649ca546400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:28.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.257+0000 7f2ecd400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca664c10 0x5649ca668500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.257+0000 7f2ecd400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca664c10 0x5649ca668500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41078/0 (socket says 172.21.15.79:41078) 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.257+0000 7f2ecd400700 1 -- 172.21.15.79:0/2506019257 learned_addr learned my addr 172.21.15.79:0/2506019257 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.257+0000 7f2ece800700 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca668a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.258+0000 7f2ece800700 1 -- 172.21.15.79:0/2506019257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca664c10 msgr2=0x5649ca668500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.258+0000 7f2ece800700 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca664c10 0x5649ca668500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.258+0000 7f2ece800700 1 -- 172.21.15.79:0/2506019257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 msgr2=0x5649ca66b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.258+0000 7f2ece800700 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 0x5649ca66b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.258+0000 7f2ece800700 1 -- 172.21.15.79:0/2506019257 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ca4ff950 con 0x5649ca665530 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.259+0000 7f2ece800700 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca668a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00ccbc0 tx=0x7f2ec009b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607f3516fcf1873a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.262+0000 7f2eb3e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.262+0000 7f2eb3e00700 1 -- 172.21.15.79:0/2506019257 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00b3020 con 0x5649ca665530 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.262+0000 7f2eb3e00700 1 -- 172.21.15.79:0/2506019257 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2ec00a4100 con 0x5649ca665530 2024-06-03T02:29:28.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.262+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2506019257 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb8003680 con 0x5649ca665530 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.263+0000 7f2eb3e00700 1 -- 172.21.15.79:0/2506019257 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ec009c020 con 0x5649ca665530 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.263+0000 7f2edf7c7b80 10 client.26386 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.263+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2506019257 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649ca676570 con 0x5649ca665530 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.264+0000 7f2eb3e00700 1 -- 172.21.15.79:0/2506019257 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec009b920 con 0x5649ca665530 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.265+0000 7f2eb3e00700 1 -- 172.21.15.79:0/2506019257 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2ec009ad70 con 0x5649ca665530 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.265+0000 7f2edf7c7b80 10 client.26386 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.265+0000 7f2edf7c7b80 10 client.26386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.265+0000 7f2edf7c7b80 20 client.26386 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.265+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5649ca678c00 0x5649ca67b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.265+0000 7f2edf7c7b80 4 client.26386 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.265+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2506019257 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2eb8003680 con 0x5649ca678c00 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.272+0000 7f2ecde00700 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5649ca678c00 0x5649ca67b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.273+0000 7f2ecde00700 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5649ca678c00 0x5649ca67b080 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7568c1288b56238 server_cookie=4c56b1616dd57f96 in_seq=0 out_seq=0 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.274+0000 7f2eb3e00700 10 client.26386 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2eb3e00700 1 -- 172.21.15.79:0/2506019257 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f2eb8003680 con 0x5649ca678c00 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2eb3e00700 10 client.26386 handle_command_reply: tid=0 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2edf7c7b80 1 client.26386 shutdown 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2506019257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 msgr2=0x5649ca668a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca668a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00ccbc0 tx=0x7f2ec009b4e0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2506019257 shutdown_connections 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca668a40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5649ca678c00 0x5649ca67b080 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 0x5649ca66b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2506019257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca664c10 0x5649ca668500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2506019257 >> 172.21.15.79:0/2506019257 conn(0x5649ca58b1f0 msgr2=0x5649ca653810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:28.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.277+0000 7f2eb3e00700 0 client.26386 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.280+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2506019257 shutdown_connections 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.280+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2506019257 wait complete. 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.280+0000 7f2edf7c7b80 20 client.26386 trim_cache size 0 max 16384 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.321+0000 7f2edf7c7b80 1 Processor -- start 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.321+0000 7f2edf7c7b80 1 -- start start 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.322+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca65f2c0 0x5649ca65ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.322+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca664c10 0x5649ca66ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.322+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca674700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.322+0000 7f2edf7c7b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac00ea40 con 0x5649ca664c10 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.322+0000 7f2edf7c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac010580 con 0x5649ca65f2c0 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.322+0000 7f2edf7c7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac00f110 con 0x5649ca665530 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.322+0000 7f2ece800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca674700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.322+0000 7f2ece800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca674700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33658/0 (socket says 172.21.15.79:33658) 2024-06-03T02:29:28.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.322+0000 7f2ece800700 1 -- 172.21.15.79:0/2314101419 learned_addr learned my addr 172.21.15.79:0/2314101419 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.323+0000 7f2ecde00700 1 --2- 172.21.15.79:0/2314101419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca65f2c0 0x5649ca65ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.324+0000 7f2ece800700 1 -- 172.21.15.79:0/2314101419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca65f2c0 msgr2=0x5649ca65ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.324+0000 7f2ece800700 1 --2- 172.21.15.79:0/2314101419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca65f2c0 0x5649ca65ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.324+0000 7f2ece800700 1 -- 172.21.15.79:0/2314101419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca664c10 msgr2=0x5649ca66ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.324+0000 7f2ece800700 1 --2- 172.21.15.79:0/2314101419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca664c10 0x5649ca66ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.324+0000 7f2ece800700 1 -- 172.21.15.79:0/2314101419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ca666a70 con 0x5649ca665530 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.324+0000 7f2ece800700 1 --2- 172.21.15.79:0/2314101419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca674700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00982a0 tx=0x7f2ec0005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c64fc4052460945a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.324+0000 7f2ebea00700 1 -- 172.21.15.79:0/2314101419 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00ac030 con 0x5649ca665530 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.324+0000 7f2ebea00700 1 -- 172.21.15.79:0/2314101419 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec009b920 con 0x5649ca665530 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.324+0000 7f2ebea00700 1 -- 172.21.15.79:0/2314101419 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00b3030 con 0x5649ca665530 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.325+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2314101419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 msgr2=0x5649ca674700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.325+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2314101419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca674700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00982a0 tx=0x7f2ec0005ec0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.326+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2314101419 shutdown_connections 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.326+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2314101419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca665530 0x5649ca674700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.326+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2314101419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca65f2c0 0x5649ca65ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.326+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/2314101419 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca664c10 0x5649ca66ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.326+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2314101419 >> 172.21.15.79:0/2314101419 conn(0x5649ca58b1f0 msgr2=0x5649ca673ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.326+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2314101419 shutdown_connections 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.326+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/2314101419 wait complete. 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.327+0000 7f2edf7c7b80 1 -- start start 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.327+0000 7f2edf7c7b80 1 Processor -- start 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.329+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 0x5649ca66f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.329+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca664c10 0x5649ca66fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.329+0000 7f2ecde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 0x5649ca66f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.330+0000 7f2edf7c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca6e2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.330+0000 7f2ecde00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 0x5649ca66f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47258/0 (socket says 172.21.15.79:47258) 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.330+0000 7f2ecd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca664c10 0x5649ca66fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.330+0000 7f2ecd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca664c10 0x5649ca66fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33666/0 (socket says 172.21.15.79:33666) 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.330+0000 7f2ecde00700 1 -- 172.21.15.79:0/1587903883 learned_addr learned my addr 172.21.15.79:0/1587903883 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.330+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac00feb0 con 0x5649ca65f2c0 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.330+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac00f7e0 con 0x5649ca665530 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.330+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac00ca90 con 0x5649ca664c10 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.330+0000 7f2ece800700 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca6e2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.331+0000 7f2edf7c7b80 10 client.0 _ll_register_callbacks cb 0x5649ca546400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.331+0000 7f2ece800700 1 -- 172.21.15.79:0/1587903883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca664c10 msgr2=0x5649ca66fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.331+0000 7f2ece800700 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca664c10 0x5649ca66fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.331+0000 7f2ece800700 1 -- 172.21.15.79:0/1587903883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 msgr2=0x5649ca66f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.331+0000 7f2ece800700 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 0x5649ca66f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.332+0000 7f2ece800700 1 -- 172.21.15.79:0/1587903883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ca6e2e70 con 0x5649ca665530 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.332+0000 7f2ece800700 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca6e2930 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5649ca656b60 tx=0x7f2ec009b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76e9bd61fb7ba68c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.332+0000 7f2eb3e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:29:28.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.333+0000 7f2eb3e00700 1 -- 172.21.15.79:0/1587903883 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00ac070 con 0x5649ca665530 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.333+0000 7f2eb3e00700 1 -- 172.21.15.79:0/1587903883 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2ec00b4d20 con 0x5649ca665530 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.333+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec009b920 con 0x5649ca665530 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2eb3e00700 1 -- 172.21.15.79:0/1587903883 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ec00a4590 con 0x5649ca665530 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2edf7c7b80 10 client.26387 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649ca6e2e70 con 0x5649ca665530 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2eb3e00700 1 -- 172.21.15.79:0/1587903883 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec009b920 con 0x5649ca665530 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2eb3e00700 1 -- 172.21.15.79:0/1587903883 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2ec00a0e60 con 0x5649ca665530 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2edf7c7b80 10 client.26387 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2edf7c7b80 10 client.26387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2edf7c7b80 20 client.26387 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5649ca6e4350 0x5649ca6e6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2edf7c7b80 4 client.26387 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.334+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5649ca5be080 con 0x5649ca6e4350 2024-06-03T02:29:28.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.337+0000 7f2ecde00700 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5649ca6e4350 0x5649ca6e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.338+0000 7f2ecde00700 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5649ca6e4350 0x5649ca6e6740 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b00ca9675e652d3 server_cookie=4c7a4767ade3ef76 in_seq=0 out_seq=0 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.338+0000 7f2eb3e00700 10 client.26387 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.339+0000 7f2eb3e00700 1 -- 172.21.15.79:0/1587903883 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5649ca5be080 con 0x5649ca6e4350 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.339+0000 7f2eb3e00700 10 client.26387 handle_command_reply: tid=0 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.339+0000 7f2edf7c7b80 1 client.26387 shutdown 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.340+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 msgr2=0x5649ca6e2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.340+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca6e2930 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5649ca656b60 tx=0x7f2ec009b820 comp rx=0 tx=0).stop 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.340+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 shutdown_connections 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.340+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649ca665530 0x5649ca6e2930 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.340+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5649ca6e4350 0x5649ca6e6740 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.340+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5649ca65f2c0 0x5649ca66f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.340+0000 7f2edf7c7b80 1 --2- 172.21.15.79:0/1587903883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649ca664c10 0x5649ca66fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.340+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 >> 172.21.15.79:0/1587903883 conn(0x5649ca58b1f0 msgr2=0x5649ca6642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:28.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.340+0000 7f2eb3e00700 0 client.26387 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.341+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 shutdown_connections 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.341+0000 7f2edf7c7b80 1 -- 172.21.15.79:0/1587903883 wait complete. 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.341+0000 7f2edf7c7b80 20 client.26387 trim_cache size 0 max 16384 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 -- 172.21.15.79:0/3099921188 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2eb40a2d70 msgr2=0x7f2eb40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2eb40a2d70 0x7f2eb40a5230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8004b10 tx=0x7f2ec8009920 comp rx=0 tx=0).stop 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 -- 172.21.15.79:0/3099921188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 msgr2=0x7f2ed006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 0x7f2ed006cd10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000e970 tx=0x7f2ec000ed30 comp rx=0 tx=0).stop 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 -- 172.21.15.79:0/3099921188 shutdown_connections 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed0108650 0x7f2ed006cd10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2eb40a2d70 0x7f2eb40a5230 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ed01027b0 0x7f2ed0073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 --2- 172.21.15.79:0/3099921188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ed0103170 0x7f2ed0074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.344+0000 7f2ed4e00700 1 -- 172.21.15.79:0/3099921188 >> 172.21.15.79:0/3099921188 conn(0x7f2ed007a420 msgr2=0x7f2ed01012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.345+0000 7f2ed4e00700 1 -- 172.21.15.79:0/3099921188 shutdown_connections 2024-06-03T02:29:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:28.345+0000 7f2ed4e00700 1 -- 172.21.15.79:0/3099921188 wait complete. 2024-06-03T02:29:29.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:29:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:29 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:29:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:29 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:29:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:29 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:29:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:29 smithi079 ceph-mon[31819]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.5 MiB/s wr, 523 op/s 2024-06-03T02:29:30.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:29 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:29:30.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:29 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:29:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:29 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:29:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:29 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:29:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:29 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:29:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:29 smithi135 ceph-mon[36653]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.5 MiB/s wr, 523 op/s 2024-06-03T02:29:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:29 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:29:30.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:29 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:29:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:29 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:29:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:29 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:29:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:29 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:29:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:29 smithi123 ceph-mon[36640]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.5 MiB/s wr, 523 op/s 2024-06-03T02:29:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:29 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:29:30.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:29 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:29:30.419 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:29:32.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:31 smithi079 ceph-mon[31819]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 420 op/s 2024-06-03T02:29:32.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:31 smithi135 ceph-mon[36653]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 420 op/s 2024-06-03T02:29:32.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:31 smithi123 ceph-mon[36640]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 420 op/s 2024-06-03T02:29:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 565.0 s, 610.8 tps, lat 172.885 ms stddev 225.310 2024-06-03T02:29:33.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.902+0000 7f4867200700 1 -- 172.21.15.79:0/1004301911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 msgr2=0x7f4860104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:33.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.902+0000 7f4867200700 1 --2- 172.21.15.79:0/1004301911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 0x7f4860104b50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f484c009990 tx=0x7f484c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.902+0000 7f4867200700 1 -- 172.21.15.79:0/1004301911 shutdown_connections 2024-06-03T02:29:33.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.902+0000 7f4867200700 1 --2- 172.21.15.79:0/1004301911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4860105090 0x7f48601115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.902+0000 7f4867200700 1 --2- 172.21.15.79:0/1004301911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f486010a6d0 0x7f486010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.902+0000 7f4867200700 1 --2- 172.21.15.79:0/1004301911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 0x7f4860104b50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.902+0000 7f4867200700 1 -- 172.21.15.79:0/1004301911 >> 172.21.15.79:0/1004301911 conn(0x7f48601001c0 msgr2=0x7f48601025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:33.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.902+0000 7f4867200700 1 -- 172.21.15.79:0/1004301911 shutdown_connections 2024-06-03T02:29:33.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.903+0000 7f4867200700 1 -- 172.21.15.79:0/1004301911 wait complete. 2024-06-03T02:29:33.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.904+0000 7f4867200700 1 Processor -- start 2024-06-03T02:29:33.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.905+0000 7f4867200700 1 -- start start 2024-06-03T02:29:33.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.905+0000 7f4867200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 0x7f486019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.905+0000 7f4867200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4860105090 0x7f486019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.905+0000 7f4867200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f486010a6d0 0x7f486019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.905+0000 7f4867200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4860114a60 con 0x7f4860105090 2024-06-03T02:29:33.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.906+0000 7f4867200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48601148e0 con 0x7f486010a6d0 2024-06-03T02:29:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.906+0000 7f4867200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4860114be0 con 0x7f48601046d0 2024-06-03T02:29:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.906+0000 7f485fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 0x7f486019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.906+0000 7f485fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 0x7f486019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33672/0 (socket says 172.21.15.79:33672) 2024-06-03T02:29:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.906+0000 7f485fe00700 1 -- 172.21.15.79:0/2714883146 learned_addr learned my addr 172.21.15.79:0/2714883146 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:33.906 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:33 smithi079 ceph-mon[31819]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 488 op/s 2024-06-03T02:29:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.906+0000 7f485fe00700 1 -- 172.21.15.79:0/2714883146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f486010a6d0 msgr2=0x7f486019f720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:29:33.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.906+0000 7f485fe00700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f486010a6d0 0x7f486019f720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.906+0000 7f485fe00700 1 -- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4860105090 msgr2=0x7f486019b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:29:33.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.908+0000 7f485f400700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4860105090 0x7f486019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.908+0000 7f485fe00700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4860105090 0x7f486019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.908+0000 7f485fe00700 1 -- 172.21.15.79:0/2714883146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486019fe60 con 0x7f48601046d0 2024-06-03T02:29:33.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.909+0000 7f485fe00700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 0x7f486019ae50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f485000e920 tx=0x7f485000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:33.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.909+0000 7f485cc00700 1 -- 172.21.15.79:0/2714883146 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485000cbd0 con 0x7f48601046d0 2024-06-03T02:29:33.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.910+0000 7f485cc00700 1 -- 172.21.15.79:0/2714883146 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485000cd70 con 0x7f48601046d0 2024-06-03T02:29:33.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.910+0000 7f4867200700 1 -- 172.21.15.79:0/2714883146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48601a0150 con 0x7f48601046d0 2024-06-03T02:29:33.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.910+0000 7f4867200700 1 -- 172.21.15.79:0/2714883146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48601a79d0 con 0x7f48601046d0 2024-06-03T02:29:33.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.910+0000 7f485cc00700 1 -- 172.21.15.79:0/2714883146 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500186a0 con 0x7f48601046d0 2024-06-03T02:29:33.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.921+0000 7f4871aceb80 1 Processor -- start 2024-06-03T02:29:33.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.921+0000 7f4871aceb80 1 -- start start 2024-06-03T02:29:33.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.922+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b4b90 0x55ed798b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.922+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b54b0 0x55ed798aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.922+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798af680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.922+0000 7f4871aceb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ed797bda10 con 0x55ed798b4b90 2024-06-03T02:29:33.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.922+0000 7f4871aceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed79800b00 con 0x55ed798b54b0 2024-06-03T02:29:33.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.922+0000 7f4871aceb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed797bb8f0 con 0x55ed798af240 2024-06-03T02:29:33.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.922+0000 7f485cc00700 1 -- 172.21.15.79:0/2714883146 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4850018860 con 0x7f48601046d0 2024-06-03T02:29:33.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.923+0000 7f485f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.923+0000 7f4865200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b4b90 0x55ed798b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.923+0000 7f485f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33686/0 (socket says 172.21.15.79:33686) 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.923+0000 7f485f400700 1 -- 172.21.15.79:0/2526459935 learned_addr learned my addr 172.21.15.79:0/2526459935 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.926+0000 7f485f400700 1 -- 172.21.15.79:0/2526459935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b54b0 msgr2=0x55ed798aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.926+0000 7f485f400700 1 --2- 172.21.15.79:0/2526459935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b54b0 0x55ed798aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.926+0000 7f485f400700 1 -- 172.21.15.79:0/2526459935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b4b90 msgr2=0x55ed798b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.926+0000 7f485f400700 1 --2- 172.21.15.79:0/2526459935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b4b90 0x55ed798b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.926+0000 7f485f400700 1 -- 172.21.15.79:0/2526459935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed797a8f90 con 0x55ed798af240 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.926+0000 7f485cc00700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48440a2ca0 0x7f48440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.927+0000 7f485cc00700 1 -- 172.21.15.79:0/2714883146 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f48500cf600 con 0x7f48601046d0 2024-06-03T02:29:33.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.927+0000 7f4865200700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48440a2ca0 0x7f48440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.928+0000 7f485f400700 1 --2- 172.21.15.79:0/2526459935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798af680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b7a0 tx=0x7f484c00b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f415c0122f9687b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:33.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.928+0000 7f4855600700 1 -- 172.21.15.79:0/2526459935 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c026070 con 0x55ed798af240 2024-06-03T02:29:33.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.928+0000 7f4855600700 1 -- 172.21.15.79:0/2526459935 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f484c009510 con 0x55ed798af240 2024-06-03T02:29:33.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.928+0000 7f4855600700 1 -- 172.21.15.79:0/2526459935 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c003ab0 con 0x55ed798af240 2024-06-03T02:29:33.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.930+0000 7f4865200700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48440a2ca0 0x7f48440a5160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f485800f540 tx=0x7f485800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.931+0000 7f4871aceb80 1 -- 172.21.15.79:0/2526459935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 msgr2=0x55ed798af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.931+0000 7f4871aceb80 1 --2- 172.21.15.79:0/2526459935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798af680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b7a0 tx=0x7f484c00b7d0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.931+0000 7f4871aceb80 1 -- 172.21.15.79:0/2526459935 shutdown_connections 2024-06-03T02:29:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.931+0000 7f4871aceb80 1 --2- 172.21.15.79:0/2526459935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798af680 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.931+0000 7f4871aceb80 1 --2- 172.21.15.79:0/2526459935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b4b90 0x55ed798b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.931+0000 7f4871aceb80 1 --2- 172.21.15.79:0/2526459935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b54b0 0x55ed798aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.931+0000 7f4871aceb80 1 -- 172.21.15.79:0/2526459935 >> 172.21.15.79:0/2526459935 conn(0x55ed797db170 msgr2=0x55ed798a9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.931+0000 7f4871aceb80 1 -- 172.21.15.79:0/2526459935 shutdown_connections 2024-06-03T02:29:33.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.931+0000 7f4871aceb80 1 -- 172.21.15.79:0/2526459935 wait complete. 2024-06-03T02:29:33.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.932+0000 7f4871aceb80 1 -- start start 2024-06-03T02:29:33.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.932+0000 7f4871aceb80 1 Processor -- start 2024-06-03T02:29:33.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.933+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.933+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b4b90 0x55ed798c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.933+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 0x55ed798c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.933+0000 7f4871aceb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ed7979cf60 con 0x55ed798b54b0 2024-06-03T02:29:33.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.933+0000 7f4871aceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed797fcac0 con 0x55ed798b4b90 2024-06-03T02:29:33.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.933+0000 7f4871aceb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed798c4640 con 0x55ed798af240 2024-06-03T02:29:33.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.933+0000 7f4871aceb80 10 client.0 _ll_register_callbacks cb 0x55ed79796380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:33.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.934+0000 7f485f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b4b90 0x55ed798c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.934+0000 7f485fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.934+0000 7f485f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b4b90 0x55ed798c0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41120/0 (socket says 172.21.15.79:41120) 2024-06-03T02:29:33.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.934+0000 7f485f400700 1 -- 172.21.15.79:0/233716125 learned_addr learned my addr 172.21.15.79:0/233716125 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.935+0000 7f4865200700 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 0x55ed798c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.936+0000 7f485f400700 1 -- 172.21.15.79:0/233716125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 msgr2=0x55ed798c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.936+0000 7f485f400700 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.936+0000 7f485f400700 1 -- 172.21.15.79:0/233716125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 msgr2=0x55ed798c12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.936+0000 7f485f400700 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 0x55ed798c12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.936+0000 7f485f400700 1 -- 172.21.15.79:0/233716125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7974f8d0 con 0x55ed798b4b90 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.937+0000 7f485f400700 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b4b90 0x55ed798c0db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f484c024040 tx=0x7f484c005660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e90e9973d6ccfc5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.937+0000 7f484a000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.937+0000 7f484a000700 1 -- 172.21.15.79:0/233716125 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c02f060 con 0x55ed798b4b90 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.937+0000 7f4871aceb80 1 -- 172.21.15.79:0/233716125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f484c009510 con 0x55ed798b4b90 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.937+0000 7f4865200700 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 0x55ed798c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:29:33.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.938+0000 7f484a000700 1 -- 172.21.15.79:0/233716125 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f484c003680 con 0x55ed798b4b90 2024-06-03T02:29:33.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.939+0000 7f484a000700 1 -- 172.21.15.79:0/233716125 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f484c026070 con 0x55ed798b4b90 2024-06-03T02:29:33.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.939+0000 7f4871aceb80 10 client.26393 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:33.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.939+0000 7f4871aceb80 1 -- 172.21.15.79:0/233716125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed798b6ec0 con 0x55ed798b4b90 2024-06-03T02:29:33.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.940+0000 7f485fe00700 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:29:33.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.940+0000 7f484a000700 1 -- 172.21.15.79:0/233716125 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f484c009510 con 0x55ed798b4b90 2024-06-03T02:29:33.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.941+0000 7f484a000700 1 -- 172.21.15.79:0/233716125 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f484c00fa60 con 0x55ed798b4b90 2024-06-03T02:29:33.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.941+0000 7f4871aceb80 10 client.26393 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:33.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.941+0000 7f4871aceb80 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:29:33.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.941+0000 7f4871aceb80 20 client.26393 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:33.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.942+0000 7f4871aceb80 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed798b93f0 0x55ed798bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.942+0000 7f4871aceb80 4 client.26393 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:29:33.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.942+0000 7f4871aceb80 1 -- 172.21.15.79:0/233716125 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed7980e000 con 0x55ed798b93f0 2024-06-03T02:29:33.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.943+0000 7f485fe00700 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed798b93f0 0x55ed798bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.945+0000 7f484a000700 10 client.26393 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:33.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.945+0000 7f485fe00700 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed798b93f0 0x55ed798bb8b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5799a4aa71ae393d server_cookie=11d55e1456d2035f in_seq=0 out_seq=0 2024-06-03T02:29:33.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.946+0000 7f484a000700 1 -- 172.21.15.79:0/233716125 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ed7980e000 con 0x55ed798b93f0 2024-06-03T02:29:33.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.946+0000 7f484a000700 10 client.26393 handle_command_reply: tid=0 2024-06-03T02:29:33.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.947+0000 7f4871aceb80 1 client.26393 shutdown 2024-06-03T02:29:33.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.949+0000 7f4871aceb80 1 -- 172.21.15.79:0/233716125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b4b90 msgr2=0x55ed798c0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:33.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.949+0000 7f4871aceb80 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b4b90 0x55ed798c0db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f484c024040 tx=0x7f484c005660 comp rx=0 tx=0).stop 2024-06-03T02:29:33.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.950+0000 7f4871aceb80 1 -- 172.21.15.79:0/233716125 shutdown_connections 2024-06-03T02:29:33.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.950+0000 7f4871aceb80 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed798b93f0 0x55ed798bb8b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.950+0000 7f4871aceb80 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 0x55ed798c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.950+0000 7f4871aceb80 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.950+0000 7f484a000700 0 client.26393 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:33.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.950+0000 7f4871aceb80 1 --2- 172.21.15.79:0/233716125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798b4b90 0x55ed798c0db0 secure :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f484c024040 tx=0x7f484c005660 comp rx=0 tx=0).stop 2024-06-03T02:29:33.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.950+0000 7f4871aceb80 1 -- 172.21.15.79:0/233716125 >> 172.21.15.79:0/233716125 conn(0x55ed797db170 msgr2=0x55ed798a3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:33.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.951+0000 7f4871aceb80 1 -- 172.21.15.79:0/233716125 shutdown_connections 2024-06-03T02:29:33.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.951+0000 7f4871aceb80 1 -- 172.21.15.79:0/233716125 wait complete. 2024-06-03T02:29:33.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.951+0000 7f4871aceb80 20 client.26393 trim_cache size 0 max 16384 2024-06-03T02:29:33.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.994+0000 7f4871aceb80 1 Processor -- start 2024-06-03T02:29:33.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.995+0000 7f4871aceb80 1 -- start start 2024-06-03T02:29:33.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.995+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.995+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 0x55ed798a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.995+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798be800 0x55ed798c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:33.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.995+0000 7f4871aceb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f484000eb40 con 0x55ed798b54b0 2024-06-03T02:29:33.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.995+0000 7f4871aceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4840010680 con 0x55ed798be800 2024-06-03T02:29:33.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.995+0000 7f4871aceb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f484000f210 con 0x55ed798af240 2024-06-03T02:29:33.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.996+0000 7f4865200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798be800 0x55ed798c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.996+0000 7f4865200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798be800 0x55ed798c35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41136/0 (socket says 172.21.15.79:41136) 2024-06-03T02:29:33.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.996+0000 7f4865200700 1 -- 172.21.15.79:0/2479856249 learned_addr learned my addr 172.21.15.79:0/2479856249 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:33.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.996+0000 7f485f400700 1 --2- 172.21.15.79:0/2479856249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 0x55ed798a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:33.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.997+0000 7f4865200700 1 -- 172.21.15.79:0/2479856249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 msgr2=0x55ed798aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:33.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.997+0000 7f4865200700 1 --2- 172.21.15.79:0/2479856249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.997+0000 7f4865200700 1 -- 172.21.15.79:0/2479856249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 msgr2=0x55ed798a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:33.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.997+0000 7f4865200700 1 --2- 172.21.15.79:0/2479856249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 0x55ed798a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.997+0000 7f4865200700 1 -- 172.21.15.79:0/2479856249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed798c13e0 con 0x55ed798be800 2024-06-03T02:29:33.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.997+0000 7f4865200700 1 --2- 172.21.15.79:0/2479856249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798be800 0x55ed798c35f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4858009140 tx=0x7f4858014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6f8d2f197e146c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:33.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.997+0000 7f4854c00700 1 -- 172.21.15.79:0/2479856249 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858004bb0 con 0x55ed798be800 2024-06-03T02:29:33.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.998+0000 7f4854c00700 1 -- 172.21.15.79:0/2479856249 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485801b030 con 0x55ed798be800 2024-06-03T02:29:33.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.998+0000 7f4854c00700 1 -- 172.21.15.79:0/2479856249 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858022bf0 con 0x55ed798be800 2024-06-03T02:29:33.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.999+0000 7f4871aceb80 1 -- 172.21.15.79:0/2479856249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798be800 msgr2=0x55ed798c35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:33.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.999+0000 7f4871aceb80 1 --2- 172.21.15.79:0/2479856249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798be800 0x55ed798c35f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4858009140 tx=0x7f4858014000 comp rx=0 tx=0).stop 2024-06-03T02:29:33.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.999+0000 7f4871aceb80 1 -- 172.21.15.79:0/2479856249 shutdown_connections 2024-06-03T02:29:33.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.999+0000 7f4871aceb80 1 --2- 172.21.15.79:0/2479856249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798be800 0x55ed798c35f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.999+0000 7f4871aceb80 1 --2- 172.21.15.79:0/2479856249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798af240 0x55ed798aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.999+0000 7f4871aceb80 1 --2- 172.21.15.79:0/2479856249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798b54b0 0x55ed798a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:33.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:33.999+0000 7f4871aceb80 1 -- 172.21.15.79:0/2479856249 >> 172.21.15.79:0/2479856249 conn(0x55ed797db170 msgr2=0x55ed798b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:34.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.000+0000 7f4871aceb80 1 -- 172.21.15.79:0/2479856249 shutdown_connections 2024-06-03T02:29:34.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.000+0000 7f4871aceb80 1 -- 172.21.15.79:0/2479856249 wait complete. 2024-06-03T02:29:34.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.001+0000 7f4871aceb80 1 -- start start 2024-06-03T02:29:34.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.001+0000 7f4871aceb80 1 Processor -- start 2024-06-03T02:29:34.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.002+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798af240 0x55ed798c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:34.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.002+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798b54b0 0x55ed798c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:34.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.002+0000 7f4871aceb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798be800 0x55ed798e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:34.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.002+0000 7f485fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798af240 0x55ed798c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:34.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.003+0000 7f4871aceb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f484000ffb0 con 0x55ed798be800 2024-06-03T02:29:34.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.003+0000 7f485f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798b54b0 0x55ed798c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:34.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.003+0000 7f4865200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798be800 0x55ed798e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:34.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.003+0000 7f485f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798b54b0 0x55ed798c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33706/0 (socket says 172.21.15.79:33706) 2024-06-03T02:29:34.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.003+0000 7f485f400700 1 -- 172.21.15.79:0/1002785376 learned_addr learned my addr 172.21.15.79:0/1002785376 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:34.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.003+0000 7f4871aceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f484000f8e0 con 0x55ed798af240 2024-06-03T02:29:34.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.003+0000 7f4871aceb80 1 -- 172.21.15.79:0/1002785376 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f484000cae0 con 0x55ed798b54b0 2024-06-03T02:29:34.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.003+0000 7f4871aceb80 10 client.0 _ll_register_callbacks cb 0x55ed79796380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:34.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.004+0000 7f4865200700 1 -- 172.21.15.79:0/1002785376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798b54b0 msgr2=0x55ed798c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:34.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.004+0000 7f4865200700 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798b54b0 0x55ed798c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.004+0000 7f4865200700 1 -- 172.21.15.79:0/1002785376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798af240 msgr2=0x55ed798c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:34.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.004+0000 7f4865200700 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798af240 0x55ed798c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.004+0000 7f4865200700 1 -- 172.21.15.79:0/1002785376 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed798e4560 con 0x55ed798be800 2024-06-03T02:29:34.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.006+0000 7f4865200700 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798be800 0x55ed798e2a30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f485800fd10 tx=0x7f4858008bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465a3dfe03d0342f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:34.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.006+0000 7f484a000700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:29:34.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.008+0000 7f484a000700 1 -- 172.21.15.79:0/1002785376 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485802f070 con 0x55ed798be800 2024-06-03T02:29:34.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.008+0000 7f484a000700 1 -- 172.21.15.79:0/1002785376 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f4858022a80 con 0x55ed798be800 2024-06-03T02:29:34.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.008+0000 7f4871aceb80 1 -- 172.21.15.79:0/1002785376 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f485801b030 con 0x55ed798be800 2024-06-03T02:29:34.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.008+0000 7f484a000700 1 -- 172.21.15.79:0/1002785376 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f485803b050 con 0x55ed798be800 2024-06-03T02:29:34.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.008+0000 7f4871aceb80 10 client.15954 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:34.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.008+0000 7f4871aceb80 1 -- 172.21.15.79:0/1002785376 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed79943ef0 con 0x55ed798be800 2024-06-03T02:29:34.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.009+0000 7f484a000700 1 -- 172.21.15.79:0/1002785376 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485801b030 con 0x55ed798be800 2024-06-03T02:29:34.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.009+0000 7f484a000700 1 -- 172.21.15.79:0/1002785376 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4858017ac0 con 0x55ed798be800 2024-06-03T02:29:34.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.009+0000 7f4871aceb80 10 client.15954 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:34.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.009+0000 7f4871aceb80 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:29:34.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.009+0000 7f4871aceb80 20 client.15954 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:34.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.009+0000 7f4871aceb80 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed799444d0 0x55ed799468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:34.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.009+0000 7f4871aceb80 4 client.15954 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:29:34.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.009+0000 7f4871aceb80 1 -- 172.21.15.79:0/1002785376 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed79808580 con 0x55ed799444d0 2024-06-03T02:29:34.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.009+0000 7f485fe00700 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed799444d0 0x55ed799468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:34.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.011+0000 7f485fe00700 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed799444d0 0x55ed799468f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537cd8b584fce133 server_cookie=678f797b5aa4c106 in_seq=0 out_seq=0 2024-06-03T02:29:34.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.011+0000 7f484a000700 10 client.15954 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:34.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f484a000700 1 -- 172.21.15.79:0/1002785376 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed79808580 con 0x55ed799444d0 2024-06-03T02:29:34.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f484a000700 10 client.15954 handle_command_reply: tid=0 2024-06-03T02:29:34.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f4871aceb80 1 client.15954 shutdown 2024-06-03T02:29:34.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f4871aceb80 1 -- 172.21.15.79:0/1002785376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798be800 msgr2=0x55ed798e2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:34.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f4871aceb80 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798be800 0x55ed798e2a30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f485800fd10 tx=0x7f4858008bb0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f4871aceb80 1 -- 172.21.15.79:0/1002785376 shutdown_connections 2024-06-03T02:29:34.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f4871aceb80 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed799444d0 0x55ed799468f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f4871aceb80 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed798be800 0x55ed798e2a30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f4871aceb80 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed798af240 0x55ed798c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f4871aceb80 1 --2- 172.21.15.79:0/1002785376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed798b54b0 0x55ed798c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f4871aceb80 1 -- 172.21.15.79:0/1002785376 >> 172.21.15.79:0/1002785376 conn(0x55ed797db170 msgr2=0x55ed798be080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:34.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.012+0000 7f484a000700 0 client.15954 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:29:34.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.013+0000 7f4871aceb80 1 -- 172.21.15.79:0/1002785376 shutdown_connections 2024-06-03T02:29:34.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.015+0000 7f4871aceb80 1 -- 172.21.15.79:0/1002785376 wait complete. 2024-06-03T02:29:34.016 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:29:34.016 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:29:34.016 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:29:34.016 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.015+0000 7f4871aceb80 20 client.15954 trim_cache size 0 max 16384 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 -- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48440a2ca0 msgr2=0x7f48440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48440a2ca0 0x7f48440a5160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f485800f540 tx=0x7f485800b040 comp rx=0 tx=0).stop 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 -- 172.21.15.79:0/2714883146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 msgr2=0x7f486019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 0x7f486019ae50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f485000e920 tx=0x7f485000ece0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 -- 172.21.15.79:0/2714883146 shutdown_connections 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f486010a6d0 0x7f486019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48601046d0 0x7f486019ae50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f48440a2ca0 0x7f48440a5160 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 --2- 172.21.15.79:0/2714883146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4860105090 0x7f486019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 -- 172.21.15.79:0/2714883146 >> 172.21.15.79:0/2714883146 conn(0x7f48601001c0 msgr2=0x7f48601006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 -- 172.21.15.79:0/2714883146 shutdown_connections 2024-06-03T02:29:34.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:34.017+0000 7f4867200700 1 -- 172.21.15.79:0/2714883146 wait complete. 2024-06-03T02:29:34.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:33 smithi135 ceph-mon[36653]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 488 op/s 2024-06-03T02:29:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:33 smithi123 ceph-mon[36640]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 488 op/s 2024-06-03T02:29:36.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:35 smithi079 ceph-mon[31819]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.8 MiB/s wr, 548 op/s 2024-06-03T02:29:36.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:35 smithi135 ceph-mon[36653]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.8 MiB/s wr, 548 op/s 2024-06-03T02:29:36.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:35 smithi123 ceph-mon[36640]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.8 MiB/s wr, 548 op/s 2024-06-03T02:29:36.644 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:29:36.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:29:37.357 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:29:37.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:37 smithi079 ceph-mon[31819]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.9 MiB/s wr, 441 op/s 2024-06-03T02:29:38.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:37 smithi135 ceph-mon[36653]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.9 MiB/s wr, 441 op/s 2024-06-03T02:29:38.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:37 smithi123 ceph-mon[36640]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.9 MiB/s wr, 441 op/s 2024-06-03T02:29:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 570.0 s, 482.2 tps, lat 203.230 ms stddev 248.563 2024-06-03T02:29:40.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:39 smithi079 ceph-mon[31819]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 595 op/s 2024-06-03T02:29:40.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:39 smithi135 ceph-mon[36653]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 595 op/s 2024-06-03T02:29:40.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:39 smithi123 ceph-mon[36640]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 595 op/s 2024-06-03T02:29:41.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.053+0000 7fd472a00700 1 -- 172.21.15.79:0/1365134514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd46c06ab30 msgr2=0x7fd46c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:41.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.053+0000 7fd472a00700 1 --2- 172.21.15.79:0/1365134514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd46c06ab30 0x7fd46c10f320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd46400b0a0 tx=0x7fd46400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.053+0000 7fd472a00700 1 -- 172.21.15.79:0/1365134514 shutdown_connections 2024-06-03T02:29:41.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.053+0000 7fd472a00700 1 --2- 172.21.15.79:0/1365134514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd46c06ab30 0x7fd46c10f320 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.053+0000 7fd472a00700 1 --2- 172.21.15.79:0/1365134514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c108500 0x7fd46c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.053+0000 7fd472a00700 1 --2- 172.21.15.79:0/1365134514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd46c06a170 0x7fd46c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.053+0000 7fd472a00700 1 -- 172.21.15.79:0/1365134514 >> 172.21.15.79:0/1365134514 conn(0x7fd46c078340 msgr2=0x7fd46c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:41.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.053+0000 7fd472a00700 1 -- 172.21.15.79:0/1365134514 shutdown_connections 2024-06-03T02:29:41.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.053+0000 7fd472a00700 1 -- 172.21.15.79:0/1365134514 wait complete. 2024-06-03T02:29:41.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.054+0000 7fd472a00700 1 Processor -- start 2024-06-03T02:29:41.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.054+0000 7fd472a00700 1 -- start start 2024-06-03T02:29:41.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.055+0000 7fd472a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd46c06a170 0x7fd46c10d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:41.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.055+0000 7fd472a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd46c06ab30 0x7fd46c10d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:41.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.055+0000 7fd472a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c108500 0x7fd46c1a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:41.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.055+0000 7fd472a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c1127b0 con 0x7fd46c06ab30 2024-06-03T02:29:41.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.055+0000 7fd472a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c112630 con 0x7fd46c108500 2024-06-03T02:29:41.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.055+0000 7fd472a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c112930 con 0x7fd46c06a170 2024-06-03T02:29:41.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd470a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c108500 0x7fd46c1a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:41.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd46be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd46c06a170 0x7fd46c10d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:41.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd470a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c108500 0x7fd46c1a3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47010/0 (socket says 172.21.15.79:47010) 2024-06-03T02:29:41.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd470a00700 1 -- 172.21.15.79:0/2102110656 learned_addr learned my addr 172.21.15.79:0/2102110656 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:41.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd470a00700 1 -- 172.21.15.79:0/2102110656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd46c06a170 msgr2=0x7fd46c10d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:41.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd470a00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd46c06a170 0x7fd46c10d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd470a00700 1 -- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd46c06ab30 msgr2=0x7fd46c10d830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:29:41.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd46b400700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd46c06ab30 0x7fd46c10d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:41.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd470a00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd46c06ab30 0x7fd46c10d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd470a00700 1 -- 172.21.15.79:0/2102110656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c1a4180 con 0x7fd46c108500 2024-06-03T02:29:41.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.056+0000 7fd46b400700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd46c06ab30 0x7fd46c10d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:29:41.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.057+0000 7fd470a00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c108500 0x7fd46c1a3a40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd46400b780 tx=0x7fd464004da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:41.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.057+0000 7fd468c00700 1 -- 172.21.15.79:0/2102110656 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464021070 con 0x7fd46c108500 2024-06-03T02:29:41.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.057+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46c1a4410 con 0x7fd46c108500 2024-06-03T02:29:41.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.057+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46c1abd30 con 0x7fd46c108500 2024-06-03T02:29:41.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.057+0000 7fd468c00700 1 -- 172.21.15.79:0/2102110656 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd464007c40 con 0x7fd46c108500 2024-06-03T02:29:41.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.057+0000 7fd468c00700 1 -- 172.21.15.79:0/2102110656 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464005370 con 0x7fd46c108500 2024-06-03T02:29:41.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.058+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46c069180 con 0x7fd46c108500 2024-06-03T02:29:41.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.063+0000 7fd468c00700 1 -- 172.21.15.79:0/2102110656 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd464024070 con 0x7fd46c108500 2024-06-03T02:29:41.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.065+0000 7fd468c00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4500a2cf0 0x7fd4500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:41.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.065+0000 7fd468c00700 1 -- 172.21.15.79:0/2102110656 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd46400f020 con 0x7fd46c108500 2024-06-03T02:29:41.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.066+0000 7fd46be00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4500a2cf0 0x7fd4500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:41.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.066+0000 7fd46be00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4500a2cf0 0x7fd4500a51b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd45c005fd0 tx=0x7fd45c005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:41.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.071+0000 7fd468c00700 1 -- 172.21.15.79:0/2102110656 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd46402b020 con 0x7fd46c108500 2024-06-03T02:29:41.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.522+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd46c1abfc0 con 0x7fd46c108500 2024-06-03T02:29:41.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.524+0000 7fd468c00700 1 -- 172.21.15.79:0/2102110656 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7fd464007df0 con 0x7fd46c108500 2024-06-03T02:29:41.525 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:29:41.525 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:29:41.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.530+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4500a2cf0 msgr2=0x7fd4500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:41.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.530+0000 7fd472a00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4500a2cf0 0x7fd4500a51b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd45c005fd0 tx=0x7fd45c005ce0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.530+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c108500 msgr2=0x7fd46c1a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:41.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.530+0000 7fd472a00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c108500 0x7fd46c1a3a40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd46400b780 tx=0x7fd464004da0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.531+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 shutdown_connections 2024-06-03T02:29:41.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.531+0000 7fd472a00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c108500 0x7fd46c1a3a40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.531+0000 7fd472a00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd46c06a170 0x7fd46c10d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.531+0000 7fd472a00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd4500a2cf0 0x7fd4500a51b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.531+0000 7fd472a00700 1 --2- 172.21.15.79:0/2102110656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd46c06ab30 0x7fd46c10d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:41.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.531+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 >> 172.21.15.79:0/2102110656 conn(0x7fd46c078340 msgr2=0x7fd46c101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:41.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.532+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 shutdown_connections 2024-06-03T02:29:41.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:41.532+0000 7fd472a00700 1 -- 172.21.15.79:0/2102110656 wait complete. 2024-06-03T02:29:41.547 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:29:42.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:41 smithi079 ceph-mon[31819]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-06-03T02:29:42.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:41 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2102110656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:29:42.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:41 smithi135 ceph-mon[36653]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-06-03T02:29:42.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:41 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2102110656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:29:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:41 smithi123 ceph-mon[36640]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-06-03T02:29:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:41 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2102110656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:29:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 575.0 s, 541.8 tps, lat 186.692 ms stddev 225.473 2024-06-03T02:29:44.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:43 smithi079 ceph-mon[31819]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-06-03T02:29:44.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:43 smithi135 ceph-mon[36653]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-06-03T02:29:44.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:43 smithi123 ceph-mon[36640]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-06-03T02:29:45.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:29:45.847 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:29:46.035 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:45 smithi079 ceph-mon[31819]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-06-03T02:29:46.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:45 smithi135 ceph-mon[36653]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-06-03T02:29:46.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:45 smithi123 ceph-mon[36640]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-06-03T02:29:47.812 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:47 smithi079 ceph-mon[31819]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.5 MiB/s wr, 418 op/s 2024-06-03T02:29:48.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:47 smithi135 ceph-mon[36653]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.5 MiB/s wr, 418 op/s 2024-06-03T02:29:48.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:47 smithi123 ceph-mon[36640]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.5 MiB/s wr, 418 op/s 2024-06-03T02:29:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 580.0 s, 466.4 tps, lat 216.914 ms stddev 305.790 2024-06-03T02:29:49.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:48 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:29:49.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:48 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:29:49.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:48 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:29:49.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:49 smithi079 ceph-mon[31819]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 577 op/s 2024-06-03T02:29:50.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:49 smithi135 ceph-mon[36653]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 577 op/s 2024-06-03T02:29:50.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:49 smithi123 ceph-mon[36640]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 577 op/s 2024-06-03T02:29:50.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.612+0000 7f8b42a00700 1 -- 172.21.15.79:0/548560821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b3c0772e0 msgr2=0x7f8b3c112e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:50.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.612+0000 7f8b42a00700 1 --2- 172.21.15.79:0/548560821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b3c0772e0 0x7f8b3c112e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009990 tx=0x7f8b28009ca0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.612+0000 7f8b42a00700 1 -- 172.21.15.79:0/548560821 shutdown_connections 2024-06-03T02:29:50.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.612+0000 7f8b42a00700 1 --2- 172.21.15.79:0/548560821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b3c113370 0x7f8b3c115760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.612+0000 7f8b42a00700 1 --2- 172.21.15.79:0/548560821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c0769c0 0x7f8b3c076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.612+0000 7f8b42a00700 1 --2- 172.21.15.79:0/548560821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b3c0772e0 0x7f8b3c112e30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.612+0000 7f8b42a00700 1 -- 172.21.15.79:0/548560821 >> 172.21.15.79:0/548560821 conn(0x7f8b3c1001c0 msgr2=0x7f8b3c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.614+0000 7f8b42a00700 1 -- 172.21.15.79:0/548560821 shutdown_connections 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.614+0000 7f8b42a00700 1 -- 172.21.15.79:0/548560821 wait complete. 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.616+0000 7f8b42a00700 1 Processor -- start 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.616+0000 7f8b42a00700 1 -- start start 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.616+0000 7f8b42a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c0769c0 0x7f8b3c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.616+0000 7f8b42a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b3c0772e0 0x7f8b3c19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.617+0000 7f8b42a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b3c113370 0x7f8b3c1a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.617+0000 7f8b42a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c118e80 con 0x7f8b3c0772e0 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.617+0000 7f8b42a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c118d00 con 0x7f8b3c0769c0 2024-06-03T02:29:50.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.617+0000 7f8b42a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c119000 con 0x7f8b3c113370 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.617+0000 7f8b40a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b3c113370 0x7f8b3c1a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.617+0000 7f8b3be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c0769c0 0x7f8b3c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.617+0000 7f8b3be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c0769c0 0x7f8b3c19f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:57634/0 (socket says 172.21.15.79:57634) 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.617+0000 7f8b3be00700 1 -- 172.21.15.79:0/2924110412 learned_addr learned my addr 172.21.15.79:0/2924110412 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.618+0000 7f8b40a00700 1 -- 172.21.15.79:0/2924110412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c0769c0 msgr2=0x7f8b3c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.618+0000 7f8b40a00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c0769c0 0x7f8b3c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.618+0000 7f8b40a00700 1 -- 172.21.15.79:0/2924110412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b3c0772e0 msgr2=0x7f8b3c19f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.618+0000 7f8b40a00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b3c0772e0 0x7f8b3c19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.618+0000 7f8b40a00700 1 -- 172.21.15.79:0/2924110412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c1a4400 con 0x7f8b3c113370 2024-06-03T02:29:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.619+0000 7f8b40a00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b3c113370 0x7f8b3c1a3cc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c07acb0 tx=0x7f8b3400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:50.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.619+0000 7f8b38c00700 1 -- 172.21.15.79:0/2924110412 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3400ddd0 con 0x7f8b3c113370 2024-06-03T02:29:50.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.619+0000 7f8b38c00700 1 -- 172.21.15.79:0/2924110412 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b34009e20 con 0x7f8b3c113370 2024-06-03T02:29:50.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.619+0000 7f8b38c00700 1 -- 172.21.15.79:0/2924110412 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34015660 con 0x7f8b3c113370 2024-06-03T02:29:50.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.620+0000 7f8b3be00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c0769c0 0x7f8b3c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:29:50.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.621+0000 7f8b42a00700 1 -- 172.21.15.79:0/2924110412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3c1a46f0 con 0x7f8b3c113370 2024-06-03T02:29:50.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.621+0000 7f8b42a00700 1 -- 172.21.15.79:0/2924110412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3c06beb0 con 0x7f8b3c113370 2024-06-03T02:29:50.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.632+0000 7f8b38c00700 1 -- 172.21.15.79:0/2924110412 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8b34004dd0 con 0x7f8b3c113370 2024-06-03T02:29:50.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.634+0000 7f8b38c00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b200a2d20 0x7f8b200a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.634+0000 7f8b38c00700 1 -- 172.21.15.79:0/2924110412 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8b340ceca0 con 0x7f8b3c113370 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.635+0000 7f8b4d330b80 1 Processor -- start 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.635+0000 7f8b4d330b80 1 -- start start 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.635+0000 7f8b3be00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b200a2d20 0x7f8b200a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aedf6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aee01300 0x5653aee036f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b3be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b3be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:57646/0 (socket says 172.21.15.79:57646) 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b3be00700 1 -- 172.21.15.79:0/1453428152 learned_addr learned my addr 172.21.15.79:0/1453428152 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1453428152 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5653aed05a10 con 0x5653aee01300 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1453428152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653aed48b00 con 0x5653aed72e40 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1453428152 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653aed038f0 con 0x5653aedf65b0 2024-06-03T02:29:50.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.636+0000 7f8b3b400700 1 --2- 172.21.15.79:0/1453428152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aedf6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.637+0000 7f8b3be00700 1 -- 172.21.15.79:0/1453428152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 msgr2=0x5653aedf6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.637+0000 7f8b3be00700 1 --2- 172.21.15.79:0/1453428152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aedf6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.637+0000 7f8b3be00700 1 -- 172.21.15.79:0/1453428152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aee01300 msgr2=0x5653aee036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.637+0000 7f8b3be00700 1 --2- 172.21.15.79:0/1453428152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aee01300 0x5653aee036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.637+0000 7f8b3be00700 1 -- 172.21.15.79:0/1453428152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653aecf0f90 con 0x5653aed72e40 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.638+0000 7f8b3be00700 1 --2- 172.21.15.79:0/1453428152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf6070 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c013d20 tx=0x7f8b2c00f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0dcb7f372291ed server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.638+0000 7f8b3be00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b200a2d20 0x7f8b200a51e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00f990 tx=0x7f8b2c011000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.639+0000 7f8b31600700 1 -- 172.21.15.79:0/1453428152 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0130e0 con 0x5653aed72e40 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.639+0000 7f8b31600700 1 -- 172.21.15.79:0/1453428152 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c023070 con 0x5653aed72e40 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.639+0000 7f8b31600700 1 -- 172.21.15.79:0/1453428152 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0130e0 con 0x5653aed72e40 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.641+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1453428152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 msgr2=0x5653aedf6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.641+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1453428152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf6070 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c013d20 tx=0x7f8b2c00f640 comp rx=0 tx=0).stop 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.643+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1453428152 shutdown_connections 2024-06-03T02:29:50.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.643+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1453428152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aee01300 0x5653aee036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.643+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1453428152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf6070 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.643+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1453428152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aedf6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.643+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1453428152 >> 172.21.15.79:0/1453428152 conn(0x5653aed23170 msgr2=0x5653aedf24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.643+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1453428152 shutdown_connections 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.644+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1453428152 wait complete. 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.645+0000 7f8b4d330b80 1 -- start start 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.645+0000 7f8b4d330b80 1 Processor -- start 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aed72e40 0x5653aee00b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aedf65b0 0x5653aedfde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aedfe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b4d330b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5653aece4f60 con 0x5653aedf65b0 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b4d330b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653aed44ac0 con 0x5653aee01300 2024-06-03T02:29:50.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b4d330b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653aee03f60 con 0x5653aed72e40 2024-06-03T02:29:50.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b4d330b80 10 client.0 _ll_register_callbacks cb 0x5653aece2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:50.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b40a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aedfe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b40a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aedfe360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:57656/0 (socket says 172.21.15.79:57656) 2024-06-03T02:29:50.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.646+0000 7f8b40a00700 1 -- 172.21.15.79:0/1600330363 learned_addr learned my addr 172.21.15.79:0/1600330363 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:50.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.647+0000 7f8b3be00700 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aed72e40 0x5653aee00b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.647+0000 7f8b40a00700 1 -- 172.21.15.79:0/1600330363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aed72e40 msgr2=0x5653aee00b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:50.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.647+0000 7f8b40a00700 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aed72e40 0x5653aee00b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.647+0000 7f8b40a00700 1 -- 172.21.15.79:0/1600330363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aedf65b0 msgr2=0x5653aedfde20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:29:50.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.648+0000 7f8b3b400700 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aedf65b0 0x5653aedfde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.648+0000 7f8b40a00700 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aedf65b0 0x5653aedfde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.648+0000 7f8b40a00700 1 -- 172.21.15.79:0/1600330363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653aec978d0 con 0x5653aee01300 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.649+0000 7f8b40a00700 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aedfe360 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8b3409b080 tx=0x7f8b3409b1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d342aa4a2f915c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.649+0000 7f8b26000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.649+0000 7f8b26000700 1 -- 172.21.15.79:0/1600330363 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b340cc4a0 con 0x5653aee01300 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.649+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1600330363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b2c023070 con 0x5653aee01300 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.649+0000 7f8b26000700 1 -- 172.21.15.79:0/1600330363 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8b340bb0c0 con 0x5653aee01300 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.649+0000 7f8b26000700 1 -- 172.21.15.79:0/1600330363 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b3409f160 con 0x5653aee01300 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.650+0000 7f8b4d330b80 10 client.26414 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.650+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1600330363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5653aee0e7e0 con 0x5653aee01300 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.652+0000 7f8b26000700 1 -- 172.21.15.79:0/1600330363 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3409b350 con 0x5653aee01300 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.652+0000 7f8b26000700 1 -- 172.21.15.79:0/1600330363 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8b340b5020 con 0x5653aee01300 2024-06-03T02:29:50.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.652+0000 7f8b4d330b80 10 client.26414 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.652+0000 7f8b4d330b80 10 client.26414 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.652+0000 7f8b4d330b80 20 client.26414 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.652+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5653aee10d70 0x5653aee13160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.652+0000 7f8b4d330b80 4 client.26414 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.652+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1600330363 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8b2c023070 con 0x5653aee10d70 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.653+0000 7f8b3be00700 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5653aee10d70 0x5653aee13160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.654+0000 7f8b3be00700 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5653aee10d70 0x5653aee13160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a8bcdb8b6b18c8c server_cookie=ccaf08f83c32e221 in_seq=0 out_seq=0 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.655+0000 7f8b26000700 10 client.26414 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.656+0000 7f8b26000700 1 -- 172.21.15.79:0/1600330363 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f8b2c023070 con 0x5653aee10d70 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.656+0000 7f8b26000700 10 client.26414 handle_command_reply: tid=0 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.656+0000 7f8b4d330b80 1 client.26414 shutdown 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.657+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1600330363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 msgr2=0x5653aedfe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.657+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aedfe360 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8b3409b080 tx=0x7f8b3409b1a0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.658+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1600330363 shutdown_connections 2024-06-03T02:29:50.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.658+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5653aee10d70 0x5653aee13160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.658+0000 7f8b26000700 0 client.26414 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.658+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aedfe360 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.658+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aed72e40 0x5653aee00b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.658+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1600330363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aedf65b0 0x5653aedfde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.658+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1600330363 >> 172.21.15.79:0/1600330363 conn(0x5653aed23170 msgr2=0x5653aedf1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.658+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1600330363 shutdown_connections 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.658+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1600330363 wait complete. 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.659+0000 7f8b4d330b80 20 client.26414 trim_cache size 0 max 16384 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.683+0000 7f8b4d330b80 1 Processor -- start 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.683+0000 7f8b4d330b80 1 -- start start 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aedfcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aee01300 0x5653aedf15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b4d330b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1bd370 con 0x5653aee01300 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b4d330b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1beeb0 con 0x5653aed72e40 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b4d330b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1bda40 con 0x5653aedf65b0 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aedfcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:57664/0 (socket says 172.21.15.79:57664) 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b40a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aee01300 0x5653aedf15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aedfcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52472/0 (socket says 172.21.15.79:52472) 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3be00700 1 -- 172.21.15.79:0/1531055540 learned_addr learned my addr 172.21.15.79:0/1531055540 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3be00700 1 -- 172.21.15.79:0/1531055540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 msgr2=0x5653aedfcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3be00700 1 --2- 172.21.15.79:0/1531055540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aedfcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3be00700 1 -- 172.21.15.79:0/1531055540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aee01300 msgr2=0x5653aedf15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3be00700 1 --2- 172.21.15.79:0/1531055540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aee01300 0x5653aedf15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.684+0000 7f8b3be00700 1 -- 172.21.15.79:0/1531055540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653aedf6e30 con 0x5653aed72e40 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.685+0000 7f8b3be00700 1 --2- 172.21.15.79:0/1531055540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf2620 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c013d20 tx=0x7f8b2c0136d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcad1bf3df5b6361 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.685+0000 7f8b30c00700 1 -- 172.21.15.79:0/1531055540 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0130e0 con 0x5653aed72e40 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.685+0000 7f8b30c00700 1 -- 172.21.15.79:0/1531055540 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c023040 con 0x5653aed72e40 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.685+0000 7f8b30c00700 1 -- 172.21.15.79:0/1531055540 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c005440 con 0x5653aed72e40 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1531055540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 msgr2=0x5653aedf2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1531055540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf2620 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c013d20 tx=0x7f8b2c0136d0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1531055540 shutdown_connections 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1531055540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aee01300 0x5653aedf15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1531055540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aed72e40 0x5653aedf2620 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/1531055540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aedfcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1531055540 >> 172.21.15.79:0/1531055540 conn(0x5653aed23170 msgr2=0x5653aedf0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1531055540 shutdown_connections 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 -- 172.21.15.79:0/1531055540 wait complete. 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.686+0000 7f8b4d330b80 1 -- start start 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.687+0000 7f8b4d330b80 1 Processor -- start 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.687+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aed72e40 0x5653aee016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.687+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aee01c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.687+0000 7f8b4d330b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aee02160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.687+0000 7f8b4d330b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1be7e0 con 0x5653aed72e40 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b4d330b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1be110 con 0x5653aee01300 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b4d330b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c072a50 con 0x5653aedf65b0 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b40a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aee02160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b4d330b80 10 client.0 _ll_register_callbacks cb 0x5653aece2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b40a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aee02160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:57680/0 (socket says 172.21.15.79:57680) 2024-06-03T02:29:50.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b40a00700 1 -- 172.21.15.79:0/2194118560 learned_addr learned my addr 172.21.15.79:0/2194118560 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b3b400700 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aee01c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b3be00700 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aed72e40 0x5653aee016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b40a00700 1 -- 172.21.15.79:0/2194118560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 msgr2=0x5653aee01c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b40a00700 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aee01c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b40a00700 1 -- 172.21.15.79:0/2194118560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aed72e40 msgr2=0x5653aee016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b40a00700 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aed72e40 0x5653aee016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b40a00700 1 -- 172.21.15.79:0/2194118560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653aee081c0 con 0x5653aee01300 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b3be00700 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aed72e40 0x5653aee016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b40a00700 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aee02160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8b34097a70 tx=0x7f8b340055a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503aa0a63128bae0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b26000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.688+0000 7f8b26000700 1 -- 172.21.15.79:0/2194118560 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3409aec0 con 0x5653aee01300 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b4d330b80 1 -- 172.21.15.79:0/2194118560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b2c023040 con 0x5653aee01300 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b26000700 1 -- 172.21.15.79:0/2194118560 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8b340bb0b0 con 0x5653aee01300 2024-06-03T02:29:50.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b26000700 1 -- 172.21.15.79:0/2194118560 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b3409a5e0 con 0x5653aee01300 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b4d330b80 10 client.26420 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b4d330b80 1 -- 172.21.15.79:0/2194118560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5653aee094c0 con 0x5653aee01300 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b26000700 1 -- 172.21.15.79:0/2194118560 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c023040 con 0x5653aee01300 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b26000700 1 -- 172.21.15.79:0/2194118560 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8b340b5040 con 0x5653aee01300 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b4d330b80 10 client.26420 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b4d330b80 10 client.26420 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b4d330b80 20 client.26420 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5653aee10d70 0x5653aee7d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b4d330b80 4 client.26420 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b4d330b80 1 -- 172.21.15.79:0/2194118560 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653aed56000 con 0x5653aee10d70 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.689+0000 7f8b3be00700 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5653aee10d70 0x5653aee7d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.690+0000 7f8b3be00700 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5653aee10d70 0x5653aee7d990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23458088c2dda86f server_cookie=1ec97d8a017f1b5 in_seq=0 out_seq=0 2024-06-03T02:29:50.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.690+0000 7f8b26000700 10 client.26420 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.690+0000 7f8b26000700 1 -- 172.21.15.79:0/2194118560 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653aed56000 con 0x5653aee10d70 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.690+0000 7f8b26000700 10 client.26420 handle_command_reply: tid=0 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.690+0000 7f8b4d330b80 1 client.26420 shutdown 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 -- 172.21.15.79:0/2194118560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 msgr2=0x5653aee02160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aee02160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8b34097a70 tx=0x7f8b340055a0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 -- 172.21.15.79:0/2194118560 shutdown_connections 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5653aee10d70 0x5653aee7d990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653aee01300 0x5653aee02160 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5653aed72e40 0x5653aee016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b26000700 0 client.26420 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 --2- 172.21.15.79:0/2194118560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653aedf65b0 0x5653aee01c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 -- 172.21.15.79:0/2194118560 >> 172.21.15.79:0/2194118560 conn(0x5653aed23170 msgr2=0x5653aedf1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 -- 172.21.15.79:0/2194118560 shutdown_connections 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 1 -- 172.21.15.79:0/2194118560 wait complete. 2024-06-03T02:29:50.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.691+0000 7f8b4d330b80 20 client.26420 trim_cache size 0 max 16384 2024-06-03T02:29:50.996 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:29:50.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 -- 172.21.15.79:0/2924110412 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b200a2d20 msgr2=0x7f8b200a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:50.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b200a2d20 0x7f8b200a51e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00f990 tx=0x7f8b2c011000 comp rx=0 tx=0).stop 2024-06-03T02:29:50.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 -- 172.21.15.79:0/2924110412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b3c113370 msgr2=0x7f8b3c1a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:50.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b3c113370 0x7f8b3c1a3cc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c07acb0 tx=0x7f8b3400d3b0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 -- 172.21.15.79:0/2924110412 shutdown_connections 2024-06-03T02:29:50.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b3c113370 0x7f8b3c1a3cc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c0769c0 0x7f8b3c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b200a2d20 0x7f8b200a51e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 --2- 172.21.15.79:0/2924110412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b3c0772e0 0x7f8b3c19f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:50.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.693+0000 7f8b42a00700 1 -- 172.21.15.79:0/2924110412 >> 172.21.15.79:0/2924110412 conn(0x7f8b3c1001c0 msgr2=0x7f8b3c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:50.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.694+0000 7f8b42a00700 1 -- 172.21.15.79:0/2924110412 shutdown_connections 2024-06-03T02:29:50.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:50.694+0000 7f8b42a00700 1 -- 172.21.15.79:0/2924110412 wait complete. 2024-06-03T02:29:52.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:51 smithi135 ceph-mon[36653]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-06-03T02:29:52.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:51 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:29:52.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:51 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:29:52.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:51 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:29:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:51 smithi123 ceph-mon[36640]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-06-03T02:29:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:51 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:29:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:51 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:29:52.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:51 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:29:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:51 smithi079 ceph-mon[31819]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-06-03T02:29:52.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:51 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:29:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:51 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:29:52.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:51 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:29:52.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:29:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 585.0 s, 445.8 tps, lat 228.555 ms stddev 274.853 2024-06-03T02:29:53.358 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:29:54.138 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:53 smithi079 ceph-mon[31819]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.9 MiB/s wr, 450 op/s 2024-06-03T02:29:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:53 smithi123 ceph-mon[36640]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.9 MiB/s wr, 450 op/s 2024-06-03T02:29:54.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:53 smithi135 ceph-mon[36653]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.9 MiB/s wr, 450 op/s 2024-06-03T02:29:56.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:55 smithi079 ceph-mon[31819]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-06-03T02:29:56.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:55 smithi123 ceph-mon[36640]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-06-03T02:29:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:55 smithi135 ceph-mon[36653]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-06-03T02:29:58.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.124+0000 7f0782800700 1 -- 172.21.15.79:0/2150285862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c10ddd0 msgr2=0x7f077c1158f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:58.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.124+0000 7f0782800700 1 --2- 172.21.15.79:0/2150285862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c10ddd0 0x7f077c1158f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f077400b0a0 tx=0x7f077400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.125+0000 7f0782800700 1 -- 172.21.15.79:0/2150285862 shutdown_connections 2024-06-03T02:29:58.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.125+0000 7f0782800700 1 --2- 172.21.15.79:0/2150285862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c10ddd0 0x7f077c1158f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.125+0000 7f0782800700 1 --2- 172.21.15.79:0/2150285862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f077c109530 0x7f077c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.125+0000 7f0782800700 1 --2- 172.21.15.79:0/2150285862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c109ee0 0x7f077c10d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.125+0000 7f0782800700 1 -- 172.21.15.79:0/2150285862 >> 172.21.15.79:0/2150285862 conn(0x7f077c0781c0 msgr2=0x7f077c07a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:58.127 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:57 smithi079 ceph-mon[31819]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 384 op/s 2024-06-03T02:29:58.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.128+0000 7f0782800700 1 -- 172.21.15.79:0/2150285862 shutdown_connections 2024-06-03T02:29:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.128+0000 7f0782800700 1 -- 172.21.15.79:0/2150285862 wait complete. 2024-06-03T02:29:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.130+0000 7f0782800700 1 Processor -- start 2024-06-03T02:29:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.130+0000 7f0782800700 1 -- start start 2024-06-03T02:29:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.132+0000 7f0782800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c109530 0x7f077c19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.132+0000 7f0782800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c109ee0 0x7f077c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.132+0000 7f0782800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f077c10ddd0 0x7f077c1a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.132+0000 7f0781400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c109530 0x7f077c19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.132+0000 7f0781400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c109530 0x7f077c19f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54718/0 (socket says 172.21.15.79:54718) 2024-06-03T02:29:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.132+0000 7f0781400700 1 -- 172.21.15.79:0/598498325 learned_addr learned my addr 172.21.15.79:0/598498325 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.133+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f077c118d50 con 0x7f077c10ddd0 2024-06-03T02:29:58.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.133+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f077c118bd0 con 0x7f077c109ee0 2024-06-03T02:29:58.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.133+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f077c118ed0 con 0x7f077c109530 2024-06-03T02:29:58.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.133+0000 7f0780a00700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c109ee0 0x7f077c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.135+0000 7f0781e00700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f077c10ddd0 0x7f077c1a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.135+0000 7f0781400700 1 -- 172.21.15.79:0/598498325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c109ee0 msgr2=0x7f077c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:58.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.135+0000 7f0781400700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c109ee0 0x7f077c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.135+0000 7f0781400700 1 -- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f077c10ddd0 msgr2=0x7f077c1a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:58.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.135+0000 7f0781400700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f077c10ddd0 0x7f077c1a39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.135+0000 7f0781400700 1 -- 172.21.15.79:0/598498325 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c1a4120 con 0x7f077c109530 2024-06-03T02:29:58.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.136+0000 7f0781400700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c109530 0x7f077c19f160 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f076800d980 tx=0x7f076800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:58.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.136+0000 7f077a000700 1 -- 172.21.15.79:0/598498325 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768016070 con 0x7f077c109530 2024-06-03T02:29:58.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.136+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077c1a4410 con 0x7f077c109530 2024-06-03T02:29:58.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.137+0000 7f077a000700 1 -- 172.21.15.79:0/598498325 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f076800be30 con 0x7f077c109530 2024-06-03T02:29:58.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.137+0000 7f077a000700 1 -- 172.21.15.79:0/598498325 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076800f3c0 con 0x7f077c109530 2024-06-03T02:29:58.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.137+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077c1abc90 con 0x7f077c109530 2024-06-03T02:29:58.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.146+0000 7f077a000700 1 -- 172.21.15.79:0/598498325 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0768019070 con 0x7f077c109530 2024-06-03T02:29:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.152+0000 7f077a000700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f07640a2ca0 0x7f07640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.152+0000 7f078e73fb80 1 Processor -- start 2024-06-03T02:29:58.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.152+0000 7f077a000700 1 -- 172.21.15.79:0/598498325 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f07680cef00 con 0x7f077c109530 2024-06-03T02:29:58.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.152+0000 7f0780a00700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f07640a2ca0 0x7f07640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.153+0000 7f0780a00700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f07640a2ca0 0x7f07640a5160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f077c07b7f0 tx=0x7f07700073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:58.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.154+0000 7f078e73fb80 1 -- start start 2024-06-03T02:29:58.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.154+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468c42e40 0x55b468cc8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.154+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cc8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.154+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cd3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.154+0000 7f078e73fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b468bd5a10 con 0x55b468c42e40 2024-06-03T02:29:58.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.154+0000 7f078e73fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b468c18b00 con 0x55b468cc8960 2024-06-03T02:29:58.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.154+0000 7f078e73fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b468bd38f0 con 0x55b468cd1570 2024-06-03T02:29:58.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.155+0000 7f0780a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cc8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.155+0000 7f0780a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cc8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49400/0 (socket says 172.21.15.79:49400) 2024-06-03T02:29:58.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.155+0000 7f0780a00700 1 -- 172.21.15.79:0/1016364300 learned_addr learned my addr 172.21.15.79:0/1016364300 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:58.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.157+0000 7f0781e00700 1 --2- 172.21.15.79:0/1016364300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cd3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.159+0000 7f0781e00700 1 -- 172.21.15.79:0/1016364300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 msgr2=0x55b468cc8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:58.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.159+0000 7f0781e00700 1 --2- 172.21.15.79:0/1016364300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cc8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.159+0000 7f0781e00700 1 -- 172.21.15.79:0/1016364300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468c42e40 msgr2=0x55b468cc8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:58.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.159+0000 7f0781e00700 1 --2- 172.21.15.79:0/1016364300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468c42e40 0x55b468cc8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.159+0000 7f0781e00700 1 -- 172.21.15.79:0/1016364300 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b468bc0f90 con 0x55b468cd1570 2024-06-03T02:29:58.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.160+0000 7f0781e00700 1 --2- 172.21.15.79:0/1016364300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cd3960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fd0 tx=0x7f0774009d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b364e1b27d44d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:58.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.162+0000 7f076ea00700 1 -- 172.21.15.79:0/1016364300 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774009350 con 0x55b468cd1570 2024-06-03T02:29:58.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.163+0000 7f076ea00700 1 -- 172.21.15.79:0/1016364300 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07740074e0 con 0x55b468cd1570 2024-06-03T02:29:58.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.163+0000 7f076ea00700 1 -- 172.21.15.79:0/1016364300 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077401c680 con 0x55b468cd1570 2024-06-03T02:29:58.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.164+0000 7f078e73fb80 1 -- 172.21.15.79:0/1016364300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 msgr2=0x55b468cd3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:58.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.164+0000 7f078e73fb80 1 --2- 172.21.15.79:0/1016364300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cd3960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fd0 tx=0x7f0774009d10 comp rx=0 tx=0).stop 2024-06-03T02:29:58.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.169+0000 7f078e73fb80 1 -- 172.21.15.79:0/1016364300 shutdown_connections 2024-06-03T02:29:58.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.169+0000 7f078e73fb80 1 --2- 172.21.15.79:0/1016364300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cd3960 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.169+0000 7f078e73fb80 1 --2- 172.21.15.79:0/1016364300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468c42e40 0x55b468cc8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.169+0000 7f078e73fb80 1 --2- 172.21.15.79:0/1016364300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cc8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.170+0000 7f078e73fb80 1 -- 172.21.15.79:0/1016364300 >> 172.21.15.79:0/1016364300 conn(0x55b468bf3170 msgr2=0x55b468cc2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:58.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.170+0000 7f078e73fb80 1 -- 172.21.15.79:0/1016364300 shutdown_connections 2024-06-03T02:29:58.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.170+0000 7f078e73fb80 1 -- 172.21.15.79:0/1016364300 wait complete. 2024-06-03T02:29:58.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.171+0000 7f078e73fb80 1 -- start start 2024-06-03T02:29:58.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.171+0000 7f078e73fb80 1 Processor -- start 2024-06-03T02:29:58.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.173+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468c42e40 0x55b468cd0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.173+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.173+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cd1570 0x55b468cce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.173+0000 7f078e73fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b468bb4f60 con 0x55b468cd1570 2024-06-03T02:29:58.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.173+0000 7f078e73fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b468c14ac0 con 0x55b468cc8960 2024-06-03T02:29:58.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.173+0000 7f078e73fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b468cd4120 con 0x55b468c42e40 2024-06-03T02:29:58.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.173+0000 7f078e73fb80 10 client.0 _ll_register_callbacks cb 0x55b468bb2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:58.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.174+0000 7f0781e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cd1570 0x55b468cce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.174+0000 7f0781e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cd1570 0x55b468cce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56066/0 (socket says 172.21.15.79:56066) 2024-06-03T02:29:58.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.174+0000 7f0781e00700 1 -- 172.21.15.79:0/4268512041 learned_addr learned my addr 172.21.15.79:0/4268512041 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:58.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.175+0000 7f0781e00700 1 -- 172.21.15.79:0/4268512041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468c42e40 msgr2=0x55b468cd0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:58.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.175+0000 7f0781e00700 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468c42e40 0x55b468cd0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.175+0000 7f0781e00700 1 -- 172.21.15.79:0/4268512041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 msgr2=0x55b468cce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:58.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.175+0000 7f0781e00700 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.176+0000 7f0781e00700 1 -- 172.21.15.79:0/4268512041 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b468b678d0 con 0x55b468cd1570 2024-06-03T02:29:58.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.176+0000 7f0781e00700 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cd1570 0x55b468cce5d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0774004c80 tx=0x7f0774004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bb5fff1649929a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:58.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.176+0000 7f0763400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:29:58.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.180+0000 7f0763400700 1 -- 172.21.15.79:0/4268512041 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077402b070 con 0x55b468cd1570 2024-06-03T02:29:58.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.180+0000 7f0763400700 1 -- 172.21.15.79:0/4268512041 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0774007aa0 con 0x55b468cd1570 2024-06-03T02:29:58.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.180+0000 7f078e73fb80 1 -- 172.21.15.79:0/4268512041 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07740074e0 con 0x55b468cd1570 2024-06-03T02:29:58.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.181+0000 7f0763400700 1 -- 172.21.15.79:0/4268512041 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0774035050 con 0x55b468cd1570 2024-06-03T02:29:58.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.181+0000 7f078e73fb80 10 client.15972 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:58.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.181+0000 7f078e73fb80 1 -- 172.21.15.79:0/4268512041 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b468cde8f0 con 0x55b468cd1570 2024-06-03T02:29:58.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.182+0000 7f0763400700 1 -- 172.21.15.79:0/4268512041 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07740074e0 con 0x55b468cd1570 2024-06-03T02:29:58.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.183+0000 7f0763400700 1 -- 172.21.15.79:0/4268512041 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0774011070 con 0x55b468cd1570 2024-06-03T02:29:58.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.183+0000 7f078e73fb80 10 client.15972 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:58.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.183+0000 7f078e73fb80 10 client.15972 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:29:58.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.183+0000 7f078e73fb80 20 client.15972 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:58.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.183+0000 7f078e73fb80 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b468ce0dc0 0x55b468ce31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.183+0000 7f078e73fb80 4 client.15972 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:29:58.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.183+0000 7f078e73fb80 1 -- 172.21.15.79:0/4268512041 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b468c26000 con 0x55b468ce0dc0 2024-06-03T02:29:58.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.186+0000 7f0781400700 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b468ce0dc0 0x55b468ce31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.188+0000 7f0781400700 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b468ce0dc0 0x55b468ce31b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb2e965ff26e6b89 server_cookie=ee0175480b738515 in_seq=0 out_seq=0 2024-06-03T02:29:58.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.188+0000 7f0763400700 10 client.15972 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:29:58.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.189+0000 7f0763400700 1 -- 172.21.15.79:0/4268512041 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b468c26000 con 0x55b468ce0dc0 2024-06-03T02:29:58.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.189+0000 7f0763400700 10 client.15972 handle_command_reply: tid=0 2024-06-03T02:29:58.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.190+0000 7f078e73fb80 1 client.15972 shutdown 2024-06-03T02:29:58.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.190+0000 7f078e73fb80 1 -- 172.21.15.79:0/4268512041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cd1570 msgr2=0x55b468cce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.190+0000 7f078e73fb80 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cd1570 0x55b468cce5d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0774004c80 tx=0x7f0774004d60 comp rx=0 tx=0).stop 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.190+0000 7f078e73fb80 1 -- 172.21.15.79:0/4268512041 shutdown_connections 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.190+0000 7f078e73fb80 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b468ce0dc0 0x55b468ce31b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.191+0000 7f0763400700 0 client.15972 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.191+0000 7f078e73fb80 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cd1570 0x55b468cce5d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.191+0000 7f078e73fb80 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468c42e40 0x55b468cd0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.191+0000 7f078e73fb80 1 --2- 172.21.15.79:0/4268512041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.191+0000 7f078e73fb80 1 -- 172.21.15.79:0/4268512041 >> 172.21.15.79:0/4268512041 conn(0x55b468bf3170 msgr2=0x55b468cc2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.191+0000 7f078e73fb80 1 -- 172.21.15.79:0/4268512041 shutdown_connections 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.191+0000 7f078e73fb80 1 -- 172.21.15.79:0/4268512041 wait complete. 2024-06-03T02:29:58.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.191+0000 7f078e73fb80 20 client.15972 trim_cache size 0 max 16384 2024-06-03T02:29:58.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:57 smithi123 ceph-mon[36640]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 384 op/s 2024-06-03T02:29:58.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.225+0000 7f078e73fb80 1 Processor -- start 2024-06-03T02:29:58.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.225+0000 7f078e73fb80 1 -- start start 2024-06-03T02:29:58.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468c42e40 0x55b468ccd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cd08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cc9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f078e73fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1b4720 con 0x55b468c42e40 2024-06-03T02:29:58.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f078e73fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1b6260 con 0x55b468cc8960 2024-06-03T02:29:58.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f078e73fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1b4df0 con 0x55b468cd1570 2024-06-03T02:29:58.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f0781e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cc9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f0781400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468c42e40 0x55b468ccd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f0781e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cc9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54746/0 (socket says 172.21.15.79:54746) 2024-06-03T02:29:58.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f0781e00700 1 -- 172.21.15.79:0/1183141417 learned_addr learned my addr 172.21.15.79:0/1183141417 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:58.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.226+0000 7f0780a00700 1 --2- 172.21.15.79:0/1183141417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cd08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.227+0000 7f0780a00700 1 -- 172.21.15.79:0/1183141417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 msgr2=0x55b468cc9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:58.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.227+0000 7f0780a00700 1 --2- 172.21.15.79:0/1183141417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cc9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.227+0000 7f0780a00700 1 -- 172.21.15.79:0/1183141417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468c42e40 msgr2=0x55b468ccd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:58.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.227+0000 7f0780a00700 1 --2- 172.21.15.79:0/1183141417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468c42e40 0x55b468ccd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.227+0000 7f0780a00700 1 -- 172.21.15.79:0/1183141417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b468cc1520 con 0x55b468cc8960 2024-06-03T02:29:58.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.227+0000 7f0781e00700 1 --2- 172.21.15.79:0/1183141417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cc9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:29:58.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.228+0000 7f0780a00700 1 --2- 172.21.15.79:0/1183141417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cd08d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55b468ccf660 tx=0x7f0770013070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e540a513d92c6f6d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:58.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.228+0000 7f076e000700 1 -- 172.21.15.79:0/1183141417 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770011820 con 0x55b468cc8960 2024-06-03T02:29:58.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.228+0000 7f076e000700 1 -- 172.21.15.79:0/1183141417 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0770023070 con 0x55b468cc8960 2024-06-03T02:29:58.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.229+0000 7f076e000700 1 -- 172.21.15.79:0/1183141417 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077001f430 con 0x55b468cc8960 2024-06-03T02:29:58.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.230+0000 7f078e73fb80 1 -- 172.21.15.79:0/1183141417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 msgr2=0x55b468cd08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:58.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.230+0000 7f078e73fb80 1 --2- 172.21.15.79:0/1183141417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cd08d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55b468ccf660 tx=0x7f0770013070 comp rx=0 tx=0).stop 2024-06-03T02:29:58.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.230+0000 7f078e73fb80 1 -- 172.21.15.79:0/1183141417 shutdown_connections 2024-06-03T02:29:58.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.230+0000 7f078e73fb80 1 --2- 172.21.15.79:0/1183141417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cc9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.230+0000 7f078e73fb80 1 --2- 172.21.15.79:0/1183141417 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468c42e40 0x55b468ccd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.230+0000 7f078e73fb80 1 --2- 172.21.15.79:0/1183141417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468cc8960 0x55b468cd08d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.230+0000 7f078e73fb80 1 -- 172.21.15.79:0/1183141417 >> 172.21.15.79:0/1183141417 conn(0x55b468bf3170 msgr2=0x55b468cc8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:58.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.231+0000 7f078e73fb80 1 -- 172.21.15.79:0/1183141417 shutdown_connections 2024-06-03T02:29:58.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.231+0000 7f078e73fb80 1 -- 172.21.15.79:0/1183141417 wait complete. 2024-06-03T02:29:58.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.232+0000 7f078e73fb80 1 -- start start 2024-06-03T02:29:58.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.232+0000 7f078e73fb80 1 Processor -- start 2024-06-03T02:29:58.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.233+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468c42e40 0x55b468cd4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.233+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cc8960 0x55b468cd4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.233+0000 7f078e73fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cd4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.233+0000 7f078e73fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1b5b90 con 0x55b468cc8960 2024-06-03T02:29:58.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.233+0000 7f078e73fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1b54c0 con 0x55b468c42e40 2024-06-03T02:29:58.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.233+0000 7f078e73fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1b26c0 con 0x55b468cd1570 2024-06-03T02:29:58.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.233+0000 7f078e73fb80 10 client.0 _ll_register_callbacks cb 0x55b468bb2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:29:58.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.234+0000 7f0781400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468c42e40 0x55b468cd4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.234+0000 7f0781400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468c42e40 0x55b468cd4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49418/0 (socket says 172.21.15.79:49418) 2024-06-03T02:29:58.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.234+0000 7f0781400700 1 -- 172.21.15.79:0/34571594 learned_addr learned my addr 172.21.15.79:0/34571594 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:29:58.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.234+0000 7f0781e00700 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cd4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.235+0000 7f0780a00700 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cc8960 0x55b468cd4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.235+0000 7f0781400700 1 -- 172.21.15.79:0/34571594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 msgr2=0x55b468cd4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:58.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.235+0000 7f0781400700 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cd4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.235+0000 7f0781400700 1 -- 172.21.15.79:0/34571594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cc8960 msgr2=0x55b468cd4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:29:58.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.235+0000 7f0781400700 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cc8960 0x55b468cd4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.236+0000 7f0781400700 1 -- 172.21.15.79:0/34571594 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b468d49a20 con 0x55b468c42e40 2024-06-03T02:29:58.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.239+0000 7f0780a00700 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cc8960 0x55b468cd4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:29:58.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.239+0000 7f0781400700 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468c42e40 0x55b468cd4410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07680cc650 tx=0x7f076809b170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ea61b4b5bfcee02 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:29:58.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.239+0000 7f0763400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:29:58.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.240+0000 7f0763400700 1 -- 172.21.15.79:0/34571594 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07680b3070 con 0x55b468c42e40 2024-06-03T02:29:58.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.240+0000 7f0763400700 1 -- 172.21.15.79:0/34571594 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0768010640 con 0x55b468c42e40 2024-06-03T02:29:58.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.240+0000 7f078e73fb80 1 -- 172.21.15.79:0/34571594 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0770023070 con 0x55b468c42e40 2024-06-03T02:29:58.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.241+0000 7f0763400700 1 -- 172.21.15.79:0/34571594 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07680b4260 con 0x55b468c42e40 2024-06-03T02:29:58.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.241+0000 7f078e73fb80 10 client.26435 fetch_fsmap learned FSMap version 30 2024-06-03T02:29:58.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.241+0000 7f078e73fb80 1 -- 172.21.15.79:0/34571594 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b468d4ad00 con 0x55b468c42e40 2024-06-03T02:29:58.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.242+0000 7f0763400700 1 -- 172.21.15.79:0/34571594 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0770023070 con 0x55b468c42e40 2024-06-03T02:29:58.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.244+0000 7f0763400700 1 -- 172.21.15.79:0/34571594 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0768084b70 con 0x55b468c42e40 2024-06-03T02:29:58.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.244+0000 7f078e73fb80 10 client.26435 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:29:58.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.244+0000 7f078e73fb80 10 client.26435 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:29:58.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.244+0000 7f078e73fb80 20 client.26435 populate_metadata read hostname 'smithi079' 2024-06-03T02:29:58.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.244+0000 7f078e73fb80 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b468d4b360 0x55b468d4d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:29:58.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.244+0000 7f078e73fb80 4 client.26435 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:29:58.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.244+0000 7f078e73fb80 1 -- 172.21.15.79:0/34571594 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b468c20580 con 0x55b468d4b360 2024-06-03T02:29:58.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.244+0000 7f0780a00700 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b468d4b360 0x55b468d4d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:29:58.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.245+0000 7f0763400700 10 client.26435 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:29:58.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.245+0000 7f0780a00700 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b468d4b360 0x55b468d4d750 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4d131fe1110d835 server_cookie=d849a021323cdc88 in_seq=0 out_seq=0 2024-06-03T02:29:58.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.245+0000 7f0763400700 1 -- 172.21.15.79:0/34571594 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b468c20580 con 0x55b468d4b360 2024-06-03T02:29:58.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.245+0000 7f0763400700 10 client.26435 handle_command_reply: tid=0 2024-06-03T02:29:58.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f078e73fb80 1 client.26435 shutdown 2024-06-03T02:29:58.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f078e73fb80 1 -- 172.21.15.79:0/34571594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468c42e40 msgr2=0x55b468cd4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:58.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f078e73fb80 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468c42e40 0x55b468cd4410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07680cc650 tx=0x7f076809b170 comp rx=0 tx=0).stop 2024-06-03T02:29:58.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f078e73fb80 1 -- 172.21.15.79:0/34571594 shutdown_connections 2024-06-03T02:29:58.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f078e73fb80 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b468cd1570 0x55b468cd4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f078e73fb80 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b468c42e40 0x55b468cd4410 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f078e73fb80 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b468d4b360 0x55b468d4d750 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f078e73fb80 1 --2- 172.21.15.79:0/34571594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b468cc8960 0x55b468cd4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f0763400700 0 client.26435 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:29:58.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.246+0000 7f078e73fb80 1 -- 172.21.15.79:0/34571594 >> 172.21.15.79:0/34571594 conn(0x55b468bf3170 msgr2=0x55b468cc8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:58.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.247+0000 7f078e73fb80 1 -- 172.21.15.79:0/34571594 shutdown_connections 2024-06-03T02:29:58.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.247+0000 7f078e73fb80 1 -- 172.21.15.79:0/34571594 wait complete. 2024-06-03T02:29:58.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.247+0000 7f078e73fb80 20 client.26435 trim_cache size 0 max 16384 2024-06-03T02:29:58.249 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f07640a2ca0 msgr2=0x7f07640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f07640a2ca0 0x7f07640a5160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f077c07b7f0 tx=0x7f07700073d0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c109530 msgr2=0x7f077c19f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c109530 0x7f077c19f160 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f076800d980 tx=0x7f076800dc90 comp rx=0 tx=0).stop 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 shutdown_connections 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f07640a2ca0 0x7f07640a5160 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f077c10ddd0 0x7f077c1a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c109530 0x7f077c19f160 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 --2- 172.21.15.79:0/598498325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f077c109ee0 0x7f077c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:29:58.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 >> 172.21.15.79:0/598498325 conn(0x7f077c0781c0 msgr2=0x7f077c078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:29:58.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 shutdown_connections 2024-06-03T02:29:58.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:29:58.250+0000 7f0782800700 1 -- 172.21.15.79:0/598498325 wait complete. 2024-06-03T02:29:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 590.0 s, 561.2 tps, lat 169.768 ms stddev 183.658 2024-06-03T02:29:58.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:57 smithi135 ceph-mon[36653]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 384 op/s 2024-06-03T02:30:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:29:59 smithi123 ceph-mon[36640]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.3 MiB/s wr, 577 op/s 2024-06-03T02:30:00.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:29:59 smithi079 ceph-mon[31819]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.3 MiB/s wr, 577 op/s 2024-06-03T02:30:00.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:29:59 smithi135 ceph-mon[36653]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.3 MiB/s wr, 577 op/s 2024-06-03T02:30:00.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:30:01.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:00 smithi079 ceph-mon[31819]: overall HEALTH_OK 2024-06-03T02:30:01.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:00 smithi123 ceph-mon[36640]: overall HEALTH_OK 2024-06-03T02:30:01.335 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:30:01.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:00 smithi135 ceph-mon[36653]: overall HEALTH_OK 2024-06-03T02:30:02.007 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:01 smithi079 ceph-mon[31819]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.6 MiB/s wr, 480 op/s 2024-06-03T02:30:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:01 smithi123 ceph-mon[36640]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.6 MiB/s wr, 480 op/s 2024-06-03T02:30:02.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:01 smithi135 ceph-mon[36653]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.6 MiB/s wr, 480 op/s 2024-06-03T02:30:03.312 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 595.0 s, 458.3 tps, lat 224.674 ms stddev 252.497 2024-06-03T02:30:04.048 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:03 smithi079 ceph-mon[31819]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 495 op/s 2024-06-03T02:30:04.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:03 smithi123 ceph-mon[36640]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 495 op/s 2024-06-03T02:30:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:03 smithi135 ceph-mon[36653]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 495 op/s 2024-06-03T02:30:04.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.735+0000 7f6ded400700 1 -- 172.21.15.79:0/4183666333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6de810ee90 msgr2=0x7f6de8111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:04.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.735+0000 7f6ded400700 1 --2- 172.21.15.79:0/4183666333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6de810ee90 0x7f6de8111280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6de00098f0 tx=0x7f6de0009c00 comp rx=0 tx=0).stop 2024-06-03T02:30:04.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.736+0000 7f6ded400700 1 -- 172.21.15.79:0/4183666333 shutdown_connections 2024-06-03T02:30:04.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.736+0000 7f6ded400700 1 --2- 172.21.15.79:0/4183666333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6de810ee90 0x7f6de8111280 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.736+0000 7f6ded400700 1 --2- 172.21.15.79:0/4183666333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6de80746c0 0x7f6de8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.736+0000 7f6ded400700 1 --2- 172.21.15.79:0/4183666333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6de8074fe0 0x7f6de810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.736+0000 7f6ded400700 1 -- 172.21.15.79:0/4183666333 >> 172.21.15.79:0/4183666333 conn(0x7f6de80795f0 msgr2=0x7f6de8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:04.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.736+0000 7f6ded400700 1 -- 172.21.15.79:0/4183666333 shutdown_connections 2024-06-03T02:30:04.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.736+0000 7f6ded400700 1 -- 172.21.15.79:0/4183666333 wait complete. 2024-06-03T02:30:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.737+0000 7f6ded400700 1 Processor -- start 2024-06-03T02:30:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.737+0000 7f6ded400700 1 -- start start 2024-06-03T02:30:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.738+0000 7f6ded400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6de80746c0 0x7f6de819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.738+0000 7f6ded400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6de8074fe0 0x7f6de819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.738+0000 7f6ded400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6de810ee90 0x7f6de819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.738+0000 7f6ded400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8114a10 con 0x7f6de80746c0 2024-06-03T02:30:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.738+0000 7f6ded400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8114890 con 0x7f6de8074fe0 2024-06-03T02:30:04.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.738+0000 7f6ded400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8114b90 con 0x7f6de810ee90 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.738+0000 7f6ddfe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6de8074fe0 0x7f6de819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.739+0000 7f6ddfe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6de8074fe0 0x7f6de819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49440/0 (socket says 172.21.15.79:49440) 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.739+0000 7f6ddfe00700 1 -- 172.21.15.79:0/362706432 learned_addr learned my addr 172.21.15.79:0/362706432 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.739+0000 7f6ddfe00700 1 -- 172.21.15.79:0/362706432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6de810ee90 msgr2=0x7f6de819f6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.739+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6de810ee90 0x7f6de819f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.739+0000 7f6ddfe00700 1 -- 172.21.15.79:0/362706432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6de80746c0 msgr2=0x7f6de819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.739+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6de80746c0 0x7f6de819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.739+0000 7f6ddfe00700 1 -- 172.21.15.79:0/362706432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de819fe30 con 0x7f6de8074fe0 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.739+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6de8074fe0 0x7f6de819b360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009f90 tx=0x7f6dd400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:04.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.740+0000 7f6ddf400700 1 -- 172.21.15.79:0/362706432 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4004220 con 0x7f6de8074fe0 2024-06-03T02:30:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.740+0000 7f6ddf400700 1 -- 172.21.15.79:0/362706432 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd40043c0 con 0x7f6de8074fe0 2024-06-03T02:30:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.740+0000 7f6ded400700 1 -- 172.21.15.79:0/362706432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de81a0120 con 0x7f6de8074fe0 2024-06-03T02:30:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.740+0000 7f6ded400700 1 -- 172.21.15.79:0/362706432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de81024e0 con 0x7f6de8074fe0 2024-06-03T02:30:04.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.740+0000 7f6ddf400700 1 -- 172.21.15.79:0/362706432 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4003c20 con 0x7f6de8074fe0 2024-06-03T02:30:04.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.744+0000 7f6ddf400700 1 -- 172.21.15.79:0/362706432 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6dd4026020 con 0x7f6de8074fe0 2024-06-03T02:30:04.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.746+0000 7f6ddf400700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dcc0a2df0 0x7f6dcc0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.747+0000 7f6ddf400700 1 -- 172.21.15.79:0/362706432 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6dd40d76b0 con 0x7f6de8074fe0 2024-06-03T02:30:04.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.747+0000 7f6de6800700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dcc0a2df0 0x7f6dcc0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.748+0000 7f6df7dbdb80 1 Processor -- start 2024-06-03T02:30:04.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.748+0000 7f6de6800700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dcc0a2df0 0x7f6dcc0a52b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80062a0 tx=0x7f6dd80061f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:04.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.748+0000 7f6df7dbdb80 1 -- start start 2024-06-03T02:30:04.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.748+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78149b90 0x563b78149f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b7814a4b0 0x563b78143d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78144680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563b78052a10 con 0x563b7814a4b0 2024-06-03T02:30:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b78095b00 con 0x563b78149b90 2024-06-03T02:30:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b780508f0 con 0x563b78144240 2024-06-03T02:30:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6de7200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6de7200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54762/0 (socket says 172.21.15.79:54762) 2024-06-03T02:30:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6de6800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78149b90 0x563b78149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6de7200700 1 -- 172.21.15.79:0/912776851 learned_addr learned my addr 172.21.15.79:0/912776851 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/912776851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b7814a4b0 0x563b78143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6de7200700 1 -- 172.21.15.79:0/912776851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78149b90 msgr2=0x563b78149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6de7200700 1 --2- 172.21.15.79:0/912776851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78149b90 0x563b78149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6de7200700 1 -- 172.21.15.79:0/912776851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b7814a4b0 msgr2=0x563b78143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6de7200700 1 --2- 172.21.15.79:0/912776851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b7814a4b0 0x563b78143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.749+0000 7f6de7200700 1 -- 172.21.15.79:0/912776851 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b7803df90 con 0x563b78144240 2024-06-03T02:30:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.750+0000 7f6de7200700 1 --2- 172.21.15.79:0/912776851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78144680 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009810 tx=0x7f6de0009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa5f58b40a2de01 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.750+0000 7f6dd3e00700 1 -- 172.21.15.79:0/912776851 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0027070 con 0x563b78144240 2024-06-03T02:30:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.750+0000 7f6dd3e00700 1 -- 172.21.15.79:0/912776851 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de000f940 con 0x563b78144240 2024-06-03T02:30:04.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.750+0000 7f6dd3e00700 1 -- 172.21.15.79:0/912776851 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de00046d0 con 0x563b78144240 2024-06-03T02:30:04.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.751+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/912776851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 msgr2=0x563b78144680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:04.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.751+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/912776851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78144680 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009810 tx=0x7f6de0009610 comp rx=0 tx=0).stop 2024-06-03T02:30:04.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.752+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/912776851 shutdown_connections 2024-06-03T02:30:04.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.752+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/912776851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78144680 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.752+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/912776851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78149b90 0x563b78149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.752+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/912776851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b7814a4b0 0x563b78143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.752+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/912776851 >> 172.21.15.79:0/912776851 conn(0x563b78070170 msgr2=0x563b7813eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.753+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/912776851 shutdown_connections 2024-06-03T02:30:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.753+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/912776851 wait complete. 2024-06-03T02:30:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.754+0000 7f6df7dbdb80 1 -- start start 2024-06-03T02:30:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.754+0000 7f6df7dbdb80 1 Processor -- start 2024-06-03T02:30:04.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.756+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78144240 0x563b781501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78149b90 0x563b7814d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b7814da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563b78031f60 con 0x563b78144240 2024-06-03T02:30:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b78091ac0 con 0x563b7814a4b0 2024-06-03T02:30:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b78150e30 con 0x563b78149b90 2024-06-03T02:30:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6df7dbdb80 10 client.0 _ll_register_callbacks cb 0x563b7802b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6de7200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b7814da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6de7200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b7814da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49470/0 (socket says 172.21.15.79:49470) 2024-06-03T02:30:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6de7200700 1 -- 172.21.15.79:0/553928389 learned_addr learned my addr 172.21.15.79:0/553928389 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.757+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78149b90 0x563b7814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.758+0000 7f6de6800700 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78144240 0x563b781501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.758+0000 7f6ddfe00700 1 -- 172.21.15.79:0/553928389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 msgr2=0x563b7814da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.758+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b7814da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.758+0000 7f6ddfe00700 1 -- 172.21.15.79:0/553928389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78144240 msgr2=0x563b781501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.758+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78144240 0x563b781501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.758+0000 7f6ddfe00700 1 -- 172.21.15.79:0/553928389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b77fe48d0 con 0x563b78149b90 2024-06-03T02:30:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.759+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78149b90 0x563b7814d510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400c5b0 tx=0x7f6dd40a3590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80782c4ca16a42ef server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.759+0000 7f6dd0c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:30:04.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.760+0000 7f6dd0c00700 1 -- 172.21.15.79:0/553928389 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40bc030 con 0x563b78149b90 2024-06-03T02:30:04.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.760+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/553928389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de000f940 con 0x563b78149b90 2024-06-03T02:30:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.760+0000 7f6dd0c00700 1 -- 172.21.15.79:0/553928389 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6dd40a3d60 con 0x563b78149b90 2024-06-03T02:30:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.760+0000 7f6dd0c00700 1 -- 172.21.15.79:0/553928389 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6dd40a7eb0 con 0x563b78149b90 2024-06-03T02:30:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.760+0000 7f6df7dbdb80 10 client.26452 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.760+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/553928389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b7815b650 con 0x563b78149b90 2024-06-03T02:30:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.761+0000 7f6dd0c00700 1 -- 172.21.15.79:0/553928389 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de000f940 con 0x563b78149b90 2024-06-03T02:30:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.761+0000 7f6dd0c00700 1 -- 172.21.15.79:0/553928389 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6dd40a2f50 con 0x563b78149b90 2024-06-03T02:30:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.761+0000 7f6df7dbdb80 10 client.26452 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.761+0000 7f6df7dbdb80 10 client.26452 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:30:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.761+0000 7f6df7dbdb80 20 client.26452 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.761+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563b7815dbe0 0x563b7815ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.762+0000 7f6df7dbdb80 4 client.26452 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:30:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.762+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/553928389 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b780a3000 con 0x563b7815dbe0 2024-06-03T02:30:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.762+0000 7f6de6800700 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563b7815dbe0 0x563b7815ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.763+0000 7f6de6800700 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563b7815dbe0 0x563b7815ffd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9591095273f1d9f server_cookie=6a95d503fb5254bb in_seq=0 out_seq=0 2024-06-03T02:30:04.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.763+0000 7f6dd0c00700 10 client.26452 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.763+0000 7f6dd0c00700 1 -- 172.21.15.79:0/553928389 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563b780a3000 con 0x563b7815dbe0 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.763+0000 7f6dd0c00700 10 client.26452 handle_command_reply: tid=0 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6df7dbdb80 1 client.26452 shutdown 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/553928389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78149b90 msgr2=0x563b7814d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78149b90 0x563b7814d510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400c5b0 tx=0x7f6dd40a3590 comp rx=0 tx=0).stop 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/553928389 shutdown_connections 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563b7815dbe0 0x563b7815ffd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b7814da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78144240 0x563b781501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/553928389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78149b90 0x563b7814d510 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6dd0c00700 0 client.26452 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:30:04.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.764+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/553928389 >> 172.21.15.79:0/553928389 conn(0x563b78070170 msgr2=0x563b78138790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:04.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.765+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/553928389 shutdown_connections 2024-06-03T02:30:04.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.765+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/553928389 wait complete. 2024-06-03T02:30:04.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.765+0000 7f6df7dbdb80 20 client.26452 trim_cache size 0 max 16384 2024-06-03T02:30:04.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.788+0000 7f6df7dbdb80 1 Processor -- start 2024-06-03T02:30:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6df7dbdb80 1 -- start start 2024-06-03T02:30:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78143d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 0x563b7814aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b78139130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc800ebd0 con 0x563b78149b90 2024-06-03T02:30:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8010710 con 0x563b7814a4b0 2024-06-03T02:30:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc800f2a0 con 0x563b78144240 2024-06-03T02:30:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6de6800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6de7200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b78139130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6de6800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54782/0 (socket says 172.21.15.79:54782) 2024-06-03T02:30:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6de7200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b78139130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49480/0 (socket says 172.21.15.79:49480) 2024-06-03T02:30:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6de6800700 1 -- 172.21.15.79:0/2867105431 learned_addr learned my addr 172.21.15.79:0/2867105431 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.789+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/2867105431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 0x563b7814aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.790+0000 7f6de7200700 1 -- 172.21.15.79:0/2867105431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 msgr2=0x563b78143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.790+0000 7f6de7200700 1 --2- 172.21.15.79:0/2867105431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.790+0000 7f6de7200700 1 -- 172.21.15.79:0/2867105431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 msgr2=0x563b7814aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:04.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.790+0000 7f6de7200700 1 --2- 172.21.15.79:0/2867105431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 0x563b7814aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.790+0000 7f6de7200700 1 -- 172.21.15.79:0/2867105431 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b78160600 con 0x563b7814a4b0 2024-06-03T02:30:04.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.790+0000 7f6de6800700 1 --2- 172.21.15.79:0/2867105431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:30:04.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.790+0000 7f6de7200700 1 --2- 172.21.15.79:0/2867105431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b78139130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009680 tx=0x7f6de0009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e788c46d6a108fc9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:04.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.790+0000 7f6dd3400700 1 -- 172.21.15.79:0/2867105431 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0027070 con 0x563b7814a4b0 2024-06-03T02:30:04.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.791+0000 7f6dd3400700 1 -- 172.21.15.79:0/2867105431 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de0010cd0 con 0x563b7814a4b0 2024-06-03T02:30:04.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.791+0000 7f6dd3400700 1 -- 172.21.15.79:0/2867105431 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de00038e0 con 0x563b7814a4b0 2024-06-03T02:30:04.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.792+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/2867105431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 msgr2=0x563b78139130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:04.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.792+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/2867105431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b78139130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009680 tx=0x7f6de0009f50 comp rx=0 tx=0).stop 2024-06-03T02:30:04.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.792+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/2867105431 shutdown_connections 2024-06-03T02:30:04.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.792+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/2867105431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b7814a4b0 0x563b78139130 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.792+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/2867105431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b78144240 0x563b78143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.792+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/2867105431 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 0x563b7814aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.792+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/2867105431 >> 172.21.15.79:0/2867105431 conn(0x563b78070170 msgr2=0x563b78148e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:04.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.792+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/2867105431 shutdown_connections 2024-06-03T02:30:04.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.792+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/2867105431 wait complete. 2024-06-03T02:30:04.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.793+0000 7f6df7dbdb80 1 -- start start 2024-06-03T02:30:04.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.793+0000 7f6df7dbdb80 1 Processor -- start 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.793+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78144240 0x563b78156920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.793+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 0x563b781652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.793+0000 7f6df7dbdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7814a4b0 0x563b78165840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.793+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8010040 con 0x563b78149b90 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.793+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc800f970 con 0x563b78144240 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.793+0000 7f6df7dbdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc800cb70 con 0x563b7814a4b0 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.793+0000 7f6df7dbdb80 10 client.0 _ll_register_callbacks cb 0x563b7802b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6de6800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78144240 0x563b78156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6de7200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7814a4b0 0x563b78165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6de6800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78144240 0x563b78156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49496/0 (socket says 172.21.15.79:49496) 2024-06-03T02:30:04.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6de6800700 1 -- 172.21.15.79:0/1912593478 learned_addr learned my addr 172.21.15.79:0/1912593478 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:04.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 0x563b781652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6de6800700 1 -- 172.21.15.79:0/1912593478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7814a4b0 msgr2=0x563b78165840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:04.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6de6800700 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7814a4b0 0x563b78165840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6de6800700 1 -- 172.21.15.79:0/1912593478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 msgr2=0x563b781652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:04.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6de6800700 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 0x563b781652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6de6800700 1 -- 172.21.15.79:0/1912593478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b781660b0 con 0x563b78144240 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.794+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 0x563b781652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.795+0000 7f6de6800700 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78144240 0x563b78156920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8015640 tx=0x7f6dd80096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ade5fdc8f57f5a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.795+0000 7f6dd0c00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.795+0000 7f6dd0c00700 1 -- 172.21.15.79:0/1912593478 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8032060 con 0x563b78144240 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.795+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/1912593478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de0010cd0 con 0x563b78144240 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.795+0000 7f6dd0c00700 1 -- 172.21.15.79:0/1912593478 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6dd802d440 con 0x563b78144240 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.795+0000 7f6dd0c00700 1 -- 172.21.15.79:0/1912593478 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6dd8039050 con 0x563b78144240 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.795+0000 7f6df7dbdb80 10 client.26453 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:04.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.795+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/1912593478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b781c89f0 con 0x563b78144240 2024-06-03T02:30:04.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.796+0000 7f6dd0c00700 1 -- 172.21.15.79:0/1912593478 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de0010cd0 con 0x563b78144240 2024-06-03T02:30:04.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.796+0000 7f6dd0c00700 1 -- 172.21.15.79:0/1912593478 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6dd8025830 con 0x563b78144240 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.796+0000 7f6df7dbdb80 10 client.26453 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.796+0000 7f6df7dbdb80 10 client.26453 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.796+0000 7f6df7dbdb80 20 client.26453 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.796+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563b781c90e0 0x563b781cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.796+0000 7f6df7dbdb80 4 client.26453 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.796+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/1912593478 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b7809d580 con 0x563b781c90e0 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.796+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563b781c90e0 0x563b781cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6ddfe00700 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563b781c90e0 0x563b781cb500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=999fe9643caa4dad server_cookie=4c50c269e2d5c585 in_seq=0 out_seq=0 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6dd0c00700 10 client.26453 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6dd0c00700 1 -- 172.21.15.79:0/1912593478 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b7809d580 con 0x563b781c90e0 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6dd0c00700 10 client.26453 handle_command_reply: tid=0 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 client.26453 shutdown 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/1912593478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78144240 msgr2=0x563b78156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78144240 0x563b78156920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8015640 tx=0x7f6dd80096e0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/1912593478 shutdown_connections 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7814a4b0 0x563b78165840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x563b781c90e0 0x563b781cb500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6dd0c00700 0 client.26453 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b78144240 0x563b78156920 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 --2- 172.21.15.79:0/1912593478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563b78149b90 0x563b781652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/1912593478 >> 172.21.15.79:0/1912593478 conn(0x563b78070170 msgr2=0x563b7814ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/1912593478 shutdown_connections 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.797+0000 7f6df7dbdb80 1 -- 172.21.15.79:0/1912593478 wait complete. 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.798+0000 7f6df7dbdb80 20 client.26453 trim_cache size 0 max 16384 2024-06-03T02:30:04.800 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:30:04.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.798+0000 7f6ded400700 1 -- 172.21.15.79:0/362706432 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dcc0a2df0 msgr2=0x7f6dcc0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:04.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.798+0000 7f6ded400700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dcc0a2df0 0x7f6dcc0a52b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80062a0 tx=0x7f6dd80061f0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.798+0000 7f6ded400700 1 -- 172.21.15.79:0/362706432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6de8074fe0 msgr2=0x7f6de819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:04.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.798+0000 7f6ded400700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6de8074fe0 0x7f6de819b360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009f90 tx=0x7f6dd400ee90 comp rx=0 tx=0).stop 2024-06-03T02:30:04.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.799+0000 7f6ded400700 1 -- 172.21.15.79:0/362706432 shutdown_connections 2024-06-03T02:30:04.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.799+0000 7f6ded400700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6de810ee90 0x7f6de819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.799+0000 7f6ded400700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6dcc0a2df0 0x7f6dcc0a52b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.799+0000 7f6ded400700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6de80746c0 0x7f6de819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.799+0000 7f6ded400700 1 --2- 172.21.15.79:0/362706432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6de8074fe0 0x7f6de819b360 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:04.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.799+0000 7f6ded400700 1 -- 172.21.15.79:0/362706432 >> 172.21.15.79:0/362706432 conn(0x7f6de80795f0 msgr2=0x7f6de80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:04.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.799+0000 7f6ded400700 1 -- 172.21.15.79:0/362706432 shutdown_connections 2024-06-03T02:30:04.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:04.799+0000 7f6ded400700 1 -- 172.21.15.79:0/362706432 wait complete. 2024-06-03T02:30:05.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:30:06.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:05 smithi079 ceph-mon[31819]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 617 op/s 2024-06-03T02:30:06.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:05 smithi123 ceph-mon[36640]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 617 op/s 2024-06-03T02:30:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:05 smithi135 ceph-mon[36653]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 617 op/s 2024-06-03T02:30:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:30:08.201 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:07 smithi079 ceph-mon[31819]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.4 MiB/s wr, 477 op/s 2024-06-03T02:30:08.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:07 smithi123 ceph-mon[36640]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.4 MiB/s wr, 477 op/s 2024-06-03T02:30:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 600.0 s, 498.5 tps, lat 199.505 ms stddev 227.390 2024-06-03T02:30:08.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:07 smithi135 ceph-mon[36653]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.4 MiB/s wr, 477 op/s 2024-06-03T02:30:10.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:09 smithi079 ceph-mon[31819]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.9 MiB/s wr, 624 op/s 2024-06-03T02:30:10.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:09 smithi123 ceph-mon[36640]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.9 MiB/s wr, 624 op/s 2024-06-03T02:30:10.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:09 smithi135 ceph-mon[36653]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.9 MiB/s wr, 624 op/s 2024-06-03T02:30:11.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.894+0000 7f12d0000700 1 -- 172.21.15.79:0/3975010401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c810a6d0 msgr2=0x7f12c810aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:11.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.894+0000 7f12d0000700 1 --2- 172.21.15.79:0/3975010401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c810a6d0 0x7f12c810aab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009990 tx=0x7f12bc009ca0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.894+0000 7f12d0000700 1 -- 172.21.15.79:0/3975010401 shutdown_connections 2024-06-03T02:30:11.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.894+0000 7f12d0000700 1 --2- 172.21.15.79:0/3975010401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12c8105090 0x7f12c81115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.894+0000 7f12d0000700 1 --2- 172.21.15.79:0/3975010401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c810a6d0 0x7f12c810aab0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.894+0000 7f12d0000700 1 --2- 172.21.15.79:0/3975010401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12c81046d0 0x7f12c8104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.894+0000 7f12d0000700 1 -- 172.21.15.79:0/3975010401 >> 172.21.15.79:0/3975010401 conn(0x7f12c81001c0 msgr2=0x7f12c81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:11.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.895+0000 7f12d0000700 1 -- 172.21.15.79:0/3975010401 shutdown_connections 2024-06-03T02:30:11.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.895+0000 7f12d0000700 1 -- 172.21.15.79:0/3975010401 wait complete. 2024-06-03T02:30:11.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.896+0000 7f12d0000700 1 Processor -- start 2024-06-03T02:30:11.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.896+0000 7f12d0000700 1 -- start start 2024-06-03T02:30:11.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12d0000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12c81046d0 0x7f12c819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12d0000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12c8105090 0x7f12c819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12d0000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c810a6d0 0x7f12c819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12d0000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8114ab0 con 0x7f12c8105090 2024-06-03T02:30:11.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12d0000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8114930 con 0x7f12c810a6d0 2024-06-03T02:30:11.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12d0000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8114c30 con 0x7f12c81046d0 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12cd600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12c81046d0 0x7f12c819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12cd600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12c81046d0 0x7f12c819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37528/0 (socket says 172.21.15.79:37528) 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12cd600700 1 -- 172.21.15.79:0/949474243 learned_addr learned my addr 172.21.15.79:0/949474243 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12ce000700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c810a6d0 0x7f12c819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12ccc00700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12c8105090 0x7f12c819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12cd600700 1 -- 172.21.15.79:0/949474243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c810a6d0 msgr2=0x7f12c819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12cd600700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c810a6d0 0x7f12c819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12cd600700 1 -- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12c8105090 msgr2=0x7f12c819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12cd600700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12c8105090 0x7f12c819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12cd600700 1 -- 172.21.15.79:0/949474243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c819fe70 con 0x7f12c81046d0 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.897+0000 7f12ccc00700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12c8105090 0x7f12c819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.898+0000 7f12cd600700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12c81046d0 0x7f12c819ae60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000c00 tx=0x7f12bc00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.898+0000 7f12c6000700 1 -- 172.21.15.79:0/949474243 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc027070 con 0x7f12c81046d0 2024-06-03T02:30:11.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.898+0000 7f12c6000700 1 -- 172.21.15.79:0/949474243 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12bc003d70 con 0x7f12c81046d0 2024-06-03T02:30:11.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.898+0000 7f12c6000700 1 -- 172.21.15.79:0/949474243 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00f460 con 0x7f12c81046d0 2024-06-03T02:30:11.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.898+0000 7f12d0000700 1 -- 172.21.15.79:0/949474243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c81a0100 con 0x7f12c81046d0 2024-06-03T02:30:11.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.898+0000 7f12d0000700 1 -- 172.21.15.79:0/949474243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c81a79e0 con 0x7f12c81046d0 2024-06-03T02:30:11.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.902+0000 7f12c6000700 1 -- 172.21.15.79:0/949474243 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f12bc02a070 con 0x7f12c81046d0 2024-06-03T02:30:11.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.903+0000 7f12c6000700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12b00a2c20 0x7f12b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.904+0000 7f12c6000700 1 -- 172.21.15.79:0/949474243 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f12bc0d0150 con 0x7f12c81046d0 2024-06-03T02:30:11.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.905+0000 7f12da902b80 1 Processor -- start 2024-06-03T02:30:11.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.906+0000 7f12da902b80 1 -- start start 2024-06-03T02:30:11.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.906+0000 7f12da902b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23220f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.906+0000 7f12da902b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd2321ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.906+0000 7f12da902b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2321b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.906+0000 7f12da902b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cd23129a10 con 0x55cd23220b90 2024-06-03T02:30:11.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.906+0000 7f12da902b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cd2316cb00 con 0x55cd232214b0 2024-06-03T02:30:11.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.906+0000 7f12da902b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cd231278f0 con 0x55cd2321b240 2024-06-03T02:30:11.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.907+0000 7f12ccc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd2321ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.907+0000 7f12ccc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd2321ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38018/0 (socket says 172.21.15.79:38018) 2024-06-03T02:30:11.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.907+0000 7f12ccc00700 1 -- 172.21.15.79:0/1658725323 learned_addr learned my addr 172.21.15.79:0/1658725323 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:11.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.907+0000 7f12ce000700 1 --2- 172.21.15.79:0/1658725323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2321b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.907+0000 7f12ccc00700 1 -- 172.21.15.79:0/1658725323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 msgr2=0x55cd2321b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:11.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.907+0000 7f12ccc00700 1 --2- 172.21.15.79:0/1658725323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2321b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.907+0000 7f12ccc00700 1 -- 172.21.15.79:0/1658725323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 msgr2=0x55cd23220f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:11.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.907+0000 7f12ccc00700 1 --2- 172.21.15.79:0/1658725323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23220f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.907+0000 7f12ccc00700 1 -- 172.21.15.79:0/1658725323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd23114f90 con 0x55cd232214b0 2024-06-03T02:30:11.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.908+0000 7f12ccc00700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12b00a2c20 0x7f12b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.909+0000 7f12ccc00700 1 --2- 172.21.15.79:0/1658725323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd2321ad00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009db0 tx=0x7f12b40116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315743166a8e9dea server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:11.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.910+0000 7f12baa00700 1 -- 172.21.15.79:0/1658725323 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4022070 con 0x55cd232214b0 2024-06-03T02:30:11.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.910+0000 7f12baa00700 1 -- 172.21.15.79:0/1658725323 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b4016070 con 0x55cd232214b0 2024-06-03T02:30:11.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.910+0000 7f12baa00700 1 -- 172.21.15.79:0/1658725323 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b401c680 con 0x55cd232214b0 2024-06-03T02:30:11.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.911+0000 7f12da902b80 1 -- 172.21.15.79:0/1658725323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 msgr2=0x55cd2321ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:11.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.911+0000 7f12ccc00700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12b00a2c20 0x7f12b00a50e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f12b401f000 tx=0x7f12b401c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:11.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.911+0000 7f12da902b80 1 --2- 172.21.15.79:0/1658725323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd2321ad00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009db0 tx=0x7f12b40116c0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.911+0000 7f12da902b80 1 -- 172.21.15.79:0/1658725323 shutdown_connections 2024-06-03T02:30:11.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.911+0000 7f12da902b80 1 --2- 172.21.15.79:0/1658725323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2321b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.911+0000 7f12da902b80 1 --2- 172.21.15.79:0/1658725323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23220f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.911+0000 7f12da902b80 1 --2- 172.21.15.79:0/1658725323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd2321ad00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.911+0000 7f12da902b80 1 -- 172.21.15.79:0/1658725323 >> 172.21.15.79:0/1658725323 conn(0x55cd23147170 msgr2=0x55cd23215ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:11.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.912+0000 7f12da902b80 1 -- 172.21.15.79:0/1658725323 shutdown_connections 2024-06-03T02:30:11.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.912+0000 7f12da902b80 1 -- 172.21.15.79:0/1658725323 wait complete. 2024-06-03T02:30:11.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.912+0000 7f12da902b80 1 -- start start 2024-06-03T02:30:11.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.913+0000 7f12da902b80 1 Processor -- start 2024-06-03T02:30:11.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.913+0000 7f12da902b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd2321b240 0x55cd232298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.913+0000 7f12da902b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23229de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12da902b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd232214b0 0x55cd2322f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12da902b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55cd23108f60 con 0x55cd23220b90 2024-06-03T02:30:11.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12cd600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd2321b240 0x55cd232298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12da902b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cd23168ac0 con 0x55cd2321b240 2024-06-03T02:30:11.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12da902b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cd23221d30 con 0x55cd232214b0 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12cd600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd2321b240 0x55cd232298a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38022/0 (socket says 172.21.15.79:38022) 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12cd600700 1 -- 172.21.15.79:0/1684476996 learned_addr learned my addr 172.21.15.79:0/1684476996 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12da902b80 10 client.0 _ll_register_callbacks cb 0x55cd23102380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12ccc00700 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23229de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12ce000700 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd232214b0 0x55cd2322f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12cd600700 1 -- 172.21.15.79:0/1684476996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd232214b0 msgr2=0x55cd2322f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12cd600700 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd232214b0 0x55cd2322f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12cd600700 1 -- 172.21.15.79:0/1684476996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 msgr2=0x55cd23229de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12cd600700 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23229de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12cd600700 1 -- 172.21.15.79:0/1684476996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd230bb8d0 con 0x55cd2321b240 2024-06-03T02:30:11.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.914+0000 7f12ccc00700 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23229de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:30:11.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.915+0000 7f12cd600700 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd2321b240 0x55cd232298a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f12bc09c5c0 tx=0x7f12bc09c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb33c9dc3d72e6c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:11.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.915+0000 7f12af400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:30:11.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.915+0000 7f12af400700 1 -- 172.21.15.79:0/1684476996 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc09bf90 con 0x55cd2321b240 2024-06-03T02:30:11.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.915+0000 7f12af400700 1 -- 172.21.15.79:0/1684476996 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f12bc09a930 con 0x55cd2321b240 2024-06-03T02:30:11.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.915+0000 7f12da902b80 1 -- 172.21.15.79:0/1684476996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b4016070 con 0x55cd2321b240 2024-06-03T02:30:11.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.915+0000 7f12af400700 1 -- 172.21.15.79:0/1684476996 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12bc09cd30 con 0x55cd2321b240 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.916+0000 7f12da902b80 10 client.26465 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.916+0000 7f12da902b80 1 -- 172.21.15.79:0/1684476996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd232438a0 con 0x55cd2321b240 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.916+0000 7f12af400700 1 -- 172.21.15.79:0/1684476996 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b4016070 con 0x55cd2321b240 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.917+0000 7f12af400700 1 -- 172.21.15.79:0/1684476996 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f12bc085bc0 con 0x55cd2321b240 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.917+0000 7f12da902b80 10 client.26465 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.917+0000 7f12da902b80 10 client.26465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.917+0000 7f12da902b80 20 client.26465 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.917+0000 7f12da902b80 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55cd23245d70 0x55cd23248160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.917+0000 7f12da902b80 4 client.26465 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:30:11.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.917+0000 7f12da902b80 1 -- 172.21.15.79:0/1684476996 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd2317a000 con 0x55cd23245d70 2024-06-03T02:30:11.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.918+0000 7f12ccc00700 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55cd23245d70 0x55cd23248160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.919+0000 7f12ccc00700 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55cd23245d70 0x55cd23248160 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f20689f14e83e30 server_cookie=edebf7e3061296cd in_seq=0 out_seq=0 2024-06-03T02:30:11.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.919+0000 7f12af400700 10 client.26465 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:30:11.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.920+0000 7f12af400700 1 -- 172.21.15.79:0/1684476996 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cd2317a000 con 0x55cd23245d70 2024-06-03T02:30:11.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.920+0000 7f12af400700 10 client.26465 handle_command_reply: tid=0 2024-06-03T02:30:11.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.920+0000 7f12da902b80 1 client.26465 shutdown 2024-06-03T02:30:11.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.921+0000 7f12da902b80 1 -- 172.21.15.79:0/1684476996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd2321b240 msgr2=0x55cd232298a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:11.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.921+0000 7f12da902b80 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd2321b240 0x55cd232298a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f12bc09c5c0 tx=0x7f12bc09c530 comp rx=0 tx=0).stop 2024-06-03T02:30:11.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.922+0000 7f12da902b80 1 -- 172.21.15.79:0/1684476996 shutdown_connections 2024-06-03T02:30:11.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.922+0000 7f12da902b80 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55cd23245d70 0x55cd23248160 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.922+0000 7f12af400700 0 client.26465 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:30:11.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.922+0000 7f12da902b80 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd232214b0 0x55cd2322f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.922+0000 7f12da902b80 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd2321b240 0x55cd232298a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.922+0000 7f12da902b80 1 --2- 172.21.15.79:0/1684476996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23229de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.922+0000 7f12da902b80 1 -- 172.21.15.79:0/1684476996 >> 172.21.15.79:0/1684476996 conn(0x55cd23147170 msgr2=0x55cd2320f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:11.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.923+0000 7f12da902b80 1 -- 172.21.15.79:0/1684476996 shutdown_connections 2024-06-03T02:30:11.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.923+0000 7f12da902b80 1 -- 172.21.15.79:0/1684476996 wait complete. 2024-06-03T02:30:11.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.923+0000 7f12da902b80 20 client.26465 trim_cache size 0 max 16384 2024-06-03T02:30:11.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.952+0000 7f12da902b80 1 Processor -- start 2024-06-03T02:30:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.952+0000 7f12da902b80 1 -- start start 2024-06-03T02:30:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.952+0000 7f12da902b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2322f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.952+0000 7f12da902b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23210130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.952+0000 7f12da902b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd232226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.952+0000 7f12da902b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f12c81abef0 con 0x55cd23220b90 2024-06-03T02:30:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.952+0000 7f12da902b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12c81ada30 con 0x55cd232214b0 2024-06-03T02:30:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.952+0000 7f12da902b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12c81ac5c0 con 0x55cd2321b240 2024-06-03T02:30:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.953+0000 7f12ce000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd232226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.955+0000 7f12cd600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2322f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.955+0000 7f12cd600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2322f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37542/0 (socket says 172.21.15.79:37542) 2024-06-03T02:30:11.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.955+0000 7f12cd600700 1 -- 172.21.15.79:0/3825722917 learned_addr learned my addr 172.21.15.79:0/3825722917 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:11.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.955+0000 7f12ccc00700 1 --2- 172.21.15.79:0/3825722917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23210130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.955+0000 7f12cd600700 1 -- 172.21.15.79:0/3825722917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 msgr2=0x55cd232226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:11.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.955+0000 7f12cd600700 1 --2- 172.21.15.79:0/3825722917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd232226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.955+0000 7f12cd600700 1 -- 172.21.15.79:0/3825722917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 msgr2=0x55cd23210130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:11.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.955+0000 7f12cd600700 1 --2- 172.21.15.79:0/3825722917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23210130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.955+0000 7f12cd600700 1 -- 172.21.15.79:0/3825722917 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd23224a30 con 0x55cd2321b240 2024-06-03T02:30:11.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.959+0000 7f12cd600700 1 --2- 172.21.15.79:0/3825722917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2322f550 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f12bc085f80 tx=0x7f12bc022a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=effbeed7ca0f5abd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.959+0000 7f12ba000700 1 -- 172.21.15.79:0/3825722917 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc09bf90 con 0x55cd2321b240 2024-06-03T02:30:11.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.959+0000 7f12ba000700 1 -- 172.21.15.79:0/3825722917 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12bc09c630 con 0x55cd2321b240 2024-06-03T02:30:11.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.959+0000 7f12ba000700 1 -- 172.21.15.79:0/3825722917 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc09a930 con 0x55cd2321b240 2024-06-03T02:30:11.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.961+0000 7f12da902b80 1 -- 172.21.15.79:0/3825722917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 msgr2=0x55cd2322f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:11.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.961+0000 7f12da902b80 1 --2- 172.21.15.79:0/3825722917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2322f550 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f12bc085f80 tx=0x7f12bc022a30 comp rx=0 tx=0).stop 2024-06-03T02:30:11.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.961+0000 7f12da902b80 1 -- 172.21.15.79:0/3825722917 shutdown_connections 2024-06-03T02:30:11.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.961+0000 7f12da902b80 1 --2- 172.21.15.79:0/3825722917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd232226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.961+0000 7f12da902b80 1 --2- 172.21.15.79:0/3825722917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd2321b240 0x55cd2322f550 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.961+0000 7f12da902b80 1 --2- 172.21.15.79:0/3825722917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd23220b90 0x55cd23210130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.961+0000 7f12da902b80 1 -- 172.21.15.79:0/3825722917 >> 172.21.15.79:0/3825722917 conn(0x55cd23147170 msgr2=0x55cd23221fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:11.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.961+0000 7f12da902b80 1 -- 172.21.15.79:0/3825722917 shutdown_connections 2024-06-03T02:30:11.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.964+0000 7f12da902b80 1 -- 172.21.15.79:0/3825722917 wait complete. 2024-06-03T02:30:11.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.964+0000 7f12da902b80 1 -- start start 2024-06-03T02:30:11.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.964+0000 7f12da902b80 1 Processor -- start 2024-06-03T02:30:11.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.966+0000 7f12da902b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd2321b240 0x55cd23259c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.966+0000 7f12da902b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd23220b90 0x55cd23228ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.966+0000 7f12da902b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd232290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.967+0000 7f12da902b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f12c81ad360 con 0x55cd2321b240 2024-06-03T02:30:11.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.967+0000 7f12da902b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12c81acc90 con 0x55cd232214b0 2024-06-03T02:30:11.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.967+0000 7f12da902b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12c81a9fe0 con 0x55cd23220b90 2024-06-03T02:30:11.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.967+0000 7f12da902b80 10 client.0 _ll_register_callbacks cb 0x55cd23102380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:11.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.968+0000 7f12ce000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd232290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.968+0000 7f12cd600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd2321b240 0x55cd23259c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.968+0000 7f12cd600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd2321b240 0x55cd23259c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33744/0 (socket says 172.21.15.79:33744) 2024-06-03T02:30:11.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.968+0000 7f12cd600700 1 -- 172.21.15.79:0/3442431332 learned_addr learned my addr 172.21.15.79:0/3442431332 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:11.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.968+0000 7f12ce000700 1 -- 172.21.15.79:0/3442431332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd23220b90 msgr2=0x55cd23228ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:11.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.968+0000 7f12ce000700 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd23220b90 0x55cd23228ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.969+0000 7f12ce000700 1 -- 172.21.15.79:0/3442431332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd2321b240 msgr2=0x55cd23259c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:11.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.969+0000 7f12ce000700 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd2321b240 0x55cd23259c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.969+0000 7f12ce000700 1 -- 172.21.15.79:0/3442431332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd232296a0 con 0x55cd232214b0 2024-06-03T02:30:11.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.971+0000 7f12ce000700 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd232290e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f12c0007ed0 tx=0x7f12c00176f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43b652e9df564b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:11.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.971+0000 7f12af400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:30:11.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.971+0000 7f12af400700 1 -- 172.21.15.79:0/3442431332 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c0003c10 con 0x55cd232214b0 2024-06-03T02:30:11.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.972+0000 7f12af400700 1 -- 172.21.15.79:0/3442431332 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f12c0003db0 con 0x55cd232214b0 2024-06-03T02:30:11.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.972+0000 7f12da902b80 1 -- 172.21.15.79:0/3442431332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12bc09c630 con 0x55cd232214b0 2024-06-03T02:30:11.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.973+0000 7f12af400700 1 -- 172.21.15.79:0/3442431332 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12c0004030 con 0x55cd232214b0 2024-06-03T02:30:11.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.973+0000 7f12da902b80 10 client.26471 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:11.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.973+0000 7f12da902b80 1 -- 172.21.15.79:0/3442431332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd232c1610 con 0x55cd232214b0 2024-06-03T02:30:11.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.973+0000 7f12af400700 1 -- 172.21.15.79:0/3442431332 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12bc09c630 con 0x55cd232214b0 2024-06-03T02:30:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.975+0000 7f12af400700 1 -- 172.21.15.79:0/3442431332 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f12c002b910 con 0x55cd232214b0 2024-06-03T02:30:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.975+0000 7f12da902b80 10 client.26471 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.975+0000 7f12da902b80 10 client.26471 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:30:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.975+0000 7f12da902b80 20 client.26471 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.975+0000 7f12da902b80 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55cd232c1c70 0x55cd232c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.975+0000 7f12da902b80 4 client.26471 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:30:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.975+0000 7f12da902b80 1 -- 172.21.15.79:0/3442431332 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd23174580 con 0x55cd232c1c70 2024-06-03T02:30:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.976+0000 7f12cd600700 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55cd232c1c70 0x55cd232c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:11.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.980+0000 7f12cd600700 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55cd232c1c70 0x55cd232c40b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df8324b8b7b7c4e server_cookie=528f3ca6a97216e9 in_seq=0 out_seq=0 2024-06-03T02:30:11.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.980+0000 7f12af400700 10 client.26471 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:30:11.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12af400700 1 -- 172.21.15.79:0/3442431332 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd23174580 con 0x55cd232c1c70 2024-06-03T02:30:11.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12af400700 10 client.26471 handle_command_reply: tid=0 2024-06-03T02:30:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 client.26471 shutdown 2024-06-03T02:30:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 -- 172.21.15.79:0/3442431332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 msgr2=0x55cd232290e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd232290e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f12c0007ed0 tx=0x7f12c00176f0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 -- 172.21.15.79:0/3442431332 shutdown_connections 2024-06-03T02:30:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd232214b0 0x55cd232290e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55cd2321b240 0x55cd23259c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55cd232c1c70 0x55cd232c40b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 --2- 172.21.15.79:0/3442431332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd23220b90 0x55cd23228ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 -- 172.21.15.79:0/3442431332 >> 172.21.15.79:0/3442431332 conn(0x55cd23147170 msgr2=0x55cd2321f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:11.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12af400700 0 client.26471 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:30:11.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.981+0000 7f12da902b80 1 -- 172.21.15.79:0/3442431332 shutdown_connections 2024-06-03T02:30:11.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.982+0000 7f12da902b80 1 -- 172.21.15.79:0/3442431332 wait complete. 2024-06-03T02:30:11.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.982+0000 7f12da902b80 20 client.26471 trim_cache size 0 max 16384 2024-06-03T02:30:11.984 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:30:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 -- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12b00a2c20 msgr2=0x7f12b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12b00a2c20 0x7f12b00a50e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f12b401f000 tx=0x7f12b401c000 comp rx=0 tx=0).stop 2024-06-03T02:30:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 -- 172.21.15.79:0/949474243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12c81046d0 msgr2=0x7f12c819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12c81046d0 0x7f12c819ae60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000c00 tx=0x7f12bc00bab0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 -- 172.21.15.79:0/949474243 shutdown_connections 2024-06-03T02:30:11.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c810a6d0 0x7f12c819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12c81046d0 0x7f12c819ae60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f12b00a2c20 0x7f12b00a50e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 --2- 172.21.15.79:0/949474243 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12c8105090 0x7f12c819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:11.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.983+0000 7f12d0000700 1 -- 172.21.15.79:0/949474243 >> 172.21.15.79:0/949474243 conn(0x7f12c81001c0 msgr2=0x7f12c8100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:11.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.984+0000 7f12d0000700 1 -- 172.21.15.79:0/949474243 shutdown_connections 2024-06-03T02:30:11.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:11.985+0000 7f12d0000700 1 -- 172.21.15.79:0/949474243 wait complete. 2024-06-03T02:30:12.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:11 smithi079 ceph-mon[31819]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 492 op/s 2024-06-03T02:30:12.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:11 smithi123 ceph-mon[36640]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 492 op/s 2024-06-03T02:30:12.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:11 smithi135 ceph-mon[36653]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 492 op/s 2024-06-03T02:30:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 605.0 s, 507.6 tps, lat 198.440 ms stddev 222.771 2024-06-03T02:30:14.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:13 smithi123 ceph-mon[36640]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 487 op/s 2024-06-03T02:30:14.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:13 smithi079 ceph-mon[31819]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 487 op/s 2024-06-03T02:30:14.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:13 smithi135 ceph-mon[36653]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 487 op/s 2024-06-03T02:30:14.755 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:30:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:30:16.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:15 smithi079 ceph-mon[31819]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 571 op/s 2024-06-03T02:30:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:15 smithi123 ceph-mon[36640]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 571 op/s 2024-06-03T02:30:16.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:15 smithi135 ceph-mon[36653]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 571 op/s 2024-06-03T02:30:18.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:17 smithi123 ceph-mon[36640]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 405 op/s 2024-06-03T02:30:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 610.0 s, 478.0 tps, lat 203.890 ms stddev 244.394 2024-06-03T02:30:18.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:17 smithi079 ceph-mon[31819]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 405 op/s 2024-06-03T02:30:18.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:17 smithi135 ceph-mon[36653]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 405 op/s 2024-06-03T02:30:20.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:19 smithi123 ceph-mon[36640]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.0 MiB/s wr, 539 op/s 2024-06-03T02:30:20.294 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:19 smithi079 ceph-mon[31819]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.0 MiB/s wr, 539 op/s 2024-06-03T02:30:20.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:19 smithi135 ceph-mon[36653]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.0 MiB/s wr, 539 op/s 2024-06-03T02:30:20.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.405+0000 7f6a38e00700 1 -- 172.21.15.79:0/541523365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a34104e90 msgr2=0x7f6a34105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:20.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.405+0000 7f6a38e00700 1 --2- 172.21.15.79:0/541523365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a34104e90 0x7f6a34105320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009990 tx=0x7f6a2c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.406+0000 7f6a38e00700 1 -- 172.21.15.79:0/541523365 shutdown_connections 2024-06-03T02:30:20.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.406+0000 7f6a38e00700 1 --2- 172.21.15.79:0/541523365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a34104e90 0x7f6a34105320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.406+0000 7f6a38e00700 1 --2- 172.21.15.79:0/541523365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34107380 0x7f6a3406a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.406+0000 7f6a38e00700 1 --2- 172.21.15.79:0/541523365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3406a6b0 0x7f6a3406ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.406+0000 7f6a38e00700 1 -- 172.21.15.79:0/541523365 >> 172.21.15.79:0/541523365 conn(0x7f6a340782c0 msgr2=0x7f6a340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:20.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.406+0000 7f6a38e00700 1 -- 172.21.15.79:0/541523365 shutdown_connections 2024-06-03T02:30:20.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.406+0000 7f6a38e00700 1 -- 172.21.15.79:0/541523365 wait complete. 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.408+0000 7f6a38e00700 1 Processor -- start 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.409+0000 7f6a38e00700 1 -- start start 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.410+0000 7f6a38e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3406a6b0 0x7f6a34115760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.410+0000 7f6a38e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a34104e90 0x7f6a34115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.410+0000 7f6a38e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34107380 0x7f6a3410ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.410+0000 7f6a2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a34104e90 0x7f6a34115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.410+0000 7f6a2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a34104e90 0x7f6a34115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53082/0 (socket says 172.21.15.79:53082) 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.410+0000 7f6a32800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34107380 0x7f6a3410ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.411+0000 7f6a32800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34107380 0x7f6a3410ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51164/0 (socket says 172.21.15.79:51164) 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.411+0000 7f6a31e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3406a6b0 0x7f6a34115760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.411+0000 7f6a31e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3406a6b0 0x7f6a34115760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53764/0 (socket says 172.21.15.79:53764) 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.411+0000 7f6a32800700 1 -- 172.21.15.79:0/2428611505 learned_addr learned my addr 172.21.15.79:0/2428611505 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:20.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.411+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34118da0 con 0x7f6a34107380 2024-06-03T02:30:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.411+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34118c20 con 0x7f6a3406a6b0 2024-06-03T02:30:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.411+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34118f20 con 0x7f6a34104e90 2024-06-03T02:30:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.412+0000 7f6a32800700 1 -- 172.21.15.79:0/2428611505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a34104e90 msgr2=0x7f6a34115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.412+0000 7f6a32800700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a34104e90 0x7f6a34115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.412+0000 7f6a32800700 1 -- 172.21.15.79:0/2428611505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3406a6b0 msgr2=0x7f6a34115760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:20.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.412+0000 7f6a32800700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3406a6b0 0x7f6a34115760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.412+0000 7f6a32800700 1 -- 172.21.15.79:0/2428611505 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3410f2b0 con 0x7f6a34107380 2024-06-03T02:30:20.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.413+0000 7f6a31e00700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3406a6b0 0x7f6a34115760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:30:20.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.416+0000 7f6a32800700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34107380 0x7f6a3410ea40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b820 tx=0x7f6a2c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:20.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.417+0000 7f6a2aa00700 1 -- 172.21.15.79:0/2428611505 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c003890 con 0x7f6a34107380 2024-06-03T02:30:20.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.417+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3410f540 con 0x7f6a34107380 2024-06-03T02:30:20.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.417+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a34113460 con 0x7f6a34107380 2024-06-03T02:30:20.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.418+0000 7f6a2aa00700 1 -- 172.21.15.79:0/2428611505 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c004830 con 0x7f6a34107380 2024-06-03T02:30:20.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.418+0000 7f6a2aa00700 1 -- 172.21.15.79:0/2428611505 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c005440 con 0x7f6a34107380 2024-06-03T02:30:20.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.424+0000 7f6a436fab80 1 Processor -- start 2024-06-03T02:30:20.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.424+0000 7f6a2aa00700 1 -- 172.21.15.79:0/2428611505 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6a2c009d10 con 0x7f6a34107380 2024-06-03T02:30:20.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.424+0000 7f6a436fab80 1 -- start start 2024-06-03T02:30:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.425+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242ab99070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.425+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242ab99a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.425+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242aba66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.425+0000 7f6a436fab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56242aaa8a10 con 0x56242aba4300 2024-06-03T02:30:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.425+0000 7f6a436fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56242aaebb00 con 0x56242ab15e40 2024-06-03T02:30:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.425+0000 7f6a436fab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56242aaa68f0 con 0x56242ab995b0 2024-06-03T02:30:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.426+0000 7f6a31e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242ab99a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.426+0000 7f6a31e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242ab99a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53092/0 (socket says 172.21.15.79:53092) 2024-06-03T02:30:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.426+0000 7f6a31e00700 1 -- 172.21.15.79:0/2853220280 learned_addr learned my addr 172.21.15.79:0/2853220280 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.426+0000 7f6a2aa00700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a180a2ca0 0x7f6a180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.426+0000 7f6a2be00700 1 --2- 172.21.15.79:0/2853220280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242ab99070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.426+0000 7f6a2be00700 1 --2- 172.21.15.79:0/2853220280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242aba66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.427+0000 7f6a2aa00700 1 -- 172.21.15.79:0/2428611505 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6a2c0d01f0 con 0x7f6a34107380 2024-06-03T02:30:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.427+0000 7f6a2be00700 1 -- 172.21.15.79:0/2853220280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 msgr2=0x56242ab99a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.427+0000 7f6a2be00700 1 --2- 172.21.15.79:0/2853220280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242ab99a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.427+0000 7f6a2be00700 1 -- 172.21.15.79:0/2853220280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 msgr2=0x56242aba66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:20.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.427+0000 7f6a2be00700 1 --2- 172.21.15.79:0/2853220280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242aba66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.427+0000 7f6a2be00700 1 -- 172.21.15.79:0/2853220280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242aa93f90 con 0x56242ab15e40 2024-06-03T02:30:20.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.427+0000 7f6a2be00700 1 --2- 172.21.15.79:0/2853220280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242ab99070 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6a34110040 tx=0x7f6a1c0096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8abbba502ad1b6f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.427+0000 7f6a32800700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a180a2ca0 0x7f6a180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.428+0000 7f6a23400700 1 -- 172.21.15.79:0/2853220280 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c004030 con 0x56242ab15e40 2024-06-03T02:30:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.428+0000 7f6a23400700 1 -- 172.21.15.79:0/2853220280 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c004830 con 0x56242ab15e40 2024-06-03T02:30:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.428+0000 7f6a23400700 1 -- 172.21.15.79:0/2853220280 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c004030 con 0x56242ab15e40 2024-06-03T02:30:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.428+0000 7f6a32800700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a180a2ca0 0x7f6a180a5160 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c098b90 tx=0x7f6a2c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.429+0000 7f6a436fab80 1 -- 172.21.15.79:0/2853220280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 msgr2=0x56242ab99070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.429+0000 7f6a436fab80 1 --2- 172.21.15.79:0/2853220280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242ab99070 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6a34110040 tx=0x7f6a1c0096d0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.430+0000 7f6a436fab80 1 -- 172.21.15.79:0/2853220280 shutdown_connections 2024-06-03T02:30:20.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.430+0000 7f6a436fab80 1 --2- 172.21.15.79:0/2853220280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242aba66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.430+0000 7f6a436fab80 1 --2- 172.21.15.79:0/2853220280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242ab99070 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.430+0000 7f6a436fab80 1 --2- 172.21.15.79:0/2853220280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242ab99a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.430+0000 7f6a436fab80 1 -- 172.21.15.79:0/2853220280 >> 172.21.15.79:0/2853220280 conn(0x56242aac6170 msgr2=0x56242ab954b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:20.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.430+0000 7f6a436fab80 1 -- 172.21.15.79:0/2853220280 shutdown_connections 2024-06-03T02:30:20.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.430+0000 7f6a436fab80 1 -- 172.21.15.79:0/2853220280 wait complete. 2024-06-03T02:30:20.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.431+0000 7f6a436fab80 1 -- start start 2024-06-03T02:30:20.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.431+0000 7f6a436fab80 1 Processor -- start 2024-06-03T02:30:20.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.432+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242ab15e40 0x56242aba3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.432+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242aba0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.432+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242aba4300 0x56242aba12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.432+0000 7f6a436fab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56242aa87f60 con 0x56242ab15e40 2024-06-03T02:30:20.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.432+0000 7f6a436fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56242aae7ac0 con 0x56242ab995b0 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.432+0000 7f6a436fab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56242aba6fc0 con 0x56242aba4300 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.432+0000 7f6a436fab80 10 client.0 _ll_register_callbacks cb 0x56242aa85720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.433+0000 7f6a2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242aba4300 0x56242aba12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.433+0000 7f6a2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242aba4300 0x56242aba12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53104/0 (socket says 172.21.15.79:53104) 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.433+0000 7f6a2be00700 1 -- 172.21.15.79:0/3593954902 learned_addr learned my addr 172.21.15.79:0/3593954902 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.434+0000 7f6a2be00700 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242ab15e40 0x56242aba3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.434+0000 7f6a31e00700 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242aba0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.434+0000 7f6a31e00700 1 -- 172.21.15.79:0/3593954902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242aba4300 msgr2=0x56242aba12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.434+0000 7f6a31e00700 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242aba4300 0x56242aba12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.434+0000 7f6a31e00700 1 -- 172.21.15.79:0/3593954902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242ab15e40 msgr2=0x56242aba3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.434+0000 7f6a31e00700 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242ab15e40 0x56242aba3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.434+0000 7f6a31e00700 1 -- 172.21.15.79:0/3593954902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242aa3a8d0 con 0x56242ab995b0 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.434+0000 7f6a2be00700 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242aba4300 0x56242aba12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.435+0000 7f6a31e00700 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242aba0d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6a24004630 tx=0x7f6a24004660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d002cc960f834b37 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:20.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.435+0000 7f6a17e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:30:20.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.435+0000 7f6a17e00700 1 -- 172.21.15.79:0/3593954902 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2401a070 con 0x56242ab995b0 2024-06-03T02:30:20.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.435+0000 7f6a17e00700 1 -- 172.21.15.79:0/3593954902 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6a24007a30 con 0x56242ab995b0 2024-06-03T02:30:20.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.435+0000 7f6a436fab80 1 -- 172.21.15.79:0/3593954902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a1c004830 con 0x56242ab995b0 2024-06-03T02:30:20.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.436+0000 7f6a17e00700 1 -- 172.21.15.79:0/3593954902 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a24024050 con 0x56242ab995b0 2024-06-03T02:30:20.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.437+0000 7f6a436fab80 10 client.26483 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:20.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.437+0000 7f6a436fab80 1 -- 172.21.15.79:0/3593954902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56242abb1780 con 0x56242ab995b0 2024-06-03T02:30:20.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.438+0000 7f6a17e00700 1 -- 172.21.15.79:0/3593954902 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c004830 con 0x56242ab995b0 2024-06-03T02:30:20.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.438+0000 7f6a17e00700 1 -- 172.21.15.79:0/3593954902 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6a24016430 con 0x56242ab995b0 2024-06-03T02:30:20.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.439+0000 7f6a436fab80 10 client.26483 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:20.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.439+0000 7f6a436fab80 10 client.26483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:30:20.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.439+0000 7f6a436fab80 20 client.26483 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:20.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.439+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56242abb3dc0 0x56242abb61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.439+0000 7f6a436fab80 4 client.26483 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:30:20.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.439+0000 7f6a436fab80 1 -- 172.21.15.79:0/3593954902 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56242aaf9000 con 0x56242abb3dc0 2024-06-03T02:30:20.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.439+0000 7f6a2be00700 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56242abb3dc0 0x56242abb61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.443+0000 7f6a2be00700 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56242abb3dc0 0x56242abb61b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b185a944b66970f server_cookie=f3547c8f922d3bb6 in_seq=0 out_seq=0 2024-06-03T02:30:20.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.443+0000 7f6a17e00700 10 client.26483 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:20.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.444+0000 7f6a17e00700 1 -- 172.21.15.79:0/3593954902 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56242aaf9000 con 0x56242abb3dc0 2024-06-03T02:30:20.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.444+0000 7f6a17e00700 10 client.26483 handle_command_reply: tid=0 2024-06-03T02:30:20.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.444+0000 7f6a436fab80 1 client.26483 shutdown 2024-06-03T02:30:20.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.445+0000 7f6a436fab80 1 -- 172.21.15.79:0/3593954902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 msgr2=0x56242aba0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:20.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.445+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242aba0d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6a24004630 tx=0x7f6a24004660 comp rx=0 tx=0).stop 2024-06-03T02:30:20.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a436fab80 1 -- 172.21.15.79:0/3593954902 shutdown_connections 2024-06-03T02:30:20.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242aba4300 0x56242aba12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56242abb3dc0 0x56242abb61b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242ab15e40 0x56242aba3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a17e00700 0 client.26483 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:20.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3593954902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242aba0d80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a436fab80 1 -- 172.21.15.79:0/3593954902 >> 172.21.15.79:0/3593954902 conn(0x56242aac6170 msgr2=0x56242ab94740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:20.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a436fab80 1 -- 172.21.15.79:0/3593954902 shutdown_connections 2024-06-03T02:30:20.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a436fab80 1 -- 172.21.15.79:0/3593954902 wait complete. 2024-06-03T02:30:20.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.446+0000 7f6a436fab80 20 client.26483 trim_cache size 0 max 16384 2024-06-03T02:30:20.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.474+0000 7f6a436fab80 1 Processor -- start 2024-06-03T02:30:20.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.474+0000 7f6a436fab80 1 -- start start 2024-06-03T02:30:20.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.474+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab15e40 0x56242aba56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.474+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242ab9bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.474+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242ab9c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.474+0000 7f6a436fab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341bd250 con 0x56242aba4300 2024-06-03T02:30:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.474+0000 7f6a436fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341bed90 con 0x56242ab995b0 2024-06-03T02:30:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.474+0000 7f6a436fab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341bd920 con 0x56242ab15e40 2024-06-03T02:30:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.475+0000 7f6a2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab15e40 0x56242aba56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.475+0000 7f6a2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab15e40 0x56242aba56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53120/0 (socket says 172.21.15.79:53120) 2024-06-03T02:30:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.475+0000 7f6a2be00700 1 -- 172.21.15.79:0/2986917434 learned_addr learned my addr 172.21.15.79:0/2986917434 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:20.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.475+0000 7f6a31e00700 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242ab9bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.475+0000 7f6a2be00700 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242ab9c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.476+0000 7f6a31e00700 1 -- 172.21.15.79:0/2986917434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab15e40 msgr2=0x56242aba56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.476+0000 7f6a31e00700 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab15e40 0x56242aba56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.476+0000 7f6a31e00700 1 -- 172.21.15.79:0/2986917434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 msgr2=0x56242ab9c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.476+0000 7f6a31e00700 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242ab9c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.476+0000 7f6a31e00700 1 -- 172.21.15.79:0/2986917434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242aba1c60 con 0x56242ab995b0 2024-06-03T02:30:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.476+0000 7f6a2be00700 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242ab9c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:30:20.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.476+0000 7f6a31e00700 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242ab9bfd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6a24016880 tx=0x7f6a24016640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919997812eef2f2e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.477+0000 7f6a22a00700 1 -- 172.21.15.79:0/2986917434 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2401a070 con 0x56242ab995b0 2024-06-03T02:30:20.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.477+0000 7f6a22a00700 1 -- 172.21.15.79:0/2986917434 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2402e070 con 0x56242ab995b0 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.477+0000 7f6a22a00700 1 -- 172.21.15.79:0/2986917434 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24007a90 con 0x56242ab995b0 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.477+0000 7f6a436fab80 1 -- 172.21.15.79:0/2986917434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 msgr2=0x56242ab9bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.477+0000 7f6a436fab80 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242ab9bfd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6a24016880 tx=0x7f6a24016640 comp rx=0 tx=0).stop 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.478+0000 7f6a436fab80 1 -- 172.21.15.79:0/2986917434 shutdown_connections 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.478+0000 7f6a436fab80 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242ab9c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.478+0000 7f6a436fab80 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab15e40 0x56242aba56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.478+0000 7f6a436fab80 1 --2- 172.21.15.79:0/2986917434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab995b0 0x56242ab9bfd0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.478+0000 7f6a436fab80 1 -- 172.21.15.79:0/2986917434 >> 172.21.15.79:0/2986917434 conn(0x56242aac6170 msgr2=0x56242ab9f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.478+0000 7f6a436fab80 1 -- 172.21.15.79:0/2986917434 shutdown_connections 2024-06-03T02:30:20.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.478+0000 7f6a436fab80 1 -- 172.21.15.79:0/2986917434 wait complete. 2024-06-03T02:30:20.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.478+0000 7f6a436fab80 1 -- start start 2024-06-03T02:30:20.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.479+0000 7f6a436fab80 1 Processor -- start 2024-06-03T02:30:20.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.479+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242abac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.479+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242abac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.479+0000 7f6a436fab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242ab9e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.479+0000 7f6a436fab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341be6c0 con 0x56242aba4300 2024-06-03T02:30:20.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.479+0000 7f6a436fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341bdff0 con 0x56242ab15e40 2024-06-03T02:30:20.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.479+0000 7f6a436fab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341bb1f0 con 0x56242ab995b0 2024-06-03T02:30:20.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.479+0000 7f6a436fab80 10 client.0 _ll_register_callbacks cb 0x56242aa85720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a2be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242abac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a2be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242abac280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53800/0 (socket says 172.21.15.79:53800) 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a2be00700 1 -- 172.21.15.79:0/3240676068 learned_addr learned my addr 172.21.15.79:0/3240676068 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a31e00700 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242abac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a2be00700 1 -- 172.21.15.79:0/3240676068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 msgr2=0x56242abac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a2be00700 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242abac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a2be00700 1 -- 172.21.15.79:0/3240676068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 msgr2=0x56242ab9e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a2be00700 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242ab9e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a2be00700 1 -- 172.21.15.79:0/3240676068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242abbc530 con 0x56242ab15e40 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.480+0000 7f6a31e00700 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242abac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:30:20.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.481+0000 7f6a2be00700 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242abac280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c005d90 tx=0x7f6a1c005980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c92c4eb5d4a6bd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:20.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.481+0000 7f6a17e00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:30:20.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.481+0000 7f6a17e00700 1 -- 172.21.15.79:0/3240676068 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c005090 con 0x56242ab15e40 2024-06-03T02:30:20.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.481+0000 7f6a17e00700 1 -- 172.21.15.79:0/3240676068 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6a1c005230 con 0x56242ab15e40 2024-06-03T02:30:20.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.481+0000 7f6a436fab80 1 -- 172.21.15.79:0/3240676068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2402e070 con 0x56242ab15e40 2024-06-03T02:30:20.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.482+0000 7f6a17e00700 1 -- 172.21.15.79:0/3240676068 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c005430 con 0x56242ab15e40 2024-06-03T02:30:20.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.482+0000 7f6a436fab80 10 client.26489 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:20.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.482+0000 7f6a436fab80 1 -- 172.21.15.79:0/3240676068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56242abbaf00 con 0x56242ab15e40 2024-06-03T02:30:20.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.483+0000 7f6a17e00700 1 -- 172.21.15.79:0/3240676068 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2402e070 con 0x56242ab15e40 2024-06-03T02:30:20.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.483+0000 7f6a17e00700 1 -- 172.21.15.79:0/3240676068 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6a1c00fcc0 con 0x56242ab15e40 2024-06-03T02:30:20.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.484+0000 7f6a436fab80 10 client.26489 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:20.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.484+0000 7f6a436fab80 10 client.26489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:30:20.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.484+0000 7f6a436fab80 20 client.26489 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:20.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.484+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56242abb3dc0 0x56242abbb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:20.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.484+0000 7f6a436fab80 4 client.26489 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:30:20.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.484+0000 7f6a436fab80 1 -- 172.21.15.79:0/3240676068 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56242aaf3580 con 0x56242abb3dc0 2024-06-03T02:30:20.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.485+0000 7f6a31e00700 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56242abb3dc0 0x56242abbb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:20.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.486+0000 7f6a31e00700 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56242abb3dc0 0x56242abbb430 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1637e6dd09b903 server_cookie=2cb13f1d1e88c686 in_seq=0 out_seq=0 2024-06-03T02:30:20.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.486+0000 7f6a17e00700 10 client.26489 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:20.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.486+0000 7f6a17e00700 1 -- 172.21.15.79:0/3240676068 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56242aaf3580 con 0x56242abb3dc0 2024-06-03T02:30:20.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.486+0000 7f6a17e00700 10 client.26489 handle_command_reply: tid=0 2024-06-03T02:30:20.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.486+0000 7f6a436fab80 1 client.26489 shutdown 2024-06-03T02:30:20.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.486+0000 7f6a436fab80 1 -- 172.21.15.79:0/3240676068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 msgr2=0x56242abac280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:20.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.486+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242abac280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c005d90 tx=0x7f6a1c005980 comp rx=0 tx=0).stop 2024-06-03T02:30:20.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a436fab80 1 -- 172.21.15.79:0/3240676068 shutdown_connections 2024-06-03T02:30:20.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56242abb3dc0 0x56242abbb430 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56242aba4300 0x56242ab9e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56242ab15e40 0x56242abac280 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a436fab80 1 --2- 172.21.15.79:0/3240676068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56242ab995b0 0x56242abac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a436fab80 1 -- 172.21.15.79:0/3240676068 >> 172.21.15.79:0/3240676068 conn(0x56242aac6170 msgr2=0x56242ab9f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:20.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a17e00700 0 client.26489 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:20.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a436fab80 1 -- 172.21.15.79:0/3240676068 shutdown_connections 2024-06-03T02:30:20.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a436fab80 1 -- 172.21.15.79:0/3240676068 wait complete. 2024-06-03T02:30:20.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.487+0000 7f6a436fab80 20 client.26489 trim_cache size 0 max 16384 2024-06-03T02:30:20.488 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:30:20.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.488+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a180a2ca0 msgr2=0x7f6a180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:20.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.488+0000 7f6a38e00700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a180a2ca0 0x7f6a180a5160 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c098b90 tx=0x7f6a2c01a040 comp rx=0 tx=0).stop 2024-06-03T02:30:20.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.488+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34107380 msgr2=0x7f6a3410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:20.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.488+0000 7f6a38e00700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34107380 0x7f6a3410ea40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b820 tx=0x7f6a2c00bab0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.489+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 shutdown_connections 2024-06-03T02:30:20.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.489+0000 7f6a38e00700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f6a180a2ca0 0x7f6a180a5160 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.489+0000 7f6a38e00700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6a34107380 0x7f6a3410ea40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.489+0000 7f6a38e00700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a3406a6b0 0x7f6a34115760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.489+0000 7f6a38e00700 1 --2- 172.21.15.79:0/2428611505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a34104e90 0x7f6a34115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:20.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.489+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 >> 172.21.15.79:0/2428611505 conn(0x7f6a340782c0 msgr2=0x7f6a34103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:20.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.489+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 shutdown_connections 2024-06-03T02:30:20.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:20.489+0000 7f6a38e00700 1 -- 172.21.15.79:0/2428611505 wait complete. 2024-06-03T02:30:22.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:21 smithi079 ceph-mon[31819]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 427 op/s 2024-06-03T02:30:22.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:21 smithi123 ceph-mon[36640]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 427 op/s 2024-06-03T02:30:22.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:21 smithi135 ceph-mon[36653]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 427 op/s 2024-06-03T02:30:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 615.0 s, 520.0 tps, lat 198.328 ms stddev 249.765 2024-06-03T02:30:23.442 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:30:23.442 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:30:24.149 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:30:24.324 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:23 smithi079 ceph-mon[31819]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 421 op/s 2024-06-03T02:30:24.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:23 smithi135 ceph-mon[36653]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 421 op/s 2024-06-03T02:30:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:23 smithi123 ceph-mon[36640]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 421 op/s 2024-06-03T02:30:26.241 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:25 smithi079 ceph-mon[31819]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 506 op/s 2024-06-03T02:30:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:25 smithi135 ceph-mon[36653]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 506 op/s 2024-06-03T02:30:26.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:25 smithi123 ceph-mon[36640]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 506 op/s 2024-06-03T02:30:28.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:27 smithi079 ceph-mon[31819]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 365 op/s 2024-06-03T02:30:28.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.226+0000 7fd98de00700 1 -- 172.21.15.79:0/1795608185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9881155e0 msgr2=0x7fd9881179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:28.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.226+0000 7fd98de00700 1 --2- 172.21.15.79:0/1795608185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9881155e0 0x7fd9881179d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd980009f90 tx=0x7fd98000b330 comp rx=0 tx=0).stop 2024-06-03T02:30:28.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.226+0000 7fd98de00700 1 -- 172.21.15.79:0/1795608185 shutdown_connections 2024-06-03T02:30:28.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.226+0000 7fd98de00700 1 --2- 172.21.15.79:0/1795608185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9881155e0 0x7fd9881179d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.226+0000 7fd98de00700 1 --2- 172.21.15.79:0/1795608185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd988109490 0x7fd988109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.226+0000 7fd98de00700 1 --2- 172.21.15.79:0/1795608185 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd988109db0 0x7fd9881150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.226+0000 7fd98de00700 1 -- 172.21.15.79:0/1795608185 >> 172.21.15.79:0/1795608185 conn(0x7fd98806d600 msgr2=0x7fd98806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:28.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.227+0000 7fd98de00700 1 -- 172.21.15.79:0/1795608185 shutdown_connections 2024-06-03T02:30:28.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.227+0000 7fd98de00700 1 -- 172.21.15.79:0/1795608185 wait complete. 2024-06-03T02:30:28.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.231+0000 7fd98de00700 1 Processor -- start 2024-06-03T02:30:28.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.233+0000 7fd98de00700 1 -- start start 2024-06-03T02:30:28.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.235+0000 7fd98de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd988109490 0x7fd9881a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.235+0000 7fd98de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd988109db0 0x7fd9881a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.235+0000 7fd98de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9881155e0 0x7fd9881a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.235+0000 7fd986000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd988109db0 0x7fd9881a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.235+0000 7fd98de00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd98811b040 con 0x7fd988109490 2024-06-03T02:30:28.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.235+0000 7fd98de00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd98811aec0 con 0x7fd9881155e0 2024-06-03T02:30:28.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.235+0000 7fd98de00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd98811b1c0 con 0x7fd988109db0 2024-06-03T02:30:28.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.236+0000 7fd986000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd988109db0 0x7fd9881a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54438/0 (socket says 172.21.15.79:54438) 2024-06-03T02:30:28.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.236+0000 7fd986000700 1 -- 172.21.15.79:0/1351479962 learned_addr learned my addr 172.21.15.79:0/1351479962 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:28.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.236+0000 7fd986a00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd988109490 0x7fd9881a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.236+0000 7fd987400700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9881155e0 0x7fd9881a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.236+0000 7fd987400700 1 -- 172.21.15.79:0/1351479962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd988109db0 msgr2=0x7fd9881a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:28.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.236+0000 7fd987400700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd988109db0 0x7fd9881a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.236+0000 7fd987400700 1 -- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd988109490 msgr2=0x7fd9881a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:28.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.236+0000 7fd987400700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd988109490 0x7fd9881a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.236+0000 7fd987400700 1 -- 172.21.15.79:0/1351479962 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9881a8680 con 0x7fd9881155e0 2024-06-03T02:30:28.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.237+0000 7fd987400700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9881155e0 0x7fd9881a7f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd980009f90 tx=0x7fd98000cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:28.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.237+0000 7fd98ca00700 1 -- 172.21.15.79:0/1351479962 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980021070 con 0x7fd9881155e0 2024-06-03T02:30:28.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.237+0000 7fd98ca00700 1 -- 172.21.15.79:0/1351479962 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd980007ac0 con 0x7fd9881155e0 2024-06-03T02:30:28.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.237+0000 7fd98ca00700 1 -- 172.21.15.79:0/1351479962 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9800052a0 con 0x7fd9881155e0 2024-06-03T02:30:28.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.238+0000 7fd98de00700 1 -- 172.21.15.79:0/1351479962 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9881a8910 con 0x7fd9881155e0 2024-06-03T02:30:28.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.238+0000 7fd98de00700 1 -- 172.21.15.79:0/1351479962 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9881b01f0 con 0x7fd9881155e0 2024-06-03T02:30:28.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.248+0000 7fd98ca00700 1 -- 172.21.15.79:0/1351479962 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd980024070 con 0x7fd9881155e0 2024-06-03T02:30:28.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.251+0000 7fd98ca00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd96c0a2df0 0x7fd96c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.251+0000 7fd98ca00700 1 -- 172.21.15.79:0/1351479962 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd98000f070 con 0x7fd9881155e0 2024-06-03T02:30:28.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.252+0000 7fd986a00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd96c0a2df0 0x7fd96c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.252+0000 7fd986a00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd96c0a2df0 0x7fd96c0a52b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd978005fd0 tx=0x7fd978009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:28.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.259+0000 7fd99869cb80 1 Processor -- start 2024-06-03T02:30:28.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.259+0000 7fd99869cb80 1 -- start start 2024-06-03T02:30:28.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.261+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388cc10 0x562ef388cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.262+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3886d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.264+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef38872c0 0x562ef3887700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.264+0000 7fd986000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3886d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.264+0000 7fd99869cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562ef3795a90 con 0x562ef388cc10 2024-06-03T02:30:28.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.264+0000 7fd99869cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ef37d8b80 con 0x562ef388d530 2024-06-03T02:30:28.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.264+0000 7fd99869cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ef3793970 con 0x562ef38872c0 2024-06-03T02:30:28.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.264+0000 7fd986000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3886d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35680/0 (socket says 172.21.15.79:35680) 2024-06-03T02:30:28.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.264+0000 7fd986000700 1 -- 172.21.15.79:0/2041438128 learned_addr learned my addr 172.21.15.79:0/2041438128 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:28.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.264+0000 7fd987400700 1 --2- 172.21.15.79:0/2041438128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef38872c0 0x562ef3887700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.265+0000 7fd986000700 1 -- 172.21.15.79:0/2041438128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef38872c0 msgr2=0x562ef3887700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:28.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.265+0000 7fd986000700 1 --2- 172.21.15.79:0/2041438128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef38872c0 0x562ef3887700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.265+0000 7fd986000700 1 -- 172.21.15.79:0/2041438128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388cc10 msgr2=0x562ef388cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:28.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.265+0000 7fd986000700 1 --2- 172.21.15.79:0/2041438128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388cc10 0x562ef388cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.266+0000 7fd986000700 1 -- 172.21.15.79:0/2041438128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef3781010 con 0x562ef388d530 2024-06-03T02:30:28.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.267+0000 7fd986000700 1 --2- 172.21.15.79:0/2041438128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3886d80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd97400acc0 tx=0x7fd97400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3690438b98466fb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:28.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.267+0000 7fd97cc00700 1 -- 172.21.15.79:0/2041438128 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97401d070 con 0x562ef388d530 2024-06-03T02:30:28.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.267+0000 7fd97cc00700 1 -- 172.21.15.79:0/2041438128 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd97400fe20 con 0x562ef388d530 2024-06-03T02:30:28.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.267+0000 7fd97cc00700 1 -- 172.21.15.79:0/2041438128 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974010850 con 0x562ef388d530 2024-06-03T02:30:28.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.271+0000 7fd99869cb80 1 -- 172.21.15.79:0/2041438128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 msgr2=0x562ef3886d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:28.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.271+0000 7fd99869cb80 1 --2- 172.21.15.79:0/2041438128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3886d80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd97400acc0 tx=0x7fd97400f3c0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.271+0000 7fd99869cb80 1 -- 172.21.15.79:0/2041438128 shutdown_connections 2024-06-03T02:30:28.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.271+0000 7fd99869cb80 1 --2- 172.21.15.79:0/2041438128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef38872c0 0x562ef3887700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.271+0000 7fd99869cb80 1 --2- 172.21.15.79:0/2041438128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388cc10 0x562ef388cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.271+0000 7fd99869cb80 1 --2- 172.21.15.79:0/2041438128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3886d80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.271+0000 7fd99869cb80 1 -- 172.21.15.79:0/2041438128 >> 172.21.15.79:0/2041438128 conn(0x562ef37b31f0 msgr2=0x562ef3882070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:28.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.271+0000 7fd99869cb80 1 -- 172.21.15.79:0/2041438128 shutdown_connections 2024-06-03T02:30:28.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.271+0000 7fd99869cb80 1 -- 172.21.15.79:0/2041438128 wait complete. 2024-06-03T02:30:28.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.272+0000 7fd99869cb80 1 -- start start 2024-06-03T02:30:28.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.273+0000 7fd99869cb80 1 Processor -- start 2024-06-03T02:30:28.275 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:27 smithi123 ceph-mon[36640]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 365 op/s 2024-06-03T02:30:28.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.276+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef3893360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.277+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef38906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.277+0000 7fd986a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef3893360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.277+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef3890bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.277+0000 7fd986a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef3893360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35686/0 (socket says 172.21.15.79:35686) 2024-06-03T02:30:28.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.277+0000 7fd986000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef38906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.277+0000 7fd986000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef38906b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54462/0 (socket says 172.21.15.79:54462) 2024-06-03T02:30:28.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.277+0000 7fd99869cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562ef3774fe0 con 0x562ef388d530 2024-06-03T02:30:28.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.277+0000 7fd99869cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ef37d4b40 con 0x562ef38872c0 2024-06-03T02:30:28.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.277+0000 7fd99869cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ef3893f10 con 0x562ef388cc10 2024-06-03T02:30:28.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.278+0000 7fd99869cb80 10 client.0 _ll_register_callbacks cb 0x562ef376e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:28.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.278+0000 7fd987400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef3890bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.278+0000 7fd987400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef3890bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35460/0 (socket says 172.21.15.79:35460) 2024-06-03T02:30:28.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.278+0000 7fd987400700 1 -- 172.21.15.79:0/4241519239 learned_addr learned my addr 172.21.15.79:0/4241519239 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:28.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.278+0000 7fd987400700 1 -- 172.21.15.79:0/4241519239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 msgr2=0x562ef38906b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:28.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.278+0000 7fd987400700 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef38906b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.278+0000 7fd987400700 1 -- 172.21.15.79:0/4241519239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 msgr2=0x562ef3893360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:28.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.278+0000 7fd987400700 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef3893360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.278+0000 7fd987400700 1 -- 172.21.15.79:0/4241519239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef3727950 con 0x562ef388d530 2024-06-03T02:30:28.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.280+0000 7fd986a00700 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef3893360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-03T02:30:28.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.280+0000 7fd986000700 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef38906b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-03T02:30:28.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.280+0000 7fd987400700 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef3890bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd98000d010 tx=0x7fd98009c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209916ed42fc8c3e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:28.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.280+0000 7fd971600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:30:28.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.280+0000 7fd971600700 1 -- 172.21.15.79:0/4241519239 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9800a4ac0 con 0x562ef388d530 2024-06-03T02:30:28.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.280+0000 7fd971600700 1 -- 172.21.15.79:0/4241519239 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd9800cd170 con 0x562ef388d530 2024-06-03T02:30:28.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.281+0000 7fd99869cb80 1 -- 172.21.15.79:0/4241519239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd97400fe20 con 0x562ef388d530 2024-06-03T02:30:28.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.281+0000 7fd971600700 1 -- 172.21.15.79:0/4241519239 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd980007420 con 0x562ef388d530 2024-06-03T02:30:28.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.282+0000 7fd99869cb80 10 client.16026 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:28.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.282+0000 7fd99869cb80 1 -- 172.21.15.79:0/4241519239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562ef389e770 con 0x562ef388d530 2024-06-03T02:30:28.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.283+0000 7fd971600700 1 -- 172.21.15.79:0/4241519239 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd97400fe20 con 0x562ef388d530 2024-06-03T02:30:28.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.283+0000 7fd971600700 1 -- 172.21.15.79:0/4241519239 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd980011070 con 0x562ef388d530 2024-06-03T02:30:28.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.283+0000 7fd99869cb80 10 client.16026 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:28.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.283+0000 7fd99869cb80 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:30:28.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.283+0000 7fd99869cb80 20 client.16026 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:28.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.283+0000 7fd99869cb80 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562ef38a0d80 0x562ef38a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.283+0000 7fd99869cb80 4 client.16026 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:30:28.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.283+0000 7fd99869cb80 1 -- 172.21.15.79:0/4241519239 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ef37e6080 con 0x562ef38a0d80 2024-06-03T02:30:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.284+0000 7fd986a00700 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562ef38a0d80 0x562ef38a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.284+0000 7fd971600700 10 client.16026 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.285+0000 7fd986a00700 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562ef38a0d80 0x562ef38a31c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62774831806de00e server_cookie=b8ddb9816607aec0 in_seq=0 out_seq=0 2024-06-03T02:30:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.286+0000 7fd971600700 1 -- 172.21.15.79:0/4241519239 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562ef37e6080 con 0x562ef38a0d80 2024-06-03T02:30:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.286+0000 7fd971600700 10 client.16026 handle_command_reply: tid=0 2024-06-03T02:30:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.287+0000 7fd99869cb80 1 client.16026 shutdown 2024-06-03T02:30:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.287+0000 7fd99869cb80 1 -- 172.21.15.79:0/4241519239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 msgr2=0x562ef3890bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.287+0000 7fd99869cb80 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef3890bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd98000d010 tx=0x7fd98009c710 comp rx=0 tx=0).stop 2024-06-03T02:30:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.287+0000 7fd99869cb80 1 -- 172.21.15.79:0/4241519239 shutdown_connections 2024-06-03T02:30:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.287+0000 7fd99869cb80 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562ef38a0d80 0x562ef38a31c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.288+0000 7fd99869cb80 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef3890bf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.288+0000 7fd971600700 0 client.16026 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:28.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.288+0000 7fd99869cb80 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef3893360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.288+0000 7fd99869cb80 1 --2- 172.21.15.79:0/4241519239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef38906b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.288+0000 7fd99869cb80 1 -- 172.21.15.79:0/4241519239 >> 172.21.15.79:0/4241519239 conn(0x562ef37b31f0 msgr2=0x562ef387b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:28.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.288+0000 7fd99869cb80 1 -- 172.21.15.79:0/4241519239 shutdown_connections 2024-06-03T02:30:28.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.288+0000 7fd99869cb80 1 -- 172.21.15.79:0/4241519239 wait complete. 2024-06-03T02:30:28.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.288+0000 7fd99869cb80 20 client.16026 trim_cache size 0 max 16384 2024-06-03T02:30:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 620.0 s, 459.6 tps, lat 199.010 ms stddev 228.523 2024-06-03T02:30:28.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.336+0000 7fd99869cb80 1 Processor -- start 2024-06-03T02:30:28.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd99869cb80 1 -- start start 2024-06-03T02:30:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef38872c0 0x562ef3886d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef387c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3892ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd99869cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881b4990 con 0x562ef38872c0 2024-06-03T02:30:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd99869cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881b64d0 con 0x562ef388d530 2024-06-03T02:30:28.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd99869cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881b5060 con 0x562ef388cc10 2024-06-03T02:30:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd986000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef387c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd986000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef387c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54474/0 (socket says 172.21.15.79:54474) 2024-06-03T02:30:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.337+0000 7fd986000700 1 -- 172.21.15.79:0/292454129 learned_addr learned my addr 172.21.15.79:0/292454129 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.338+0000 7fd986a00700 1 --2- 172.21.15.79:0/292454129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef38872c0 0x562ef3886d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.338+0000 7fd987400700 1 --2- 172.21.15.79:0/292454129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3892ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.338+0000 7fd986000700 1 -- 172.21.15.79:0/292454129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 msgr2=0x562ef3892ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.338+0000 7fd986000700 1 --2- 172.21.15.79:0/292454129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3892ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.338+0000 7fd986000700 1 -- 172.21.15.79:0/292454129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef38872c0 msgr2=0x562ef3886d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.338+0000 7fd986000700 1 --2- 172.21.15.79:0/292454129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef38872c0 0x562ef3886d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.338+0000 7fd986000700 1 -- 172.21.15.79:0/292454129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef388d240 con 0x562ef388cc10 2024-06-03T02:30:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.339+0000 7fd986000700 1 --2- 172.21.15.79:0/292454129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef387c1b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd97400a2a0 tx=0x7fd974005950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db8fe33618e8796 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.339+0000 7fd973e00700 1 -- 172.21.15.79:0/292454129 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97401d040 con 0x562ef388cc10 2024-06-03T02:30:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.339+0000 7fd973e00700 1 -- 172.21.15.79:0/292454129 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd974028070 con 0x562ef388cc10 2024-06-03T02:30:28.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.339+0000 7fd973e00700 1 -- 172.21.15.79:0/292454129 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974004ba0 con 0x562ef388cc10 2024-06-03T02:30:28.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.340+0000 7fd99869cb80 1 -- 172.21.15.79:0/292454129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 msgr2=0x562ef387c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.340+0000 7fd99869cb80 1 --2- 172.21.15.79:0/292454129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef387c1b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd97400a2a0 tx=0x7fd974005950 comp rx=0 tx=0).stop 2024-06-03T02:30:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.341+0000 7fd99869cb80 1 -- 172.21.15.79:0/292454129 shutdown_connections 2024-06-03T02:30:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.341+0000 7fd99869cb80 1 --2- 172.21.15.79:0/292454129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef388d530 0x562ef3892ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.341+0000 7fd99869cb80 1 --2- 172.21.15.79:0/292454129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef38872c0 0x562ef3886d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.341+0000 7fd99869cb80 1 --2- 172.21.15.79:0/292454129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef387c1b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.341+0000 7fd99869cb80 1 -- 172.21.15.79:0/292454129 >> 172.21.15.79:0/292454129 conn(0x562ef37b31f0 msgr2=0x562ef388bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.341+0000 7fd99869cb80 1 -- 172.21.15.79:0/292454129 shutdown_connections 2024-06-03T02:30:28.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.341+0000 7fd99869cb80 1 -- 172.21.15.79:0/292454129 wait complete. 2024-06-03T02:30:28.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.341+0000 7fd99869cb80 1 -- start start 2024-06-03T02:30:28.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.342+0000 7fd99869cb80 1 Processor -- start 2024-06-03T02:30:28.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.344+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef38991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.344+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef3899710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:27 smithi135 ceph-mon[36653]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 365 op/s 2024-06-03T02:30:28.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.345+0000 7fd986000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef3899710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.345+0000 7fd986000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef3899710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54488/0 (socket says 172.21.15.79:54488) 2024-06-03T02:30:28.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.345+0000 7fd99869cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef38961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.345+0000 7fd99869cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881b5e00 con 0x562ef388d530 2024-06-03T02:30:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.345+0000 7fd99869cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881b5730 con 0x562ef38872c0 2024-06-03T02:30:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.345+0000 7fd99869cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9881b2930 con 0x562ef388cc10 2024-06-03T02:30:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.345+0000 7fd99869cb80 10 client.0 _ll_register_callbacks cb 0x562ef376e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:28.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd986a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef38991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd986a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef38991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35702/0 (socket says 172.21.15.79:35702) 2024-06-03T02:30:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd986a00700 1 -- 172.21.15.79:0/1320074289 learned_addr learned my addr 172.21.15.79:0/1320074289 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd987400700 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef38961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd986a00700 1 -- 172.21.15.79:0/1320074289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 msgr2=0x562ef3899710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd986a00700 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef3899710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd986a00700 1 -- 172.21.15.79:0/1320074289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 msgr2=0x562ef38961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd986a00700 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef38961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd986a00700 1 -- 172.21.15.79:0/1320074289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef38966e0 con 0x562ef38872c0 2024-06-03T02:30:28.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.346+0000 7fd986000700 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef3899710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-03T02:30:28.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.348+0000 7fd986a00700 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef38991d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd978011770 tx=0x7fd9780047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7740e4a6adf0a67 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:28.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.348+0000 7fd971600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:30:28.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.348+0000 7fd971600700 1 -- 172.21.15.79:0/1320074289 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9780272d0 con 0x562ef38872c0 2024-06-03T02:30:28.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.349+0000 7fd99869cb80 1 -- 172.21.15.79:0/1320074289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd974028070 con 0x562ef38872c0 2024-06-03T02:30:28.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.349+0000 7fd971600700 1 -- 172.21.15.79:0/1320074289 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd978027470 con 0x562ef38872c0 2024-06-03T02:30:28.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.351+0000 7fd971600700 1 -- 172.21.15.79:0/1320074289 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd978046050 con 0x562ef38872c0 2024-06-03T02:30:28.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.351+0000 7fd99869cb80 10 client.26504 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:28.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.351+0000 7fd99869cb80 1 -- 172.21.15.79:0/1320074289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562ef390c290 con 0x562ef38872c0 2024-06-03T02:30:28.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.352+0000 7fd971600700 1 -- 172.21.15.79:0/1320074289 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd974028070 con 0x562ef38872c0 2024-06-03T02:30:28.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.352+0000 7fd971600700 1 -- 172.21.15.79:0/1320074289 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd97801f970 con 0x562ef38872c0 2024-06-03T02:30:28.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.353+0000 7fd99869cb80 10 client.26504 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:28.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.353+0000 7fd99869cb80 10 client.26504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:30:28.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.353+0000 7fd99869cb80 20 client.26504 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:28.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.353+0000 7fd99869cb80 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562ef38a0d80 0x562ef390e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:28.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.353+0000 7fd99869cb80 4 client.26504 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:30:28.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.353+0000 7fd99869cb80 1 -- 172.21.15.79:0/1320074289 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ef37e0600 con 0x562ef38a0d80 2024-06-03T02:30:28.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.355+0000 7fd986000700 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562ef38a0d80 0x562ef390e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:28.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.355+0000 7fd986000700 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562ef38a0d80 0x562ef390e9e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ae9b60e7fbe5f5 server_cookie=26b605f055a83261 in_seq=0 out_seq=0 2024-06-03T02:30:28.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.356+0000 7fd971600700 10 client.26504 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:28.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.357+0000 7fd971600700 1 -- 172.21.15.79:0/1320074289 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ef37e0600 con 0x562ef38a0d80 2024-06-03T02:30:28.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.357+0000 7fd971600700 10 client.26504 handle_command_reply: tid=0 2024-06-03T02:30:28.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.357+0000 7fd99869cb80 1 client.26504 shutdown 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.359+0000 7fd99869cb80 1 -- 172.21.15.79:0/1320074289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 msgr2=0x562ef38991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.359+0000 7fd99869cb80 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef38991d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd978011770 tx=0x7fd9780047c0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.359+0000 7fd99869cb80 1 -- 172.21.15.79:0/1320074289 shutdown_connections 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.359+0000 7fd99869cb80 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562ef38a0d80 0x562ef390e9e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.359+0000 7fd99869cb80 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ef388d530 0x562ef38961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.359+0000 7fd99869cb80 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ef38872c0 0x562ef38991d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.359+0000 7fd99869cb80 1 --2- 172.21.15.79:0/1320074289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef388cc10 0x562ef3899710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.359+0000 7fd99869cb80 1 -- 172.21.15.79:0/1320074289 >> 172.21.15.79:0/1320074289 conn(0x562ef37b31f0 msgr2=0x562ef388b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.359+0000 7fd971600700 0 client.26504 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.360+0000 7fd99869cb80 1 -- 172.21.15.79:0/1320074289 shutdown_connections 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.360+0000 7fd99869cb80 1 -- 172.21.15.79:0/1320074289 wait complete. 2024-06-03T02:30:28.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.360+0000 7fd99869cb80 20 client.26504 trim_cache size 0 max 16384 2024-06-03T02:30:28.362 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:30:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:30:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "fae9d7b7-3177-4c1e-8935-7cb7a03ee836", 2024-06-03T02:30:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:30:28.362 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:30:28.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.361+0000 7fd98de00700 1 -- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd96c0a2df0 msgr2=0x7fd96c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:28.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.361+0000 7fd98de00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd96c0a2df0 0x7fd96c0a52b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd978005fd0 tx=0x7fd978009310 comp rx=0 tx=0).stop 2024-06-03T02:30:28.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.361+0000 7fd98de00700 1 -- 172.21.15.79:0/1351479962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9881155e0 msgr2=0x7fd9881a7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:28.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.361+0000 7fd98de00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9881155e0 0x7fd9881a7f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd980009f90 tx=0x7fd98000cfa0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.362+0000 7fd98de00700 1 -- 172.21.15.79:0/1351479962 shutdown_connections 2024-06-03T02:30:28.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.362+0000 7fd98de00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9881155e0 0x7fd9881a7f40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.362+0000 7fd98de00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd96c0a2df0 0x7fd96c0a52b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.362+0000 7fd98de00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd988109490 0x7fd9881a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.362+0000 7fd98de00700 1 --2- 172.21.15.79:0/1351479962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd988109db0 0x7fd9881a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:28.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.362+0000 7fd98de00700 1 -- 172.21.15.79:0/1351479962 >> 172.21.15.79:0/1351479962 conn(0x7fd98806d600 msgr2=0x7fd988071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:28.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.364+0000 7fd98de00700 1 -- 172.21.15.79:0/1351479962 shutdown_connections 2024-06-03T02:30:28.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:28.364+0000 7fd98de00700 1 -- 172.21.15.79:0/1351479962 wait complete. 2024-06-03T02:30:29.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:30:30.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:29 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:30:30.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:29 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:30:30.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:29 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:30:30.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:29 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:30:30.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:29 smithi079 ceph-mon[31819]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 518 op/s 2024-06-03T02:30:30.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:29 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:30:30.319 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:30:30.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:29 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:30:30.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:29 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:30:30.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:29 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:30:30.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:29 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:30:30.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:29 smithi135 ceph-mon[36653]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 518 op/s 2024-06-03T02:30:30.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:29 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:30:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:29 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:30:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:29 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:30:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:29 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:30:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:29 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:30:30.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:29 smithi123 ceph-mon[36640]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 518 op/s 2024-06-03T02:30:30.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:29 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:30:32.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:32 smithi079 ceph-mon[31819]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 430 op/s 2024-06-03T02:30:32.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:32 smithi135 ceph-mon[36653]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 430 op/s 2024-06-03T02:30:32.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:32 smithi123 ceph-mon[36640]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 430 op/s 2024-06-03T02:30:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 625.0 s, 507.8 tps, lat 209.711 ms stddev 298.434 2024-06-03T02:30:34.146 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:34 smithi079 ceph-mon[31819]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.2 MiB/s wr, 453 op/s 2024-06-03T02:30:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:34 smithi135 ceph-mon[36653]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.2 MiB/s wr, 453 op/s 2024-06-03T02:30:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:34 smithi123 ceph-mon[36640]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.2 MiB/s wr, 453 op/s 2024-06-03T02:30:34.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.551+0000 7efd1a200700 1 -- 172.21.15.79:0/1627163879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd141084e0 msgr2=0x7efd141088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:34.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.551+0000 7efd1a200700 1 --2- 172.21.15.79:0/1627163879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd141084e0 0x7efd141088c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efd04009990 tx=0x7efd04009ca0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.551+0000 7efd1a200700 1 -- 172.21.15.79:0/1627163879 shutdown_connections 2024-06-03T02:30:34.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.551+0000 7efd1a200700 1 --2- 172.21.15.79:0/1627163879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1406ab10 0x7efd1410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.551+0000 7efd1a200700 1 --2- 172.21.15.79:0/1627163879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd141084e0 0x7efd141088c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.551+0000 7efd1a200700 1 --2- 172.21.15.79:0/1627163879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd1406a170 0x7efd1406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.551+0000 7efd1a200700 1 -- 172.21.15.79:0/1627163879 >> 172.21.15.79:0/1627163879 conn(0x7efd140782c0 msgr2=0x7efd140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:34.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.551+0000 7efd1a200700 1 -- 172.21.15.79:0/1627163879 shutdown_connections 2024-06-03T02:30:34.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.551+0000 7efd1a200700 1 -- 172.21.15.79:0/1627163879 wait complete. 2024-06-03T02:30:34.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.552+0000 7efd1a200700 1 Processor -- start 2024-06-03T02:30:34.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.553+0000 7efd1a200700 1 -- start start 2024-06-03T02:30:34.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.554+0000 7efd1a200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1406a170 0x7efd14111f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.554+0000 7efd1a200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1406ab10 0x7efd14112480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.554+0000 7efd13400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1406a170 0x7efd14111f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.554+0000 7efd1a200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd141084e0 0x7efd141176c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.554+0000 7efd1a200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efd14110520 con 0x7efd141084e0 2024-06-03T02:30:34.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.555+0000 7efd12a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1406ab10 0x7efd14112480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.555+0000 7efd1a200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd141103a0 con 0x7efd1406ab10 2024-06-03T02:30:34.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.555+0000 7efd1a200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd141106a0 con 0x7efd1406a170 2024-06-03T02:30:34.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.555+0000 7efd12a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1406ab10 0x7efd14112480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35710/0 (socket says 172.21.15.79:35710) 2024-06-03T02:30:34.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.555+0000 7efd12a00700 1 -- 172.21.15.79:0/3172779424 learned_addr learned my addr 172.21.15.79:0/3172779424 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:34.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.555+0000 7efd13e00700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd141084e0 0x7efd141176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.555+0000 7efd13400700 1 -- 172.21.15.79:0/3172779424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1406ab10 msgr2=0x7efd14112480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:34.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.555+0000 7efd13400700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1406ab10 0x7efd14112480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.555+0000 7efd13400700 1 -- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd141084e0 msgr2=0x7efd141176c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:34.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.557+0000 7efd13400700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd141084e0 0x7efd141176c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.557+0000 7efd13400700 1 -- 172.21.15.79:0/3172779424 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd14117f70 con 0x7efd1406a170 2024-06-03T02:30:34.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.557+0000 7efd13e00700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd141084e0 0x7efd141176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:30:34.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.557+0000 7efd13400700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1406a170 0x7efd14111f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd040059d0 tx=0x7efd04009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:34.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.558+0000 7efd0be00700 1 -- 172.21.15.79:0/3172779424 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04027070 con 0x7efd1406a170 2024-06-03T02:30:34.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.558+0000 7efd0be00700 1 -- 172.21.15.79:0/3172779424 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd04004050 con 0x7efd1406a170 2024-06-03T02:30:34.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.558+0000 7efd0be00700 1 -- 172.21.15.79:0/3172779424 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04022cf0 con 0x7efd1406a170 2024-06-03T02:30:34.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.558+0000 7efd1a200700 1 -- 172.21.15.79:0/3172779424 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd14118200 con 0x7efd1406a170 2024-06-03T02:30:34.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.558+0000 7efd1a200700 1 -- 172.21.15.79:0/3172779424 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd141b4470 con 0x7efd1406a170 2024-06-03T02:30:34.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.566+0000 7efd0be00700 1 -- 172.21.15.79:0/3172779424 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efd04022470 con 0x7efd1406a170 2024-06-03T02:30:34.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.568+0000 7efd0be00700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efcf80a2dd0 0x7efcf80a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.569+0000 7efd0be00700 1 -- 172.21.15.79:0/3172779424 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7efd040d0270 con 0x7efd1406a170 2024-06-03T02:30:34.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.569+0000 7efd24b4fb80 1 Processor -- start 2024-06-03T02:30:34.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.569+0000 7efd12a00700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efcf80a2dd0 0x7efcf80a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.570+0000 7efd24b4fb80 1 -- start start 2024-06-03T02:30:34.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.571+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80aaa1e40 0x55f80ab25070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.571+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab255b0 0x55f80ab25a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.571+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.571+0000 7efd24b4fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f80aa34a10 con 0x55f80ab30300 2024-06-03T02:30:34.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.571+0000 7efd24b4fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f80aa77b00 con 0x55f80aaa1e40 2024-06-03T02:30:34.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.571+0000 7efd24b4fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f80aa328f0 con 0x55f80ab255b0 2024-06-03T02:30:34.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.571+0000 7efd12a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab255b0 0x55f80ab25a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.572+0000 7efd12a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab255b0 0x55f80ab25a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54516/0 (socket says 172.21.15.79:54516) 2024-06-03T02:30:34.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.572+0000 7efd12a00700 1 -- 172.21.15.79:0/380200997 learned_addr learned my addr 172.21.15.79:0/380200997 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:34.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.571+0000 7efd13400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80aaa1e40 0x55f80ab25070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.573+0000 7efd13400700 1 -- 172.21.15.79:0/380200997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab255b0 msgr2=0x55f80ab25a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:34.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.573+0000 7efd13400700 1 --2- 172.21.15.79:0/380200997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab255b0 0x55f80ab25a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.573+0000 7efd13400700 1 -- 172.21.15.79:0/380200997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 msgr2=0x55f80ab326f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:34.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.573+0000 7efd13400700 1 --2- 172.21.15.79:0/380200997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.573+0000 7efd13400700 1 -- 172.21.15.79:0/380200997 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f80aa1ff90 con 0x55f80aaa1e40 2024-06-03T02:30:34.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.576+0000 7efd12a00700 1 --2- 172.21.15.79:0/380200997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab255b0 0x55f80ab25a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:30:34.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.576+0000 7efd12a00700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efcf80a2dd0 0x7efcf80a5290 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efd000135f0 tx=0x7efd000139b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:34.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.576+0000 7efd13400700 1 --2- 172.21.15.79:0/380200997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80aaa1e40 0x55f80ab25070 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efd04006010 tx=0x7efd04005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3282923f6e8aad server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:34.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.576+0000 7efd08c00700 1 -- 172.21.15.79:0/380200997 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0409c0b0 con 0x55f80aaa1e40 2024-06-03T02:30:34.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.577+0000 7efd08c00700 1 -- 172.21.15.79:0/380200997 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0409ce50 con 0x55f80aaa1e40 2024-06-03T02:30:34.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.577+0000 7efd08c00700 1 -- 172.21.15.79:0/380200997 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd040a5100 con 0x55f80aaa1e40 2024-06-03T02:30:34.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.580+0000 7efd24b4fb80 1 -- 172.21.15.79:0/380200997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80aaa1e40 msgr2=0x55f80ab25070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:34.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.580+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/380200997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80aaa1e40 0x55f80ab25070 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efd04006010 tx=0x7efd04005ef0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.581+0000 7efd24b4fb80 1 -- 172.21.15.79:0/380200997 shutdown_connections 2024-06-03T02:30:34.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.581+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/380200997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.582+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/380200997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80aaa1e40 0x55f80ab25070 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.582+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/380200997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab255b0 0x55f80ab25a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.582+0000 7efd24b4fb80 1 -- 172.21.15.79:0/380200997 >> 172.21.15.79:0/380200997 conn(0x55f80aa52170 msgr2=0x55f80ab214b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:34.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.583+0000 7efd24b4fb80 1 -- 172.21.15.79:0/380200997 shutdown_connections 2024-06-03T02:30:34.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.583+0000 7efd24b4fb80 1 -- 172.21.15.79:0/380200997 wait complete. 2024-06-03T02:30:34.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.584+0000 7efd24b4fb80 1 -- start start 2024-06-03T02:30:34.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.585+0000 7efd24b4fb80 1 Processor -- start 2024-06-03T02:30:34.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.588+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 0x55f80ab2fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.588+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab2ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.588+0000 7efd13400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 0x55f80ab2fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.588+0000 7efd13400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 0x55f80ab2fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54520/0 (socket says 172.21.15.79:54520) 2024-06-03T02:30:34.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.588+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab2d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.589+0000 7efd13400700 1 -- 172.21.15.79:0/3795694450 learned_addr learned my addr 172.21.15.79:0/3795694450 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:34.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.589+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f80aa13f60 con 0x55f80ab30300 2024-06-03T02:30:34.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.589+0000 7efd12a00700 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab2ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.589+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f80aa73ac0 con 0x55f80ab255b0 2024-06-03T02:30:34.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.589+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f80ab32d70 con 0x55f80aaa1e40 2024-06-03T02:30:34.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.589+0000 7efd24b4fb80 10 client.0 _ll_register_callbacks cb 0x55f80aa11720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:34.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.590+0000 7efd12a00700 1 -- 172.21.15.79:0/3795694450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 msgr2=0x55f80ab2fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:34.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.590+0000 7efd12a00700 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 0x55f80ab2fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.590+0000 7efd12a00700 1 -- 172.21.15.79:0/3795694450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 msgr2=0x55f80ab2d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:34.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.590+0000 7efd12a00700 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab2d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.590+0000 7efd12a00700 1 -- 172.21.15.79:0/3795694450 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f80a9c68d0 con 0x55f80ab255b0 2024-06-03T02:30:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.591+0000 7efd12a00700 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab2ce50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efd0001b000 tx=0x7efd000047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8854501f8d619efd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.591+0000 7efcfd600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:30:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.591+0000 7efcfd600700 1 -- 172.21.15.79:0/3795694450 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00020450 con 0x55f80ab255b0 2024-06-03T02:30:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.591+0000 7efcfd600700 1 -- 172.21.15.79:0/3795694450 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7efd000205f0 con 0x55f80ab255b0 2024-06-03T02:30:34.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.592+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd0409ce50 con 0x55f80ab255b0 2024-06-03T02:30:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.592+0000 7efcfd600700 1 -- 172.21.15.79:0/3795694450 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd0002f050 con 0x55f80ab255b0 2024-06-03T02:30:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.592+0000 7efd24b4fb80 10 client.26516 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.592+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f80ab3d570 con 0x55f80ab255b0 2024-06-03T02:30:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.593+0000 7efcfd600700 1 -- 172.21.15.79:0/3795694450 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0409ce50 con 0x55f80ab255b0 2024-06-03T02:30:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.593+0000 7efcfd600700 1 -- 172.21.15.79:0/3795694450 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efd000117c0 con 0x55f80ab255b0 2024-06-03T02:30:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.593+0000 7efd24b4fb80 10 client.26516 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:34.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.593+0000 7efd24b4fb80 10 client.26516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:30:34.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.593+0000 7efd24b4fb80 20 client.26516 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:34.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.593+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f80ab3fc00 0x55f80ab42040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.593+0000 7efd24b4fb80 4 client.26516 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:30:34.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.593+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f80aa85000 con 0x55f80ab3fc00 2024-06-03T02:30:34.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.595+0000 7efd13400700 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f80ab3fc00 0x55f80ab42040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.597+0000 7efcfd600700 10 client.26516 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:34.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.600+0000 7efd13400700 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f80ab3fc00 0x55f80ab42040 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717404bca9f7909b server_cookie=26f07b7850b65b1 in_seq=0 out_seq=0 2024-06-03T02:30:34.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.601+0000 7efcfd600700 1 -- 172.21.15.79:0/3795694450 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f80aa85000 con 0x55f80ab3fc00 2024-06-03T02:30:34.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.601+0000 7efcfd600700 10 client.26516 handle_command_reply: tid=0 2024-06-03T02:30:34.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.602+0000 7efd24b4fb80 1 client.26516 shutdown 2024-06-03T02:30:34.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.602+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 msgr2=0x55f80ab2ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:34.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.602+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab2ce50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efd0001b000 tx=0x7efd000047c0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.603+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 shutdown_connections 2024-06-03T02:30:34.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.603+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f80ab3fc00 0x55f80ab42040 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.603+0000 7efcfd600700 0 client.26516 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:34.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.603+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab2d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.603+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 0x55f80ab2fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.603+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3795694450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab2ce50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.603+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 >> 172.21.15.79:0/3795694450 conn(0x55f80aa52170 msgr2=0x55f80ab20740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:34.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.604+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 shutdown_connections 2024-06-03T02:30:34.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.604+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3795694450 wait complete. 2024-06-03T02:30:34.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.604+0000 7efd24b4fb80 20 client.26516 trim_cache size 0 max 16384 2024-06-03T02:30:34.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.653+0000 7efd24b4fb80 1 Processor -- start 2024-06-03T02:30:34.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.653+0000 7efd24b4fb80 1 -- start start 2024-06-03T02:30:34.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.654+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 0x55f80ab2f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.654+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.654+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab31c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.654+0000 7efd24b4fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efd141bd050 con 0x55f80ab30300 2024-06-03T02:30:34.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.654+0000 7efd24b4fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd141beb90 con 0x55f80ab255b0 2024-06-03T02:30:34.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.654+0000 7efd24b4fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd141bd720 con 0x55f80aaa1e40 2024-06-03T02:30:34.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.654+0000 7efd12a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.654+0000 7efd12a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab316f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:35738/0 (socket says 172.21.15.79:35738) 2024-06-03T02:30:34.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.654+0000 7efd12a00700 1 -- 172.21.15.79:0/3916426392 learned_addr learned my addr 172.21.15.79:0/3916426392 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:34.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.656+0000 7efd12a00700 1 -- 172.21.15.79:0/3916426392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 msgr2=0x55f80ab2f690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:30:34.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.656+0000 7efd12a00700 1 --2- 172.21.15.79:0/3916426392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 0x55f80ab2f690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.656+0000 7efd12a00700 1 -- 172.21.15.79:0/3916426392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 msgr2=0x55f80ab31c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:30:34.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.658+0000 7efd13e00700 1 --2- 172.21.15.79:0/3916426392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab31c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.659+0000 7efd12a00700 1 --2- 172.21.15.79:0/3916426392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab31c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.659+0000 7efd12a00700 1 -- 172.21.15.79:0/3916426392 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f80ab28280 con 0x55f80ab255b0 2024-06-03T02:30:34.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.659+0000 7efd13e00700 1 --2- 172.21.15.79:0/3916426392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab31c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:30:34.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.660+0000 7efd12a00700 1 --2- 172.21.15.79:0/3916426392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab316f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efd00013db0 tx=0x7efd0000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c827d1663f2e1e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:34.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.663+0000 7efcffe00700 1 -- 172.21.15.79:0/3916426392 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00013270 con 0x55f80ab255b0 2024-06-03T02:30:34.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.664+0000 7efcffe00700 1 -- 172.21.15.79:0/3916426392 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd00024030 con 0x55f80ab255b0 2024-06-03T02:30:34.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.664+0000 7efcffe00700 1 -- 172.21.15.79:0/3916426392 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00020dc0 con 0x55f80ab255b0 2024-06-03T02:30:34.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.666+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3916426392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 msgr2=0x55f80ab316f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:34.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.666+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3916426392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab316f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efd00013db0 tx=0x7efd0000b670 comp rx=0 tx=0).stop 2024-06-03T02:30:34.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.666+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3916426392 shutdown_connections 2024-06-03T02:30:34.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.666+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3916426392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80ab30300 0x55f80ab31c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.666+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3916426392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80aaa1e40 0x55f80ab2f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.666+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3916426392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab255b0 0x55f80ab316f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.666+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3916426392 >> 172.21.15.79:0/3916426392 conn(0x55f80aa52170 msgr2=0x55f80ab2b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:34.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.666+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3916426392 shutdown_connections 2024-06-03T02:30:34.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.667+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3916426392 wait complete. 2024-06-03T02:30:34.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.668+0000 7efd24b4fb80 1 -- start start 2024-06-03T02:30:34.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.668+0000 7efd24b4fb80 1 Processor -- start 2024-06-03T02:30:34.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.669+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80aaa1e40 0x55f80ab37fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.669+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab30300 0x55f80ab384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.669+0000 7efd24b4fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab3fc00 0x55f80ab32f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.669+0000 7efd24b4fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efd141be4c0 con 0x55f80aaa1e40 2024-06-03T02:30:34.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.669+0000 7efd24b4fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd141bddf0 con 0x55f80ab30300 2024-06-03T02:30:34.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.669+0000 7efd24b4fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd141baff0 con 0x55f80ab3fc00 2024-06-03T02:30:34.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.669+0000 7efd24b4fb80 10 client.0 _ll_register_callbacks cb 0x55f80aa11720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:34.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.670+0000 7efd13e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab3fc00 0x55f80ab32f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.670+0000 7efd13e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab3fc00 0x55f80ab32f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:54524/0 (socket says 172.21.15.79:54524) 2024-06-03T02:30:34.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.670+0000 7efd13e00700 1 -- 172.21.15.79:0/3011718601 learned_addr learned my addr 172.21.15.79:0/3011718601 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:34.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.670+0000 7efd12a00700 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab30300 0x55f80ab384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.670+0000 7efd13400700 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80aaa1e40 0x55f80ab37fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.671+0000 7efd13e00700 1 -- 172.21.15.79:0/3011718601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab30300 msgr2=0x55f80ab384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:34.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.671+0000 7efd13e00700 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab30300 0x55f80ab384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.671+0000 7efd13e00700 1 -- 172.21.15.79:0/3011718601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80aaa1e40 msgr2=0x55f80ab37fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:34.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.671+0000 7efd13e00700 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80aaa1e40 0x55f80ab37fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.671+0000 7efd13e00700 1 -- 172.21.15.79:0/3011718601 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f80ab33490 con 0x55f80ab3fc00 2024-06-03T02:30:34.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.671+0000 7efd13e00700 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab3fc00 0x55f80ab32f50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00e820 tx=0x7efd0c00ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b960b04e102dabe3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:34.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.671+0000 7efd12a00700 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab30300 0x55f80ab384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:30:34.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.671+0000 7efcfd600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:30:34.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.672+0000 7efd13400700 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80aaa1e40 0x55f80ab37fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:30:34.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.673+0000 7efcfd600700 1 -- 172.21.15.79:0/3011718601 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c017070 con 0x55f80ab3fc00 2024-06-03T02:30:34.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.673+0000 7efcfd600700 1 -- 172.21.15.79:0/3011718601 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7efd0c013870 con 0x55f80ab3fc00 2024-06-03T02:30:34.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.673+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3011718601 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd00024030 con 0x55f80ab3fc00 2024-06-03T02:30:34.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.673+0000 7efcfd600700 1 -- 172.21.15.79:0/3011718601 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd0c021050 con 0x55f80ab3fc00 2024-06-03T02:30:34.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.674+0000 7efd24b4fb80 10 client.26515 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:34.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.674+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3011718601 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f80abaaf10 con 0x55f80ab3fc00 2024-06-03T02:30:34.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.675+0000 7efcfd600700 1 -- 172.21.15.79:0/3011718601 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd00024030 con 0x55f80ab3fc00 2024-06-03T02:30:34.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.675+0000 7efcfd600700 1 -- 172.21.15.79:0/3011718601 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efd0c00c3c0 con 0x55f80ab3fc00 2024-06-03T02:30:34.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.676+0000 7efd24b4fb80 10 client.26515 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:34.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.676+0000 7efd24b4fb80 10 client.26515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:30:34.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.676+0000 7efd24b4fb80 20 client.26515 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:34.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.676+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f80abab5a0 0x55f80abad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:34.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.677+0000 7efd24b4fb80 4 client.26515 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:30:34.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.677+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3011718601 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f80aa7f580 con 0x55f80abab5a0 2024-06-03T02:30:34.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.678+0000 7efd13400700 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f80abab5a0 0x55f80abad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:34.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd13400700 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f80abab5a0 0x55f80abad9c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78cb2a86617ee287 server_cookie=efabeec0c2e38ec5 in_seq=0 out_seq=0 2024-06-03T02:30:34.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efcfd600700 10 client.26515 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:34.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efcfd600700 1 -- 172.21.15.79:0/3011718601 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f80aa7f580 con 0x55f80abab5a0 2024-06-03T02:30:34.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efcfd600700 10 client.26515 handle_command_reply: tid=0 2024-06-03T02:30:34.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd24b4fb80 1 client.26515 shutdown 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3011718601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab3fc00 msgr2=0x55f80ab32f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab3fc00 0x55f80ab32f50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00e820 tx=0x7efd0c00ebe0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3011718601 shutdown_connections 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f80ab3fc00 0x55f80ab32f50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55f80abab5a0 0x55f80abad9c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f80aaa1e40 0x55f80ab37fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd24b4fb80 1 --2- 172.21.15.79:0/3011718601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f80ab30300 0x55f80ab384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.680+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3011718601 >> 172.21.15.79:0/3011718601 conn(0x55f80aa52170 msgr2=0x55f80ab1feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.681+0000 7efcfd600700 0 client.26515 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.681+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3011718601 shutdown_connections 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.681+0000 7efd24b4fb80 1 -- 172.21.15.79:0/3011718601 wait complete. 2024-06-03T02:30:34.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.681+0000 7efd24b4fb80 20 client.26515 trim_cache size 0 max 16384 2024-06-03T02:30:34.683 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:30:34.683 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:30:34.683 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:30:34.683 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:30:34.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.683+0000 7efd1a200700 1 -- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efcf80a2dd0 msgr2=0x7efcf80a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:34.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.683+0000 7efd1a200700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efcf80a2dd0 0x7efcf80a5290 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efd000135f0 tx=0x7efd000139b0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.683+0000 7efd1a200700 1 -- 172.21.15.79:0/3172779424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1406a170 msgr2=0x7efd14111f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:34.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.683+0000 7efd1a200700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1406a170 0x7efd14111f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd040059d0 tx=0x7efd04009f90 comp rx=0 tx=0).stop 2024-06-03T02:30:34.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.687+0000 7efd1a200700 1 -- 172.21.15.79:0/3172779424 shutdown_connections 2024-06-03T02:30:34.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.687+0000 7efd1a200700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7efcf80a2dd0 0x7efcf80a5290 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.687+0000 7efd1a200700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd141084e0 0x7efd141176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.687+0000 7efd1a200700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1406a170 0x7efd14111f40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.687+0000 7efd1a200700 1 --2- 172.21.15.79:0/3172779424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1406ab10 0x7efd14112480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:34.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.687+0000 7efd1a200700 1 -- 172.21.15.79:0/3172779424 >> 172.21.15.79:0/3172779424 conn(0x7efd140782c0 msgr2=0x7efd140ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:34.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.687+0000 7efd1a200700 1 -- 172.21.15.79:0/3172779424 shutdown_connections 2024-06-03T02:30:34.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:34.688+0000 7efd1a200700 1 -- 172.21.15.79:0/3172779424 wait complete. 2024-06-03T02:30:36.158 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:30:36.158 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:30:36.341 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:36 smithi079 ceph-mon[31819]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 546 op/s 2024-06-03T02:30:36.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:36 smithi135 ceph-mon[36653]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 546 op/s 2024-06-03T02:30:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:36 smithi123 ceph-mon[36640]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 546 op/s 2024-06-03T02:30:37.113 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:30:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 630.0 s, 478.8 tps, lat 201.486 ms stddev 263.459 2024-06-03T02:30:38.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:38 smithi079 ceph-mon[31819]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 406 op/s 2024-06-03T02:30:38.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:38 smithi135 ceph-mon[36653]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 406 op/s 2024-06-03T02:30:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:38 smithi123 ceph-mon[36640]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 406 op/s 2024-06-03T02:30:40.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:40 smithi079 ceph-mon[31819]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.5 MiB/s wr, 531 op/s 2024-06-03T02:30:40.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:40 smithi135 ceph-mon[36653]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.5 MiB/s wr, 531 op/s 2024-06-03T02:30:40.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:40 smithi123 ceph-mon[36640]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.5 MiB/s wr, 531 op/s 2024-06-03T02:30:42.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:42 smithi079 ceph-mon[31819]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.4 MiB/s wr, 449 op/s 2024-06-03T02:30:42.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:42 smithi135 ceph-mon[36653]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.4 MiB/s wr, 449 op/s 2024-06-03T02:30:42.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:42 smithi123 ceph-mon[36640]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.4 MiB/s wr, 449 op/s 2024-06-03T02:30:42.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.368+0000 7f73f2200700 1 -- 172.21.15.79:0/1467911947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73ec100d70 msgr2=0x7f73ec1011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.368+0000 7f73f2200700 1 --2- 172.21.15.79:0/1467911947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73ec100d70 0x7f73ec1011d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f73d40098f0 tx=0x7f73d4009c00 comp rx=0 tx=0).stop 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.370+0000 7f73f2200700 1 -- 172.21.15.79:0/1467911947 shutdown_connections 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.370+0000 7f73f2200700 1 --2- 172.21.15.79:0/1467911947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec1130a0 0x7f73ec115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.370+0000 7f73f2200700 1 --2- 172.21.15.79:0/1467911947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73ec108360 0x7f73ec100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.370+0000 7f73f2200700 1 --2- 172.21.15.79:0/1467911947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73ec100d70 0x7f73ec1011d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.370+0000 7f73f2200700 1 -- 172.21.15.79:0/1467911947 >> 172.21.15.79:0/1467911947 conn(0x7f73ec076130 msgr2=0x7f73ec076540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.371+0000 7f73f2200700 1 -- 172.21.15.79:0/1467911947 shutdown_connections 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.371+0000 7f73f2200700 1 -- 172.21.15.79:0/1467911947 wait complete. 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.373+0000 7f73f2200700 1 Processor -- start 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.373+0000 7f73f2200700 1 -- start start 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.374+0000 7f73f2200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec100d70 0x7f73ec19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:42.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.374+0000 7f73f2200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73ec108360 0x7f73ec19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:42.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.374+0000 7f73f2200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73ec1130a0 0x7f73ec1a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:42.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.374+0000 7f73f2200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec118d90 con 0x7f73ec1130a0 2024-06-03T02:30:42.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.374+0000 7f73f2200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec118c10 con 0x7f73ec100d70 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.374+0000 7f73f2200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec118f10 con 0x7f73ec108360 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.375+0000 7f73f1800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73ec1130a0 0x7f73ec1a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.375+0000 7f73f1800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73ec1130a0 0x7f73ec1a3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37544/0 (socket says 172.21.15.79:37544) 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.375+0000 7f73f1800700 1 -- 172.21.15.79:0/2481767169 learned_addr learned my addr 172.21.15.79:0/2481767169 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.376+0000 7f73f1800700 1 -- 172.21.15.79:0/2481767169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73ec108360 msgr2=0x7f73ec19f7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.376+0000 7f73f1800700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73ec108360 0x7f73ec19f7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.376+0000 7f73f1800700 1 -- 172.21.15.79:0/2481767169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec100d70 msgr2=0x7f73ec19f260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.376+0000 7f73f1800700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec100d70 0x7f73ec19f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.376+0000 7f73f1800700 1 -- 172.21.15.79:0/2481767169 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec1a4270 con 0x7f73ec1130a0 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.377+0000 7f73f1800700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73ec1130a0 0x7f73ec1a3b30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f73e400ef90 tx=0x7f73e400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.377+0000 7f73ea000700 1 -- 172.21.15.79:0/2481767169 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e400cd10 con 0x7f73ec1130a0 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.377+0000 7f73ea000700 1 -- 172.21.15.79:0/2481767169 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e4004bb0 con 0x7f73ec1130a0 2024-06-03T02:30:42.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.377+0000 7f73ea000700 1 -- 172.21.15.79:0/2481767169 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4004280 con 0x7f73ec1130a0 2024-06-03T02:30:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.379+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec1a4500 con 0x7f73ec1130a0 2024-06-03T02:30:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.379+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a4900 con 0x7f73ec1130a0 2024-06-03T02:30:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.385+0000 7f73ea000700 1 -- 172.21.15.79:0/2481767169 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f73e4012050 con 0x7f73ec1130a0 2024-06-03T02:30:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.388+0000 7f73ea000700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.389+0000 7f73ea000700 1 -- 172.21.15.79:0/2481767169 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f73e40ce920 con 0x7f73ec1130a0 2024-06-03T02:30:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.391+0000 7f73f0e00700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.396+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec069180 con 0x7f73ec1130a0 2024-06-03T02:30:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.396+0000 7f73f0e00700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009730 tx=0x7f73dc006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.409+0000 7f73ea000700 1 -- 172.21.15.79:0/2481767169 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73e404e850 con 0x7f73ec1130a0 2024-06-03T02:30:42.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.879+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73ec1abf00 con 0x7f73ec1130a0 2024-06-03T02:30:42.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.881+0000 7f73ea000700 1 -- 172.21.15.79:0/2481767169 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f73e4010040 con 0x7f73ec1130a0 2024-06-03T02:30:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:30:42.882 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:30:42.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73cc0a2ca0 msgr2=0x7f73cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:42.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009730 tx=0x7f73dc006cb0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73ec1130a0 msgr2=0x7f73ec1a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:42.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73ec1130a0 0x7f73ec1a3b30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f73e400ef90 tx=0x7f73e400c450 comp rx=0 tx=0).stop 2024-06-03T02:30:42.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 shutdown_connections 2024-06-03T02:30:42.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73ec1130a0 0x7f73ec1a3b30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec100d70 0x7f73ec19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 --2- 172.21.15.79:0/2481767169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73ec108360 0x7f73ec19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:42.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.884+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 >> 172.21.15.79:0/2481767169 conn(0x7f73ec076130 msgr2=0x7f73ec0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:42.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.885+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 shutdown_connections 2024-06-03T02:30:42.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:42.885+0000 7f73f2200700 1 -- 172.21.15.79:0/2481767169 wait complete. 2024-06-03T02:30:42.887 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:30:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 635.0 s, 466.2 tps, lat 218.725 ms stddev 294.803 2024-06-03T02:30:43.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:43 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2481767169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:30:43.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:43 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2481767169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:30:43.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:43 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2481767169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:30:44.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:44 smithi079 ceph-mon[31819]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 451 op/s 2024-06-03T02:30:44.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:44 smithi135 ceph-mon[36653]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 451 op/s 2024-06-03T02:30:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:44 smithi123 ceph-mon[36640]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 451 op/s 2024-06-03T02:30:45.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:30:45.808 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:30:46.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:46 smithi079 ceph-mon[31819]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 510 op/s 2024-06-03T02:30:46.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:46 smithi135 ceph-mon[36653]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 510 op/s 2024-06-03T02:30:46.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:46 smithi123 ceph-mon[36640]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 510 op/s 2024-06-03T02:30:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 640.0 s, 397.0 tps, lat 232.136 ms stddev 303.042 2024-06-03T02:30:48.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:48 smithi079 ceph-mon[31819]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.3 MiB/s wr, 384 op/s 2024-06-03T02:30:48.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:48 smithi135 ceph-mon[36653]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.3 MiB/s wr, 384 op/s 2024-06-03T02:30:48.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:48 smithi123 ceph-mon[36640]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.3 MiB/s wr, 384 op/s 2024-06-03T02:30:50.174 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:50 smithi079 ceph-mon[31819]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 497 op/s 2024-06-03T02:30:50.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:50 smithi135 ceph-mon[36653]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 497 op/s 2024-06-03T02:30:50.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:50 smithi123 ceph-mon[36640]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 497 op/s 2024-06-03T02:30:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.480+0000 7f5a18200700 1 -- 172.21.15.79:0/2894659049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a1006a6e0 msgr2=0x7f5a1006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.480+0000 7f5a18200700 1 --2- 172.21.15.79:0/2894659049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a1006a6e0 0x7f5a1006ab40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0098f0 tx=0x7f59fc009c00 comp rx=0 tx=0).stop 2024-06-03T02:30:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.482+0000 7f5a18200700 1 -- 172.21.15.79:0/2894659049 shutdown_connections 2024-06-03T02:30:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.483+0000 7f5a18200700 1 --2- 172.21.15.79:0/2894659049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a10104ed0 0x7f5a10105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.483+0000 7f5a18200700 1 --2- 172.21.15.79:0/2894659049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a101073c0 0x7f5a1006a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.483+0000 7f5a18200700 1 --2- 172.21.15.79:0/2894659049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a1006a6e0 0x7f5a1006ab40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.483+0000 7f5a18200700 1 -- 172.21.15.79:0/2894659049 >> 172.21.15.79:0/2894659049 conn(0x7f5a10078340 msgr2=0x7f5a10078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:50.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.484+0000 7f5a18200700 1 -- 172.21.15.79:0/2894659049 shutdown_connections 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.484+0000 7f5a18200700 1 -- 172.21.15.79:0/2894659049 wait complete. 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.486+0000 7f5a18200700 1 Processor -- start 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.488+0000 7f5a18200700 1 -- start start 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.489+0000 7f5a18200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a1006a6e0 0x7f5a1019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.489+0000 7f5a18200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a10104ed0 0x7f5a1019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.489+0000 7f5a18200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a101073c0 0x7f5a101a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.489+0000 7f5a15800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a1006a6e0 0x7f5a1019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.489+0000 7f5a15800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a1006a6e0 0x7f5a1019f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33470/0 (socket says 172.21.15.79:33470) 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.489+0000 7f5a14e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a10104ed0 0x7f5a1019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a15800700 1 -- 172.21.15.79:0/2550276996 learned_addr learned my addr 172.21.15.79:0/2550276996 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10118db0 con 0x7f5a10104ed0 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10118c30 con 0x7f5a1006a6e0 2024-06-03T02:30:50.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10118f30 con 0x7f5a101073c0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a16200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a101073c0 0x7f5a101a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a16200700 1 -- 172.21.15.79:0/2550276996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a1006a6e0 msgr2=0x7f5a1019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a16200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a1006a6e0 0x7f5a1019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a16200700 1 -- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a10104ed0 msgr2=0x7f5a1019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a16200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a10104ed0 0x7f5a1019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.490+0000 7f5a16200700 1 -- 172.21.15.79:0/2550276996 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a101a4290 con 0x7f5a101073c0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.491+0000 7f5a14e00700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a10104ed0 0x7f5a1019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.491+0000 7f5a16200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a101073c0 0x7f5a101a3b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e940 tx=0x7f5a0400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.491+0000 7f5a0e000700 1 -- 172.21.15.79:0/2550276996 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04004220 con 0x7f5a101073c0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.492+0000 7f5a0e000700 1 -- 172.21.15.79:0/2550276996 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a04004d10 con 0x7f5a101073c0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.492+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a101a4580 con 0x7f5a101073c0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.492+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a101abe00 con 0x7f5a101073c0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.492+0000 7f5a0e000700 1 -- 172.21.15.79:0/2550276996 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04004220 con 0x7f5a101073c0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.499+0000 7f5a0e000700 1 -- 172.21.15.79:0/2550276996 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5a0400cc80 con 0x7f5a101073c0 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.501+0000 7f5a0e000700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f59f80a2f20 0x7f59f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.502+0000 7f5a0e000700 1 -- 172.21.15.79:0/2550276996 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5a040cec80 con 0x7f5a101073c0 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.502+0000 7f5a15800700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f59f80a2f20 0x7f59f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.504+0000 7f5a22b1ab80 1 Processor -- start 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.504+0000 7f5a15800700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f59f80a2f20 0x7f59f80a53e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5a080094f0 tx=0x7f5a08009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.504+0000 7f5a22b1ab80 1 -- start start 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.504+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d75b0e40 0x5590d7634070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.505+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d76345b0 0x5590d7634a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.505+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d76416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.505+0000 7f5a22b1ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5590d7543a10 con 0x5590d75b0e40 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.505+0000 7f5a22b1ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5590d7586b00 con 0x5590d76345b0 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.505+0000 7f5a22b1ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5590d75418f0 con 0x5590d763f300 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.505+0000 7f5a16200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d76416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.505+0000 7f5a16200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d76416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56328/0 (socket says 172.21.15.79:56328) 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.505+0000 7f5a16200700 1 -- 172.21.15.79:0/811705502 learned_addr learned my addr 172.21.15.79:0/811705502 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.507+0000 7f5a14e00700 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d76345b0 0x5590d7634a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.507+0000 7f5a15800700 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d75b0e40 0x5590d7634070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.508+0000 7f5a16200700 1 -- 172.21.15.79:0/811705502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d76345b0 msgr2=0x5590d7634a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.508+0000 7f5a16200700 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d76345b0 0x5590d7634a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.508+0000 7f5a16200700 1 -- 172.21.15.79:0/811705502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d75b0e40 msgr2=0x5590d7634070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.508+0000 7f5a16200700 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d75b0e40 0x5590d7634070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.508+0000 7f5a16200700 1 -- 172.21.15.79:0/811705502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d752ef90 con 0x5590d763f300 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.508+0000 7f5a14e00700 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d76345b0 0x5590d7634a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.509+0000 7f5a16200700 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d76416f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a040a3810 tx=0x7f5a0401e380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de1b25c79831b821 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.509+0000 7f5a02a00700 1 -- 172.21.15.79:0/811705502 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0401c040 con 0x5590d763f300 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.509+0000 7f5a02a00700 1 -- 172.21.15.79:0/811705502 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0400c760 con 0x5590d763f300 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.509+0000 7f5a02a00700 1 -- 172.21.15.79:0/811705502 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a040043c0 con 0x5590d763f300 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.510+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/811705502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 msgr2=0x5590d76416f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.510+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d76416f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a040a3810 tx=0x7f5a0401e380 comp rx=0 tx=0).stop 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.511+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/811705502 shutdown_connections 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.511+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d76416f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.511+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d75b0e40 0x5590d7634070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.511+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/811705502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d76345b0 0x5590d7634a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.511+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/811705502 >> 172.21.15.79:0/811705502 conn(0x5590d7561170 msgr2=0x5590d76304b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.511+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/811705502 shutdown_connections 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.512+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/811705502 wait complete. 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.513+0000 7f5a22b1ab80 1 -- start start 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.514+0000 7f5a22b1ab80 1 Processor -- start 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.515+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d763e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.516+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d76345b0 0x5590d763bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.516+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d763f300 0x5590d763c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.516+0000 7f5a15800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d763e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.517+0000 7f5a15800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d763e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33486/0 (socket says 172.21.15.79:33486) 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.517+0000 7f5a15800700 1 -- 172.21.15.79:0/3362408114 learned_addr learned my addr 172.21.15.79:0/3362408114 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.517+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5590d7522f60 con 0x5590d763f300 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.517+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5590d7582ac0 con 0x5590d75b0e40 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.517+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5590d7641fc0 con 0x5590d76345b0 2024-06-03T02:30:50.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.517+0000 7f5a14e00700 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d76345b0 0x5590d763bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.517+0000 7f5a22b1ab80 10 client.0 _ll_register_callbacks cb 0x5590d7520720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.517+0000 7f5a16200700 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d763f300 0x5590d763c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.518+0000 7f5a16200700 1 -- 172.21.15.79:0/3362408114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d76345b0 msgr2=0x5590d763bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.518+0000 7f5a16200700 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d76345b0 0x5590d763bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.518+0000 7f5a16200700 1 -- 172.21.15.79:0/3362408114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 msgr2=0x5590d763e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.518+0000 7f5a16200700 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d763e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.518+0000 7f5a16200700 1 -- 172.21.15.79:0/3362408114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d74d58d0 con 0x5590d763f300 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.519+0000 7f5a15800700 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d763e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.519+0000 7f5a16200700 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d763f300 0x5590d763c1d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a0409b730 tx=0x7f5a0409b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d5ef510b4f4e0a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.521+0000 7f59f7400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.522+0000 7f59f7400700 1 -- 172.21.15.79:0/3362408114 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0401c070 con 0x5590d763f300 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.522+0000 7f59f7400700 1 -- 172.21.15.79:0/3362408114 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5a0409f120 con 0x5590d763f300 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.522+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0400c760 con 0x5590d763f300 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.523+0000 7f59f7400700 1 -- 172.21.15.79:0/3362408114 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a040cc3f0 con 0x5590d763f300 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.523+0000 7f5a22b1ab80 10 client.16047 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.523+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5590d764c630 con 0x5590d763f300 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.524+0000 7f59f7400700 1 -- 172.21.15.79:0/3362408114 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0400c760 con 0x5590d763f300 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.524+0000 7f59f7400700 1 -- 172.21.15.79:0/3362408114 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5a040840a0 con 0x5590d763f300 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.526+0000 7f5a22b1ab80 10 client.16047 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.526+0000 7f5a22b1ab80 10 client.16047 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.526+0000 7f5a22b1ab80 20 client.16047 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.526+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5590d764eb00 0x5590d7650ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.526+0000 7f5a22b1ab80 4 client.16047 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.526+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590d7594000 con 0x5590d764eb00 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.526+0000 7f5a15800700 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5590d764eb00 0x5590d7650ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.527+0000 7f59f7400700 10 client.16047 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.527+0000 7f5a15800700 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5590d764eb00 0x5590d7650ef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=80796e71c6a050a2 server_cookie=c525b7a7c9e527a1 in_seq=0 out_seq=0 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.528+0000 7f59f7400700 1 -- 172.21.15.79:0/3362408114 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5590d7594000 con 0x5590d764eb00 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.528+0000 7f59f7400700 10 client.16047 handle_command_reply: tid=0 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.528+0000 7f5a22b1ab80 1 client.16047 shutdown 2024-06-03T02:30:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.528+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d763f300 msgr2=0x5590d763c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.528+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d763f300 0x5590d763c1d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a0409b730 tx=0x7f5a0409b0d0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.529+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 shutdown_connections 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.529+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5590d764eb00 0x5590d7650ef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.529+0000 7f59f7400700 0 client.16047 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.529+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d763f300 0x5590d763c1d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.529+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d763e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.529+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/3362408114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d76345b0 0x5590d763bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.529+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 >> 172.21.15.79:0/3362408114 conn(0x5590d7561170 msgr2=0x5590d762f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.530+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 shutdown_connections 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.530+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/3362408114 wait complete. 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.531+0000 7f5a22b1ab80 20 client.16047 trim_cache size 0 max 16384 2024-06-03T02:30:50.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.562+0000 7f5a22b1ab80 1 Processor -- start 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.562+0000 7f5a22b1ab80 1 -- start start 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.562+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d7634ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.562+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 0x5590d762fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.563+0000 7f5a15800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d7634ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.563+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d7630400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.563+0000 7f5a15800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d7634ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33490/0 (socket says 172.21.15.79:33490) 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.563+0000 7f5a22b1ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f59f000eb90 con 0x5590d76345b0 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.563+0000 7f5a22b1ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59f00106d0 con 0x5590d75b0e40 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.563+0000 7f5a22b1ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59f000f260 con 0x5590d763f300 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.563+0000 7f5a15800700 1 -- 172.21.15.79:0/612723132 learned_addr learned my addr 172.21.15.79:0/612723132 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.563+0000 7f5a16200700 1 --2- 172.21.15.79:0/612723132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d7630400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.564+0000 7f5a15800700 1 -- 172.21.15.79:0/612723132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 msgr2=0x5590d7630400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.564+0000 7f5a15800700 1 --2- 172.21.15.79:0/612723132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d7630400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.564+0000 7f5a15800700 1 -- 172.21.15.79:0/612723132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 msgr2=0x5590d762fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.564+0000 7f5a15800700 1 --2- 172.21.15.79:0/612723132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 0x5590d762fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.564+0000 7f5a15800700 1 -- 172.21.15.79:0/612723132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d764b290 con 0x5590d75b0e40 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.565+0000 7f5a16200700 1 --2- 172.21.15.79:0/612723132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d7630400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.565+0000 7f5a15800700 1 --2- 172.21.15.79:0/612723132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d7634ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009d80 tx=0x7f5a0800b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd5eda44e7409e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.565+0000 7f5a02000700 1 -- 172.21.15.79:0/612723132 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08015ea0 con 0x5590d75b0e40 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.565+0000 7f5a02000700 1 -- 172.21.15.79:0/612723132 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a08007e50 con 0x5590d75b0e40 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.565+0000 7f5a02000700 1 -- 172.21.15.79:0/612723132 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080258c0 con 0x5590d75b0e40 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.566+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/612723132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 msgr2=0x5590d7634ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.566+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/612723132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d7634ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009d80 tx=0x7f5a0800b050 comp rx=0 tx=0).stop 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.566+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/612723132 shutdown_connections 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.566+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/612723132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d7630400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.566+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/612723132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d7634ad0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.566+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/612723132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 0x5590d762fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.566+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/612723132 >> 172.21.15.79:0/612723132 conn(0x5590d7561170 msgr2=0x5590d763a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.567+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/612723132 shutdown_connections 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.567+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/612723132 wait complete. 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.568+0000 7f5a22b1ab80 1 -- start start 2024-06-03T02:30:50.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.569+0000 7f5a22b1ab80 1 Processor -- start 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.569+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d76516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.569+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 0x5590d7651bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.569+0000 7f5a22b1ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d76b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.569+0000 7f5a22b1ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0010000 con 0x5590d76345b0 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.569+0000 7f5a22b1ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59f000f930 con 0x5590d75b0e40 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.569+0000 7f5a22b1ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59f000cb30 con 0x5590d763f300 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.569+0000 7f5a22b1ab80 10 client.0 _ll_register_callbacks cb 0x5590d7520720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a15800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d76516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a15800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d76516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33496/0 (socket says 172.21.15.79:33496) 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a15800700 1 -- 172.21.15.79:0/4279366620 learned_addr learned my addr 172.21.15.79:0/4279366620 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a14e00700 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 0x5590d7651bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a15800700 1 -- 172.21.15.79:0/4279366620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 msgr2=0x5590d76b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a15800700 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d76b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a15800700 1 -- 172.21.15.79:0/4279366620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 msgr2=0x5590d7651bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a15800700 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 0x5590d7651bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a15800700 1 -- 172.21.15.79:0/4279366620 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d76b1a60 con 0x5590d75b0e40 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a14e00700 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 0x5590d7651bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f5a15800700 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d76516b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5a08007bf0 tx=0x7f5a08007cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e7ea8e3e10a414f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f59f7400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.570+0000 7f59f7400700 1 -- 172.21.15.79:0/4279366620 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0802e460 con 0x5590d75b0e40 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.571+0000 7f59f7400700 1 -- 172.21.15.79:0/4279366620 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5a0802e600 con 0x5590d75b0e40 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.571+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/4279366620 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a08007e50 con 0x5590d75b0e40 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.571+0000 7f59f7400700 1 -- 172.21.15.79:0/4279366620 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a08041050 con 0x5590d75b0e40 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.571+0000 7f5a22b1ab80 10 client.26540 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.571+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/4279366620 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5590d76b8e00 con 0x5590d75b0e40 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.571+0000 7f59f7400700 1 -- 172.21.15.79:0/4279366620 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a08007e50 con 0x5590d75b0e40 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.572+0000 7f59f7400700 1 -- 172.21.15.79:0/4279366620 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5a08005040 con 0x5590d75b0e40 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.572+0000 7f5a22b1ab80 10 client.26540 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.572+0000 7f5a22b1ab80 10 client.26540 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:30:50.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.572+0000 7f5a22b1ab80 20 client.26540 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.572+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5590d764eb00 0x5590d76bb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.572+0000 7f5a22b1ab80 4 client.26540 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.572+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/4279366620 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590d758e580 con 0x5590d764eb00 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.572+0000 7f5a14e00700 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5590d764eb00 0x5590d76bb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.573+0000 7f5a14e00700 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5590d764eb00 0x5590d76bb520 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84038b4b79503948 server_cookie=45a7471e3bc036b7 in_seq=0 out_seq=0 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.573+0000 7f59f7400700 10 client.26540 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.573+0000 7f59f7400700 1 -- 172.21.15.79:0/4279366620 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590d758e580 con 0x5590d764eb00 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.573+0000 7f59f7400700 10 client.26540 handle_command_reply: tid=0 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.573+0000 7f5a22b1ab80 1 client.26540 shutdown 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.573+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/4279366620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 msgr2=0x5590d76516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.573+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d76516b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5a08007bf0 tx=0x7f5a08007cd0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/4279366620 shutdown_connections 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5590d764eb00 0x5590d76bb520 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590d763f300 0x5590d76b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590d75b0e40 0x5590d76516b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f5a22b1ab80 1 --2- 172.21.15.79:0/4279366620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5590d76345b0 0x5590d7651bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f59f7400700 0 client.26540 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:30:50.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/4279366620 >> 172.21.15.79:0/4279366620 conn(0x5590d7561170 msgr2=0x5590d763d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:50.756 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:30:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/4279366620 shutdown_connections 2024-06-03T02:30:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f5a22b1ab80 1 -- 172.21.15.79:0/4279366620 wait complete. 2024-06-03T02:30:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.574+0000 7f5a22b1ab80 20 client.26540 trim_cache size 0 max 16384 2024-06-03T02:30:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f59f80a2f20 msgr2=0x7f59f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f59f80a2f20 0x7f59f80a53e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5a080094f0 tx=0x7f5a08009040 comp rx=0 tx=0).stop 2024-06-03T02:30:50.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a101073c0 msgr2=0x7f5a101a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a101073c0 0x7f5a101a3b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e940 tx=0x7f5a0400ec50 comp rx=0 tx=0).stop 2024-06-03T02:30:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 shutdown_connections 2024-06-03T02:30:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a101073c0 0x7f5a101a3b50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a1006a6e0 0x7f5a1019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f59f80a2f20 0x7f59f80a53e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 --2- 172.21.15.79:0/2550276996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a10104ed0 0x7f5a1019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:50.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.575+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 >> 172.21.15.79:0/2550276996 conn(0x7f5a10078340 msgr2=0x7f5a10103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:50.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.576+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 shutdown_connections 2024-06-03T02:30:50.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:50.576+0000 7f5a18200700 1 -- 172.21.15.79:0/2550276996 wait complete. 2024-06-03T02:30:51.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:30:52.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:52 smithi135 ceph-mon[36653]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 434 op/s 2024-06-03T02:30:52.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:52 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:30:52.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:52 smithi123 ceph-mon[36640]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 434 op/s 2024-06-03T02:30:52.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:52 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:30:52.566 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:30:52.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:52 smithi079 ceph-mon[31819]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 434 op/s 2024-06-03T02:30:52.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:52 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:30:53.312 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 645.0 s, 409.9 tps, lat 255.283 ms stddev 403.911 2024-06-03T02:30:54.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:54 smithi079 ceph-mon[31819]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 MiB/s wr, 403 op/s 2024-06-03T02:30:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:54 smithi123 ceph-mon[36640]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 MiB/s wr, 403 op/s 2024-06-03T02:30:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:54 smithi135 ceph-mon[36653]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 MiB/s wr, 403 op/s 2024-06-03T02:30:56.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.450+0000 7f5181e00700 1 -- 172.21.15.79:0/1813731690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f517c0746c0 msgr2=0x7f517c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:56.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.450+0000 7f5181e00700 1 --2- 172.21.15.79:0/1813731690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f517c0746c0 0x7f517c074aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f516c009990 tx=0x7f516c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.451+0000 7f5181e00700 1 -- 172.21.15.79:0/1813731690 shutdown_connections 2024-06-03T02:30:56.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.451+0000 7f5181e00700 1 --2- 172.21.15.79:0/1813731690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f517c10ee90 0x7f517c111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.451+0000 7f5181e00700 1 --2- 172.21.15.79:0/1813731690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f517c0746c0 0x7f517c074aa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.451+0000 7f5181e00700 1 --2- 172.21.15.79:0/1813731690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f517c074fe0 0x7f517c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.451+0000 7f5181e00700 1 -- 172.21.15.79:0/1813731690 >> 172.21.15.79:0/1813731690 conn(0x7f517c0795f0 msgr2=0x7f517c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:56.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.452+0000 7f5181e00700 1 -- 172.21.15.79:0/1813731690 shutdown_connections 2024-06-03T02:30:56.453 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:56 smithi079 ceph-mon[31819]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 456 op/s 2024-06-03T02:30:56.453 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:56 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:30:56.453 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:56 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:30:56.453 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:56 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:30:56.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.452+0000 7f5181e00700 1 -- 172.21.15.79:0/1813731690 wait complete. 2024-06-03T02:30:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.453+0000 7f5181e00700 1 Processor -- start 2024-06-03T02:30:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.454+0000 7f5181e00700 1 -- start start 2024-06-03T02:30:56.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.454+0000 7f5181e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f517c0746c0 0x7f517c10c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.454+0000 7f5181e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f517c074fe0 0x7f517c10cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.454+0000 7f5181e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f517c10ee90 0x7f517c10de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.454+0000 7f5181e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f517c102870 con 0x7f517c074fe0 2024-06-03T02:30:56.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.454+0000 7f5181e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1026f0 con 0x7f517c10ee90 2024-06-03T02:30:56.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.454+0000 7f5181e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1029f0 con 0x7f517c0746c0 2024-06-03T02:30:56.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517a000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f517c074fe0 0x7f517c10cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517a000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f517c074fe0 0x7f517c10cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49894/0 (socket says 172.21.15.79:49894) 2024-06-03T02:30:56.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517a000700 1 -- 172.21.15.79:0/1569759353 learned_addr learned my addr 172.21.15.79:0/1569759353 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:56.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517b400700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f517c10ee90 0x7f517c10de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517aa00700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f517c0746c0 0x7f517c10c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517b400700 1 -- 172.21.15.79:0/1569759353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f517c0746c0 msgr2=0x7f517c10c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517b400700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f517c0746c0 0x7f517c10c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517b400700 1 -- 172.21.15.79:0/1569759353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f517c074fe0 msgr2=0x7f517c10cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517b400700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f517c074fe0 0x7f517c10cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.455+0000 7f517b400700 1 -- 172.21.15.79:0/1569759353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c10e570 con 0x7f517c10ee90 2024-06-03T02:30:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.456+0000 7f517b400700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f517c10ee90 0x7f517c10de00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f517400efc0 tx=0x7f517400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.456+0000 7f5180a00700 1 -- 172.21.15.79:0/1569759353 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740092e0 con 0x7f517c10ee90 2024-06-03T02:30:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.456+0000 7f5181e00700 1 -- 172.21.15.79:0/1569759353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517c10e800 con 0x7f517c10ee90 2024-06-03T02:30:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.456+0000 7f5181e00700 1 -- 172.21.15.79:0/1569759353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517c1abd80 con 0x7f517c10ee90 2024-06-03T02:30:56.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.457+0000 7f5180a00700 1 -- 172.21.15.79:0/1569759353 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5174004bb0 con 0x7f517c10ee90 2024-06-03T02:30:56.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.457+0000 7f5180a00700 1 -- 172.21.15.79:0/1569759353 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174004280 con 0x7f517c10ee90 2024-06-03T02:30:56.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.461+0000 7f5180a00700 1 -- 172.21.15.79:0/1569759353 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f517401b030 con 0x7f517c10ee90 2024-06-03T02:30:56.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.462+0000 7f518c769b80 1 Processor -- start 2024-06-03T02:30:56.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.463+0000 7f5180a00700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51600a2d20 0x7f51600a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.463+0000 7f5180a00700 1 -- 172.21.15.79:0/1569759353 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f51740cee20 con 0x7f517c10ee90 2024-06-03T02:30:56.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.464+0000 7f518c769b80 1 -- start start 2024-06-03T02:30:56.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.464+0000 7f517a000700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51600a2d20 0x7f51600a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f518c769b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8f62e40 0x5596b8fe6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f518c769b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8fe65b0 0x5596b8fe6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f518c769b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 0x5596b8ff36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f518c769b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5596b8ef5a10 con 0x5596b8ff1300 2024-06-03T02:30:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f518c769b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5596b8f38b00 con 0x5596b8fe65b0 2024-06-03T02:30:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f518c769b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596b8ef38f0 con 0x5596b8f62e40 2024-06-03T02:30:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f517aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8f62e40 0x5596b8fe6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f517a000700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51600a2d20 0x7f51600a51e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f517c110b90 tx=0x7f5168009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f517aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8f62e40 0x5596b8fe6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59936/0 (socket says 172.21.15.79:59936) 2024-06-03T02:30:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f517aa00700 1 -- 172.21.15.79:0/1125217977 learned_addr learned my addr 172.21.15.79:0/1125217977 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.465+0000 7f517b400700 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 0x5596b8ff36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.466+0000 7f517b400700 1 -- 172.21.15.79:0/1125217977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8f62e40 msgr2=0x5596b8fe6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.466+0000 7f517b400700 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8f62e40 0x5596b8fe6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.466+0000 7f517b400700 1 -- 172.21.15.79:0/1125217977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8fe65b0 msgr2=0x5596b8fe6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.466+0000 7f517a000700 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8fe65b0 0x5596b8fe6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.466+0000 7f517b400700 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8fe65b0 0x5596b8fe6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.466+0000 7f517b400700 1 -- 172.21.15.79:0/1125217977 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b8ee0f90 con 0x5596b8ff1300 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.466+0000 7f517b400700 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 0x5596b8ff36f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f51740cc510 tx=0x7f517409ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a55862c00a00a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.466+0000 7f517aa00700 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8f62e40 0x5596b8fe6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.467+0000 7f5170c00700 1 -- 172.21.15.79:0/1125217977 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517409b300 con 0x5596b8ff1300 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.467+0000 7f5170c00700 1 -- 172.21.15.79:0/1125217977 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f517409f6b0 con 0x5596b8ff1300 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.468+0000 7f518c769b80 1 -- 172.21.15.79:0/1125217977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 msgr2=0x5596b8ff36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.468+0000 7f518c769b80 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 0x5596b8ff36f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f51740cc510 tx=0x7f517409ac60 comp rx=0 tx=0).stop 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.468+0000 7f518c769b80 1 -- 172.21.15.79:0/1125217977 shutdown_connections 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.468+0000 7f518c769b80 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 0x5596b8ff36f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.468+0000 7f518c769b80 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8f62e40 0x5596b8fe6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.468+0000 7f518c769b80 1 --2- 172.21.15.79:0/1125217977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8fe65b0 0x5596b8fe6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.468+0000 7f518c769b80 1 -- 172.21.15.79:0/1125217977 >> 172.21.15.79:0/1125217977 conn(0x5596b8f13170 msgr2=0x5596b8fe24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:56.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.468+0000 7f518c769b80 1 -- 172.21.15.79:0/1125217977 shutdown_connections 2024-06-03T02:30:56.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.468+0000 7f518c769b80 1 -- 172.21.15.79:0/1125217977 wait complete. 2024-06-03T02:30:56.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.469+0000 7f518c769b80 1 -- start start 2024-06-03T02:30:56.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.469+0000 7f518c769b80 1 Processor -- start 2024-06-03T02:30:56.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f518c769b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f518c769b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 0x5596b8fedca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f518c769b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f518c769b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5596b8ed4f60 con 0x5596b8fe65b0 2024-06-03T02:30:56.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f518c769b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5596b8f34ac0 con 0x5596b8f62e40 2024-06-03T02:30:56.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f518c769b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596b8ff4010 con 0x5596b8ff1300 2024-06-03T02:30:56.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f518c769b80 10 client.0 _ll_register_callbacks cb 0x5596b8ed2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:56.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f517aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f517b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f517b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fee1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59938/0 (socket says 172.21.15.79:59938) 2024-06-03T02:30:56.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.470+0000 7f517b400700 1 -- 172.21.15.79:0/3068675920 learned_addr learned my addr 172.21.15.79:0/3068675920 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:56.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.471+0000 7f517a000700 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 0x5596b8fedca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.472+0000 7f517b400700 1 -- 172.21.15.79:0/3068675920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 msgr2=0x5596b8ff09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:56.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.472+0000 7f517b400700 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.472+0000 7f517b400700 1 -- 172.21.15.79:0/3068675920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 msgr2=0x5596b8fedca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:56.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.472+0000 7f517b400700 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 0x5596b8fedca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.472+0000 7f517b400700 1 -- 172.21.15.79:0/3068675920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b8e878d0 con 0x5596b8ff1300 2024-06-03T02:30:56.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.472+0000 7f517b400700 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fee1e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5174097cd0 tx=0x7f51740bb480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=540a387585d9cf44 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:56.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.472+0000 7f5165600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:30:56.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.473+0000 7f5165600700 1 -- 172.21.15.79:0/3068675920 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517409b300 con 0x5596b8ff1300 2024-06-03T02:30:56.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.473+0000 7f5165600700 1 -- 172.21.15.79:0/3068675920 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5174004d60 con 0x5596b8ff1300 2024-06-03T02:30:56.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:56 smithi123 ceph-mon[36640]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 456 op/s 2024-06-03T02:30:56.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:56 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:30:56.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:56 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:30:56.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:56 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:30:56.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.473+0000 7f518c769b80 1 -- 172.21.15.79:0/3068675920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517409f6b0 con 0x5596b8ff1300 2024-06-03T02:30:56.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.475+0000 7f5165600700 1 -- 172.21.15.79:0/3068675920 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51740a3ee0 con 0x5596b8ff1300 2024-06-03T02:30:56.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.475+0000 7f518c769b80 10 client.26542 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:56.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.475+0000 7f518c769b80 1 -- 172.21.15.79:0/3068675920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596b8ffe660 con 0x5596b8ff1300 2024-06-03T02:30:56.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.476+0000 7f5165600700 1 -- 172.21.15.79:0/3068675920 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51740a6080 con 0x5596b8ff1300 2024-06-03T02:30:56.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.476+0000 7f5165600700 1 -- 172.21.15.79:0/3068675920 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f517407e020 con 0x5596b8ff1300 2024-06-03T02:30:56.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.476+0000 7f518c769b80 10 client.26542 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:56.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.476+0000 7f518c769b80 10 client.26542 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:30:56.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.476+0000 7f518c769b80 20 client.26542 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:56.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.476+0000 7f518c769b80 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5596b9000dd0 0x5596b90031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.476+0000 7f518c769b80 4 client.26542 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:30:56.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.476+0000 7f518c769b80 1 -- 172.21.15.79:0/3068675920 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596b8f46000 con 0x5596b9000dd0 2024-06-03T02:30:56.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.477+0000 7f517aa00700 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5596b9000dd0 0x5596b90031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.482+0000 7f517aa00700 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5596b9000dd0 0x5596b90031f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=36cfd1a230597bff server_cookie=25d62ab187c1d5eb in_seq=0 out_seq=0 2024-06-03T02:30:56.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.482+0000 7f5165600700 10 client.26542 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:30:56.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.483+0000 7f5165600700 1 -- 172.21.15.79:0/3068675920 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5596b8f46000 con 0x5596b9000dd0 2024-06-03T02:30:56.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.483+0000 7f5165600700 10 client.26542 handle_command_reply: tid=0 2024-06-03T02:30:56.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.483+0000 7f518c769b80 1 client.26542 shutdown 2024-06-03T02:30:56.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.485+0000 7f518c769b80 1 -- 172.21.15.79:0/3068675920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 msgr2=0x5596b8fee1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:56.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.485+0000 7f518c769b80 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fee1e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5174097cd0 tx=0x7f51740bb480 comp rx=0 tx=0).stop 2024-06-03T02:30:56.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.486+0000 7f518c769b80 1 -- 172.21.15.79:0/3068675920 shutdown_connections 2024-06-03T02:30:56.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.486+0000 7f518c769b80 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fee1e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.486+0000 7f518c769b80 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.486+0000 7f518c769b80 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5596b9000dd0 0x5596b90031f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.486+0000 7f518c769b80 1 --2- 172.21.15.79:0/3068675920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 0x5596b8fedca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.486+0000 7f518c769b80 1 -- 172.21.15.79:0/3068675920 >> 172.21.15.79:0/3068675920 conn(0x5596b8f13170 msgr2=0x5596b8fe1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:56.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.486+0000 7f5165600700 0 client.26542 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:30:56.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.487+0000 7f518c769b80 1 -- 172.21.15.79:0/3068675920 shutdown_connections 2024-06-03T02:30:56.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.487+0000 7f518c769b80 1 -- 172.21.15.79:0/3068675920 wait complete. 2024-06-03T02:30:56.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.488+0000 7f518c769b80 20 client.26542 trim_cache size 0 max 16384 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.526+0000 7f518c769b80 1 Processor -- start 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.526+0000 7f518c769b80 1 -- start start 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f518c769b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b90031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f518c769b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 0x5596b8ff3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f518c769b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fefbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f518c769b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f515c00eb00 con 0x5596b8fe65b0 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f518c769b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f515c010640 con 0x5596b8f62e40 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f518c769b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f515c00f1d0 con 0x5596b8ff1300 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f517aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b90031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f517a000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 0x5596b8ff3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f517a000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 0x5596b8ff3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49936/0 (socket says 172.21.15.79:49936) 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f517a000700 1 -- 172.21.15.79:0/2097928409 learned_addr learned my addr 172.21.15.79:0/2097928409 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.527+0000 7f517b400700 1 --2- 172.21.15.79:0/2097928409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fefbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.528+0000 7f517b400700 1 -- 172.21.15.79:0/2097928409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 msgr2=0x5596b90031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.528+0000 7f517b400700 1 --2- 172.21.15.79:0/2097928409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b90031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.528+0000 7f517b400700 1 -- 172.21.15.79:0/2097928409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 msgr2=0x5596b8ff3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.528+0000 7f517b400700 1 --2- 172.21.15.79:0/2097928409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 0x5596b8ff3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.528+0000 7f517b400700 1 -- 172.21.15.79:0/2097928409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b8fee5a0 con 0x5596b8ff1300 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.528+0000 7f517b400700 1 --2- 172.21.15.79:0/2097928409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fefbe0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5174004b80 tx=0x7f517409ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77900a58c4f04b9d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.530+0000 7f5167e00700 1 -- 172.21.15.79:0/2097928409 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517409b300 con 0x5596b8ff1300 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.530+0000 7f5167e00700 1 -- 172.21.15.79:0/2097928409 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f517409f6b0 con 0x5596b8ff1300 2024-06-03T02:30:56.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.530+0000 7f5167e00700 1 -- 172.21.15.79:0/2097928409 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517409a750 con 0x5596b8ff1300 2024-06-03T02:30:56.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.532+0000 7f518c769b80 1 -- 172.21.15.79:0/2097928409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 msgr2=0x5596b8fefbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:56.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.532+0000 7f518c769b80 1 --2- 172.21.15.79:0/2097928409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fefbe0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5174004b80 tx=0x7f517409ac60 comp rx=0 tx=0).stop 2024-06-03T02:30:56.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.534+0000 7f518c769b80 1 -- 172.21.15.79:0/2097928409 shutdown_connections 2024-06-03T02:30:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.534+0000 7f518c769b80 1 --2- 172.21.15.79:0/2097928409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8ff1300 0x5596b8fefbe0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.534+0000 7f518c769b80 1 --2- 172.21.15.79:0/2097928409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b90031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.534+0000 7f518c769b80 1 --2- 172.21.15.79:0/2097928409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8fe65b0 0x5596b8ff3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.534+0000 7f518c769b80 1 -- 172.21.15.79:0/2097928409 >> 172.21.15.79:0/2097928409 conn(0x5596b8f13170 msgr2=0x5596b8fef4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.535+0000 7f518c769b80 1 -- 172.21.15.79:0/2097928409 shutdown_connections 2024-06-03T02:30:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.535+0000 7f518c769b80 1 -- 172.21.15.79:0/2097928409 wait complete. 2024-06-03T02:30:56.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.537+0000 7f518c769b80 1 -- start start 2024-06-03T02:30:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.537+0000 7f518c769b80 1 Processor -- start 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.538+0000 7f518c769b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.538+0000 7f518c769b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8fe65b0 0x5596b8ff4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.538+0000 7f518c769b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 0x5596b8ff4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.538+0000 7f518c769b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f515c00ff70 con 0x5596b8ff1300 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.538+0000 7f518c769b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f515c00f8a0 con 0x5596b8f62e40 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.538+0000 7f518c769b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f515c00caa0 con 0x5596b8fe65b0 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.538+0000 7f518c769b80 10 client.0 _ll_register_callbacks cb 0x5596b8ed2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8fe65b0 0x5596b8ff4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40310/0 (socket says 172.21.15.79:40310) 2024-06-03T02:30:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8fe65b0 0x5596b8ff4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59970/0 (socket says 172.21.15.79:59970) 2024-06-03T02:30:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517aa00700 1 -- 172.21.15.79:0/1913250131 learned_addr learned my addr 172.21.15.79:0/1913250131 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:30:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517aa00700 1 -- 172.21.15.79:0/1913250131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8fe65b0 msgr2=0x5596b8ff4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517aa00700 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8fe65b0 0x5596b8ff4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517aa00700 1 -- 172.21.15.79:0/1913250131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 msgr2=0x5596b8ff4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:30:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517aa00700 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 0x5596b8ff4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.539+0000 7f517aa00700 1 -- 172.21.15.79:0/1913250131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b9069b80 con 0x5596b8f62e40 2024-06-03T02:30:56.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.542+0000 7f517aa00700 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff42e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f516c0098e0 tx=0x7f516c005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b668025ff4c413 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:30:56.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.542+0000 7f5165600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:30:56.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.542+0000 7f5165600700 1 -- 172.21.15.79:0/1913250131 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c0052b0 con 0x5596b8f62e40 2024-06-03T02:30:56.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.542+0000 7f518c769b80 1 -- 172.21.15.79:0/1913250131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517409f6b0 con 0x5596b8f62e40 2024-06-03T02:30:56.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.542+0000 7f5165600700 1 -- 172.21.15.79:0/1913250131 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f516c005450 con 0x5596b8f62e40 2024-06-03T02:30:56.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.543+0000 7f5165600700 1 -- 172.21.15.79:0/1913250131 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f516c037050 con 0x5596b8f62e40 2024-06-03T02:30:56.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.543+0000 7f518c769b80 10 client.26555 fetch_fsmap learned FSMap version 30 2024-06-03T02:30:56.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.543+0000 7f518c769b80 1 -- 172.21.15.79:0/1913250131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596b906aee0 con 0x5596b8f62e40 2024-06-03T02:30:56.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.543+0000 7f5165600700 1 -- 172.21.15.79:0/1913250131 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f517409f6b0 con 0x5596b8f62e40 2024-06-03T02:30:56.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.546+0000 7f5165600700 1 -- 172.21.15.79:0/1913250131 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f516c003680 con 0x5596b8f62e40 2024-06-03T02:30:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.546+0000 7f518c769b80 10 client.26555 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:30:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.546+0000 7f518c769b80 10 client.26555 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:30:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.546+0000 7f518c769b80 20 client.26555 populate_metadata read hostname 'smithi079' 2024-06-03T02:30:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.546+0000 7f518c769b80 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5596b906b5a0 0x5596b906d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:30:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.546+0000 7f518c769b80 4 client.26555 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:30:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.546+0000 7f518c769b80 1 -- 172.21.15.79:0/1913250131 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596b8f40580 con 0x5596b906b5a0 2024-06-03T02:30:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.546+0000 7f517a000700 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5596b906b5a0 0x5596b906d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:30:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.547+0000 7f517a000700 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5596b906b5a0 0x5596b906d9e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff36ebee609f165d server_cookie=96a8a9d3dbab3b1 in_seq=0 out_seq=0 2024-06-03T02:30:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.547+0000 7f5165600700 10 client.26555 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:30:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.547+0000 7f5165600700 1 -- 172.21.15.79:0/1913250131 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596b8f40580 con 0x5596b906b5a0 2024-06-03T02:30:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.547+0000 7f5165600700 10 client.26555 handle_command_reply: tid=0 2024-06-03T02:30:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.548+0000 7f518c769b80 1 client.26555 shutdown 2024-06-03T02:30:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.548+0000 7f518c769b80 1 -- 172.21.15.79:0/1913250131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 msgr2=0x5596b8ff42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:56.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.548+0000 7f518c769b80 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff42e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f516c0098e0 tx=0x7f516c005ab0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.548+0000 7f518c769b80 1 -- 172.21.15.79:0/1913250131 shutdown_connections 2024-06-03T02:30:56.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.548+0000 7f518c769b80 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x5596b906b5a0 0x5596b906d9e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.549+0000 7f5165600700 0 client.26555 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:30:56.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.549+0000 7f518c769b80 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5596b8ff1300 0x5596b8ff4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.549+0000 7f518c769b80 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596b8f62e40 0x5596b8ff42e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.549+0000 7f518c769b80 1 --2- 172.21.15.79:0/1913250131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596b8fe65b0 0x5596b8ff4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.549+0000 7f518c769b80 1 -- 172.21.15.79:0/1913250131 >> 172.21.15.79:0/1913250131 conn(0x5596b8f13170 msgr2=0x5596b8fe1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:56.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.549+0000 7f518c769b80 1 -- 172.21.15.79:0/1913250131 shutdown_connections 2024-06-03T02:30:56.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.549+0000 7f518c769b80 1 -- 172.21.15.79:0/1913250131 wait complete. 2024-06-03T02:30:56.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.549+0000 7f518c769b80 20 client.26555 trim_cache size 0 max 16384 2024-06-03T02:30:56.552 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:30:56.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.552+0000 7f5181e00700 1 -- 172.21.15.79:0/1569759353 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51600a2d20 msgr2=0x7f51600a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:56.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.552+0000 7f5181e00700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51600a2d20 0x7f51600a51e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f517c110b90 tx=0x7f5168009260 comp rx=0 tx=0).stop 2024-06-03T02:30:56.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.552+0000 7f5181e00700 1 -- 172.21.15.79:0/1569759353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f517c10ee90 msgr2=0x7f517c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:30:56.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.552+0000 7f5181e00700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f517c10ee90 0x7f517c10de00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f517400efc0 tx=0x7f517400c3a0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.553+0000 7f5181e00700 1 -- 172.21.15.79:0/1569759353 shutdown_connections 2024-06-03T02:30:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.553+0000 7f5181e00700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f517c10ee90 0x7f517c10de00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.553+0000 7f5181e00700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f517c0746c0 0x7f517c10c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.553+0000 7f5181e00700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f51600a2d20 0x7f51600a51e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.553+0000 7f5181e00700 1 --2- 172.21.15.79:0/1569759353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f517c074fe0 0x7f517c10cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:30:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.553+0000 7f5181e00700 1 -- 172.21.15.79:0/1569759353 >> 172.21.15.79:0/1569759353 conn(0x7f517c0795f0 msgr2=0x7f517c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:30:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.553+0000 7f5181e00700 1 -- 172.21.15.79:0/1569759353 shutdown_connections 2024-06-03T02:30:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:30:56.553+0000 7f5181e00700 1 -- 172.21.15.79:0/1569759353 wait complete. 2024-06-03T02:30:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:56 smithi135 ceph-mon[36653]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 456 op/s 2024-06-03T02:30:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:56 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:30:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:56 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:30:56.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:56 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:30:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 650.0 s, 449.5 tps, lat 233.302 ms stddev 305.936 2024-06-03T02:30:58.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:30:58 smithi123 ceph-mon[36640]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 363 op/s 2024-06-03T02:30:58.529 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:30:58 smithi079 ceph-mon[31819]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 363 op/s 2024-06-03T02:30:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:30:58 smithi135 ceph-mon[36653]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 363 op/s 2024-06-03T02:30:58.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:30:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:31:00.324 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:00 smithi079 ceph-mon[31819]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 448 op/s 2024-06-03T02:31:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:00 smithi123 ceph-mon[36640]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 448 op/s 2024-06-03T02:31:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:00 smithi135 ceph-mon[36653]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 448 op/s 2024-06-03T02:31:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.408+0000 7f9f5de00700 1 -- 172.21.15.79:0/1460394445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f58074830 msgr2=0x7f9f58074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.408+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1460394445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f58074830 0x7f9f58074c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009990 tx=0x7f9f48009ca0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.409+0000 7f9f5de00700 1 -- 172.21.15.79:0/1460394445 shutdown_connections 2024-06-03T02:31:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.409+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1460394445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f581133d0 0x7f9f581157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.409+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1460394445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f58074830 0x7f9f58074c10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.409+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1460394445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58075150 0x7f9f58112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.409+0000 7f9f5de00700 1 -- 172.21.15.79:0/1460394445 >> 172.21.15.79:0/1460394445 conn(0x7f9f580fdf10 msgr2=0x7f9f58100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.410+0000 7f9f5de00700 1 -- 172.21.15.79:0/1460394445 shutdown_connections 2024-06-03T02:31:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.410+0000 7f9f5de00700 1 -- 172.21.15.79:0/1460394445 wait complete. 2024-06-03T02:31:02.412 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:02 smithi079 ceph-mon[31819]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 400 op/s 2024-06-03T02:31:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.412+0000 7f9f5de00700 1 Processor -- start 2024-06-03T02:31:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.412+0000 7f9f5de00700 1 -- start start 2024-06-03T02:31:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f58074830 0x7f9f5810cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58075150 0x7f9f5810d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f581133d0 0x7f9f5810d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5de00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58107460 con 0x7f9f58074830 2024-06-03T02:31:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5de00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581072e0 con 0x7f9f58075150 2024-06-03T02:31:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5de00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581075e0 con 0x7f9f581133d0 2024-06-03T02:31:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f581133d0 0x7f9f5810d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f581133d0 0x7f9f5810d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60000/0 (socket says 172.21.15.79:60000) 2024-06-03T02:31:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5d400700 1 -- 172.21.15.79:0/1840016553 learned_addr learned my addr 172.21.15.79:0/1840016553 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58075150 0x7f9f5810d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5d400700 1 -- 172.21.15.79:0/1840016553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58075150 msgr2=0x7f9f5810d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.413+0000 7f9f5d400700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58075150 0x7f9f5810d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.414+0000 7f9f5d400700 1 -- 172.21.15.79:0/1840016553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f58074830 msgr2=0x7f9f5810cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.414+0000 7f9f5d400700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f58074830 0x7f9f5810cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.414+0000 7f9f5d400700 1 -- 172.21.15.79:0/1840016553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f581b3db0 con 0x7f9f581133d0 2024-06-03T02:31:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.414+0000 7f9f5d400700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f581133d0 0x7f9f5810d9e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000ef90 tx=0x7f9f5000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:02.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.415+0000 7f9f56000700 1 -- 172.21.15.79:0/1840016553 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f500092e0 con 0x7f9f581133d0 2024-06-03T02:31:02.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.415+0000 7f9f56000700 1 -- 172.21.15.79:0/1840016553 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f50004bb0 con 0x7f9f581133d0 2024-06-03T02:31:02.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.415+0000 7f9f56000700 1 -- 172.21.15.79:0/1840016553 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50004280 con 0x7f9f581133d0 2024-06-03T02:31:02.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.416+0000 7f9f5de00700 1 -- 172.21.15.79:0/1840016553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f581b4040 con 0x7f9f581133d0 2024-06-03T02:31:02.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.416+0000 7f9f5de00700 1 -- 172.21.15.79:0/1840016553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f581b4530 con 0x7f9f581133d0 2024-06-03T02:31:02.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.421+0000 7f9f56000700 1 -- 172.21.15.79:0/1840016553 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9f5001b030 con 0x7f9f581133d0 2024-06-03T02:31:02.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.423+0000 7f9f56000700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9f3c0a2df0 0x7f9f3c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.423+0000 7f9f56000700 1 -- 172.21.15.79:0/1840016553 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f9f500cef40 con 0x7f9f581133d0 2024-06-03T02:31:02.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.425+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9f3c0a2df0 0x7f9f3c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.427+0000 7f9f69dc9b80 1 Processor -- start 2024-06-03T02:31:02.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.427+0000 7f9f69dc9b80 1 -- start start 2024-06-03T02:31:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9f3c0a2df0 0x7f9f3c0a52b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006010 tx=0x7f9f4800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648703e6e40 0x56487046c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x56487046cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564870475570 0x564870477960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564870379a10 con 0x564870475570 2024-06-03T02:31:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5648703bcb00 con 0x56487046c960 2024-06-03T02:31:02.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648703778f0 con 0x5648703e6e40 2024-06-03T02:31:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f5ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648703e6e40 0x56487046c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f4fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x56487046cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f5ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648703e6e40 0x56487046c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60012/0 (socket says 172.21.15.79:60012) 2024-06-03T02:31:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.429+0000 7f9f5ca00700 1 -- 172.21.15.79:0/1779286877 learned_addr learned my addr 172.21.15.79:0/1779286877 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.430+0000 7f9f4fe00700 1 -- 172.21.15.79:0/1779286877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648703e6e40 msgr2=0x56487046c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:02.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.430+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/1779286877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648703e6e40 0x56487046c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.430+0000 7f9f4fe00700 1 -- 172.21.15.79:0/1779286877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564870475570 msgr2=0x564870477960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:31:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.430+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/1779286877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564870475570 0x564870477960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.430+0000 7f9f4fe00700 1 -- 172.21.15.79:0/1779286877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564870364f90 con 0x56487046c960 2024-06-03T02:31:02.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.431+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/1779286877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x56487046cde0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9f5810e7b0 tx=0x7f9f40009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb442069fe5bda1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.431+0000 7f9f4e000700 1 -- 172.21.15.79:0/1779286877 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40017070 con 0x56487046c960 2024-06-03T02:31:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.431+0000 7f9f4e000700 1 -- 172.21.15.79:0/1779286877 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4000ed20 con 0x56487046c960 2024-06-03T02:31:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.431+0000 7f9f4e000700 1 -- 172.21.15.79:0/1779286877 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40013e60 con 0x56487046c960 2024-06-03T02:31:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.433+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1779286877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 msgr2=0x56487046cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.433+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/1779286877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x56487046cde0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9f5810e7b0 tx=0x7f9f40009cf0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.434+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1779286877 shutdown_connections 2024-06-03T02:31:02.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.434+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/1779286877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564870475570 0x564870477960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.434+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/1779286877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648703e6e40 0x56487046c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.434+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/1779286877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x56487046cde0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.434+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1779286877 >> 172.21.15.79:0/1779286877 conn(0x564870397170 msgr2=0x564870466820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:02.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.435+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1779286877 shutdown_connections 2024-06-03T02:31:02.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.435+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1779286877 wait complete. 2024-06-03T02:31:02.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.435+0000 7f9f69dc9b80 1 -- start start 2024-06-03T02:31:02.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.436+0000 7f9f69dc9b80 1 Processor -- start 2024-06-03T02:31:02.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.436+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 0x564870474d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.436+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x564870472040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.436+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564870475570 0x564870472580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.436+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564870358f60 con 0x5648703e6e40 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.436+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5648703b8ac0 con 0x56487046c960 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.436+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648704781e0 con 0x564870475570 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.436+0000 7f9f69dc9b80 10 client.0 _ll_register_callbacks cb 0x564870356720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f5d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564870475570 0x564870472580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f5d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564870475570 0x564870472580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60022/0 (socket says 172.21.15.79:60022) 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 0x564870474d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f5d400700 1 -- 172.21.15.79:0/943491635 learned_addr learned my addr 172.21.15.79:0/943491635 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:02.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x564870472040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f5d400700 1 -- 172.21.15.79:0/943491635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 msgr2=0x564870472040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f5d400700 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x564870472040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f5d400700 1 -- 172.21.15.79:0/943491635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 msgr2=0x564870474d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f5d400700 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 0x564870474d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f5d400700 1 -- 172.21.15.79:0/943491635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56487030b8d0 con 0x564870475570 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.437+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x564870472040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.438+0000 7f9f5d400700 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564870475570 0x564870472580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f50097880 tx=0x7f9f5009b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c94d9b7ee717cc0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.438+0000 7f9f46a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.438+0000 7f9f46a00700 1 -- 172.21.15.79:0/943491635 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5009aa70 con 0x564870475570 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.438+0000 7f9f46a00700 1 -- 172.21.15.79:0/943491635 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f9f500ccaf0 con 0x564870475570 2024-06-03T02:31:02.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.439+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/943491635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4000ed20 con 0x564870475570 2024-06-03T02:31:02.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.439+0000 7f9f46a00700 1 -- 172.21.15.79:0/943491635 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f5009bb20 con 0x564870475570 2024-06-03T02:31:02.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.439+0000 7f9f69dc9b80 10 client.26560 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:02.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.439+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/943491635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564870482960 con 0x564870475570 2024-06-03T02:31:02.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.440+0000 7f9f46a00700 1 -- 172.21.15.79:0/943491635 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4000ed20 con 0x564870475570 2024-06-03T02:31:02.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.440+0000 7f9f46a00700 1 -- 172.21.15.79:0/943491635 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9f5009a2e0 con 0x564870475570 2024-06-03T02:31:02.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.440+0000 7f9f69dc9b80 10 client.26560 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:02.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.440+0000 7f9f69dc9b80 10 client.26560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:31:02.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.440+0000 7f9f69dc9b80 20 client.26560 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:02.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.440+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x564870484f40 0x564870487330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.440+0000 7f9f69dc9b80 4 client.26560 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:31:02.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.440+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/943491635 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648703ca000 con 0x564870484f40 2024-06-03T02:31:02.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.440+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x564870484f40 0x564870487330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.441+0000 7f9f46a00700 10 client.26560 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:31:02.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.441+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x564870484f40 0x564870487330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d5d45755781e64c server_cookie=c0439c0cac68d6d1 in_seq=0 out_seq=0 2024-06-03T02:31:02.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.442+0000 7f9f46a00700 1 -- 172.21.15.79:0/943491635 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5648703ca000 con 0x564870484f40 2024-06-03T02:31:02.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.442+0000 7f9f46a00700 10 client.26560 handle_command_reply: tid=0 2024-06-03T02:31:02.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.442+0000 7f9f69dc9b80 1 client.26560 shutdown 2024-06-03T02:31:02.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.444+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/943491635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564870475570 msgr2=0x564870472580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:02.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.444+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564870475570 0x564870472580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f50097880 tx=0x7f9f5009b960 comp rx=0 tx=0).stop 2024-06-03T02:31:02.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.444+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/943491635 shutdown_connections 2024-06-03T02:31:02.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.444+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564870475570 0x564870472580 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.444+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 0x564870474d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.444+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x564870484f40 0x564870487330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.444+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/943491635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56487046c960 0x564870472040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.444+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/943491635 >> 172.21.15.79:0/943491635 conn(0x564870397170 msgr2=0x564870466410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:02.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.444+0000 7f9f46a00700 0 client.26560 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:31:02.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.445+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/943491635 shutdown_connections 2024-06-03T02:31:02.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.445+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/943491635 wait complete. 2024-06-03T02:31:02.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.445+0000 7f9f69dc9b80 20 client.26560 trim_cache size 0 max 16384 2024-06-03T02:31:02.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:02 smithi123 ceph-mon[36640]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 400 op/s 2024-06-03T02:31:02.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.475+0000 7f9f69dc9b80 1 Processor -- start 2024-06-03T02:31:02.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f69dc9b80 1 -- start start 2024-06-03T02:31:02.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 0x5648704718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487046d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564870475570 0x564870472000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581bd1e0 con 0x5648703e6e40 2024-06-03T02:31:02.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581bed20 con 0x564870475570 2024-06-03T02:31:02.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581bd8b0 con 0x56487046c960 2024-06-03T02:31:02.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f4fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487046d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f4fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487046d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60026/0 (socket says 172.21.15.79:60026) 2024-06-03T02:31:02.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/1951916921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 0x5648704718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.476+0000 7f9f4fe00700 1 -- 172.21.15.79:0/1951916921 learned_addr learned my addr 172.21.15.79:0/1951916921 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:02.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.477+0000 7f9f5d400700 1 --2- 172.21.15.79:0/1951916921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564870475570 0x564870472000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.477+0000 7f9f4fe00700 1 -- 172.21.15.79:0/1951916921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564870475570 msgr2=0x564870472000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:02.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.477+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/1951916921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564870475570 0x564870472000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.477+0000 7f9f4fe00700 1 -- 172.21.15.79:0/1951916921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 msgr2=0x5648704718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:02.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.477+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/1951916921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 0x5648704718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.477+0000 7f9f4fe00700 1 -- 172.21.15.79:0/1951916921 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648704705b0 con 0x56487046c960 2024-06-03T02:31:02.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.478+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/1951916921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487046d460 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f400094e0 tx=0x7f9f40009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afd54fe8fb9015e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:02.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.478+0000 7f9f4d600700 1 -- 172.21.15.79:0/1951916921 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40017040 con 0x56487046c960 2024-06-03T02:31:02.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.478+0000 7f9f4d600700 1 -- 172.21.15.79:0/1951916921 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4000ec00 con 0x56487046c960 2024-06-03T02:31:02.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.478+0000 7f9f4d600700 1 -- 172.21.15.79:0/1951916921 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40013470 con 0x56487046c960 2024-06-03T02:31:02.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.479+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1951916921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 msgr2=0x56487046d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:02.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.479+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/1951916921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487046d460 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f400094e0 tx=0x7f9f40009f20 comp rx=0 tx=0).stop 2024-06-03T02:31:02.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.480+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1951916921 shutdown_connections 2024-06-03T02:31:02.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.480+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/1951916921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564870475570 0x564870472000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.480+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/1951916921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5648703e6e40 0x5648704718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.480+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/1951916921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487046d460 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.480+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1951916921 >> 172.21.15.79:0/1951916921 conn(0x564870397170 msgr2=0x564870465950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:02.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.480+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1951916921 shutdown_connections 2024-06-03T02:31:02.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.480+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/1951916921 wait complete. 2024-06-03T02:31:02.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.481+0000 7f9f69dc9b80 1 -- start start 2024-06-03T02:31:02.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.481+0000 7f9f69dc9b80 1 Processor -- start 2024-06-03T02:31:02.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648703e6e40 0x56487047d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487047da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f69dc9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564870475570 0x564870478a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581be650 con 0x564870475570 2024-06-03T02:31:02.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581bdf80 con 0x5648703e6e40 2024-06-03T02:31:02.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f69dc9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581bb180 con 0x56487046c960 2024-06-03T02:31:02.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f69dc9b80 10 client.0 _ll_register_callbacks cb 0x564870356720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:02.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f4fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487047da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f4fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487047da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60042/0 (socket says 172.21.15.79:60042) 2024-06-03T02:31:02.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.482+0000 7f9f4fe00700 1 -- 172.21.15.79:0/3175981373 learned_addr learned my addr 172.21.15.79:0/3175981373 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:02.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f5d400700 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564870475570 0x564870478a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648703e6e40 0x56487047d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f4fe00700 1 -- 172.21.15.79:0/3175981373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648703e6e40 msgr2=0x56487047d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:02.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648703e6e40 0x56487047d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f4fe00700 1 -- 172.21.15.79:0/3175981373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564870475570 msgr2=0x564870478a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:02.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564870475570 0x564870478a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f4fe00700 1 -- 172.21.15.79:0/3175981373 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564870479220 con 0x56487046c960 2024-06-03T02:31:02.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648703e6e40 0x56487047d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:31:02.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f4fe00700 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487047da10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f400094e0 tx=0x7f9f40003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6c457284d5db3df server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:02.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.483+0000 7f9f46a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:31:02.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.484+0000 7f9f46a00700 1 -- 172.21.15.79:0/3175981373 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40017040 con 0x56487046c960 2024-06-03T02:31:02.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.484+0000 7f9f46a00700 1 -- 172.21.15.79:0/3175981373 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f9f400040e0 con 0x56487046c960 2024-06-03T02:31:02.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.484+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/3175981373 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4000ec00 con 0x56487046c960 2024-06-03T02:31:02.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.485+0000 7f9f46a00700 1 -- 172.21.15.79:0/3175981373 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f400042e0 con 0x56487046c960 2024-06-03T02:31:02.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.485+0000 7f9f69dc9b80 10 client.26566 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:02.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.485+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/3175981373 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648704ef650 con 0x56487046c960 2024-06-03T02:31:02.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.485+0000 7f9f46a00700 1 -- 172.21.15.79:0/3175981373 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4000ec00 con 0x56487046c960 2024-06-03T02:31:02.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.486+0000 7f9f46a00700 1 -- 172.21.15.79:0/3175981373 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9f40006320 con 0x56487046c960 2024-06-03T02:31:02.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.486+0000 7f9f69dc9b80 10 client.26566 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:02.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.486+0000 7f9f69dc9b80 10 client.26566 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:31:02.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.486+0000 7f9f69dc9b80 20 client.26566 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:02.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.486+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5648704ef970 0x5648704f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:02.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.486+0000 7f9f69dc9b80 4 client.26566 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:31:02.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.486+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/3175981373 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648703c4580 con 0x5648704ef970 2024-06-03T02:31:02.487 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.486+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5648704ef970 0x5648704f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:02.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.488+0000 7f9f46a00700 10 client.26566 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:31:02.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.488+0000 7f9f5ca00700 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5648704ef970 0x5648704f1d90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ebb02ab546e5250 server_cookie=a2addbcd7dbe65f6 in_seq=0 out_seq=0 2024-06-03T02:31:02.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.488+0000 7f9f46a00700 1 -- 172.21.15.79:0/3175981373 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648703c4580 con 0x5648704ef970 2024-06-03T02:31:02.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.488+0000 7f9f46a00700 10 client.26566 handle_command_reply: tid=0 2024-06-03T02:31:02.489 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.488+0000 7f9f69dc9b80 1 client.26566 shutdown 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.488+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/3175981373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 msgr2=0x56487047da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.488+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487047da10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f400094e0 tx=0x7f9f40003d60 comp rx=0 tx=0).stop 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/3175981373 shutdown_connections 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564870475570 0x564870478a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x5648704ef970 0x5648704f1d90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648703e6e40 0x56487047d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f69dc9b80 1 --2- 172.21.15.79:0/3175981373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487046c960 0x56487047da10 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f400094e0 tx=0x7f9f40003d60 comp rx=0 tx=0).stop 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/3175981373 >> 172.21.15.79:0/3175981373 conn(0x564870397170 msgr2=0x564870471f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f46a00700 0 client.26566 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/3175981373 shutdown_connections 2024-06-03T02:31:02.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f69dc9b80 1 -- 172.21.15.79:0/3175981373 wait complete. 2024-06-03T02:31:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.489+0000 7f9f69dc9b80 20 client.26566 trim_cache size 0 max 16384 2024-06-03T02:31:02.491 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:31:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.491+0000 7f9f5de00700 1 -- 172.21.15.79:0/1840016553 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9f3c0a2df0 msgr2=0x7f9f3c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.491+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9f3c0a2df0 0x7f9f3c0a52b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006010 tx=0x7f9f4800b260 comp rx=0 tx=0).stop 2024-06-03T02:31:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.491+0000 7f9f5de00700 1 -- 172.21.15.79:0/1840016553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f581133d0 msgr2=0x7f9f5810d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.491+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f581133d0 0x7f9f5810d9e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000ef90 tx=0x7f9f5000c3a0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.491+0000 7f9f5de00700 1 -- 172.21.15.79:0/1840016553 shutdown_connections 2024-06-03T02:31:02.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.491+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f581133d0 0x7f9f5810d9e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.491+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9f3c0a2df0 0x7f9f3c0a52b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.491+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f58074830 0x7f9f5810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.491+0000 7f9f5de00700 1 --2- 172.21.15.79:0/1840016553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58075150 0x7f9f5810d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:02.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.492+0000 7f9f5de00700 1 -- 172.21.15.79:0/1840016553 >> 172.21.15.79:0/1840016553 conn(0x7f9f580fdf10 msgr2=0x7f9f580ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:02.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.493+0000 7f9f5de00700 1 -- 172.21.15.79:0/1840016553 shutdown_connections 2024-06-03T02:31:02.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:02.495+0000 7f9f5de00700 1 -- 172.21.15.79:0/1840016553 wait complete. 2024-06-03T02:31:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:02 smithi135 ceph-mon[36653]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 400 op/s 2024-06-03T02:31:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 655.0 s, 390.8 tps, lat 256.230 ms stddev 324.962 2024-06-03T02:31:04.399 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:04 smithi079 ceph-mon[31819]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 398 op/s 2024-06-03T02:31:04.402 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:31:04.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:04 smithi123 ceph-mon[36640]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 398 op/s 2024-06-03T02:31:04.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:04 smithi135 ceph-mon[36653]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 398 op/s 2024-06-03T02:31:05.174 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:31:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:06 smithi079 ceph-mon[31819]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 472 op/s 2024-06-03T02:31:06.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:06 smithi123 ceph-mon[36640]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 472 op/s 2024-06-03T02:31:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:06 smithi135 ceph-mon[36653]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 472 op/s 2024-06-03T02:31:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 660.0 s, 622.6 tps, lat 166.028 ms stddev 234.808 2024-06-03T02:31:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.309+0000 7fce2e400700 1 -- 172.21.15.79:0/2255052278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806a170 msgr2=0x7fce2806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.309+0000 7fce2e400700 1 --2- 172.21.15.79:0/2255052278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806a170 0x7fce2806a5f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fce14009990 tx=0x7fce14009ca0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.314+0000 7fce2e400700 1 -- 172.21.15.79:0/2255052278 shutdown_connections 2024-06-03T02:31:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.315+0000 7fce2e400700 1 --2- 172.21.15.79:0/2255052278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce2806ab30 0x7fce2810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.315+0000 7fce2e400700 1 --2- 172.21.15.79:0/2255052278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce28108500 0x7fce281088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.315+0000 7fce2e400700 1 --2- 172.21.15.79:0/2255052278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806a170 0x7fce2806a5f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.316+0000 7fce2e400700 1 -- 172.21.15.79:0/2255052278 >> 172.21.15.79:0/2255052278 conn(0x7fce28078340 msgr2=0x7fce28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.317+0000 7fce2e400700 1 -- 172.21.15.79:0/2255052278 shutdown_connections 2024-06-03T02:31:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.317+0000 7fce2e400700 1 -- 172.21.15.79:0/2255052278 wait complete. 2024-06-03T02:31:08.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.324+0000 7fce2e400700 1 Processor -- start 2024-06-03T02:31:08.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.325+0000 7fce2e400700 1 -- start start 2024-06-03T02:31:08.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.330+0000 7fce2e400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce2806a170 0x7fce281169e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.331+0000 7fce2e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806ab30 0x7fce28116f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.332+0000 7fce26a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806ab30 0x7fce28116f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.332+0000 7fce26a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806ab30 0x7fce28116f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34178/0 (socket says 172.21.15.79:34178) 2024-06-03T02:31:08.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.332+0000 7fce26a00700 1 -- 172.21.15.79:0/1231846003 learned_addr learned my addr 172.21.15.79:0/1231846003 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:08.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.332+0000 7fce2e400700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce28108500 0x7fce28111b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.332+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fce28103020 con 0x7fce2806a170 2024-06-03T02:31:08.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.332+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce28102ea0 con 0x7fce2806ab30 2024-06-03T02:31:08.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.332+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fce281031a0 con 0x7fce28108500 2024-06-03T02:31:08.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.333+0000 7fce27e00700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce28108500 0x7fce28111b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.333+0000 7fce27400700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce2806a170 0x7fce281169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.334+0000 7fce26a00700 1 -- 172.21.15.79:0/1231846003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce28108500 msgr2=0x7fce28111b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:08.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.334+0000 7fce26a00700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce28108500 0x7fce28111b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.334+0000 7fce26a00700 1 -- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce2806a170 msgr2=0x7fce281169e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:08.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.334+0000 7fce26a00700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce2806a170 0x7fce281169e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.334+0000 7fce26a00700 1 -- 172.21.15.79:0/1231846003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce28112450 con 0x7fce2806ab30 2024-06-03T02:31:08.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.334+0000 7fce27400700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce2806a170 0x7fce281169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:08.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.335+0000 7fce26a00700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806ab30 0x7fce28116f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce14005910 tx=0x7fce1400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:08.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.335+0000 7fce1fe00700 1 -- 172.21.15.79:0/1231846003 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14027070 con 0x7fce2806ab30 2024-06-03T02:31:08.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.335+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce281126e0 con 0x7fce2806ab30 2024-06-03T02:31:08.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.336+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2811c8d0 con 0x7fce2806ab30 2024-06-03T02:31:08.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.336+0000 7fce1fe00700 1 -- 172.21.15.79:0/1231846003 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce14003d70 con 0x7fce2806ab30 2024-06-03T02:31:08.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.336+0000 7fce1fe00700 1 -- 172.21.15.79:0/1231846003 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1400f6c0 con 0x7fce2806ab30 2024-06-03T02:31:08.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.342+0000 7fce1fe00700 1 -- 172.21.15.79:0/1231846003 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fce1402a070 con 0x7fce2806ab30 2024-06-03T02:31:08.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.343+0000 7fce38e24b80 1 Processor -- start 2024-06-03T02:31:08.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.344+0000 7fce38e24b80 1 -- start start 2024-06-03T02:31:08.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.345+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.345+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 0x55a8d5f64d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.345+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f65240 0x55a8d5f65680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.345+0000 7fce38e24b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5e73a10 con 0x55a8d5f6b4b0 2024-06-03T02:31:08.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.345+0000 7fce38e24b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5eb6b00 con 0x55a8d5f65240 2024-06-03T02:31:08.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.345+0000 7fce38e24b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5e718f0 con 0x55a8d5f6ab90 2024-06-03T02:31:08.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.345+0000 7fce27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.345+0000 7fce27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56324/0 (socket says 172.21.15.79:56324) 2024-06-03T02:31:08.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.345+0000 7fce27e00700 1 -- 172.21.15.79:0/1585961038 learned_addr learned my addr 172.21.15.79:0/1585961038 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:08.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.346+0000 7fce27400700 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 0x55a8d5f64d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.346+0000 7fce1fe00700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fce0c0a2d20 0x7fce0c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.346+0000 7fce1fe00700 1 -- 172.21.15.79:0/1231846003 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fce140d0350 con 0x7fce2806ab30 2024-06-03T02:31:08.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.348+0000 7fce26a00700 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f65240 0x55a8d5f65680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.348+0000 7fce27e00700 1 -- 172.21.15.79:0/1585961038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f65240 msgr2=0x55a8d5f65680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:08.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.348+0000 7fce27e00700 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f65240 0x55a8d5f65680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.348+0000 7fce27e00700 1 -- 172.21.15.79:0/1585961038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 msgr2=0x55a8d5f64d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:08.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.348+0000 7fce27e00700 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 0x55a8d5f64d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.348+0000 7fce27e00700 1 -- 172.21.15.79:0/1585961038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d5e5ef90 con 0x55a8d5f6ab90 2024-06-03T02:31:08.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.348+0000 7fce26a00700 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f65240 0x55a8d5f65680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:08.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.348+0000 7fce27e00700 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6af70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce2000f5a0 tx=0x7fce2000f8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4af479f0c512a480 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:08.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.349+0000 7fce1cc00700 1 -- 172.21.15.79:0/1585961038 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20018020 con 0x55a8d5f6ab90 2024-06-03T02:31:08.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.349+0000 7fce1cc00700 1 -- 172.21.15.79:0/1585961038 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce20014a10 con 0x55a8d5f6ab90 2024-06-03T02:31:08.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.349+0000 7fce1cc00700 1 -- 172.21.15.79:0/1585961038 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20013c80 con 0x55a8d5f6ab90 2024-06-03T02:31:08.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.350+0000 7fce38e24b80 1 -- 172.21.15.79:0/1585961038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 msgr2=0x55a8d5f6af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:08.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.350+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6af70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce2000f5a0 tx=0x7fce2000f8b0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.350+0000 7fce27e00700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fce0c0a2d20 0x7fce0c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.351+0000 7fce38e24b80 1 -- 172.21.15.79:0/1585961038 shutdown_connections 2024-06-03T02:31:08.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.351+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f65240 0x55a8d5f65680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.351+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6af70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.351+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1585961038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 0x55a8d5f64d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.351+0000 7fce38e24b80 1 -- 172.21.15.79:0/1585961038 >> 172.21.15.79:0/1585961038 conn(0x55a8d5e91170 msgr2=0x55a8d5f5fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:08.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.351+0000 7fce38e24b80 1 -- 172.21.15.79:0/1585961038 shutdown_connections 2024-06-03T02:31:08.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.351+0000 7fce38e24b80 1 -- 172.21.15.79:0/1585961038 wait complete. 2024-06-03T02:31:08.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.352+0000 7fce38e24b80 1 -- start start 2024-06-03T02:31:08.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.352+0000 7fce38e24b80 1 Processor -- start 2024-06-03T02:31:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 0x55a8d5f6ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce38e24b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5e52f60 con 0x55a8d5f6b4b0 2024-06-03T02:31:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce38e24b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5eb2ac0 con 0x55a8d5f6ab90 2024-06-03T02:31:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce38e24b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5f71e90 con 0x55a8d5f65240 2024-06-03T02:31:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce38e24b80 10 client.0 _ll_register_callbacks cb 0x55a8d5e4c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce27400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce27400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56332/0 (socket says 172.21.15.79:56332) 2024-06-03T02:31:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.353+0000 7fce27400700 1 -- 172.21.15.79:0/1182632386 learned_addr learned my addr 172.21.15.79:0/1182632386 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.355+0000 7fce27e00700 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 0x55a8d5f6ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.355+0000 7fce26a00700 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.355+0000 7fce27400700 1 -- 172.21.15.79:0/1182632386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 msgr2=0x55a8d5f6e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:08.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.356+0000 7fce27400700 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.356+0000 7fce27400700 1 -- 172.21.15.79:0/1182632386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 msgr2=0x55a8d5f6ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:08.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.356+0000 7fce27400700 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 0x55a8d5f6ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.356+0000 7fce27400700 1 -- 172.21.15.79:0/1182632386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d5e058d0 con 0x55a8d5f65240 2024-06-03T02:31:08.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.356+0000 7fce26a00700 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:31:08.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.356+0000 7fce27e00700 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 0x55a8d5f6ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:08.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.357+0000 7fce27e00700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fce0c0a2d20 0x7fce0c0a51e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fce2000a040 tx=0x7fce20020040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:08.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.357+0000 7fce27400700 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f711d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fce18011770 tx=0x7fce18011b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f83f84024313350 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:08.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.357+0000 7fce11600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:31:08.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.359+0000 7fce11600700 1 -- 172.21.15.79:0/1182632386 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18020060 con 0x55a8d5f65240 2024-06-03T02:31:08.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.359+0000 7fce38e24b80 1 -- 172.21.15.79:0/1182632386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce20014a10 con 0x55a8d5f65240 2024-06-03T02:31:08.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.360+0000 7fce11600700 1 -- 172.21.15.79:0/1182632386 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fce1800f7c0 con 0x55a8d5f65240 2024-06-03T02:31:08.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.361+0000 7fce11600700 1 -- 172.21.15.79:0/1182632386 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce18027050 con 0x55a8d5f65240 2024-06-03T02:31:08.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.361+0000 7fce38e24b80 10 client.26578 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:08.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.361+0000 7fce38e24b80 1 -- 172.21.15.79:0/1182632386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a8d5f7c640 con 0x55a8d5f65240 2024-06-03T02:31:08.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.361+0000 7fce11600700 1 -- 172.21.15.79:0/1182632386 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce20014a10 con 0x55a8d5f65240 2024-06-03T02:31:08.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.363+0000 7fce11600700 1 -- 172.21.15.79:0/1182632386 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fce18019c80 con 0x55a8d5f65240 2024-06-03T02:31:08.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.363+0000 7fce38e24b80 10 client.26578 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:08.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.363+0000 7fce38e24b80 10 client.26578 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:31:08.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.363+0000 7fce38e24b80 20 client.26578 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:08.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.363+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55a8d5f7ebd0 0x55a8d5f80fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.364+0000 7fce38e24b80 4 client.26578 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:31:08.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.364+0000 7fce38e24b80 1 -- 172.21.15.79:0/1182632386 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8d5ec4000 con 0x55a8d5f7ebd0 2024-06-03T02:31:08.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.364+0000 7fce26a00700 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55a8d5f7ebd0 0x55a8d5f80fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.365+0000 7fce26a00700 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55a8d5f7ebd0 0x55a8d5f80fe0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62f808e98845cb0 server_cookie=a12c9ee4ec378e10 in_seq=0 out_seq=0 2024-06-03T02:31:08.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.365+0000 7fce11600700 10 client.26578 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:31:08.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.366+0000 7fce11600700 1 -- 172.21.15.79:0/1182632386 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a8d5ec4000 con 0x55a8d5f7ebd0 2024-06-03T02:31:08.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce11600700 10 client.26578 handle_command_reply: tid=0 2024-06-03T02:31:08.366 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce38e24b80 1 client.26578 shutdown 2024-06-03T02:31:08.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce38e24b80 1 -- 172.21.15.79:0/1182632386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 msgr2=0x55a8d5f711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:08.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f711d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fce18011770 tx=0x7fce18011b30 comp rx=0 tx=0).stop 2024-06-03T02:31:08.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce38e24b80 1 -- 172.21.15.79:0/1182632386 shutdown_connections 2024-06-03T02:31:08.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f6b4b0 0x55a8d5f6ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55a8d5f7ebd0 0x55a8d5f80fe0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f711d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce38e24b80 1 --2- 172.21.15.79:0/1182632386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f6e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce11600700 0 client.26578 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:31:08.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.367+0000 7fce38e24b80 1 -- 172.21.15.79:0/1182632386 >> 172.21.15.79:0/1182632386 conn(0x55a8d5e91170 msgr2=0x55a8d5f59790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:08.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.368+0000 7fce38e24b80 1 -- 172.21.15.79:0/1182632386 shutdown_connections 2024-06-03T02:31:08.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.368+0000 7fce38e24b80 1 -- 172.21.15.79:0/1182632386 wait complete. 2024-06-03T02:31:08.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.368+0000 7fce38e24b80 20 client.26578 trim_cache size 0 max 16384 2024-06-03T02:31:08.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.421+0000 7fce38e24b80 1 Processor -- start 2024-06-03T02:31:08.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.421+0000 7fce38e24b80 1 -- start start 2024-06-03T02:31:08.421 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:08 smithi079 ceph-mon[31819]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 390 op/s 2024-06-03T02:31:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.424+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5e91170 0x55a8d5f64d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.424+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.424+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f5a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.424+0000 7fce38e24b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fce0800eb50 con 0x55a8d5e91170 2024-06-03T02:31:08.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.425+0000 7fce27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f5a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.425+0000 7fce27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f5a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34200/0 (socket says 172.21.15.79:34200) 2024-06-03T02:31:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.425+0000 7fce27e00700 1 -- 172.21.15.79:0/2660607074 learned_addr learned my addr 172.21.15.79:0/2660607074 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.425+0000 7fce38e24b80 1 -- 172.21.15.79:0/2660607074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce08010690 con 0x55a8d5f6ab90 2024-06-03T02:31:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.425+0000 7fce38e24b80 1 -- 172.21.15.79:0/2660607074 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fce0800f220 con 0x55a8d5f65240 2024-06-03T02:31:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.425+0000 7fce27400700 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5e91170 0x55a8d5f64d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.426+0000 7fce26a00700 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.426+0000 7fce27e00700 1 -- 172.21.15.79:0/2660607074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 msgr2=0x55a8d5f601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:08.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.426+0000 7fce27e00700 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.426+0000 7fce27e00700 1 -- 172.21.15.79:0/2660607074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5e91170 msgr2=0x55a8d5f64d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:08.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.426+0000 7fce27e00700 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5e91170 0x55a8d5f64d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.426+0000 7fce27e00700 1 -- 172.21.15.79:0/2660607074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d5f81660 con 0x55a8d5f6ab90 2024-06-03T02:31:08.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.427+0000 7fce26a00700 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:08.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.427+0000 7fce27400700 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5e91170 0x55a8d5f64d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:31:08.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.427+0000 7fce27e00700 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f5a130 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fce20011010 tx=0x7fce20014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6a9663efc4d9a5c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.428+0000 7fce13e00700 1 -- 172.21.15.79:0/2660607074 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2002f070 con 0x55a8d5f6ab90 2024-06-03T02:31:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.428+0000 7fce13e00700 1 -- 172.21.15.79:0/2660607074 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce20003cc0 con 0x55a8d5f6ab90 2024-06-03T02:31:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.428+0000 7fce13e00700 1 -- 172.21.15.79:0/2660607074 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20018040 con 0x55a8d5f6ab90 2024-06-03T02:31:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.429+0000 7fce38e24b80 1 -- 172.21.15.79:0/2660607074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 msgr2=0x55a8d5f5a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.429+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f5a130 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fce20011010 tx=0x7fce20014000 comp rx=0 tx=0).stop 2024-06-03T02:31:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.429+0000 7fce38e24b80 1 -- 172.21.15.79:0/2660607074 shutdown_connections 2024-06-03T02:31:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.429+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f5a130 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.429+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5e91170 0x55a8d5f64d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.429+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2660607074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f65240 0x55a8d5f601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.429+0000 7fce38e24b80 1 -- 172.21.15.79:0/2660607074 >> 172.21.15.79:0/2660607074 conn(0x55a8d5f71330 msgr2=0x55a8d5f5fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:08.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.430+0000 7fce38e24b80 1 -- 172.21.15.79:0/2660607074 shutdown_connections 2024-06-03T02:31:08.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.432+0000 7fce38e24b80 1 -- 172.21.15.79:0/2660607074 wait complete. 2024-06-03T02:31:08.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.432+0000 7fce38e24b80 1 -- start start 2024-06-03T02:31:08.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.432+0000 7fce38e24b80 1 Processor -- start 2024-06-03T02:31:08.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.435+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5e91170 0x55a8d5f77390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f65240 0x55a8d5f778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce27400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5e91170 0x55a8d5f77390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce38e24b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f85cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce27400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5e91170 0x55a8d5f77390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34202/0 (socket says 172.21.15.79:34202) 2024-06-03T02:31:08.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce27400700 1 -- 172.21.15.79:0/2399466484 learned_addr learned my addr 172.21.15.79:0/2399466484 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fce0800ffc0 con 0x55a8d5f65240 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce26a00700 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f65240 0x55a8d5f778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce0800f8f0 con 0x55a8d5e91170 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fce0800caf0 con 0x55a8d5f6ab90 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce38e24b80 10 client.0 _ll_register_callbacks cb 0x55a8d5e4c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.436+0000 7fce27e00700 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f85cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.437+0000 7fce27e00700 1 -- 172.21.15.79:0/2399466484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5e91170 msgr2=0x55a8d5f77390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.437+0000 7fce27e00700 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5e91170 0x55a8d5f77390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.437+0000 7fce27e00700 1 -- 172.21.15.79:0/2399466484 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f65240 msgr2=0x55a8d5f778d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.437+0000 7fce27e00700 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f65240 0x55a8d5f778d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.437+0000 7fce27e00700 1 -- 172.21.15.79:0/2399466484 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d5f86300 con 0x55a8d5f6ab90 2024-06-03T02:31:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.437+0000 7fce11600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:31:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.437+0000 7fce27e00700 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f85cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fce20011010 tx=0x7fce20014bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=987c62ffbaa005f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.438+0000 7fce11600700 1 -- 172.21.15.79:0/2399466484 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2002f070 con 0x55a8d5f6ab90 2024-06-03T02:31:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.439+0000 7fce11600700 1 -- 172.21.15.79:0/2399466484 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fce20034450 con 0x55a8d5f6ab90 2024-06-03T02:31:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.439+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce20003cc0 con 0x55a8d5f6ab90 2024-06-03T02:31:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.439+0000 7fce11600700 1 -- 172.21.15.79:0/2399466484 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce20018040 con 0x55a8d5f6ab90 2024-06-03T02:31:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.439+0000 7fce38e24b80 10 client.26584 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.439+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a8d5fe9200 con 0x55a8d5f6ab90 2024-06-03T02:31:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.440+0000 7fce11600700 1 -- 172.21.15.79:0/2399466484 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce20003cc0 con 0x55a8d5f6ab90 2024-06-03T02:31:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.440+0000 7fce11600700 1 -- 172.21.15.79:0/2399466484 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fce20025040 con 0x55a8d5f6ab90 2024-06-03T02:31:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.441+0000 7fce38e24b80 10 client.26584 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.441+0000 7fce38e24b80 10 client.26584 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:31:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.441+0000 7fce38e24b80 20 client.26584 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.441+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55a8d5f6b4b0 0x55a8d5feb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.441+0000 7fce38e24b80 4 client.26584 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:31:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.441+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8d5ebe580 con 0x55a8d5f6b4b0 2024-06-03T02:31:08.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.447+0000 7fce27400700 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55a8d5f6b4b0 0x55a8d5feb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:08.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.451+0000 7fce11600700 10 client.26584 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:31:08.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.451+0000 7fce27400700 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55a8d5f6b4b0 0x55a8d5feb550 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2509464373a3834d server_cookie=1b9327b345dab253 in_seq=0 out_seq=0 2024-06-03T02:31:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.453+0000 7fce11600700 1 -- 172.21.15.79:0/2399466484 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8d5ebe580 con 0x55a8d5f6b4b0 2024-06-03T02:31:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.453+0000 7fce11600700 10 client.26584 handle_command_reply: tid=0 2024-06-03T02:31:08.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.453+0000 7fce38e24b80 1 client.26584 shutdown 2024-06-03T02:31:08.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.454+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 msgr2=0x55a8d5f85cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:08.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.454+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f85cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fce20011010 tx=0x7fce20014bb0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.454+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 shutdown_connections 2024-06-03T02:31:08.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.454+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55a8d5f6b4b0 0x55a8d5feb550 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.454+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8d5f6ab90 0x55a8d5f85cc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.454+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d5e91170 0x55a8d5f77390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.454+0000 7fce38e24b80 1 --2- 172.21.15.79:0/2399466484 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a8d5f65240 0x55a8d5f778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.454+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 >> 172.21.15.79:0/2399466484 conn(0x55a8d5f71330 msgr2=0x55a8d5f6b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:08.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.457+0000 7fce11600700 0 client.26584 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:31:08.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.457+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 shutdown_connections 2024-06-03T02:31:08.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.457+0000 7fce38e24b80 1 -- 172.21.15.79:0/2399466484 wait complete. 2024-06-03T02:31:08.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.457+0000 7fce38e24b80 20 client.26584 trim_cache size 0 max 16384 2024-06-03T02:31:08.457 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:31:08.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.458+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fce0c0a2d20 msgr2=0x7fce0c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:08.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.458+0000 7fce2e400700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fce0c0a2d20 0x7fce0c0a51e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fce2000a040 tx=0x7fce20020040 comp rx=0 tx=0).stop 2024-06-03T02:31:08.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.458+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806ab30 msgr2=0x7fce28116f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:08.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.458+0000 7fce2e400700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806ab30 0x7fce28116f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce14005910 tx=0x7fce1400bab0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.459+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 shutdown_connections 2024-06-03T02:31:08.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.459+0000 7fce2e400700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce28108500 0x7fce28111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.459+0000 7fce2e400700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fce0c0a2d20 0x7fce0c0a51e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.459+0000 7fce2e400700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce2806a170 0x7fce281169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.459+0000 7fce2e400700 1 --2- 172.21.15.79:0/1231846003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce2806ab30 0x7fce28116f20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:08.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.459+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 >> 172.21.15.79:0/1231846003 conn(0x7fce28078340 msgr2=0x7fce280fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:08.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.459+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 shutdown_connections 2024-06-03T02:31:08.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:08.459+0000 7fce2e400700 1 -- 172.21.15.79:0/1231846003 wait complete. 2024-06-03T02:31:08.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:08 smithi123 ceph-mon[36640]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 390 op/s 2024-06-03T02:31:08.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:08 smithi135 ceph-mon[36653]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 390 op/s 2024-06-03T02:31:09.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:31:10.341 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:10 smithi079 ceph-mon[31819]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.9 MiB/s wr, 503 op/s 2024-06-03T02:31:10.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:10 smithi123 ceph-mon[36640]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.9 MiB/s wr, 503 op/s 2024-06-03T02:31:10.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:10 smithi135 ceph-mon[36653]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.9 MiB/s wr, 503 op/s 2024-06-03T02:31:10.658 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:31:12.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:12 smithi123 ceph-mon[36640]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 467 op/s 2024-06-03T02:31:12.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:12 smithi079 ceph-mon[31819]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 467 op/s 2024-06-03T02:31:12.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:12 smithi135 ceph-mon[36653]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 467 op/s 2024-06-03T02:31:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 665.0 s, 438.0 tps, lat 220.534 ms stddev 285.850 2024-06-03T02:31:14.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:14 smithi123 ceph-mon[36640]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 462 op/s 2024-06-03T02:31:14.530 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:14 smithi079 ceph-mon[31819]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 462 op/s 2024-06-03T02:31:14.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:14 smithi135 ceph-mon[36653]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 462 op/s 2024-06-03T02:31:16.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:16 smithi123 ceph-mon[36640]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.9 MiB/s wr, 540 op/s 2024-06-03T02:31:16.518 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:31:16.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.229+0000 7f66b0800700 1 -- 172.21.15.79:0/856293592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a806a6e0 msgr2=0x7f66a806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.232+0000 7f66b0800700 1 --2- 172.21.15.79:0/856293592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a806a6e0 0x7f66a806ab40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6698009990 tx=0x7f6698009ca0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.232+0000 7f66b0800700 1 -- 172.21.15.79:0/856293592 shutdown_connections 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.232+0000 7f66b0800700 1 --2- 172.21.15.79:0/856293592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a8104ed0 0x7f66a8105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.232+0000 7f66b0800700 1 --2- 172.21.15.79:0/856293592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a81073c0 0x7f66a806a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.232+0000 7f66b0800700 1 --2- 172.21.15.79:0/856293592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a806a6e0 0x7f66a806ab40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.232+0000 7f66b0800700 1 -- 172.21.15.79:0/856293592 >> 172.21.15.79:0/856293592 conn(0x7f66a8078340 msgr2=0x7f66a8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.233+0000 7f66b0800700 1 -- 172.21.15.79:0/856293592 shutdown_connections 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.233+0000 7f66b0800700 1 -- 172.21.15.79:0/856293592 wait complete. 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.236+0000 7f66b0800700 1 Processor -- start 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.236+0000 7f66b0800700 1 -- start start 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.238+0000 7f66b0800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a806a6e0 0x7f66a819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.238+0000 7f66b0800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a8104ed0 0x7f66a819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.238+0000 7f66b0800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a81073c0 0x7f66a81a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.238+0000 7f66b0800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8118e10 con 0x7f66a806a6e0 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.238+0000 7f66b0800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8118c90 con 0x7f66a8104ed0 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.238+0000 7f66b0800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8118f90 con 0x7f66a81073c0 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.239+0000 7f66ad400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a8104ed0 0x7f66a819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.239+0000 7f66ad400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a8104ed0 0x7f66a819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:51290/0 (socket says 172.21.15.79:51290) 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.239+0000 7f66ad400700 1 -- 172.21.15.79:0/4220861250 learned_addr learned my addr 172.21.15.79:0/4220861250 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.241+0000 7f66ad400700 1 -- 172.21.15.79:0/4220861250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a81073c0 msgr2=0x7f66a81a3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.241+0000 7f66ad400700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a81073c0 0x7f66a81a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.242+0000 7f66ad400700 1 -- 172.21.15.79:0/4220861250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a806a6e0 msgr2=0x7f66a819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.242+0000 7f66ad400700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a806a6e0 0x7f66a819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.242+0000 7f66ad400700 1 -- 172.21.15.79:0/4220861250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a81a42d0 con 0x7f66a8104ed0 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.243+0000 7f66ad400700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a8104ed0 0x7f66a819f800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f669800b2e0 tx=0x7f669800b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:16.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.248+0000 7f66a2a00700 1 -- 172.21.15.79:0/4220861250 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669800f990 con 0x7f66a8104ed0 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.248+0000 7f66a2a00700 1 -- 172.21.15.79:0/4220861250 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6698005510 con 0x7f66a8104ed0 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.248+0000 7f66a2a00700 1 -- 172.21.15.79:0/4220861250 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698004e20 con 0x7f66a8104ed0 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.249+0000 7f66b0800700 1 -- 172.21.15.79:0/4220861250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a81a4560 con 0x7f66a8104ed0 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.249+0000 7f66b0800700 1 -- 172.21.15.79:0/4220861250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a81a4990 con 0x7f66a8104ed0 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.253+0000 7f66a2a00700 1 -- 172.21.15.79:0/4220861250 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6698009d10 con 0x7f66a8104ed0 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.256+0000 7f66bb250b80 1 Processor -- start 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.260+0000 7f66a2a00700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f66900a2cf0 0x7f66900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.260+0000 7f66a2a00700 1 -- 172.21.15.79:0/4220861250 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f66980cffb0 con 0x7f66a8104ed0 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.261+0000 7f66ae800700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f66900a2cf0 0x7f66900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.262+0000 7f66bb250b80 1 -- start start 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.262+0000 7f66ae800700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f66900a2cf0 0x7f66900a51b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f66a400e350 tx=0x7f66a400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251125b90 0x55c251125f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2511264b0 0x55c25111fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c251120240 0x55c251120680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66bb250b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c25102ea10 con 0x55c251120240 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66bb250b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c251071b00 con 0x55c2511264b0 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66ad400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2511264b0 0x55c25111fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66ade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251125b90 0x55c251125f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66ade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251125b90 0x55c251125f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46488/0 (socket says 172.21.15.79:46488) 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66ade00700 1 -- 172.21.15.79:0/3949131541 learned_addr learned my addr 172.21.15.79:0/3949131541 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.263+0000 7f66bb250b80 1 -- 172.21.15.79:0/3949131541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c25102c8f0 con 0x55c251125b90 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.264+0000 7f66ad400700 1 -- 172.21.15.79:0/3949131541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251125b90 msgr2=0x55c251125f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.264+0000 7f66ad400700 1 --2- 172.21.15.79:0/3949131541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251125b90 0x55c251125f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.264+0000 7f66ad400700 1 -- 172.21.15.79:0/3949131541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c251120240 msgr2=0x55c251120680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.264+0000 7f66ad400700 1 --2- 172.21.15.79:0/3949131541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c251120240 0x55c251120680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.264+0000 7f66ad400700 1 -- 172.21.15.79:0/3949131541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c251019f90 con 0x55c2511264b0 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.264+0000 7f66ade00700 1 --2- 172.21.15.79:0/3949131541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251125b90 0x55c251125f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.265+0000 7f66ad400700 1 --2- 172.21.15.79:0/3949131541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2511264b0 0x55c25111fd00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f66980cd700 tx=0x7f6698009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ced8f854d1e2299 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.265+0000 7f6697400700 1 -- 172.21.15.79:0/3949131541 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669809cbf0 con 0x55c2511264b0 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.265+0000 7f6697400700 1 -- 172.21.15.79:0/3949131541 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669809c4f0 con 0x55c2511264b0 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.265+0000 7f6697400700 1 -- 172.21.15.79:0/3949131541 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66980a4c40 con 0x55c2511264b0 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.266+0000 7f66bb250b80 1 -- 172.21.15.79:0/3949131541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2511264b0 msgr2=0x55c25111fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.266+0000 7f66bb250b80 1 --2- 172.21.15.79:0/3949131541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2511264b0 0x55c25111fd00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f66980cd700 tx=0x7f6698009ca0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.266+0000 7f66bb250b80 1 -- 172.21.15.79:0/3949131541 shutdown_connections 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.266+0000 7f66bb250b80 1 --2- 172.21.15.79:0/3949131541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c251120240 0x55c251120680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.266+0000 7f66bb250b80 1 --2- 172.21.15.79:0/3949131541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251125b90 0x55c251125f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.266+0000 7f66bb250b80 1 --2- 172.21.15.79:0/3949131541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2511264b0 0x55c25111fd00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.266+0000 7f66bb250b80 1 -- 172.21.15.79:0/3949131541 >> 172.21.15.79:0/3949131541 conn(0x55c25104c170 msgr2=0x55c25111aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:16.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.267+0000 7f66bb250b80 1 -- 172.21.15.79:0/3949131541 shutdown_connections 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.267+0000 7f66bb250b80 1 -- 172.21.15.79:0/3949131541 wait complete. 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.267+0000 7f66bb250b80 1 -- start start 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.268+0000 7f66bb250b80 1 Processor -- start 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.268+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c251120240 0x55c25112c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.268+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251125b90 0x55c251129610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.268+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 0x55c251129b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.271+0000 7f66ad400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251125b90 0x55c251129610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.271+0000 7f66ad400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251125b90 0x55c251129610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:51308/0 (socket says 172.21.15.79:51308) 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.271+0000 7f66ae800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 0x55c251129b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.271+0000 7f66ae800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 0x55c251129b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46490/0 (socket says 172.21.15.79:46490) 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.271+0000 7f66ad400700 1 -- 172.21.15.79:0/345867508 learned_addr learned my addr 172.21.15.79:0/345867508 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.271+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c25100df60 con 0x55c251120240 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.271+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c25106dac0 con 0x55c251125b90 2024-06-03T02:31:16.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.271+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c25112cc40 con 0x55c2511264b0 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.272+0000 7f66bb250b80 10 client.0 _ll_register_callbacks cb 0x55c251007380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.272+0000 7f66ad400700 1 -- 172.21.15.79:0/345867508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 msgr2=0x55c251129b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.272+0000 7f66ad400700 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 0x55c251129b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.272+0000 7f66ad400700 1 -- 172.21.15.79:0/345867508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c251120240 msgr2=0x55c25112c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.272+0000 7f66ad400700 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c251120240 0x55c25112c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.272+0000 7f66ad400700 1 -- 172.21.15.79:0/345867508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c250fc08d0 con 0x55c251125b90 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.273+0000 7f668be00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.273+0000 7f66ad400700 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251125b90 0x55c251129610 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f66980a4ba0 tx=0x7f6698004820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb2eef5e4011ac13 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.273+0000 7f668be00700 1 -- 172.21.15.79:0/345867508 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66980b4070 con 0x55c251125b90 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.273+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669809c4f0 con 0x55c251125b90 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.273+0000 7f668be00700 1 -- 172.21.15.79:0/345867508 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f669800fb30 con 0x55c251125b90 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.274+0000 7f668be00700 1 -- 172.21.15.79:0/345867508 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f66980b4070 con 0x55c251125b90 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.274+0000 7f66bb250b80 10 client.26594 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:16.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.274+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2511373b0 con 0x55c251125b90 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.274+0000 7f668be00700 1 -- 172.21.15.79:0/345867508 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669809c4f0 con 0x55c251125b90 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.275+0000 7f668be00700 1 -- 172.21.15.79:0/345867508 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f66980cfc80 con 0x55c251125b90 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.275+0000 7f66bb250b80 10 client.26594 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.275+0000 7f66bb250b80 10 client.26594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.275+0000 7f66bb250b80 20 client.26594 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.275+0000 7f66bb250b80 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c251139880 0x55c25113bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.275+0000 7f66bb250b80 4 client.26594 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.275+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c25107f000 con 0x55c251139880 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.276+0000 7f66ade00700 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c251139880 0x55c25113bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.276+0000 7f66ade00700 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c251139880 0x55c25113bc70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb19a05cb2962f6 server_cookie=8aae148faad9b9c6 in_seq=0 out_seq=0 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.277+0000 7f668be00700 10 client.26594 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.277+0000 7f668be00700 1 -- 172.21.15.79:0/345867508 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c25107f000 con 0x55c251139880 2024-06-03T02:31:16.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.277+0000 7f668be00700 10 client.26594 handle_command_reply: tid=0 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.277+0000 7f66bb250b80 1 client.26594 shutdown 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.278+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251125b90 msgr2=0x55c251129610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.278+0000 7f66bb250b80 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251125b90 0x55c251129610 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f66980a4ba0 tx=0x7f6698004820 comp rx=0 tx=0).stop 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.278+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 shutdown_connections 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.278+0000 7f66bb250b80 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 0x55c251129b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.278+0000 7f66bb250b80 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c251139880 0x55c25113bc70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.278+0000 7f66bb250b80 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c251120240 0x55c25112c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.278+0000 7f66bb250b80 1 --2- 172.21.15.79:0/345867508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251125b90 0x55c251129610 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f66980a4ba0 tx=0x7f6698004820 comp rx=0 tx=0).stop 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.278+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 >> 172.21.15.79:0/345867508 conn(0x55c25104c170 msgr2=0x55c251114790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.278+0000 7f668be00700 0 client.26594 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.280+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 shutdown_connections 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.280+0000 7f66bb250b80 1 -- 172.21.15.79:0/345867508 wait complete. 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.280+0000 7f66bb250b80 20 client.26594 trim_cache size 0 max 16384 2024-06-03T02:31:16.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.322+0000 7f66bb250b80 1 Processor -- start 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.322+0000 7f66bb250b80 1 -- start start 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.322+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251120240 0x55c25111fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.322+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 0x55c251115130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.323+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c25111b100 0x55c25112bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.323+0000 7f66bb250b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81b4be0 con 0x55c25111b100 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.323+0000 7f66bb250b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81b6720 con 0x55c251120240 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.323+0000 7f66bb250b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81b52b0 con 0x55c2511264b0 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.323+0000 7f66ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251120240 0x55c25111fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.323+0000 7f66ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251120240 0x55c25111fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:51318/0 (socket says 172.21.15.79:51318) 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.323+0000 7f66ade00700 1 -- 172.21.15.79:0/564960023 learned_addr learned my addr 172.21.15.79:0/564960023 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.324+0000 7f66ae800700 1 --2- 172.21.15.79:0/564960023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c25111b100 0x55c25112bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.324+0000 7f66ade00700 1 -- 172.21.15.79:0/564960023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 msgr2=0x55c251115130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:16.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.324+0000 7f66ade00700 1 --2- 172.21.15.79:0/564960023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 0x55c251115130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.324+0000 7f66ade00700 1 -- 172.21.15.79:0/564960023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c25111b100 msgr2=0x55c25112bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.324+0000 7f66ade00700 1 --2- 172.21.15.79:0/564960023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c25111b100 0x55c25112bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.324+0000 7f66ade00700 1 -- 172.21.15.79:0/564960023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c25112a550 con 0x55c251120240 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.324+0000 7f66ae800700 1 --2- 172.21.15.79:0/564960023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c25111b100 0x55c25112bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.325+0000 7f66ade00700 1 --2- 172.21.15.79:0/564960023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251120240 0x55c25111fd00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f669c009d90 tx=0x7f669c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661443e8609d0a03 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.325+0000 7f6696a00700 1 -- 172.21.15.79:0/564960023 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c012110 con 0x55c251120240 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.325+0000 7f6696a00700 1 -- 172.21.15.79:0/564960023 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c004510 con 0x55c251120240 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.326+0000 7f66bb250b80 1 -- 172.21.15.79:0/564960023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251120240 msgr2=0x55c25111fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.326+0000 7f66bb250b80 1 --2- 172.21.15.79:0/564960023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251120240 0x55c25111fd00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f669c009d90 tx=0x7f669c009840 comp rx=0 tx=0).stop 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.327+0000 7f66bb250b80 1 -- 172.21.15.79:0/564960023 shutdown_connections 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.327+0000 7f66bb250b80 1 --2- 172.21.15.79:0/564960023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c25111b100 0x55c25112bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.327+0000 7f66bb250b80 1 --2- 172.21.15.79:0/564960023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c251120240 0x55c25111fd00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.327+0000 7f66bb250b80 1 --2- 172.21.15.79:0/564960023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2511264b0 0x55c251115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.327+0000 7f66bb250b80 1 -- 172.21.15.79:0/564960023 >> 172.21.15.79:0/564960023 conn(0x55c25104c170 msgr2=0x55c251129750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.327+0000 7f66bb250b80 1 -- 172.21.15.79:0/564960023 shutdown_connections 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.328+0000 7f66bb250b80 1 -- 172.21.15.79:0/564960023 wait complete. 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.328+0000 7f66bb250b80 1 -- start start 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.328+0000 7f66bb250b80 1 Processor -- start 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.329+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c25111b100 0x55c251140f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.329+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251120240 0x55c2511414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.329+0000 7f66bb250b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2511264b0 0x55c2511419f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.330+0000 7f66ad400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251120240 0x55c2511414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.330+0000 7f66ad400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251120240 0x55c2511414b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46510/0 (socket says 172.21.15.79:46510) 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.330+0000 7f66ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c25111b100 0x55c251140f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.330+0000 7f66ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c25111b100 0x55c251140f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:51326/0 (socket says 172.21.15.79:51326) 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.330+0000 7f66bb250b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81b6050 con 0x55c2511264b0 2024-06-03T02:31:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.330+0000 7f66bb250b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81b5980 con 0x55c25111b100 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.330+0000 7f66bb250b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81b2b20 con 0x55c251120240 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.331+0000 7f66ae800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2511264b0 0x55c2511419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.331+0000 7f66ae800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2511264b0 0x55c2511419f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54786/0 (socket says 172.21.15.79:54786) 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.331+0000 7f66ae800700 1 -- 172.21.15.79:0/2017372804 learned_addr learned my addr 172.21.15.79:0/2017372804 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.331+0000 7f66bb250b80 10 client.0 _ll_register_callbacks cb 0x55c251007380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.332+0000 7f66ae800700 1 -- 172.21.15.79:0/2017372804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251120240 msgr2=0x55c2511414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.332+0000 7f66ae800700 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251120240 0x55c2511414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.332+0000 7f66ae800700 1 -- 172.21.15.79:0/2017372804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c25111b100 msgr2=0x55c251140f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.332+0000 7f66ae800700 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c25111b100 0x55c251140f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.332+0000 7f66ae800700 1 -- 172.21.15.79:0/2017372804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2511a3350 con 0x55c2511264b0 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.332+0000 7f66ad400700 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251120240 0x55c2511414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.333+0000 7f668be00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:31:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.333+0000 7f66ae800700 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2511264b0 0x55c2511419f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f66a4015df0 tx=0x7f66a4015e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a83318ecc54676 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.334+0000 7f668be00700 1 -- 172.21.15.79:0/2017372804 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4026030 con 0x55c2511264b0 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.335+0000 7f668be00700 1 -- 172.21.15.79:0/2017372804 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f66a4004bb0 con 0x55c2511264b0 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.335+0000 7f66bb250b80 1 -- 172.21.15.79:0/2017372804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669c004510 con 0x55c2511264b0 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.335+0000 7f668be00700 1 -- 172.21.15.79:0/2017372804 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f66a4030050 con 0x55c2511264b0 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.335+0000 7f66bb250b80 10 client.16095 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.335+0000 7f66bb250b80 1 -- 172.21.15.79:0/2017372804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2511a45d0 con 0x55c2511264b0 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.336+0000 7f668be00700 1 -- 172.21.15.79:0/2017372804 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c004510 con 0x55c2511264b0 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.336+0000 7f668be00700 1 -- 172.21.15.79:0/2017372804 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f66a402b960 con 0x55c2511264b0 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.336+0000 7f66bb250b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.336+0000 7f66bb250b80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.336+0000 7f66bb250b80 20 client.16095 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.336+0000 7f66bb250b80 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c2511a4d60 0x55c2511a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.336+0000 7f66bb250b80 4 client.16095 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:31:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.336+0000 7f66bb250b80 1 -- 172.21.15.79:0/2017372804 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c251079580 con 0x55c2511a4d60 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.337+0000 7f66ade00700 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c2511a4d60 0x55c2511a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.342+0000 7f66ade00700 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c2511a4d60 0x55c2511a71a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5583a67a163feb94 server_cookie=2320e008fa2604ef in_seq=0 out_seq=0 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.342+0000 7f668be00700 10 client.16095 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.342+0000 7f668be00700 1 -- 172.21.15.79:0/2017372804 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c251079580 con 0x55c2511a4d60 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.342+0000 7f668be00700 10 client.16095 handle_command_reply: tid=0 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.345+0000 7f66bb250b80 1 client.16095 shutdown 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.346+0000 7f66bb250b80 1 -- 172.21.15.79:0/2017372804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2511264b0 msgr2=0x55c2511419f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.346+0000 7f66bb250b80 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2511264b0 0x55c2511419f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f66a4015df0 tx=0x7f66a4015e20 comp rx=0 tx=0).stop 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.346+0000 7f66bb250b80 1 -- 172.21.15.79:0/2017372804 shutdown_connections 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.346+0000 7f66bb250b80 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55c2511a4d60 0x55c2511a71a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.346+0000 7f66bb250b80 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2511264b0 0x55c2511419f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.346+0000 7f66bb250b80 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c25111b100 0x55c251140f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.346+0000 7f66bb250b80 1 --2- 172.21.15.79:0/2017372804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c251120240 0x55c2511414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.346+0000 7f66bb250b80 1 -- 172.21.15.79:0/2017372804 >> 172.21.15.79:0/2017372804 conn(0x55c25104c170 msgr2=0x55c251128ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.346+0000 7f668be00700 0 client.16095 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.347+0000 7f66bb250b80 1 -- 172.21.15.79:0/2017372804 shutdown_connections 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.347+0000 7f66bb250b80 1 -- 172.21.15.79:0/2017372804 wait complete. 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.347+0000 7f66bb250b80 20 client.16095 trim_cache size 0 max 16384 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 -- 172.21.15.79:0/4220861250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f66900a2cf0 msgr2=0x7f66900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f66900a2cf0 0x7f66900a51b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f66a400e350 tx=0x7f66a400b040 comp rx=0 tx=0).stop 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 -- 172.21.15.79:0/4220861250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a8104ed0 msgr2=0x7f66a819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a8104ed0 0x7f66a819f800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f669800b2e0 tx=0x7f669800b730 comp rx=0 tx=0).stop 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 -- 172.21.15.79:0/4220861250 shutdown_connections 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a81073c0 0x7f66a81a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f66900a2cf0 0x7f66900a51b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66a806a6e0 0x7f66a819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 --2- 172.21.15.79:0/4220861250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a8104ed0 0x7f66a819f800 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:16.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.348+0000 7f66b0800700 1 -- 172.21.15.79:0/4220861250 >> 172.21.15.79:0/4220861250 conn(0x7f66a8078340 msgr2=0x7f66a8103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:16.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.349+0000 7f66b0800700 1 -- 172.21.15.79:0/4220861250 shutdown_connections 2024-06-03T02:31:16.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:16.349+0000 7f66b0800700 1 -- 172.21.15.79:0/4220861250 wait complete. 2024-06-03T02:31:16.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:16 smithi079 ceph-mon[31819]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.9 MiB/s wr, 540 op/s 2024-06-03T02:31:16.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:16 smithi135 ceph-mon[36653]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.9 MiB/s wr, 540 op/s 2024-06-03T02:31:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 670.0 s, 555.4 tps, lat 181.012 ms stddev 236.511 2024-06-03T02:31:18.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:18 smithi123 ceph-mon[36640]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 447 op/s 2024-06-03T02:31:18.496 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:18 smithi079 ceph-mon[31819]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 447 op/s 2024-06-03T02:31:18.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:18 smithi135 ceph-mon[36653]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 447 op/s 2024-06-03T02:31:19.572 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:31:19.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:31:20.238 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:31:20.472 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:20 smithi079 ceph-mon[31819]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.4 MiB/s wr, 576 op/s 2024-06-03T02:31:20.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:20 smithi123 ceph-mon[36640]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.4 MiB/s wr, 576 op/s 2024-06-03T02:31:20.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:20 smithi135 ceph-mon[36653]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.4 MiB/s wr, 576 op/s 2024-06-03T02:31:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:22 smithi123 ceph-mon[36640]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 502 op/s 2024-06-03T02:31:22.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:22 smithi079 ceph-mon[31819]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 502 op/s 2024-06-03T02:31:22.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:22 smithi135 ceph-mon[36653]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 502 op/s 2024-06-03T02:31:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 675.0 s, 482.8 tps, lat 215.159 ms stddev 282.409 2024-06-03T02:31:24.210 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:24 smithi079 ceph-mon[31819]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 491 op/s 2024-06-03T02:31:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:24 smithi123 ceph-mon[36640]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 491 op/s 2024-06-03T02:31:24.518 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:31:24.518 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:31:24.518 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "5e9e8440-2816-4eda-92d2-f2815fa8ca5e", 2024-06-03T02:31:24.518 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:31:24.519 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:31:24.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.266+0000 7f2096a00700 1 -- 172.21.15.79:0/2380502265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20901155e0 msgr2=0x7f20901179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:24.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.266+0000 7f2096a00700 1 --2- 172.21.15.79:0/2380502265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20901155e0 0x7f20901179d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f208800b0a0 tx=0x7f208800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.268+0000 7f2096a00700 1 -- 172.21.15.79:0/2380502265 shutdown_connections 2024-06-03T02:31:24.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.268+0000 7f2096a00700 1 --2- 172.21.15.79:0/2380502265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20901155e0 0x7f20901179d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.268+0000 7f2096a00700 1 --2- 172.21.15.79:0/2380502265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2090109490 0x7f2090109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.268+0000 7f2096a00700 1 --2- 172.21.15.79:0/2380502265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090109db0 0x7f20901150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.268+0000 7f2096a00700 1 -- 172.21.15.79:0/2380502265 >> 172.21.15.79:0/2380502265 conn(0x7f209006d600 msgr2=0x7f209006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:24.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.268+0000 7f2096a00700 1 -- 172.21.15.79:0/2380502265 shutdown_connections 2024-06-03T02:31:24.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.269+0000 7f2096a00700 1 -- 172.21.15.79:0/2380502265 wait complete. 2024-06-03T02:31:24.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.270+0000 7f2096a00700 1 Processor -- start 2024-06-03T02:31:24.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.271+0000 7f2096a00700 1 -- start start 2024-06-03T02:31:24.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.274+0000 7f2096a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090109490 0x7f20901a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.274+0000 7f2096a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090109db0 0x7f20901a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.275+0000 7f208fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090109490 0x7f20901a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.275+0000 7f208fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090109490 0x7f20901a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46536/0 (socket says 172.21.15.79:46536) 2024-06-03T02:31:24.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.275+0000 7f208fe00700 1 -- 172.21.15.79:0/3001536693 learned_addr learned my addr 172.21.15.79:0/3001536693 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:24.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.275+0000 7f2096a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20901155e0 0x7f20901a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.276+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f209011aff0 con 0x7f20901155e0 2024-06-03T02:31:24.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.276+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f209011ae70 con 0x7f2090109db0 2024-06-03T02:31:24.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.275+0000 7f208f400700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090109db0 0x7f20901a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.276+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f209011b170 con 0x7f2090109490 2024-06-03T02:31:24.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.276+0000 7f2094a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20901155e0 0x7f20901a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.277+0000 7f208fe00700 1 -- 172.21.15.79:0/3001536693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090109db0 msgr2=0x7f20901a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:24.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.277+0000 7f208fe00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090109db0 0x7f20901a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.277+0000 7f208fe00700 1 -- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20901155e0 msgr2=0x7f20901a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:24.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.277+0000 7f208fe00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20901155e0 0x7f20901a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.277+0000 7f208fe00700 1 -- 172.21.15.79:0/3001536693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901a6380 con 0x7f2090109490 2024-06-03T02:31:24.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.277+0000 7f2094a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20901155e0 0x7f20901a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:31:24.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.277+0000 7f208fe00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090109490 0x7f20901a13c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f207c00c9b0 tx=0x7f207c00cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:24.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.277+0000 7f208cc00700 1 -- 172.21.15.79:0/3001536693 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c007b00 con 0x7f2090109490 2024-06-03T02:31:24.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.278+0000 7f208cc00700 1 -- 172.21.15.79:0/3001536693 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c007ca0 con 0x7f2090109490 2024-06-03T02:31:24.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.278+0000 7f208cc00700 1 -- 172.21.15.79:0/3001536693 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c00e750 con 0x7f2090109490 2024-06-03T02:31:24.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.278+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20901a6670 con 0x7f2090109490 2024-06-03T02:31:24.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.279+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209010ae30 con 0x7f2090109490 2024-06-03T02:31:24.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.295+0000 7f20a1336b80 1 Processor -- start 2024-06-03T02:31:24.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.295+0000 7f208cc00700 1 -- 172.21.15.79:0/3001536693 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f207c00f450 con 0x7f2090109490 2024-06-03T02:31:24.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.295+0000 7f20a1336b80 1 -- start start 2024-06-03T02:31:24.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.296+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faa70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.296+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 0x56365faa7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.296+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fabcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.296+0000 7f20a1336b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56365f9b6a90 con 0x56365fa23ec0 2024-06-03T02:31:24.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.296+0000 7f20a1336b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56365f9f9b80 con 0x56365faa7630 2024-06-03T02:31:24.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.296+0000 7f20a1336b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56365f9b4970 con 0x56365fabab90 2024-06-03T02:31:24.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.297+0000 7f208f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fabcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.297+0000 7f208f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fabcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46544/0 (socket says 172.21.15.79:46544) 2024-06-03T02:31:24.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.297+0000 7f208f400700 1 -- 172.21.15.79:0/2969037834 learned_addr learned my addr 172.21.15.79:0/2969037834 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:24.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.297+0000 7f2094a00700 1 --2- 172.21.15.79:0/2969037834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faa70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.297+0000 7f208f400700 1 -- 172.21.15.79:0/2969037834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 msgr2=0x56365faa7ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:31:24.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.297+0000 7f208f400700 1 --2- 172.21.15.79:0/2969037834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 0x56365faa7ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.297+0000 7f208f400700 1 -- 172.21.15.79:0/2969037834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 msgr2=0x56365faa70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:24.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.297+0000 7f208f400700 1 --2- 172.21.15.79:0/2969037834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faa70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.297+0000 7f208f400700 1 -- 172.21.15.79:0/2969037834 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365f9a2010 con 0x56365fabab90 2024-06-03T02:31:24.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.299+0000 7f208cc00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f20740a2cb0 0x7f20740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.299+0000 7f208cc00700 1 -- 172.21.15.79:0/3001536693 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f207c0ceac0 con 0x7f2090109490 2024-06-03T02:31:24.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.301+0000 7f2094a00700 1 --2- 172.21.15.79:0/2969037834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faa70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:24.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.301+0000 7f208f400700 1 --2- 172.21.15.79:0/2969037834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fabcf80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f208000cbd0 tx=0x7f208000cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea50c1a28e71e2f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:24.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.302+0000 7f2085600700 1 -- 172.21.15.79:0/2969037834 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208000ed70 con 0x56365fabab90 2024-06-03T02:31:24.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.302+0000 7f2085600700 1 -- 172.21.15.79:0/2969037834 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2080010910 con 0x56365fabab90 2024-06-03T02:31:24.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.302+0000 7f2085600700 1 -- 172.21.15.79:0/2969037834 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208001aa60 con 0x56365fabab90 2024-06-03T02:31:24.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.302+0000 7f2094a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f20740a2cb0 0x7f20740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.303+0000 7f2094a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f20740a2cb0 0x7f20740a5170 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f208800b830 tx=0x7f2088005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:24.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.304+0000 7f20a1336b80 1 -- 172.21.15.79:0/2969037834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 msgr2=0x56365fabcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:24.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.305+0000 7f20a1336b80 1 --2- 172.21.15.79:0/2969037834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fabcf80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f208000cbd0 tx=0x7f208000cee0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.306+0000 7f20a1336b80 1 -- 172.21.15.79:0/2969037834 shutdown_connections 2024-06-03T02:31:24.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.306+0000 7f20a1336b80 1 --2- 172.21.15.79:0/2969037834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fabcf80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.306+0000 7f20a1336b80 1 --2- 172.21.15.79:0/2969037834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faa70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.306+0000 7f20a1336b80 1 --2- 172.21.15.79:0/2969037834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 0x56365faa7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.306+0000 7f20a1336b80 1 -- 172.21.15.79:0/2969037834 >> 172.21.15.79:0/2969037834 conn(0x56365f9d41f0 msgr2=0x56365faa3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:24.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.306+0000 7f20a1336b80 1 -- 172.21.15.79:0/2969037834 shutdown_connections 2024-06-03T02:31:24.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.306+0000 7f20a1336b80 1 -- 172.21.15.79:0/2969037834 wait complete. 2024-06-03T02:31:24.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.307+0000 7f20a1336b80 1 -- start start 2024-06-03T02:31:24.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.308+0000 7f20a1336b80 1 Processor -- start 2024-06-03T02:31:24.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.310+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.311+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 0x56365fab75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.311+0000 7f208fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.311+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fab7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.311+0000 7f208fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faba2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54830/0 (socket says 172.21.15.79:54830) 2024-06-03T02:31:24.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.311+0000 7f208fe00700 1 -- 172.21.15.79:0/1267743497 learned_addr learned my addr 172.21.15.79:0/1267743497 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:24.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.311+0000 7f2094a00700 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fab7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56365f995fe0 con 0x56365fa23ec0 2024-06-03T02:31:24.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56365f9f5b40 con 0x56365faa7630 2024-06-03T02:31:24.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56365fabd850 con 0x56365fabab90 2024-06-03T02:31:24.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f208f400700 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 0x56365fab75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f20a1336b80 10 client.0 _ll_register_callbacks cb 0x56365f9937a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:24.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f208f400700 1 -- 172.21.15.79:0/1267743497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 msgr2=0x56365fab7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:24.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f208f400700 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fab7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f208f400700 1 -- 172.21.15.79:0/1267743497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 msgr2=0x56365faba2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:24.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f208f400700 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faba2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.312+0000 7f208f400700 1 -- 172.21.15.79:0/1267743497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365f948950 con 0x56365faa7630 2024-06-03T02:31:24.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.313+0000 7f208f400700 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 0x56365fab75d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2080005950 tx=0x7f2080010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b13e91e040becf65 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:24.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.313+0000 7f207a000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:31:24.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.313+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2080010910 con 0x56365faa7630 2024-06-03T02:31:24.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.320+0000 7f207a000700 1 -- 172.21.15.79:0/1267743497 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080004020 con 0x56365faa7630 2024-06-03T02:31:24.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.320+0000 7f207a000700 1 -- 172.21.15.79:0/1267743497 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f20800041c0 con 0x56365faa7630 2024-06-03T02:31:24.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.322+0000 7f207a000700 1 -- 172.21.15.79:0/1267743497 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f208002e050 con 0x56365faa7630 2024-06-03T02:31:24.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.322+0000 7f20a1336b80 10 client.26606 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.322+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x56365fac7e90 con 0x56365faa7630 2024-06-03T02:31:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.323+0000 7f207a000700 1 -- 172.21.15.79:0/1267743497 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2080005030 con 0x56365faa7630 2024-06-03T02:31:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.323+0000 7f20a1336b80 10 client.26606 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.323+0000 7f20a1336b80 10 client.26606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:31:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.323+0000 7f20a1336b80 20 client.26606 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.323+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56365faca360 0x56365facc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.323+0000 7f20a1336b80 4 client.26606 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:31:24.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.323+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56365fa07080 con 0x56365faca360 2024-06-03T02:31:24.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.325+0000 7f208fe00700 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56365faca360 0x56365facc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.325+0000 7f208fe00700 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56365faca360 0x56365facc750 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bde2bdabeac6b7 server_cookie=df15b0ea95d0da40 in_seq=0 out_seq=0 2024-06-03T02:31:24.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.325+0000 7f207a000700 10 client.26606 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:24.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.327+0000 7f207a000700 1 -- 172.21.15.79:0/1267743497 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56365fa07080 con 0x56365faca360 2024-06-03T02:31:24.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.327+0000 7f207a000700 10 client.26606 handle_command_reply: tid=0 2024-06-03T02:31:24.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.327+0000 7f20a1336b80 1 client.26606 shutdown 2024-06-03T02:31:24.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.327+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 msgr2=0x56365fab75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:24.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.327+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 0x56365fab75d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2080005950 tx=0x7f2080010660 comp rx=0 tx=0).stop 2024-06-03T02:31:24.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 shutdown_connections 2024-06-03T02:31:24.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365fabab90 0x56365fab7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56365faca360 0x56365facc750 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365faba2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1267743497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365faa7630 0x56365fab75d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 >> 172.21.15.79:0/1267743497 conn(0x56365f9d41f0 msgr2=0x56365faa27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:24.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f207a000700 0 client.26606 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:24.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 shutdown_connections 2024-06-03T02:31:24.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f20a1336b80 1 -- 172.21.15.79:0/1267743497 wait complete. 2024-06-03T02:31:24.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.328+0000 7f20a1336b80 20 client.26606 trim_cache size 0 max 16384 2024-06-03T02:31:24.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.375+0000 7f20a1336b80 1 Processor -- start 2024-06-03T02:31:24.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.375+0000 7f20a1336b80 1 -- start start 2024-06-03T02:31:24.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.376+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fa23ec0 0x56365fabbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.376+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 0x56365fab4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.376+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fabab90 0x56365fab4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.376+0000 7f20a1336b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b4960 con 0x56365fabab90 2024-06-03T02:31:24.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.376+0000 7f208fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fa23ec0 0x56365fabbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.376+0000 7f208fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fa23ec0 0x56365fabbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:51378/0 (socket says 172.21.15.79:51378) 2024-06-03T02:31:24.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.376+0000 7f208fe00700 1 -- 172.21.15.79:0/2672601762 learned_addr learned my addr 172.21.15.79:0/2672601762 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:24.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.376+0000 7f20a1336b80 1 -- 172.21.15.79:0/2672601762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b64a0 con 0x56365fa23ec0 2024-06-03T02:31:24.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.377+0000 7f20a1336b80 1 -- 172.21.15.79:0/2672601762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b5030 con 0x56365faa7630 2024-06-03T02:31:24.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.377+0000 7f2094a00700 1 --2- 172.21.15.79:0/2672601762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fabab90 0x56365fab4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.378+0000 7f2094a00700 1 -- 172.21.15.79:0/2672601762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 msgr2=0x56365fab4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:24.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.378+0000 7f2094a00700 1 --2- 172.21.15.79:0/2672601762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 0x56365fab4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.378+0000 7f2094a00700 1 -- 172.21.15.79:0/2672601762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fa23ec0 msgr2=0x56365fabbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:24.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.379+0000 7f2094a00700 1 --2- 172.21.15.79:0/2672601762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fa23ec0 0x56365fabbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.379+0000 7f2094a00700 1 -- 172.21.15.79:0/2672601762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365fab5b40 con 0x56365fabab90 2024-06-03T02:31:24.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.380+0000 7f2094a00700 1 --2- 172.21.15.79:0/2672601762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fabab90 0x56365fab4d90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f208800b070 tx=0x7f208801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2f86d2e3bd5bb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:24.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.383+0000 7f2084c00700 1 -- 172.21.15.79:0/2672601762 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088007940 con 0x56365fabab90 2024-06-03T02:31:24.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.384+0000 7f2084c00700 1 -- 172.21.15.79:0/2672601762 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f208802e030 con 0x56365fabab90 2024-06-03T02:31:24.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.385+0000 7f20a1336b80 1 -- 172.21.15.79:0/2672601762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fabab90 msgr2=0x56365fab4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:24.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.385+0000 7f2084c00700 1 -- 172.21.15.79:0/2672601762 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088007940 con 0x56365fabab90 2024-06-03T02:31:24.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.385+0000 7f20a1336b80 1 --2- 172.21.15.79:0/2672601762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fabab90 0x56365fab4d90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f208800b070 tx=0x7f208801d000 comp rx=0 tx=0).stop 2024-06-03T02:31:24.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.385+0000 7f20a1336b80 1 -- 172.21.15.79:0/2672601762 shutdown_connections 2024-06-03T02:31:24.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.390+0000 7f20a1336b80 1 --2- 172.21.15.79:0/2672601762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fabab90 0x56365fab4d90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.390+0000 7f20a1336b80 1 --2- 172.21.15.79:0/2672601762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fa23ec0 0x56365fabbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.390+0000 7f20a1336b80 1 --2- 172.21.15.79:0/2672601762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 0x56365fab4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.390+0000 7f20a1336b80 1 -- 172.21.15.79:0/2672601762 >> 172.21.15.79:0/2672601762 conn(0x56365f9d41f0 msgr2=0x56365fab5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:24.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.391+0000 7f20a1336b80 1 -- 172.21.15.79:0/2672601762 shutdown_connections 2024-06-03T02:31:24.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.391+0000 7f20a1336b80 1 -- 172.21.15.79:0/2672601762 wait complete. 2024-06-03T02:31:24.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.391+0000 7f20a1336b80 1 -- start start 2024-06-03T02:31:24.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.392+0000 7f20a1336b80 1 Processor -- start 2024-06-03T02:31:24.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.392+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365facfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.392+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 0x56365facc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.393+0000 7f20a1336b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fabab90 0x56365facc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.395+0000 7f20a1336b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b5dd0 con 0x56365fa23ec0 2024-06-03T02:31:24.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.395+0000 7f2094a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fabab90 0x56365facc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.395+0000 7f2094a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fabab90 0x56365facc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:51382/0 (socket says 172.21.15.79:51382) 2024-06-03T02:31:24.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.395+0000 7f2094a00700 1 -- 172.21.15.79:0/1973473674 learned_addr learned my addr 172.21.15.79:0/1973473674 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:24.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.395+0000 7f208f400700 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 0x56365facc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.395+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b5700 con 0x56365fabab90 2024-06-03T02:31:24.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.395+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b2900 con 0x56365faa7630 2024-06-03T02:31:24.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.396+0000 7f208fe00700 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365facfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.396+0000 7f20a1336b80 10 client.0 _ll_register_callbacks cb 0x56365f9937a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:24.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.396+0000 7f208f400700 1 -- 172.21.15.79:0/1973473674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fabab90 msgr2=0x56365facc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:24.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.396+0000 7f208f400700 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fabab90 0x56365facc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.396+0000 7f208f400700 1 -- 172.21.15.79:0/1973473674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 msgr2=0x56365facfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:24.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.396+0000 7f208f400700 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365facfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.396+0000 7f208f400700 1 -- 172.21.15.79:0/1973473674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365faccc10 con 0x56365faa7630 2024-06-03T02:31:24.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.398+0000 7f208f400700 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 0x56365facc090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x56365fab89c0 tx=0x7f208000cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5da6bd7ae52a1d37 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:24.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.398+0000 7f207a000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:31:24.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.398+0000 7f207a000700 1 -- 172.21.15.79:0/1973473674 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080037070 con 0x56365faa7630 2024-06-03T02:31:24.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.399+0000 7f207a000700 1 -- 172.21.15.79:0/1973473674 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f208001a630 con 0x56365faa7630 2024-06-03T02:31:24.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.399+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208802e030 con 0x56365faa7630 2024-06-03T02:31:24.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.401+0000 7f207a000700 1 -- 172.21.15.79:0/1973473674 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f208000e630 con 0x56365faa7630 2024-06-03T02:31:24.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.401+0000 7f20a1336b80 10 client.26611 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:24.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.401+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56365fb35b50 con 0x56365faa7630 2024-06-03T02:31:24.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.402+0000 7f207a000700 1 -- 172.21.15.79:0/1973473674 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2080010910 con 0x56365faa7630 2024-06-03T02:31:24.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.402+0000 7f207a000700 1 -- 172.21.15.79:0/1973473674 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f208003a020 con 0x56365faa7630 2024-06-03T02:31:24.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.402+0000 7f20a1336b80 10 client.26611 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:24.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.402+0000 7f20a1336b80 10 client.26611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:31:24.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.402+0000 7f20a1336b80 20 client.26611 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:24.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.403+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56365fb36250 0x56365fb38640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:24.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.403+0000 7f20a1336b80 4 client.26611 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:31:24.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.403+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f208802e030 con 0x56365fb36250 2024-06-03T02:31:24.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.403+0000 7f208fe00700 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56365fb36250 0x56365fb38640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:24.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.405+0000 7f208fe00700 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56365fb36250 0x56365fb38640 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=656fc4ed84162572 server_cookie=61a11e95ab3b041a in_seq=0 out_seq=0 2024-06-03T02:31:24.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.405+0000 7f207a000700 10 client.26611 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:24.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.406+0000 7f207a000700 1 -- 172.21.15.79:0/1973473674 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f208802e030 con 0x56365fb36250 2024-06-03T02:31:24.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.406+0000 7f207a000700 10 client.26611 handle_command_reply: tid=0 2024-06-03T02:31:24.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.406+0000 7f20a1336b80 1 client.26611 shutdown 2024-06-03T02:31:24.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 msgr2=0x56365facc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:24.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 0x56365facc090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x56365fab89c0 tx=0x7f208000cee0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 shutdown_connections 2024-06-03T02:31:24.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56365fabab90 0x56365facc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56365fb36250 0x56365fb38640 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56365fa23ec0 0x56365facfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f20a1336b80 1 --2- 172.21.15.79:0/1973473674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56365faa7630 0x56365facc090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 >> 172.21.15.79:0/1973473674 conn(0x56365f9d41f0 msgr2=0x56365faa1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:24.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f207a000700 0 client.26611 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:24.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.407+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 shutdown_connections 2024-06-03T02:31:24.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.408+0000 7f20a1336b80 1 -- 172.21.15.79:0/1973473674 wait complete. 2024-06-03T02:31:24.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.410+0000 7f20a1336b80 20 client.26611 trim_cache size 0 max 16384 2024-06-03T02:31:24.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f20740a2cb0 msgr2=0x7f20740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:24.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f20740a2cb0 0x7f20740a5170 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f208800b830 tx=0x7f2088005040 comp rx=0 tx=0).stop 2024-06-03T02:31:24.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090109490 msgr2=0x7f20901a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:24.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090109490 0x7f20901a13c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f207c00c9b0 tx=0x7f207c00cd70 comp rx=0 tx=0).stop 2024-06-03T02:31:24.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 shutdown_connections 2024-06-03T02:31:24.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f20740a2cb0 0x7f20740a5170 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20901155e0 0x7f20901a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090109490 0x7f20901a13c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 --2- 172.21.15.79:0/3001536693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090109db0 0x7f20901a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:24.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.412+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 >> 172.21.15.79:0/3001536693 conn(0x7f209006d600 msgr2=0x7f2090071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:24.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.413+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 shutdown_connections 2024-06-03T02:31:24.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:24.413+0000 7f2096a00700 1 -- 172.21.15.79:0/3001536693 wait complete. 2024-06-03T02:31:24.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:24 smithi135 ceph-mon[36653]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 491 op/s 2024-06-03T02:31:25.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:25 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:31:25.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:25 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:31:25.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:25 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:31:25.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:25 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:31:25.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:31:25.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:25 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:31:25.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:25 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:31:25.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:25 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:31:25.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:25 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:31:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:25 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:31:25.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:25 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:31:25.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:25 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:31:25.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:25 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:31:26.107 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:31:26.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:26 smithi079 ceph-mon[31819]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 532 op/s 2024-06-03T02:31:26.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:26 smithi123 ceph-mon[36640]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 532 op/s 2024-06-03T02:31:26.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:26 smithi135 ceph-mon[36653]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 532 op/s 2024-06-03T02:31:28.312 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 680.0 s, 437.3 tps, lat 224.388 ms stddev 260.408 2024-06-03T02:31:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:28 smithi123 ceph-mon[36640]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 435 op/s 2024-06-03T02:31:28.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:28 smithi079 ceph-mon[31819]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 435 op/s 2024-06-03T02:31:28.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:28 smithi135 ceph-mon[36653]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 435 op/s 2024-06-03T02:31:29.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:29 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:31:29.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:29 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:31:29.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:29 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:31:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.510+0000 7f5447200700 1 -- 172.21.15.79:0/660127818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544006a170 msgr2=0x7f544006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.510+0000 7f543f400700 1 -- 172.21.15.79:0/660127818 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428004230 con 0x7f544006a170 2024-06-03T02:31:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.510+0000 7f5447200700 1 --2- 172.21.15.79:0/660127818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544006a170 0x7f544006a5f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5428009990 tx=0x7f5428009ca0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.511+0000 7f5447200700 1 -- 172.21.15.79:0/660127818 shutdown_connections 2024-06-03T02:31:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.511+0000 7f5447200700 1 --2- 172.21.15.79:0/660127818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f544006ab30 0x7f544010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.511+0000 7f5447200700 1 --2- 172.21.15.79:0/660127818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5440108500 0x7f54401088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.511+0000 7f5447200700 1 --2- 172.21.15.79:0/660127818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544006a170 0x7f544006a5f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.511+0000 7f5447200700 1 -- 172.21.15.79:0/660127818 >> 172.21.15.79:0/660127818 conn(0x7f5440078340 msgr2=0x7f5440078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.511+0000 7f5447200700 1 -- 172.21.15.79:0/660127818 shutdown_connections 2024-06-03T02:31:30.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.511+0000 7f5447200700 1 -- 172.21.15.79:0/660127818 wait complete. 2024-06-03T02:31:30.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.514+0000 7f5447200700 1 Processor -- start 2024-06-03T02:31:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.514+0000 7f5447200700 1 -- start start 2024-06-03T02:31:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.515+0000 7f5447200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544006a170 0x7f544010d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.515+0000 7f5447200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f544006ab30 0x7f544010d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.515+0000 7f5447200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5440108500 0x7f54401a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.515+0000 7f5447200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5440112880 con 0x7f544006ab30 2024-06-03T02:31:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.515+0000 7f5447200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5440112700 con 0x7f544006a170 2024-06-03T02:31:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.515+0000 7f5447200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5440112a00 con 0x7f5440108500 2024-06-03T02:31:30.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.516+0000 7f5437400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f544006ab30 0x7f544010d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.516+0000 7f5437400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f544006ab30 0x7f544010d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:51414/0 (socket says 172.21.15.79:51414) 2024-06-03T02:31:30.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.516+0000 7f5437400700 1 -- 172.21.15.79:0/67258652 learned_addr learned my addr 172.21.15.79:0/67258652 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.517+0000 7f5437400700 1 -- 172.21.15.79:0/67258652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5440108500 msgr2=0x7f54401a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.517+0000 7f5437400700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5440108500 0x7f54401a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.517+0000 7f5437400700 1 -- 172.21.15.79:0/67258652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544006a170 msgr2=0x7f544010d180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:31:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.517+0000 7f543fe00700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544006a170 0x7f544010d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.518+0000 7f5437400700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544006a170 0x7f544010d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.518+0000 7f5437400700 1 -- 172.21.15.79:0/67258652 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54401a4090 con 0x7f544006ab30 2024-06-03T02:31:30.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.518+0000 7f543fe00700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544006a170 0x7f544010d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:31:30.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.520+0000 7f5437400700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f544006ab30 0x7f544010d6c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f542800b5c0 tx=0x7f5428005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:30.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.520+0000 7f543d600700 1 -- 172.21.15.79:0/67258652 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428027070 con 0x7f544006ab30 2024-06-03T02:31:30.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.521+0000 7f543d600700 1 -- 172.21.15.79:0/67258652 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5428022470 con 0x7f544006ab30 2024-06-03T02:31:30.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.521+0000 7f5447200700 1 -- 172.21.15.79:0/67258652 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54401a4320 con 0x7f544006ab30 2024-06-03T02:31:30.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.521+0000 7f5447200700 1 -- 172.21.15.79:0/67258652 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544006beb0 con 0x7f544006ab30 2024-06-03T02:31:30.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.521+0000 7f543d600700 1 -- 172.21.15.79:0/67258652 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542800f6c0 con 0x7f544006ab30 2024-06-03T02:31:30.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.527+0000 7f543d600700 1 -- 172.21.15.79:0/67258652 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f542802a070 con 0x7f544006ab30 2024-06-03T02:31:30.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.530+0000 7f543d600700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54240a2db0 0x7f54240a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.530+0000 7f5451a8bb80 1 Processor -- start 2024-06-03T02:31:30.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.530+0000 7f5451a8bb80 1 -- start start 2024-06-03T02:31:30.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.531+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac84d7e40 0x555ac855b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.531+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac855b5b0 0x555ac855ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.531+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85686f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.531+0000 7f5451a8bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555ac846aa10 con 0x555ac84d7e40 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.531+0000 7f5451a8bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ac84adb00 con 0x555ac8566300 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.531+0000 7f5451a8bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555ac84688f0 con 0x555ac855b5b0 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.531+0000 7f5445200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.531+0000 7f5445200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85686f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33462/0 (socket says 172.21.15.79:33462) 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.531+0000 7f5445200700 1 -- 172.21.15.79:0/4070033093 learned_addr learned my addr 172.21.15.79:0/4070033093 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.532+0000 7f543fe00700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54240a2db0 0x7f54240a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.532+0000 7f5445200700 1 -- 172.21.15.79:0/4070033093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac855b5b0 msgr2=0x555ac855ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.532+0000 7f5445200700 1 --2- 172.21.15.79:0/4070033093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac855b5b0 0x555ac855ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.532+0000 7f5445200700 1 -- 172.21.15.79:0/4070033093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac84d7e40 msgr2=0x555ac855b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:30.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.535+0000 7f5445200700 1 --2- 172.21.15.79:0/4070033093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac84d7e40 0x555ac855b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.535+0000 7f5445200700 1 -- 172.21.15.79:0/4070033093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac8455f90 con 0x555ac8566300 2024-06-03T02:31:30.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.535+0000 7f5445200700 1 --2- 172.21.15.79:0/4070033093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85686f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f543800a410 tx=0x7f543800a720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a876b95d7342e68 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:30.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.537+0000 7f5435600700 1 -- 172.21.15.79:0/4070033093 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438018070 con 0x555ac8566300 2024-06-03T02:31:30.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.537+0000 7f543fe00700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54240a2db0 0x7f54240a5270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5430007a60 tx=0x7f5430007380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:30.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.537+0000 7f5435600700 1 -- 172.21.15.79:0/4070033093 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54380138f0 con 0x555ac8566300 2024-06-03T02:31:30.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.537+0000 7f5435600700 1 -- 172.21.15.79:0/4070033093 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438008790 con 0x555ac8566300 2024-06-03T02:31:30.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.538+0000 7f5451a8bb80 1 -- 172.21.15.79:0/4070033093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 msgr2=0x555ac85686f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:30.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.538+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/4070033093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85686f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f543800a410 tx=0x7f543800a720 comp rx=0 tx=0).stop 2024-06-03T02:31:30.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.539+0000 7f5451a8bb80 1 -- 172.21.15.79:0/4070033093 shutdown_connections 2024-06-03T02:31:30.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.539+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/4070033093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85686f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.539+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/4070033093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac84d7e40 0x555ac855b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.539+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/4070033093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac855b5b0 0x555ac855ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.539+0000 7f5451a8bb80 1 -- 172.21.15.79:0/4070033093 >> 172.21.15.79:0/4070033093 conn(0x555ac8488170 msgr2=0x555ac85574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:30.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.541+0000 7f543d600700 1 -- 172.21.15.79:0/67258652 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f542809a850 con 0x7f544006ab30 2024-06-03T02:31:30.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.542+0000 7f5451a8bb80 1 -- 172.21.15.79:0/4070033093 shutdown_connections 2024-06-03T02:31:30.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.542+0000 7f5451a8bb80 1 -- 172.21.15.79:0/4070033093 wait complete. 2024-06-03T02:31:30.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.544+0000 7f5451a8bb80 1 -- start start 2024-06-03T02:31:30.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.544+0000 7f5451a8bb80 1 Processor -- start 2024-06-03T02:31:30.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.545+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac8565bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.545+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8566300 0x555ac8562ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.546+0000 7f543fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac8565bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.546+0000 7f543fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac8565bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60816/0 (socket says 172.21.15.79:60816) 2024-06-03T02:31:30.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.546+0000 7f543fe00700 1 -- 172.21.15.79:0/5685389 learned_addr learned my addr 172.21.15.79:0/5685389 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:30.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.546+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8563400 0x555ac8563880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.546+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555ac8449f60 con 0x555ac8566300 2024-06-03T02:31:30.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.546+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ac84a9ac0 con 0x555ac8563400 2024-06-03T02:31:30.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.546+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555ac8568fc0 con 0x555ac84d7e40 2024-06-03T02:31:30.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.546+0000 7f5445200700 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8563400 0x555ac8563880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.546+0000 7f5451a8bb80 10 client.0 _ll_register_callbacks cb 0x555ac8447720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:30.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.547+0000 7f5445200700 1 -- 172.21.15.79:0/5685389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 msgr2=0x555ac8565bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:30.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.547+0000 7f5445200700 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac8565bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.547+0000 7f5445200700 1 -- 172.21.15.79:0/5685389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8566300 msgr2=0x555ac8562ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:30.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.547+0000 7f5445200700 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8566300 0x555ac8562ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.547+0000 7f5445200700 1 -- 172.21.15.79:0/5685389 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac83fc8d0 con 0x555ac8563400 2024-06-03T02:31:30.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.547+0000 7f5445200700 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8563400 0x555ac8563880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5438008010 tx=0x7f5438013cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d1c0555a0eb6e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:30.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.547+0000 7f542e000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:31:30.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.547+0000 7f542e000700 1 -- 172.21.15.79:0/5685389 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438021020 con 0x555ac8563400 2024-06-03T02:31:30.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.547+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54380138f0 con 0x555ac8563400 2024-06-03T02:31:30.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.548+0000 7f542e000700 1 -- 172.21.15.79:0/5685389 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5438007490 con 0x555ac8563400 2024-06-03T02:31:30.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.548+0000 7f542e000700 1 -- 172.21.15.79:0/5685389 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5438018050 con 0x555ac8563400 2024-06-03T02:31:30.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.548+0000 7f5451a8bb80 10 client.26621 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:30.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.548+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ac8573c80 con 0x555ac8563400 2024-06-03T02:31:30.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.549+0000 7f542e000700 1 -- 172.21.15.79:0/5685389 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54380138f0 con 0x555ac8563400 2024-06-03T02:31:30.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.549+0000 7f542e000700 1 -- 172.21.15.79:0/5685389 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5438008620 con 0x555ac8563400 2024-06-03T02:31:30.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.549+0000 7f5451a8bb80 10 client.26621 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:30.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.549+0000 7f5451a8bb80 10 client.26621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:31:30.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.551+0000 7f5451a8bb80 20 client.26621 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:30.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.551+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555ac8576150 0x555ac8578540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.551+0000 7f5451a8bb80 4 client.26621 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:31:30.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.551+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ac84bb000 con 0x555ac8576150 2024-06-03T02:31:30.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.551+0000 7f543fe00700 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555ac8576150 0x555ac8578540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.552+0000 7f543fe00700 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555ac8576150 0x555ac8578540 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e631a83620fe server_cookie=2a8dfdf8f60283ed in_seq=0 out_seq=0 2024-06-03T02:31:30.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.552+0000 7f542e000700 10 client.26621 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:30.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.553+0000 7f542e000700 1 -- 172.21.15.79:0/5685389 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ac84bb000 con 0x555ac8576150 2024-06-03T02:31:30.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.553+0000 7f542e000700 10 client.26621 handle_command_reply: tid=0 2024-06-03T02:31:30.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.553+0000 7f5451a8bb80 1 client.26621 shutdown 2024-06-03T02:31:30.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.554+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8563400 msgr2=0x555ac8563880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:30.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8563400 0x555ac8563880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5438008010 tx=0x7f5438013cd0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 shutdown_connections 2024-06-03T02:31:30.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8563400 0x555ac8563880 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac8565bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555ac8576150 0x555ac8578540 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/5685389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8566300 0x555ac8562ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 >> 172.21.15.79:0/5685389 conn(0x555ac8488170 msgr2=0x555ac8556740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:30.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f542e000700 0 client.26621 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:30.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 shutdown_connections 2024-06-03T02:31:30.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 1 -- 172.21.15.79:0/5685389 wait complete. 2024-06-03T02:31:30.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.555+0000 7f5451a8bb80 20 client.26621 trim_cache size 0 max 16384 2024-06-03T02:31:30.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.596+0000 7f5451a8bb80 1 Processor -- start 2024-06-03T02:31:30.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.596+0000 7f5451a8bb80 1 -- start start 2024-06-03T02:31:30.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.596+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac85627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8565700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85676f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5451a8bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f541c00eb40 con 0x555ac8563400 2024-06-03T02:31:30.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5451a8bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f541c010680 con 0x555ac8566300 2024-06-03T02:31:30.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5451a8bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f541c00f210 con 0x555ac84d7e40 2024-06-03T02:31:30.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f543fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac85627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f543fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac85627c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60830/0 (socket says 172.21.15.79:60830) 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f543fe00700 1 -- 172.21.15.79:0/3454602376 learned_addr learned my addr 172.21.15.79:0/3454602376 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5445200700 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5437400700 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8565700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5445200700 1 -- 172.21.15.79:0/3454602376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 msgr2=0x555ac85627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5445200700 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac85627c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.597+0000 7f5445200700 1 -- 172.21.15.79:0/3454602376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 msgr2=0x555ac8565700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.598+0000 7f5445200700 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8565700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.598+0000 7f5445200700 1 -- 172.21.15.79:0/3454602376 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac855bf30 con 0x555ac8566300 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.598+0000 7f5445200700 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85676f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f543800a760 tx=0x7f543800ab20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b4ba1d5b7adb7ec server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.598+0000 7f5434c00700 1 -- 172.21.15.79:0/3454602376 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438021040 con 0x555ac8566300 2024-06-03T02:31:30.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.598+0000 7f5434c00700 1 -- 172.21.15.79:0/3454602376 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5438019040 con 0x555ac8566300 2024-06-03T02:31:30.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.598+0000 7f5434c00700 1 -- 172.21.15.79:0/3454602376 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380083f0 con 0x555ac8566300 2024-06-03T02:31:30.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.599+0000 7f543fe00700 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac85627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:30.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.599+0000 7f5451a8bb80 1 -- 172.21.15.79:0/3454602376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 msgr2=0x555ac85676f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:30.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.599+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85676f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f543800a760 tx=0x7f543800ab20 comp rx=0 tx=0).stop 2024-06-03T02:31:30.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.599+0000 7f5437400700 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8565700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:30.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.600+0000 7f5451a8bb80 1 -- 172.21.15.79:0/3454602376 shutdown_connections 2024-06-03T02:31:30.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.600+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac8566300 0x555ac85676f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.600+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac84d7e40 0x555ac85627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.600+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/3454602376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8565700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.600+0000 7f5451a8bb80 1 -- 172.21.15.79:0/3454602376 >> 172.21.15.79:0/3454602376 conn(0x555ac8488170 msgr2=0x555ac8556fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:30.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.602+0000 7f5451a8bb80 1 -- 172.21.15.79:0/3454602376 shutdown_connections 2024-06-03T02:31:30.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.602+0000 7f5451a8bb80 1 -- 172.21.15.79:0/3454602376 wait complete. 2024-06-03T02:31:30.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.604+0000 7f5451a8bb80 1 -- start start 2024-06-03T02:31:30.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.604+0000 7f5451a8bb80 1 Processor -- start 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.605+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac84d7e40 0x555ac8568200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.605+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8568740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.605+0000 7f5451a8bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac8566300 0x555ac85668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.605+0000 7f5451a8bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f541c00ffb0 con 0x555ac8563400 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.605+0000 7f5451a8bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f541c00f8e0 con 0x555ac84d7e40 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.605+0000 7f5451a8bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f541c00cae0 con 0x555ac8566300 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.605+0000 7f5451a8bb80 10 client.0 _ll_register_callbacks cb 0x555ac8447720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f543fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac84d7e40 0x555ac8568200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f543fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac84d7e40 0x555ac8568200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33488/0 (socket says 172.21.15.79:33488) 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f543fe00700 1 -- 172.21.15.79:0/2327807995 learned_addr learned my addr 172.21.15.79:0/2327807995 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f543fe00700 1 -- 172.21.15.79:0/2327807995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac8566300 msgr2=0x555ac85668e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:31:30.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f543fe00700 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac8566300 0x555ac85668e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f543fe00700 1 -- 172.21.15.79:0/2327807995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 msgr2=0x555ac8568740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:30.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f5437400700 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8568740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f543fe00700 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8568740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f543fe00700 1 -- 172.21.15.79:0/2327807995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac8567120 con 0x555ac84d7e40 2024-06-03T02:31:30.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.606+0000 7f5437400700 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8568740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:31:30.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.607+0000 7f543fe00700 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac84d7e40 0x555ac8568200 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f54300077a0 tx=0x7f5430004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b123848361165b67 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:30.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.608+0000 7f542e000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:31:30.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.608+0000 7f542e000700 1 -- 172.21.15.79:0/2327807995 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430030060 con 0x555ac84d7e40 2024-06-03T02:31:30.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.608+0000 7f542e000700 1 -- 172.21.15.79:0/2327807995 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5430013cd0 con 0x555ac84d7e40 2024-06-03T02:31:30.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.611+0000 7f5451a8bb80 1 -- 172.21.15.79:0/2327807995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5438019040 con 0x555ac84d7e40 2024-06-03T02:31:30.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.611+0000 7f542e000700 1 -- 172.21.15.79:0/2327807995 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f543000f040 con 0x555ac84d7e40 2024-06-03T02:31:30.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.611+0000 7f5451a8bb80 10 client.26627 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:30.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.611+0000 7f5451a8bb80 1 -- 172.21.15.79:0/2327807995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ac856dff0 con 0x555ac84d7e40 2024-06-03T02:31:30.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.612+0000 7f542e000700 1 -- 172.21.15.79:0/2327807995 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5438019040 con 0x555ac84d7e40 2024-06-03T02:31:30.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.612+0000 7f542e000700 1 -- 172.21.15.79:0/2327807995 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f543002a070 con 0x555ac84d7e40 2024-06-03T02:31:30.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.612+0000 7f5451a8bb80 10 client.26627 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:30.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.612+0000 7f5451a8bb80 10 client.26627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:31:30.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.612+0000 7f5451a8bb80 20 client.26627 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:30.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.612+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555ac856e640 0x555ac856ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:30.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.612+0000 7f5451a8bb80 4 client.26627 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:31:30.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.612+0000 7f5451a8bb80 1 -- 172.21.15.79:0/2327807995 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ac84b5580 con 0x555ac856e640 2024-06-03T02:31:30.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.612+0000 7f5437400700 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555ac856e640 0x555ac856ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:30.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.613+0000 7f5437400700 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555ac856e640 0x555ac856ea70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6747ca77976074 server_cookie=e6ddcfd6bd7c9673 in_seq=0 out_seq=0 2024-06-03T02:31:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.613+0000 7f542e000700 10 client.26627 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.613+0000 7f542e000700 1 -- 172.21.15.79:0/2327807995 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ac84b5580 con 0x555ac856e640 2024-06-03T02:31:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.613+0000 7f542e000700 10 client.26627 handle_command_reply: tid=0 2024-06-03T02:31:30.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.613+0000 7f5451a8bb80 1 client.26627 shutdown 2024-06-03T02:31:30.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 -- 172.21.15.79:0/2327807995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac84d7e40 msgr2=0x555ac8568200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:30.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac84d7e40 0x555ac8568200 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f54300077a0 tx=0x7f5430004750 comp rx=0 tx=0).stop 2024-06-03T02:31:30.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 -- 172.21.15.79:0/2327807995 shutdown_connections 2024-06-03T02:31:30.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac8566300 0x555ac85668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac84d7e40 0x555ac8568200 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x555ac856e640 0x555ac856ea70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 --2- 172.21.15.79:0/2327807995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555ac8563400 0x555ac8568740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 -- 172.21.15.79:0/2327807995 >> 172.21.15.79:0/2327807995 conn(0x555ac8488170 msgr2=0x555ac8556740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f542e000700 0 client.26627 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:31:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 -- 172.21.15.79:0/2327807995 shutdown_connections 2024-06-03T02:31:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 1 -- 172.21.15.79:0/2327807995 wait complete. 2024-06-03T02:31:30.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.614+0000 7f5451a8bb80 20 client.26627 trim_cache size 0 max 16384 2024-06-03T02:31:30.620 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:31:30.620 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:31:30.620 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:31:30.621 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:31:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.615+0000 7f5447200700 1 -- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54240a2db0 msgr2=0x7f54240a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.615+0000 7f5447200700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54240a2db0 0x7f54240a5270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5430007a60 tx=0x7f5430007380 comp rx=0 tx=0).stop 2024-06-03T02:31:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.616+0000 7f5447200700 1 -- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f544006ab30 msgr2=0x7f544010d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.616+0000 7f5447200700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f544006ab30 0x7f544010d6c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f542800b5c0 tx=0x7f5428005990 comp rx=0 tx=0).stop 2024-06-03T02:31:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.616+0000 7f5447200700 1 -- 172.21.15.79:0/67258652 shutdown_connections 2024-06-03T02:31:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.616+0000 7f5447200700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5440108500 0x7f54401a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.616+0000 7f5447200700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544006a170 0x7f544010d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.616+0000 7f5447200700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54240a2db0 0x7f54240a5270 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.616+0000 7f5447200700 1 --2- 172.21.15.79:0/67258652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f544006ab30 0x7f544010d6c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.616+0000 7f5447200700 1 -- 172.21.15.79:0/67258652 >> 172.21.15.79:0/67258652 conn(0x7f5440078340 msgr2=0x7f5440102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:30.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.618+0000 7f5447200700 1 -- 172.21.15.79:0/67258652 shutdown_connections 2024-06-03T02:31:30.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:30.618+0000 7f5447200700 1 -- 172.21.15.79:0/67258652 wait complete. 2024-06-03T02:31:30.645 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:30 smithi079 ceph-mon[31819]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 536 op/s 2024-06-03T02:31:30.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:30 smithi123 ceph-mon[36640]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 536 op/s 2024-06-03T02:31:30.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:30 smithi135 ceph-mon[36653]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 536 op/s 2024-06-03T02:31:32.656 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:32 smithi079 ceph-mon[31819]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 430 op/s 2024-06-03T02:31:32.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:32 smithi123 ceph-mon[36640]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 430 op/s 2024-06-03T02:31:32.726 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:31:32.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:31:32.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:32 smithi135 ceph-mon[36653]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 430 op/s 2024-06-03T02:31:33.312 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 685.0 s, 449.6 tps, lat 221.447 ms stddev 256.970 2024-06-03T02:31:33.398 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:31:34.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:34 smithi123 ceph-mon[36640]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 439 op/s 2024-06-03T02:31:34.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:34 smithi079 ceph-mon[31819]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 439 op/s 2024-06-03T02:31:34.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:34 smithi135 ceph-mon[36653]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 439 op/s 2024-06-03T02:31:36.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:36 smithi123 ceph-mon[36640]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 476 op/s 2024-06-03T02:31:36.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:36 smithi079 ceph-mon[31819]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 476 op/s 2024-06-03T02:31:36.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:36 smithi135 ceph-mon[36653]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 476 op/s 2024-06-03T02:31:38.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 690.0 s, 443.3 tps, lat 219.379 ms stddev 246.834 2024-06-03T02:31:38.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.018+0000 7f7090400700 1 -- 172.21.15.79:0/108581043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7088100870 msgr2=0x7f7088100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.018+0000 7f7090400700 1 --2- 172.21.15.79:0/108581043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7088100870 0x7f7088100c50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f707c00b0d0 tx=0x7f707c00b3e0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.018+0000 7f7090400700 1 -- 172.21.15.79:0/108581043 shutdown_connections 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.018+0000 7f7090400700 1 --2- 172.21.15.79:0/108581043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7088108950 0x7f708810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.018+0000 7f7090400700 1 --2- 172.21.15.79:0/108581043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7088100870 0x7f7088100c50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.018+0000 7f7090400700 1 --2- 172.21.15.79:0/108581043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7088101220 0x7f7088108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.018+0000 7f7090400700 1 -- 172.21.15.79:0/108581043 >> 172.21.15.79:0/108581043 conn(0x7f7088077590 msgr2=0x7f70880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.018+0000 7f7090400700 1 -- 172.21.15.79:0/108581043 shutdown_connections 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.018+0000 7f7090400700 1 -- 172.21.15.79:0/108581043 wait complete. 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.019+0000 7f7090400700 1 Processor -- start 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.019+0000 7f7090400700 1 -- start start 2024-06-03T02:31:38.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f7090400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7088100870 0x7f708810f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f7090400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7088101220 0x7f708810c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f7090400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7088108950 0x7f708810c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f7090400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7088114a00 con 0x7f7088108950 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f7090400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7088114880 con 0x7f7088101220 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f7090400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7088114b80 con 0x7f7088100870 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708d000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7088101220 0x7f708810c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708da00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7088100870 0x7f708810f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708d000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7088101220 0x7f708810c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:50352/0 (socket says 172.21.15.79:50352) 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708d000700 1 -- 172.21.15.79:0/959186093 learned_addr learned my addr 172.21.15.79:0/959186093 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708e400700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7088108950 0x7f708810c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708d000700 1 -- 172.21.15.79:0/959186093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7088100870 msgr2=0x7f708810f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708d000700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7088100870 0x7f708810f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708d000700 1 -- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7088108950 msgr2=0x7f708810c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708d000700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7088108950 0x7f708810c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.020+0000 7f708d000700 1 -- 172.21.15.79:0/959186093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708810d0e0 con 0x7f7088101220 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.021+0000 7f708d000700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7088101220 0x7f708810c2a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f707400ea30 tx=0x7f707400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.021+0000 7f7086000700 1 -- 172.21.15.79:0/959186093 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707400cbd0 con 0x7f7088101220 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.021+0000 7f7086000700 1 -- 172.21.15.79:0/959186093 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7074004500 con 0x7f7088101220 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.021+0000 7f7086000700 1 -- 172.21.15.79:0/959186093 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074010430 con 0x7f7088101220 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.021+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70881a1040 con 0x7f7088101220 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.022+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70881a1530 con 0x7f7088101220 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.025+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70881019a0 con 0x7f7088101220 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.027+0000 7f7086000700 1 -- 172.21.15.79:0/959186093 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f70740105d0 con 0x7f7088101220 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.029+0000 7f7086000700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f70700a2d20 0x7f70700a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.029+0000 7f7086000700 1 -- 172.21.15.79:0/959186093 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f70740cf650 con 0x7f7088101220 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.030+0000 7f708da00700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f70700a2d20 0x7f70700a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:38.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.030+0000 7f708da00700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f70700a2d20 0x7f70700a51e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f707c000f80 tx=0x7f707c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:38.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.032+0000 7f7086000700 1 -- 172.21.15.79:0/959186093 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f707409c1a0 con 0x7f7088101220 2024-06-03T02:31:38.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.500+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70881a1fc0 con 0x7f7088101220 2024-06-03T02:31:38.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.501+0000 7f7086000700 1 -- 172.21.15.79:0/959186093 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f707409baa0 con 0x7f7088101220 2024-06-03T02:31:38.503 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:31:38.504 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:31:38.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.506+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f70700a2d20 msgr2=0x7f70700a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:38.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.506+0000 7f7090400700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f70700a2d20 0x7f70700a51e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f707c000f80 tx=0x7f707c00b040 comp rx=0 tx=0).stop 2024-06-03T02:31:38.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.506+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7088101220 msgr2=0x7f708810c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:38.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.506+0000 7f7090400700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7088101220 0x7f708810c2a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f707400ea30 tx=0x7f707400ed40 comp rx=0 tx=0).stop 2024-06-03T02:31:38.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.507+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 shutdown_connections 2024-06-03T02:31:38.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.507+0000 7f7090400700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f70700a2d20 0x7f70700a51e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.507+0000 7f7090400700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7088108950 0x7f708810c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.507+0000 7f7090400700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7088100870 0x7f708810f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.507+0000 7f7090400700 1 --2- 172.21.15.79:0/959186093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7088101220 0x7f708810c2a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:38.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.507+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 >> 172.21.15.79:0/959186093 conn(0x7f7088077590 msgr2=0x7f70880ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:38.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.509+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 shutdown_connections 2024-06-03T02:31:38.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:38.509+0000 7f7090400700 1 -- 172.21.15.79:0/959186093 wait complete. 2024-06-03T02:31:38.511 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:31:38.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:38 smithi079 ceph-mon[31819]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.8 MiB/s wr, 413 op/s 2024-06-03T02:31:38.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:38 smithi123 ceph-mon[36640]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.8 MiB/s wr, 413 op/s 2024-06-03T02:31:38.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:38 smithi135 ceph-mon[36653]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.8 MiB/s wr, 413 op/s 2024-06-03T02:31:39.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:39 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/959186093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:31:39.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:39 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/959186093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:31:39.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:39 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/959186093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:31:40.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:40 smithi123 ceph-mon[36640]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 471 op/s 2024-06-03T02:31:40.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:40 smithi079 ceph-mon[31819]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 471 op/s 2024-06-03T02:31:40.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:31:40.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:40 smithi135 ceph-mon[36653]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 471 op/s 2024-06-03T02:31:41.520 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:31:42.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:42 smithi123 ceph-mon[36640]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 406 op/s 2024-06-03T02:31:42.736 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:42 smithi079 ceph-mon[31819]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 406 op/s 2024-06-03T02:31:42.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:42 smithi135 ceph-mon[36653]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 406 op/s 2024-06-03T02:31:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 695.0 s, 484.2 tps, lat 212.113 ms stddev 280.606 2024-06-03T02:31:44.670 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:44 smithi079 ceph-mon[31819]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 446 op/s 2024-06-03T02:31:44.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:44 smithi135 ceph-mon[36653]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 446 op/s 2024-06-03T02:31:44.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:44 smithi123 ceph-mon[36640]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 446 op/s 2024-06-03T02:31:45.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.178+0000 7f8ba9a00700 1 -- 172.21.15.79:0/3080535804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ba41024e0 msgr2=0x7f8ba4102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:45.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.178+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/3080535804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ba41024e0 0x7f8ba4102960 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098f0 tx=0x7f8b8c009c00 comp rx=0 tx=0).stop 2024-06-03T02:31:45.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.179+0000 7f8ba9a00700 1 -- 172.21.15.79:0/3080535804 shutdown_connections 2024-06-03T02:31:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.179+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/3080535804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba4102ea0 0x7f8ba410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.179+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/3080535804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba41085f0 0x7f8ba41089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.179+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/3080535804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ba41024e0 0x7f8ba4102960 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098f0 tx=0x7f8b8c009c00 comp rx=0 tx=0).stop 2024-06-03T02:31:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.179+0000 7f8ba9a00700 1 -- 172.21.15.79:0/3080535804 >> 172.21.15.79:0/3080535804 conn(0x7f8ba4079640 msgr2=0x7f8ba4079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.180+0000 7f8ba9a00700 1 -- 172.21.15.79:0/3080535804 shutdown_connections 2024-06-03T02:31:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.181+0000 7f8ba9a00700 1 -- 172.21.15.79:0/3080535804 wait complete. 2024-06-03T02:31:45.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.183+0000 7f8ba9a00700 1 Processor -- start 2024-06-03T02:31:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.186+0000 7f8ba9a00700 1 -- start start 2024-06-03T02:31:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.187+0000 7f8ba9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba4102ea0 0x7f8ba4116c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.187+0000 7f8ba9a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba41085f0 0x7f8ba4117170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.187+0000 7f8ba9a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ba411c950 0x7f8ba411ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.187+0000 7f8ba9a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba4076330 con 0x7f8ba411c950 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.187+0000 7f8ba9a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba40761b0 con 0x7f8ba41085f0 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.187+0000 7f8ba9a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba40764b0 con 0x7f8ba4102ea0 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.187+0000 7f8ba2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba4102ea0 0x7f8ba4116c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.187+0000 7f8ba2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba4102ea0 0x7f8ba4116c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55078/0 (socket says 172.21.15.79:55078) 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.187+0000 7f8ba2a00700 1 -- 172.21.15.79:0/962188377 learned_addr learned my addr 172.21.15.79:0/962188377 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.188+0000 7f8ba3400700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ba411c950 0x7f8ba411ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.188+0000 7f8ba2000700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba41085f0 0x7f8ba4117170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.189+0000 7f8ba2000700 1 -- 172.21.15.79:0/962188377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba4102ea0 msgr2=0x7f8ba4116c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.189+0000 7f8ba2000700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba4102ea0 0x7f8ba4116c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.189+0000 7f8ba2000700 1 -- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ba411c950 msgr2=0x7f8ba411ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.189+0000 7f8ba2000700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ba411c950 0x7f8ba411ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.189+0000 7f8ba2000700 1 -- 172.21.15.79:0/962188377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba4111da0 con 0x7f8ba41085f0 2024-06-03T02:31:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.189+0000 7f8ba2000700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba41085f0 0x7f8ba4117170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c008000 tx=0x7f8b8c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.193+0000 7f8b9b400700 1 -- 172.21.15.79:0/962188377 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c005320 con 0x7f8ba41085f0 2024-06-03T02:31:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.193+0000 7f8b9b400700 1 -- 172.21.15.79:0/962188377 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8c0054c0 con 0x7f8ba41085f0 2024-06-03T02:31:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.193+0000 7f8b9b400700 1 -- 172.21.15.79:0/962188377 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c017400 con 0x7f8ba41085f0 2024-06-03T02:31:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.193+0000 7f8ba9a00700 1 -- 172.21.15.79:0/962188377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba4112030 con 0x7f8ba41085f0 2024-06-03T02:31:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.193+0000 7f8ba9a00700 1 -- 172.21.15.79:0/962188377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba4112490 con 0x7f8ba41085f0 2024-06-03T02:31:45.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.200+0000 7f8bb42e4b80 1 Processor -- start 2024-06-03T02:31:45.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.201+0000 7f8b9b400700 1 -- 172.21.15.79:0/962188377 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8b8c003680 con 0x7f8ba41085f0 2024-06-03T02:31:45.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.202+0000 7f8bb42e4b80 1 -- start start 2024-06-03T02:31:45.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.203+0000 7f8b9b400700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b880a2da0 0x7f8b880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.204+0000 7f8ba3400700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b880a2da0 0x7f8b880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.204+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x5645676a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.204+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 0x5645676a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.204+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a1240 0x5645676a1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.204+0000 7f8bb42e4b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5645675afa10 con 0x5645676a1240 2024-06-03T02:31:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.204+0000 7f8bb42e4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5645675f2b00 con 0x5645676a74b0 2024-06-03T02:31:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.204+0000 7f8bb42e4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5645675ad8f0 con 0x5645676a6b90 2024-06-03T02:31:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.205+0000 7f8ba2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x5645676a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.205+0000 7f8b9b400700 1 -- 172.21.15.79:0/962188377 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8b8c0cfea0 con 0x7f8ba41085f0 2024-06-03T02:31:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.205+0000 7f8ba2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x5645676a6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55094/0 (socket says 172.21.15.79:55094) 2024-06-03T02:31:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.205+0000 7f8ba2a00700 1 -- 172.21.15.79:0/1035405457 learned_addr learned my addr 172.21.15.79:0/1035405457 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.205+0000 7f8ba2000700 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 0x5645676a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.206+0000 7f8ba3400700 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a1240 0x5645676a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.206+0000 7f8ba2a00700 1 -- 172.21.15.79:0/1035405457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 msgr2=0x5645676a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.206+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 0x5645676a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.206+0000 7f8ba2a00700 1 -- 172.21.15.79:0/1035405457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a1240 msgr2=0x5645676a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.206+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a1240 0x5645676a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.206+0000 7f8ba2a00700 1 -- 172.21.15.79:0/1035405457 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456759af90 con 0x5645676a6b90 2024-06-03T02:31:45.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.206+0000 7f8ba2000700 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 0x5645676a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:31:45.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.206+0000 7f8ba3400700 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a1240 0x5645676a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:31:45.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.207+0000 7f8ba3400700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b880a2da0 0x7f8b880a5260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c007fd0 tx=0x7f8b9c007d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:45.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.208+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x5645676a6f70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009d40 tx=0x7f8b940116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0078cb15f3ce2c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:45.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.209+0000 7f8b93e00700 1 -- 172.21.15.79:0/1035405457 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94012450 con 0x5645676a6b90 2024-06-03T02:31:45.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.209+0000 7f8b93e00700 1 -- 172.21.15.79:0/1035405457 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b94012ad0 con 0x5645676a6b90 2024-06-03T02:31:45.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.209+0000 7f8b93e00700 1 -- 172.21.15.79:0/1035405457 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9401aa50 con 0x5645676a6b90 2024-06-03T02:31:45.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.210+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/1035405457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 msgr2=0x5645676a6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:45.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.210+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x5645676a6f70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009d40 tx=0x7f8b940116c0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.210+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/1035405457 shutdown_connections 2024-06-03T02:31:45.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.211+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a1240 0x5645676a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.211+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x5645676a6f70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.211+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/1035405457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 0x5645676a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.211+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/1035405457 >> 172.21.15.79:0/1035405457 conn(0x5645675cd170 msgr2=0x56456769bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.212+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/1035405457 shutdown_connections 2024-06-03T02:31:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.212+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/1035405457 wait complete. 2024-06-03T02:31:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.213+0000 7f8bb42e4b80 1 -- start start 2024-06-03T02:31:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.213+0000 7f8bb42e4b80 1 Processor -- start 2024-06-03T02:31:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.214+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.214+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a6b90 0x5645676aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.214+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 0x5645676aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.215+0000 7f8ba2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.215+0000 7f8ba2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55108/0 (socket says 172.21.15.79:55108) 2024-06-03T02:31:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.215+0000 7f8ba2000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a6b90 0x5645676aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.215+0000 7f8ba2000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a6b90 0x5645676aa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:45566/0 (socket says 172.21.15.79:45566) 2024-06-03T02:31:45.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.215+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56456758ef60 con 0x5645676a6b90 2024-06-03T02:31:45.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.215+0000 7f8ba2a00700 1 -- 172.21.15.79:0/918629033 learned_addr learned my addr 172.21.15.79:0/918629033 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:45.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.215+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5645675eeac0 con 0x5645676a74b0 2024-06-03T02:31:45.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.215+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5645676ade90 con 0x5645676a1240 2024-06-03T02:31:45.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.215+0000 7f8bb42e4b80 10 client.0 _ll_register_callbacks cb 0x564567588380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:45.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.216+0000 7f8ba2a00700 1 -- 172.21.15.79:0/918629033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 msgr2=0x5645676aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:45.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.216+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 0x5645676aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.216+0000 7f8ba2a00700 1 -- 172.21.15.79:0/918629033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a6b90 msgr2=0x5645676aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.216+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a6b90 0x5645676aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.216+0000 7f8ba2a00700 1 -- 172.21.15.79:0/918629033 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645675418d0 con 0x5645676a1240 2024-06-03T02:31:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.219+0000 7f8ba2000700 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a6b90 0x5645676aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:31:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.220+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676ad140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b9401a5d0 tx=0x7f8b94005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=872ad5c07dc5a156 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.220+0000 7f8b90c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:31:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.221+0000 7f8b90c00700 1 -- 172.21.15.79:0/918629033 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94029020 con 0x5645676a1240 2024-06-03T02:31:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.221+0000 7f8b90c00700 1 -- 172.21.15.79:0/918629033 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8b94012660 con 0x5645676a1240 2024-06-03T02:31:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.221+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b94012ad0 con 0x5645676a1240 2024-06-03T02:31:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.221+0000 7f8b90c00700 1 -- 172.21.15.79:0/918629033 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b94012860 con 0x5645676a1240 2024-06-03T02:31:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.221+0000 7f8bb42e4b80 10 client.26632 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.221+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645676b84f0 con 0x5645676a1240 2024-06-03T02:31:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.222+0000 7f8b90c00700 1 -- 172.21.15.79:0/918629033 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b94012ad0 con 0x5645676a1240 2024-06-03T02:31:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.222+0000 7f8b90c00700 1 -- 172.21.15.79:0/918629033 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8b9400f040 con 0x5645676a1240 2024-06-03T02:31:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.224+0000 7f8bb42e4b80 10 client.26632 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.224+0000 7f8bb42e4b80 10 client.26632 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:31:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.224+0000 7f8bb42e4b80 20 client.26632 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.224+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5645676baa20 0x5645676bce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.224+0000 7f8bb42e4b80 4 client.26632 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:31:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.224+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564567600000 con 0x5645676baa20 2024-06-03T02:31:45.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.225+0000 7f8ba2000700 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5645676baa20 0x5645676bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.226+0000 7f8ba2000700 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5645676baa20 0x5645676bce10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c77cd05f08b83593 server_cookie=f505e9e6da6a9f3e in_seq=0 out_seq=0 2024-06-03T02:31:45.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.226+0000 7f8b90c00700 10 client.26632 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:31:45.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.227+0000 7f8b90c00700 1 -- 172.21.15.79:0/918629033 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564567600000 con 0x5645676baa20 2024-06-03T02:31:45.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.227+0000 7f8b90c00700 10 client.26632 handle_command_reply: tid=0 2024-06-03T02:31:45.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.227+0000 7f8bb42e4b80 1 client.26632 shutdown 2024-06-03T02:31:45.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.228+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 msgr2=0x5645676ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:45.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.228+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676ad140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b9401a5d0 tx=0x7f8b94005e10 comp rx=0 tx=0).stop 2024-06-03T02:31:45.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.230+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 shutdown_connections 2024-06-03T02:31:45.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.230+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5645676baa20 0x5645676bce10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.230+0000 7f8b90c00700 0 client.26632 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:31:45.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.230+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a74b0 0x5645676aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.230+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676ad140 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.230+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/918629033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a6b90 0x5645676aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.230+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 >> 172.21.15.79:0/918629033 conn(0x5645675cd170 msgr2=0x564567695790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:45.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.231+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 shutdown_connections 2024-06-03T02:31:45.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.231+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/918629033 wait complete. 2024-06-03T02:31:45.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.231+0000 7f8bb42e4b80 20 client.26632 trim_cache size 0 max 16384 2024-06-03T02:31:45.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.268+0000 7f8bb42e4b80 1 Processor -- start 2024-06-03T02:31:45.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.268+0000 7f8bb42e4b80 1 -- start start 2024-06-03T02:31:45.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.268+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a1240 0x5645676a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.268+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x56456769c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.268+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a74b0 0x564567696130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.268+0000 7f8bb42e4b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8000eb40 con 0x5645676a74b0 2024-06-03T02:31:45.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.269+0000 7f8bb42e4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b80010680 con 0x5645676a1240 2024-06-03T02:31:45.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.269+0000 7f8bb42e4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8000f210 con 0x5645676a6b90 2024-06-03T02:31:45.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.269+0000 7f8ba2000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x56456769c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.269+0000 7f8ba2000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x56456769c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55120/0 (socket says 172.21.15.79:55120) 2024-06-03T02:31:45.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.269+0000 7f8ba2000700 1 -- 172.21.15.79:0/2260200695 learned_addr learned my addr 172.21.15.79:0/2260200695 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:45.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.269+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/2260200695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a1240 0x5645676a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.270+0000 7f8ba2000700 1 -- 172.21.15.79:0/2260200695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a1240 msgr2=0x5645676a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:45.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.270+0000 7f8ba2000700 1 --2- 172.21.15.79:0/2260200695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a1240 0x5645676a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.270+0000 7f8ba2000700 1 -- 172.21.15.79:0/2260200695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a74b0 msgr2=0x564567696130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:45.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.270+0000 7f8ba2000700 1 --2- 172.21.15.79:0/2260200695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a74b0 0x564567696130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.270+0000 7f8ba2000700 1 -- 172.21.15.79:0/2260200695 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645676a89b0 con 0x5645676a6b90 2024-06-03T02:31:45.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.270+0000 7f8ba2000700 1 --2- 172.21.15.79:0/2260200695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x56456769c1a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c098e30 tx=0x7f8b8c009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0fdf282fc83d532 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:45.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.271+0000 7f8b93400700 1 -- 172.21.15.79:0/2260200695 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c09c460 con 0x5645676a6b90 2024-06-03T02:31:45.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.271+0000 7f8b93400700 1 -- 172.21.15.79:0/2260200695 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8c0a0810 con 0x5645676a6b90 2024-06-03T02:31:45.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.272+0000 7f8b93400700 1 -- 172.21.15.79:0/2260200695 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0a4bb0 con 0x5645676a6b90 2024-06-03T02:31:45.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.272+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/2260200695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 msgr2=0x56456769c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:45.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.273+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/2260200695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x56456769c1a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c098e30 tx=0x7f8b8c009630 comp rx=0 tx=0).stop 2024-06-03T02:31:45.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.273+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/2260200695 shutdown_connections 2024-06-03T02:31:45.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.273+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/2260200695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a74b0 0x564567696130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.273+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/2260200695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a1240 0x5645676a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.273+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/2260200695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a6b90 0x56456769c1a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.273+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/2260200695 >> 172.21.15.79:0/2260200695 conn(0x5645675cd170 msgr2=0x56456769ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:45.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.273+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/2260200695 shutdown_connections 2024-06-03T02:31:45.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.274+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/2260200695 wait complete. 2024-06-03T02:31:45.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.274+0000 7f8bb42e4b80 1 -- start start 2024-06-03T02:31:45.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.275+0000 7f8bb42e4b80 1 Processor -- start 2024-06-03T02:31:45.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.275+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.275+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a6b90 0x5645676b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.275+0000 7f8bb42e4b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a74b0 0x5645676c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.275+0000 7f8bb42e4b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8000ffb0 con 0x5645676a74b0 2024-06-03T02:31:45.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.275+0000 7f8bb42e4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8000f8e0 con 0x5645676a6b90 2024-06-03T02:31:45.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.275+0000 7f8bb42e4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8000cae0 con 0x5645676a1240 2024-06-03T02:31:45.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.275+0000 7f8bb42e4b80 10 client.0 _ll_register_callbacks cb 0x564567588380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:45.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.276+0000 7f8ba2000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a6b90 0x5645676b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.276+0000 7f8ba2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.276+0000 7f8ba3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a74b0 0x5645676c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.276+0000 7f8ba2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55132/0 (socket says 172.21.15.79:55132) 2024-06-03T02:31:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.276+0000 7f8ba2a00700 1 -- 172.21.15.79:0/79098688 learned_addr learned my addr 172.21.15.79:0/79098688 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.277+0000 7f8ba2a00700 1 -- 172.21.15.79:0/79098688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a6b90 msgr2=0x5645676b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.277+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a6b90 0x5645676b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.277+0000 7f8ba2a00700 1 -- 172.21.15.79:0/79098688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a74b0 msgr2=0x5645676c1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.277+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a74b0 0x5645676c1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.277+0000 7f8ba2a00700 1 -- 172.21.15.79:0/79098688 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645676c2e90 con 0x5645676a1240 2024-06-03T02:31:45.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.278+0000 7f8b90c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:31:45.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.278+0000 7f8ba2a00700 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676b31b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b94000c00 tx=0x7f8b94012de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f206218919194ff server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:45.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.279+0000 7f8b90c00700 1 -- 172.21.15.79:0/79098688 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9403a030 con 0x5645676a1240 2024-06-03T02:31:45.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.279+0000 7f8b90c00700 1 -- 172.21.15.79:0/79098688 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8b9403b070 con 0x5645676a1240 2024-06-03T02:31:45.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.283+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/79098688 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b8c0a0810 con 0x5645676a1240 2024-06-03T02:31:45.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.283+0000 7f8b90c00700 1 -- 172.21.15.79:0/79098688 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b9403a030 con 0x5645676a1240 2024-06-03T02:31:45.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.284+0000 7f8bb42e4b80 10 client.26638 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:45.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.284+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/79098688 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645677247c0 con 0x5645676a1240 2024-06-03T02:31:45.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.284+0000 7f8b90c00700 1 -- 172.21.15.79:0/79098688 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b94005e80 con 0x5645676a1240 2024-06-03T02:31:45.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.286+0000 7f8b90c00700 1 -- 172.21.15.79:0/79098688 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8b94005100 con 0x5645676a1240 2024-06-03T02:31:45.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.286+0000 7f8bb42e4b80 10 client.26638 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:45.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.286+0000 7f8bb42e4b80 10 client.26638 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:31:45.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.286+0000 7f8bb42e4b80 20 client.26638 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:45.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.286+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5645676baa20 0x564567726e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:45.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.287+0000 7f8bb42e4b80 4 client.26638 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:31:45.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.287+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/79098688 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8b8c0a0810 con 0x5645676baa20 2024-06-03T02:31:45.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.287+0000 7f8ba2000700 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5645676baa20 0x564567726e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:45.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.288+0000 7f8ba2000700 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5645676baa20 0x564567726e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e5bc2c79b34c1bc server_cookie=8207a96601e87a68 in_seq=0 out_seq=0 2024-06-03T02:31:45.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.288+0000 7f8b90c00700 10 client.26638 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.288+0000 7f8b90c00700 1 -- 172.21.15.79:0/79098688 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8b8c0a0810 con 0x5645676baa20 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.288+0000 7f8b90c00700 10 client.26638 handle_command_reply: tid=0 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.288+0000 7f8bb42e4b80 1 client.26638 shutdown 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.289+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/79098688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 msgr2=0x5645676b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.289+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676b31b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b94000c00 tx=0x7f8b94012de0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.289+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/79098688 shutdown_connections 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.289+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x5645676baa20 0x564567726e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.289+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5645676a74b0 0x5645676c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.289+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645676a1240 0x5645676b31b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.289+0000 7f8b90c00700 0 client.26638 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:31:45.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.289+0000 7f8bb42e4b80 1 --2- 172.21.15.79:0/79098688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645676a6b90 0x5645676b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.289+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/79098688 >> 172.21.15.79:0/79098688 conn(0x5645675cd170 msgr2=0x5645676a7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:45.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.292+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/79098688 shutdown_connections 2024-06-03T02:31:45.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.293+0000 7f8bb42e4b80 1 -- 172.21.15.79:0/79098688 wait complete. 2024-06-03T02:31:45.296 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:31:45.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.295+0000 7f8bb42e4b80 20 client.26638 trim_cache size 0 max 16384 2024-06-03T02:31:45.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.296+0000 7f8ba9a00700 1 -- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b880a2da0 msgr2=0x7f8b880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:45.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.296+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b880a2da0 0x7f8b880a5260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c007fd0 tx=0x7f8b9c007d40 comp rx=0 tx=0).stop 2024-06-03T02:31:45.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.296+0000 7f8ba9a00700 1 -- 172.21.15.79:0/962188377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba41085f0 msgr2=0x7f8ba4117170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:45.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.296+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba41085f0 0x7f8ba4117170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c008000 tx=0x7f8b8c00b840 comp rx=0 tx=0).stop 2024-06-03T02:31:45.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.297+0000 7f8ba9a00700 1 -- 172.21.15.79:0/962188377 shutdown_connections 2024-06-03T02:31:45.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.297+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8b880a2da0 0x7f8b880a5260 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.297+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ba411c950 0x7f8ba411ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.297+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba4102ea0 0x7f8ba4116c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.297+0000 7f8ba9a00700 1 --2- 172.21.15.79:0/962188377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba41085f0 0x7f8ba4117170 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:45.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.297+0000 7f8ba9a00700 1 -- 172.21.15.79:0/962188377 >> 172.21.15.79:0/962188377 conn(0x7f8ba4079640 msgr2=0x7f8ba4101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:45.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.298+0000 7f8ba9a00700 1 -- 172.21.15.79:0/962188377 shutdown_connections 2024-06-03T02:31:45.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:45.298+0000 7f8ba9a00700 1 -- 172.21.15.79:0/962188377 wait complete. 2024-06-03T02:31:46.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:46 smithi079 ceph-mon[31819]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 474 op/s 2024-06-03T02:31:46.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:46 smithi135 ceph-mon[36653]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 474 op/s 2024-06-03T02:31:46.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:46 smithi123 ceph-mon[36640]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 474 op/s 2024-06-03T02:31:47.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:31:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 700.0 s, 494.0 tps, lat 197.266 ms stddev 243.869 2024-06-03T02:31:48.589 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:31:48.769 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:48 smithi079 ceph-mon[31819]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-06-03T02:31:48.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:48 smithi135 ceph-mon[36653]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-06-03T02:31:48.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:48 smithi123 ceph-mon[36640]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-06-03T02:31:50.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:50 smithi079 ceph-mon[31819]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 510 op/s 2024-06-03T02:31:50.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:50 smithi135 ceph-mon[36653]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 510 op/s 2024-06-03T02:31:50.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:50 smithi123 ceph-mon[36640]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 510 op/s 2024-06-03T02:31:52.792 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:52 smithi079 ceph-mon[31819]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 461 op/s 2024-06-03T02:31:52.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:52 smithi135 ceph-mon[36653]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 461 op/s 2024-06-03T02:31:52.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:52 smithi123 ceph-mon[36640]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 461 op/s 2024-06-03T02:31:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 705.0 s, 430.8 tps, lat 227.890 ms stddev 289.888 2024-06-03T02:31:53.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.500+0000 7febe1400700 1 -- 172.21.15.79:0/3198106202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd40a4de0 msgr2=0x7febd40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:53.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.500+0000 7febe1400700 1 --2- 172.21.15.79:0/3198106202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd40a4de0 0x7febd40a51c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7febd0009990 tx=0x7febd0009ca0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.501+0000 7febe1400700 1 -- 172.21.15.79:0/3198106202 shutdown_connections 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.501+0000 7febe1400700 1 --2- 172.21.15.79:0/3198106202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd40b7b00 0x7febd40b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.501+0000 7febe1400700 1 --2- 172.21.15.79:0/3198106202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd40a4de0 0x7febd40a51c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.501+0000 7febe1400700 1 --2- 172.21.15.79:0/3198106202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febd40a5700 0x7febd40b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.501+0000 7febe1400700 1 -- 172.21.15.79:0/3198106202 >> 172.21.15.79:0/3198106202 conn(0x7febd401a6a0 msgr2=0x7febd401aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.501+0000 7febe1400700 1 -- 172.21.15.79:0/3198106202 shutdown_connections 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.501+0000 7febe1400700 1 -- 172.21.15.79:0/3198106202 wait complete. 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.502+0000 7febe1400700 1 Processor -- start 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.503+0000 7febe1400700 1 -- start start 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.503+0000 7febe1400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febd40a4de0 0x7febd40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.505+0000 7febe1400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd40a5700 0x7febd40ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.505+0000 7febe1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd40b7b00 0x7febd40ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.505+0000 7febdb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd40a5700 0x7febd40ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.505+0000 7febdb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd40a5700 0x7febd40ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53704/0 (socket says 172.21.15.79:53704) 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.505+0000 7febdbe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febd40a4de0 0x7febd40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.505+0000 7febe0a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd40b7b00 0x7febd40ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.505+0000 7febdb400700 1 -- 172.21.15.79:0/233818349 learned_addr learned my addr 172.21.15.79:0/233818349 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.506+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7febd40bd6f0 con 0x7febd40a4de0 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.506+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febd40bd570 con 0x7febd40a5700 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.506+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febd40bd870 con 0x7febd40b7b00 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.507+0000 7febdb400700 1 -- 172.21.15.79:0/233818349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd40b7b00 msgr2=0x7febd40ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.508+0000 7febdb400700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd40b7b00 0x7febd40ad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.508+0000 7febdb400700 1 -- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febd40a4de0 msgr2=0x7febd40b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:53.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.508+0000 7febdb400700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febd40a4de0 0x7febd40b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.508+0000 7febdb400700 1 -- 172.21.15.79:0/233818349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd40adcd0 con 0x7febd40a5700 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.509+0000 7febdbe00700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febd40a4de0 0x7febd40b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.509+0000 7febe0a00700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd40b7b00 0x7febd40ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.510+0000 7febdb400700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd40a5700 0x7febd40ace90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7febc800d950 tx=0x7febc800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.510+0000 7febd8c00700 1 -- 172.21.15.79:0/233818349 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8016070 con 0x7febd40a5700 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.510+0000 7febd8c00700 1 -- 172.21.15.79:0/233818349 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc8009b40 con 0x7febd40a5700 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.510+0000 7febd8c00700 1 -- 172.21.15.79:0/233818349 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc800f620 con 0x7febd40a5700 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.510+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd41589f0 con 0x7febd40a5700 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.512+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd4158f30 con 0x7febd40a5700 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.517+0000 7febd8c00700 1 -- 172.21.15.79:0/233818349 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7febc8019070 con 0x7febd40a5700 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.519+0000 7febd8c00700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7febc40a2da0 0x7febc40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.519+0000 7febd8c00700 1 -- 172.21.15.79:0/233818349 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7febc80ced50 con 0x7febd40a5700 2024-06-03T02:31:53.738 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.520+0000 7febdbe00700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7febc40a2da0 0x7febc40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.521+0000 7febdbe00700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7febc40a2da0 0x7febc40a5260 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7febd0006010 tx=0x7febd00046b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.535+0000 7febed2bbb80 1 Processor -- start 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.535+0000 7febed2bbb80 1 -- start start 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.537+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 0x55b1ab025140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.537+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab025680 0x55b1ab02fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.537+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab032920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.537+0000 7febed2bbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b1aaf34a10 con 0x55b1ab024d60 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.537+0000 7febed2bbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b1aaf77b00 con 0x55b1ab025680 2024-06-03T02:31:53.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.537+0000 7febed2bbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b1aaf328f0 con 0x55b1ab030530 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.539+0000 7febe0a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab032920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.539+0000 7febe0a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab032920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55144/0 (socket says 172.21.15.79:55144) 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.539+0000 7febe0a00700 1 -- 172.21.15.79:0/2174909616 learned_addr learned my addr 172.21.15.79:0/2174909616 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.540+0000 7febdbe00700 1 --2- 172.21.15.79:0/2174909616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 0x55b1ab025140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.541+0000 7febe0a00700 1 -- 172.21.15.79:0/2174909616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab025680 msgr2=0x55b1ab02fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.541+0000 7febe0a00700 1 --2- 172.21.15.79:0/2174909616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab025680 0x55b1ab02fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.541+0000 7febe0a00700 1 -- 172.21.15.79:0/2174909616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 msgr2=0x55b1ab025140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.541+0000 7febe0a00700 1 --2- 172.21.15.79:0/2174909616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 0x55b1ab025140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.541+0000 7febe0a00700 1 -- 172.21.15.79:0/2174909616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1aaf1ff90 con 0x55b1ab030530 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.544+0000 7febe0a00700 1 --2- 172.21.15.79:0/2174909616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab032920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7febdc065ce0 tx=0x7febdc0692d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8d81117df71acd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.546+0000 7febcd600700 1 -- 172.21.15.79:0/2174909616 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc06ab60 con 0x55b1ab030530 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.546+0000 7febcd600700 1 -- 172.21.15.79:0/2174909616 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febdc07b070 con 0x55b1ab030530 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.546+0000 7febcd600700 1 -- 172.21.15.79:0/2174909616 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc0766a0 con 0x55b1ab030530 2024-06-03T02:31:53.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.547+0000 7febed2bbb80 1 -- 172.21.15.79:0/2174909616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 msgr2=0x55b1ab032920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.547+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2174909616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab032920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7febdc065ce0 tx=0x7febdc0692d0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.550+0000 7febed2bbb80 1 -- 172.21.15.79:0/2174909616 shutdown_connections 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.550+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2174909616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab032920 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.550+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2174909616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 0x55b1ab025140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.550+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2174909616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab025680 0x55b1ab02fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.550+0000 7febed2bbb80 1 -- 172.21.15.79:0/2174909616 >> 172.21.15.79:0/2174909616 conn(0x55b1aaf52170 msgr2=0x55b1ab021280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.550+0000 7febed2bbb80 1 -- 172.21.15.79:0/2174909616 shutdown_connections 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.551+0000 7febed2bbb80 1 -- 172.21.15.79:0/2174909616 wait complete. 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.552+0000 7febed2bbb80 1 -- start start 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.552+0000 7febed2bbb80 1 Processor -- start 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.556+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab02fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.557+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab02cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.557+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab030530 0x55b1ab02d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.557+0000 7febed2bbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b1aaf13f60 con 0x55b1ab030530 2024-06-03T02:31:53.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.558+0000 7febdb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab02cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.558+0000 7febdb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab02cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55150/0 (socket says 172.21.15.79:55150) 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.558+0000 7febdb400700 1 -- 172.21.15.79:0/2446370602 learned_addr learned my addr 172.21.15.79:0/2446370602 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.559+0000 7febdbe00700 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab02fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.560+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b1aaf73ac0 con 0x55b1ab024d60 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.560+0000 7febe0a00700 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab030530 0x55b1ab02d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.560+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b1ab032fc0 con 0x55b1ab025680 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.560+0000 7febed2bbb80 10 client.0 _ll_register_callbacks cb 0x55b1aaf0d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.561+0000 7febdbe00700 1 -- 172.21.15.79:0/2446370602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 msgr2=0x55b1ab02cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.561+0000 7febdbe00700 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab02cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.561+0000 7febdbe00700 1 -- 172.21.15.79:0/2446370602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab030530 msgr2=0x55b1ab02d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.561+0000 7febdbe00700 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab030530 0x55b1ab02d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.561+0000 7febdbe00700 1 -- 172.21.15.79:0/2446370602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1aaec68d0 con 0x55b1ab024d60 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.563+0000 7febdbe00700 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab02fcf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7febd0000c00 tx=0x7febd0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69182b1b3f0cfad9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.563+0000 7febc2000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.563+0000 7febc2000700 1 -- 172.21.15.79:0/2446370602 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000fde0 con 0x55b1ab024d60 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.564+0000 7febc2000700 1 -- 172.21.15.79:0/2446370602 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7febd001e030 con 0x55b1ab024d60 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.565+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febdc07b070 con 0x55b1ab024d60 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.565+0000 7febc2000700 1 -- 172.21.15.79:0/2446370602 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febd0030420 con 0x55b1ab024d60 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.566+0000 7febed2bbb80 10 client.26654 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.566+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1ab03d740 con 0x55b1ab024d60 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.566+0000 7febc2000700 1 -- 172.21.15.79:0/2446370602 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febdc07b070 con 0x55b1ab024d60 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.566+0000 7febc2000700 1 -- 172.21.15.79:0/2446370602 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7febd0025680 con 0x55b1ab024d60 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.567+0000 7febed2bbb80 10 client.26654 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.567+0000 7febed2bbb80 10 client.26654 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.567+0000 7febed2bbb80 20 client.26654 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:53.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.567+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b1ab03fdd0 0x55b1ab0421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.568+0000 7febed2bbb80 4 client.26654 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.568+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1aaf85000 con 0x55b1ab03fdd0 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.568+0000 7febdb400700 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b1ab03fdd0 0x55b1ab0421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.570+0000 7febdb400700 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b1ab03fdd0 0x55b1ab0421c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=716657e42690d078 server_cookie=714d390b44851c4a in_seq=0 out_seq=0 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.570+0000 7febc2000700 10 client.26654 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.571+0000 7febc2000700 1 -- 172.21.15.79:0/2446370602 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b1aaf85000 con 0x55b1ab03fdd0 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.571+0000 7febc2000700 10 client.26654 handle_command_reply: tid=0 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.572+0000 7febed2bbb80 1 client.26654 shutdown 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.573+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 msgr2=0x55b1ab02fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.573+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab02fcf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7febd0000c00 tx=0x7febd0009310 comp rx=0 tx=0).stop 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.573+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 shutdown_connections 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.573+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b1ab03fdd0 0x55b1ab0421c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.573+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab030530 0x55b1ab02d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.573+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab02fcf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.573+0000 7febed2bbb80 1 --2- 172.21.15.79:0/2446370602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab02cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.573+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 >> 172.21.15.79:0/2446370602 conn(0x55b1aaf52170 msgr2=0x55b1ab020e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.573+0000 7febc2000700 0 client.26654 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.574+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 shutdown_connections 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.574+0000 7febed2bbb80 1 -- 172.21.15.79:0/2446370602 wait complete. 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.574+0000 7febed2bbb80 20 client.26654 trim_cache size 0 max 16384 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.605+0000 7febed2bbb80 1 Processor -- start 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.605+0000 7febed2bbb80 1 -- start start 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.605+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab0213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.606+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab025680 0x55b1ab02f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.606+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab027320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.606+0000 7febed2bbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7febd4161b00 con 0x55b1ab025680 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.606+0000 7febed2bbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febd4163640 con 0x55b1ab024d60 2024-06-03T02:31:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.606+0000 7febed2bbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febd41621d0 con 0x55b1ab030530 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.607+0000 7febdbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab0213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.607+0000 7febdbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab0213c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53736/0 (socket says 172.21.15.79:53736) 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.607+0000 7febdbe00700 1 -- 172.21.15.79:0/3529239370 learned_addr learned my addr 172.21.15.79:0/3529239370 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.607+0000 7febe0a00700 1 --2- 172.21.15.79:0/3529239370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab027320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.607+0000 7febdbe00700 1 -- 172.21.15.79:0/3529239370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 msgr2=0x55b1ab027320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.607+0000 7febdbe00700 1 --2- 172.21.15.79:0/3529239370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab027320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.607+0000 7febdbe00700 1 -- 172.21.15.79:0/3529239370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab025680 msgr2=0x55b1ab02f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.607+0000 7febdbe00700 1 --2- 172.21.15.79:0/3529239370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab025680 0x55b1ab02f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.607+0000 7febdbe00700 1 -- 172.21.15.79:0/3529239370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1ab02d8b0 con 0x55b1ab024d60 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.609+0000 7febdbe00700 1 --2- 172.21.15.79:0/3529239370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab0213c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7febd000c010 tx=0x7febd00255b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13a9b299859c911 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.609+0000 7febccc00700 1 -- 172.21.15.79:0/3529239370 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0029070 con 0x55b1ab024d60 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.609+0000 7febccc00700 1 -- 172.21.15.79:0/3529239370 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd0025850 con 0x55b1ab024d60 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.609+0000 7febccc00700 1 -- 172.21.15.79:0/3529239370 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0023890 con 0x55b1ab024d60 2024-06-03T02:31:53.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.611+0000 7febed2bbb80 1 -- 172.21.15.79:0/3529239370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 msgr2=0x55b1ab0213c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.611+0000 7febed2bbb80 1 --2- 172.21.15.79:0/3529239370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab0213c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7febd000c010 tx=0x7febd00255b0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.611+0000 7febed2bbb80 1 -- 172.21.15.79:0/3529239370 shutdown_connections 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.611+0000 7febed2bbb80 1 --2- 172.21.15.79:0/3529239370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab030530 0x55b1ab027320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.611+0000 7febed2bbb80 1 --2- 172.21.15.79:0/3529239370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab024d60 0x55b1ab0213c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.611+0000 7febed2bbb80 1 --2- 172.21.15.79:0/3529239370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab025680 0x55b1ab02f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.611+0000 7febed2bbb80 1 -- 172.21.15.79:0/3529239370 >> 172.21.15.79:0/3529239370 conn(0x55b1aaf52170 msgr2=0x55b1ab026c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.611+0000 7febed2bbb80 1 -- 172.21.15.79:0/3529239370 shutdown_connections 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.612+0000 7febed2bbb80 1 -- 172.21.15.79:0/3529239370 wait complete. 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.612+0000 7febed2bbb80 1 -- start start 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.612+0000 7febed2bbb80 1 Processor -- start 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.614+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 0x55b1ab038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.614+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab0387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.614+0000 7febed2bbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab030530 0x55b1ab035240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.614+0000 7febed2bbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7febd4162f70 con 0x55b1ab024d60 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.614+0000 7febed2bbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febd41628a0 con 0x55b1ab030530 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.614+0000 7febed2bbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febd415faa0 con 0x55b1ab025680 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.614+0000 7febed2bbb80 10 client.0 _ll_register_callbacks cb 0x55b1aaf0d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.614+0000 7febe0a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab030530 0x55b1ab035240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.615+0000 7febe0a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab030530 0x55b1ab035240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:53750/0 (socket says 172.21.15.79:53750) 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.615+0000 7febe0a00700 1 -- 172.21.15.79:0/4162180344 learned_addr learned my addr 172.21.15.79:0/4162180344 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.615+0000 7febdb400700 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab0387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.615+0000 7febdbe00700 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 0x55b1ab038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.616+0000 7febdb400700 1 -- 172.21.15.79:0/4162180344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab030530 msgr2=0x55b1ab035240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.616+0000 7febdb400700 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab030530 0x55b1ab035240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.616+0000 7febdb400700 1 -- 172.21.15.79:0/4162180344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 msgr2=0x55b1ab038270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:53.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.616+0000 7febdb400700 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 0x55b1ab038270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.616+0000 7febdb400700 1 -- 172.21.15.79:0/4162180344 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1ab038e30 con 0x55b1ab025680 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.616+0000 7febdb400700 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab0387b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7febc809af00 tx=0x7febc809afd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d0a6aed39bad098 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.620+0000 7febc2000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.624+0000 7febc2000700 1 -- 172.21.15.79:0/4162180344 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80b4060 con 0x55b1ab025680 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.624+0000 7febc2000700 1 -- 172.21.15.79:0/4162180344 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7febc8099500 con 0x55b1ab025680 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.624+0000 7febed2bbb80 1 -- 172.21.15.79:0/4162180344 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd0025850 con 0x55b1ab025680 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febc2000700 1 -- 172.21.15.79:0/4162180344 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febc80b9050 con 0x55b1ab025680 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febed2bbb80 10 client.26653 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febed2bbb80 1 -- 172.21.15.79:0/4162180344 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1ab0aafe0 con 0x55b1ab025680 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febc2000700 1 -- 172.21.15.79:0/4162180344 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd0025850 con 0x55b1ab025680 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febc2000700 1 -- 172.21.15.79:0/4162180344 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7febc8023060 con 0x55b1ab025680 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febed2bbb80 10 client.26653 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febed2bbb80 10 client.26653 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:31:53.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febed2bbb80 20 client.26653 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febed2bbb80 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b1ab0ab6a0 0x55b1ab0adac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febed2bbb80 4 client.26653 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.625+0000 7febed2bbb80 1 -- 172.21.15.79:0/4162180344 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1aaf7f580 con 0x55b1ab0ab6a0 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.626+0000 7febdbe00700 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b1ab0ab6a0 0x55b1ab0adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.627+0000 7febdbe00700 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b1ab0ab6a0 0x55b1ab0adac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdc1f051b80c8f9a server_cookie=b6945102ac573fb in_seq=0 out_seq=0 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.627+0000 7febc2000700 10 client.26653 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.627+0000 7febc2000700 1 -- 172.21.15.79:0/4162180344 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1aaf7f580 con 0x55b1ab0ab6a0 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.627+0000 7febc2000700 10 client.26653 handle_command_reply: tid=0 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.627+0000 7febed2bbb80 1 client.26653 shutdown 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 -- 172.21.15.79:0/4162180344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 msgr2=0x55b1ab0387b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab0387b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7febc809af00 tx=0x7febc809afd0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 -- 172.21.15.79:0/4162180344 shutdown_connections 2024-06-03T02:31:53.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1ab030530 0x55b1ab035240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55b1ab0ab6a0 0x55b1ab0adac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b1ab024d60 0x55b1ab038270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 --2- 172.21.15.79:0/4162180344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1ab025680 0x55b1ab0387b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febc2000700 0 client.26653 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 -- 172.21.15.79:0/4162180344 >> 172.21.15.79:0/4162180344 conn(0x55b1aaf52170 msgr2=0x55b1ab02d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 -- 172.21.15.79:0/4162180344 shutdown_connections 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 1 -- 172.21.15.79:0/4162180344 wait complete. 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.628+0000 7febed2bbb80 20 client.26653 trim_cache size 0 max 16384 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.630+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7febc40a2da0 msgr2=0x7febc40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.630+0000 7febe1400700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7febc40a2da0 0x7febc40a5260 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7febd0006010 tx=0x7febd00046b0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.630+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd40a5700 msgr2=0x7febd40ace90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.630+0000 7febe1400700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd40a5700 0x7febd40ace90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7febc800d950 tx=0x7febc800dd10 comp rx=0 tx=0).stop 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.631+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 shutdown_connections 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.631+0000 7febe1400700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd40b7b00 0x7febd40ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.631+0000 7febe1400700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7febc40a2da0 0x7febc40a5260 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.631+0000 7febe1400700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febd40a4de0 0x7febd40b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.631+0000 7febe1400700 1 --2- 172.21.15.79:0/233818349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd40a5700 0x7febd40ace90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.631+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 >> 172.21.15.79:0/233818349 conn(0x7febd401a6a0 msgr2=0x7febd40b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.631+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 shutdown_connections 2024-06-03T02:31:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:53.631+0000 7febe1400700 1 -- 172.21.15.79:0/233818349 wait complete. 2024-06-03T02:31:54.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:31:54.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:54 smithi079 ceph-mon[31819]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 474 op/s 2024-06-03T02:31:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:54 smithi135 ceph-mon[36653]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 474 op/s 2024-06-03T02:31:54.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:54 smithi123 ceph-mon[36640]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 474 op/s 2024-06-03T02:31:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:31:56.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:56 smithi079 ceph-mon[31819]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 484 op/s 2024-06-03T02:31:56.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:56 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:31:56.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:56 smithi135 ceph-mon[36653]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 484 op/s 2024-06-03T02:31:56.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:56 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:31:56.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:56 smithi123 ceph-mon[36640]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 484 op/s 2024-06-03T02:31:56.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:56 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:31:58.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 710.0 s, 463.2 tps, lat 223.479 ms stddev 296.021 2024-06-03T02:31:58.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:58 smithi079 ceph-mon[31819]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 MiB/s wr, 401 op/s 2024-06-03T02:31:58.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:31:58.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:31:58.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:31:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:31:58.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:58 smithi135 ceph-mon[36653]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 MiB/s wr, 401 op/s 2024-06-03T02:31:58.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:31:58.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:31:58.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:31:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:31:58.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:58 smithi123 ceph-mon[36640]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 MiB/s wr, 401 op/s 2024-06-03T02:31:58.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:31:58.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:31:58.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:31:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:31:59.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.085+0000 7f31fb800700 1 -- 172.21.15.79:0/3356631520 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f410a660 msgr2=0x7f31f410aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.085+0000 7f31fb800700 1 --2- 172.21.15.79:0/3356631520 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f410a660 0x7f31f410aa40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f31e80098f0 tx=0x7f31e8009c00 comp rx=0 tx=0).stop 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.085+0000 7f31fb800700 1 -- 172.21.15.79:0/3356631520 shutdown_connections 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.085+0000 7f31fb800700 1 --2- 172.21.15.79:0/3356631520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4105020 0x7f31f4111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.085+0000 7f31fb800700 1 --2- 172.21.15.79:0/3356631520 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f410a660 0x7f31f410aa40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.085+0000 7f31fb800700 1 --2- 172.21.15.79:0/3356631520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31f4104660 0x7f31f4104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.085+0000 7f31fb800700 1 -- 172.21.15.79:0/3356631520 >> 172.21.15.79:0/3356631520 conn(0x7f31f4100190 msgr2=0x7f31f41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.085+0000 7f31fb800700 1 -- 172.21.15.79:0/3356631520 shutdown_connections 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.085+0000 7f31fb800700 1 -- 172.21.15.79:0/3356631520 wait complete. 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.086+0000 7f31fb800700 1 Processor -- start 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.086+0000 7f31fb800700 1 -- start start 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31fb800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31f4104660 0x7f31f419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31fb800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4105020 0x7f31f419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31fb800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f410a660 0x7f31f419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31fb800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4114a00 con 0x7f31f410a660 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31fb800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4114880 con 0x7f31f4105020 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31fb800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4114b80 con 0x7f31f4104660 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31f8e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31f4104660 0x7f31f419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31f3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4105020 0x7f31f419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31f8e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31f4104660 0x7f31f419af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55444/0 (socket says 172.21.15.79:55444) 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31f8e00700 1 -- 172.21.15.79:0/992715363 learned_addr learned my addr 172.21.15.79:0/992715363 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.087+0000 7f31f9800700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f410a660 0x7f31f419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.088+0000 7f31f8e00700 1 -- 172.21.15.79:0/992715363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4105020 msgr2=0x7f31f419b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.088+0000 7f31f8e00700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4105020 0x7f31f419b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.088+0000 7f31f8e00700 1 -- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f410a660 msgr2=0x7f31f419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:59.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.088+0000 7f31f8e00700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f410a660 0x7f31f419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.088+0000 7f31f8e00700 1 -- 172.21.15.79:0/992715363 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f419ffa0 con 0x7f31f4104660 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.088+0000 7f31f9800700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f410a660 0x7f31f419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.088+0000 7f31f8e00700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31f4104660 0x7f31f419af90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005950 tx=0x7f31e8005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.089+0000 7f31f1600700 1 -- 172.21.15.79:0/992715363 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e80052e0 con 0x7f31f4104660 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.089+0000 7f31f1600700 1 -- 172.21.15.79:0/992715363 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e800ba40 con 0x7f31f4104660 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.089+0000 7f31f1600700 1 -- 172.21.15.79:0/992715363 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8003cb0 con 0x7f31f4104660 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.089+0000 7f31fb800700 1 -- 172.21.15.79:0/992715363 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f41a0230 con 0x7f31f4104660 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.089+0000 7f31fb800700 1 -- 172.21.15.79:0/992715363 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f41a7b10 con 0x7f31f4104660 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.092+0000 7f31f1600700 1 -- 172.21.15.79:0/992715363 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f31e8005510 con 0x7f31f4104660 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.094+0000 7f31f1600700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f31dc0a2ca0 0x7f31dc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.094+0000 7f31f1600700 1 -- 172.21.15.79:0/992715363 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f31e80d0420 con 0x7f31f4104660 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.095+0000 7f320624ab80 1 Processor -- start 2024-06-03T02:31:59.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.095+0000 7f320624ab80 1 -- start start 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.095+0000 7f31f3e00700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f31dc0a2ca0 0x7f31dc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.096+0000 7f320624ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fda780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.096+0000 7f320624ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fd4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.096+0000 7f320624ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 0x555700fd4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.096+0000 7f320624ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555700edaa10 con 0x555700fd4a50 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.096+0000 7f320624ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555700f1db00 con 0x555700fda3a0 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.096+0000 7f320624ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555700ed88f0 con 0x555700fdacc0 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.096+0000 7f31f8e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fda780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.096+0000 7f31f8e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fda780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:40996/0 (socket says 172.21.15.79:40996) 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.096+0000 7f31f8e00700 1 -- 172.21.15.79:0/2908983733 learned_addr learned my addr 172.21.15.79:0/2908983733 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31f8e00700 1 -- 172.21.15.79:0/2908983733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 msgr2=0x555700fd4510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31f8e00700 1 --2- 172.21.15.79:0/2908983733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fd4510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31f8e00700 1 -- 172.21.15.79:0/2908983733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 msgr2=0x555700fd4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31f8e00700 1 --2- 172.21.15.79:0/2908983733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 0x555700fd4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31f8e00700 1 -- 172.21.15.79:0/2908983733 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555700ec5f90 con 0x555700fda3a0 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31f8e00700 1 --2- 172.21.15.79:0/2908983733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fda780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005e90 tx=0x7f31e8003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f39691687609e56 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31f3e00700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f31dc0a2ca0 0x7f31dc0a5160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31f4109d20 tx=0x7f31e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31e6000700 1 -- 172.21.15.79:0/2908983733 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e80a0cb0 con 0x555700fda3a0 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31e6000700 1 -- 172.21.15.79:0/2908983733 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e80a54e0 con 0x555700fda3a0 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.097+0000 7f31e6000700 1 -- 172.21.15.79:0/2908983733 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8021380 con 0x555700fda3a0 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 -- 172.21.15.79:0/2908983733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 msgr2=0x555700fda780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 --2- 172.21.15.79:0/2908983733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fda780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005e90 tx=0x7f31e8003680 comp rx=0 tx=0).stop 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 -- 172.21.15.79:0/2908983733 shutdown_connections 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 --2- 172.21.15.79:0/2908983733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 0x555700fd4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 --2- 172.21.15.79:0/2908983733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fda780 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 --2- 172.21.15.79:0/2908983733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fd4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 -- 172.21.15.79:0/2908983733 >> 172.21.15.79:0/2908983733 conn(0x555700ef8170 msgr2=0x555700fc6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 -- 172.21.15.79:0/2908983733 shutdown_connections 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 -- 172.21.15.79:0/2908983733 wait complete. 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.098+0000 7f320624ab80 1 -- start start 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f320624ab80 1 Processor -- start 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f320624ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f320624ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fda3a0 0x555700fddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f320624ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fde320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f320624ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555700eb9f60 con 0x555700fda3a0 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f320624ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555700f19ac0 con 0x555700fd4a50 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f320624ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555700fe1450 con 0x555700fdacc0 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f320624ab80 10 client.0 _ll_register_callbacks cb 0x555700eb3380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f31f8e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f31f8e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41000/0 (socket says 172.21.15.79:41000) 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.099+0000 7f31f8e00700 1 -- 172.21.15.79:0/3316441153 learned_addr learned my addr 172.21.15.79:0/3316441153 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31f3e00700 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fda3a0 0x555700fddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31f9800700 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fde320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31f8e00700 1 -- 172.21.15.79:0/3316441153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 msgr2=0x555700fde320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31f8e00700 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fde320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31f8e00700 1 -- 172.21.15.79:0/3316441153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fda3a0 msgr2=0x555700fddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31f8e00700 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fda3a0 0x555700fddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31f8e00700 1 -- 172.21.15.79:0/3316441153 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555700e6c8d0 con 0x555700fd4a50 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31f8e00700 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe0ae0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f31e809c890 tx=0x7f31e809c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1349e95165c27b7f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31daa00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31daa00700 1 -- 172.21.15.79:0/3316441153 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e80ba060 con 0x555700fd4a50 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f320624ab80 1 -- 172.21.15.79:0/3316441153 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31e80a54e0 con 0x555700fd4a50 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31daa00700 1 -- 172.21.15.79:0/3316441153 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f31e8098f70 con 0x555700fd4a50 2024-06-03T02:31:59.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f31daa00700 1 -- 172.21.15.79:0/3316441153 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f31e80bb250 con 0x555700fd4a50 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f320624ab80 10 client.26672 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.100+0000 7f320624ab80 1 -- 172.21.15.79:0/3316441153 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555700febc00 con 0x555700fd4a50 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.101+0000 7f31daa00700 1 -- 172.21.15.79:0/3316441153 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e80a54e0 con 0x555700fd4a50 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.101+0000 7f31daa00700 1 -- 172.21.15.79:0/3316441153 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f31e8087110 con 0x555700fd4a50 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.101+0000 7f320624ab80 10 client.26672 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.101+0000 7f320624ab80 10 client.26672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.101+0000 7f320624ab80 20 client.26672 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.101+0000 7f320624ab80 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x555700fedf00 0x555700ff02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.101+0000 7f320624ab80 4 client.26672 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.101+0000 7f320624ab80 1 -- 172.21.15.79:0/3316441153 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555700f2b000 con 0x555700fedf00 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.102+0000 7f31f3e00700 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x555700fedf00 0x555700ff02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.102+0000 7f31f3e00700 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x555700fedf00 0x555700ff02f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d63c05fca46d5cba server_cookie=34316f7162520425 in_seq=0 out_seq=0 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.103+0000 7f31daa00700 10 client.26672 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:31:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.103+0000 7f31daa00700 1 -- 172.21.15.79:0/3316441153 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555700f2b000 con 0x555700fedf00 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.103+0000 7f31daa00700 10 client.26672 handle_command_reply: tid=0 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.103+0000 7f320624ab80 1 client.26672 shutdown 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 -- 172.21.15.79:0/3316441153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 msgr2=0x555700fe0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe0ae0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f31e809c890 tx=0x7f31e809c800 comp rx=0 tx=0).stop 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 -- 172.21.15.79:0/3316441153 shutdown_connections 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fde320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x555700fedf00 0x555700ff02f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe0ae0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 --2- 172.21.15.79:0/3316441153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fda3a0 0x555700fddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 -- 172.21.15.79:0/3316441153 >> 172.21.15.79:0/3316441153 conn(0x555700ef8170 msgr2=0x555700fc0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f31daa00700 0 client.26672 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 -- 172.21.15.79:0/3316441153 shutdown_connections 2024-06-03T02:31:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 1 -- 172.21.15.79:0/3316441153 wait complete. 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.104+0000 7f320624ab80 20 client.26672 trim_cache size 0 max 16384 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.125+0000 7f320624ab80 1 Processor -- start 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.125+0000 7f320624ab80 1 -- start start 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.125+0000 7f320624ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 0x555700fea530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.125+0000 7f320624ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fc1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.125+0000 7f320624ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fdb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.125+0000 7f320624ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41ac170 con 0x555700fd4a50 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.125+0000 7f320624ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41adcb0 con 0x555700fda3a0 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.125+0000 7f320624ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41ac840 con 0x555700fdacc0 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fc1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f9800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fdb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f8e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 0x555700fea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fc1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41006/0 (socket says 172.21.15.79:41006) 2024-06-03T02:31:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f9800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fdb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:55480/0 (socket says 172.21.15.79:55480) 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f3e00700 1 -- 172.21.15.79:0/3621981399 learned_addr learned my addr 172.21.15.79:0/3621981399 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f3e00700 1 -- 172.21.15.79:0/3621981399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 msgr2=0x555700fdb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f3e00700 1 --2- 172.21.15.79:0/3621981399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fdb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f3e00700 1 -- 172.21.15.79:0/3621981399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 msgr2=0x555700fea530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f3e00700 1 --2- 172.21.15.79:0/3621981399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 0x555700fea530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f3e00700 1 -- 172.21.15.79:0/3621981399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555700fdc350 con 0x555700fda3a0 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f8e00700 1 --2- 172.21.15.79:0/3621981399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 0x555700fea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f9800700 1 --2- 172.21.15.79:0/3621981399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fdb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.126+0000 7f31f3e00700 1 --2- 172.21.15.79:0/3621981399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fc1130 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x555700fc7e20 tx=0x7f31e0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f392729a5cbe18 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.127+0000 7f31e5600700 1 -- 172.21.15.79:0/3621981399 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0012d10 con 0x555700fda3a0 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.127+0000 7f31e5600700 1 -- 172.21.15.79:0/3621981399 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e002f070 con 0x555700fda3a0 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.127+0000 7f31e5600700 1 -- 172.21.15.79:0/3621981399 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0020690 con 0x555700fda3a0 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 -- 172.21.15.79:0/3621981399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 msgr2=0x555700fc1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:59.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 --2- 172.21.15.79:0/3621981399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fc1130 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x555700fc7e20 tx=0x7f31e0014000 comp rx=0 tx=0).stop 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 -- 172.21.15.79:0/3621981399 shutdown_connections 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 --2- 172.21.15.79:0/3621981399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fdacc0 0x555700fdb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 --2- 172.21.15.79:0/3621981399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fd4a50 0x555700fea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 --2- 172.21.15.79:0/3621981399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fda3a0 0x555700fc1130 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 -- 172.21.15.79:0/3621981399 >> 172.21.15.79:0/3621981399 conn(0x555700ef8170 msgr2=0x555700fdb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 -- 172.21.15.79:0/3621981399 shutdown_connections 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 -- 172.21.15.79:0/3621981399 wait complete. 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.128+0000 7f320624ab80 1 -- start start 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.129+0000 7f320624ab80 1 Processor -- start 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.129+0000 7f320624ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.129+0000 7f320624ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fda3a0 0x555700fe6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.129+0000 7f320624ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fdacc0 0x555700ff4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.129+0000 7f320624ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41ad5e0 con 0x555700fdacc0 2024-06-03T02:31:59.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.129+0000 7f320624ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41acf10 con 0x555700fd4a50 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.129+0000 7f320624ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31f41aa110 con 0x555700fda3a0 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.129+0000 7f320624ab80 10 client.0 _ll_register_callbacks cb 0x555700eb3380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f8e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f8e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:41014/0 (socket says 172.21.15.79:41014) 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f8e00700 1 -- 172.21.15.79:0/1522557821 learned_addr learned my addr 172.21.15.79:0/1522557821 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f3e00700 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fda3a0 0x555700fe6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f9800700 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fdacc0 0x555700ff4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f8e00700 1 -- 172.21.15.79:0/1522557821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fda3a0 msgr2=0x555700fe6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f8e00700 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fda3a0 0x555700fe6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f8e00700 1 -- 172.21.15.79:0/1522557821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fdacc0 msgr2=0x555700ff4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f8e00700 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fdacc0 0x555700ff4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f8e00700 1 -- 172.21.15.79:0/1522557821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555700ff4eb0 con 0x555700fd4a50 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.130+0000 7f31f9800700 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fdacc0 0x555700ff4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:31:59.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.131+0000 7f31f8e00700 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe66a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009fd0 tx=0x7f31e809c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=301e6ca7fa661538 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.131+0000 7f31daa00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.131+0000 7f31daa00700 1 -- 172.21.15.79:0/1522557821 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e80ba070 con 0x555700fd4a50 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.131+0000 7f320624ab80 1 -- 172.21.15.79:0/1522557821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31e002f070 con 0x555700fd4a50 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.131+0000 7f31daa00700 1 -- 172.21.15.79:0/1522557821 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f31e80b7e60 con 0x555700fd4a50 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.131+0000 7f31daa00700 1 -- 172.21.15.79:0/1522557821 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f31e809cdf0 con 0x555700fd4a50 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.131+0000 7f320624ab80 10 client.26678 fetch_fsmap learned FSMap version 30 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.131+0000 7f320624ab80 1 -- 172.21.15.79:0/1522557821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555700ff4eb0 con 0x555700fd4a50 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.132+0000 7f31daa00700 1 -- 172.21.15.79:0/1522557821 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e002f070 con 0x555700fd4a50 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.132+0000 7f31daa00700 1 -- 172.21.15.79:0/1522557821 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f31e80a5050 con 0x555700fd4a50 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.132+0000 7f320624ab80 10 client.26678 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.132+0000 7f320624ab80 10 client.26678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.132+0000 7f320624ab80 20 client.26678 populate_metadata read hostname 'smithi079' 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.132+0000 7f320624ab80 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x555701058290 0x55570105a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:31:59.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.132+0000 7f320624ab80 4 client.26678 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.132+0000 7f320624ab80 1 -- 172.21.15.79:0/1522557821 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555700f25580 con 0x555701058290 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.132+0000 7f31f3e00700 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x555701058290 0x55570105a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.133+0000 7f31f3e00700 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x555701058290 0x55570105a680 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cc3b1f774a58463 server_cookie=f711dec5902d4528 in_seq=0 out_seq=0 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.133+0000 7f31daa00700 10 client.26678 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.133+0000 7f31daa00700 1 -- 172.21.15.79:0/1522557821 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555700f25580 con 0x555701058290 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.133+0000 7f31daa00700 10 client.26678 handle_command_reply: tid=0 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.133+0000 7f320624ab80 1 client.26678 shutdown 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.134+0000 7f320624ab80 1 -- 172.21.15.79:0/1522557821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 msgr2=0x555700fe66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.134+0000 7f320624ab80 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe66a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009fd0 tx=0x7f31e809c390 comp rx=0 tx=0).stop 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.135+0000 7f320624ab80 1 -- 172.21.15.79:0/1522557821 shutdown_connections 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.135+0000 7f320624ab80 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555700fdacc0 0x555700ff4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.135+0000 7f320624ab80 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x555701058290 0x55570105a680 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.135+0000 7f320624ab80 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fd4a50 0x555700fe66a0 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009fd0 tx=0x7f31e809c390 comp rx=0 tx=0).stop 2024-06-03T02:31:59.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.135+0000 7f320624ab80 1 --2- 172.21.15.79:0/1522557821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700fda3a0 0x555700fe6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.135+0000 7f320624ab80 1 -- 172.21.15.79:0/1522557821 >> 172.21.15.79:0/1522557821 conn(0x555700ef8170 msgr2=0x555700fdb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:59.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.135+0000 7f31daa00700 0 client.26678 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:31:59.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.135+0000 7f320624ab80 1 -- 172.21.15.79:0/1522557821 shutdown_connections 2024-06-03T02:31:59.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.135+0000 7f320624ab80 1 -- 172.21.15.79:0/1522557821 wait complete. 2024-06-03T02:31:59.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f320624ab80 20 client.26678 trim_cache size 0 max 16384 2024-06-03T02:31:59.521 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 -- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f31dc0a2ca0 msgr2=0x7f31dc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f31dc0a2ca0 0x7f31dc0a5160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31f4109d20 tx=0x7f31e0009260 comp rx=0 tx=0).stop 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 -- 172.21.15.79:0/992715363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31f4104660 msgr2=0x7f31f419af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31f4104660 0x7f31f419af90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005950 tx=0x7f31e8005040 comp rx=0 tx=0).stop 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 -- 172.21.15.79:0/992715363 shutdown_connections 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f31dc0a2ca0 0x7f31dc0a5160 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f410a660 0x7f31f419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31f4104660 0x7f31f419af90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 --2- 172.21.15.79:0/992715363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f4105020 0x7f31f419b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.136+0000 7f31fb800700 1 -- 172.21.15.79:0/992715363 >> 172.21.15.79:0/992715363 conn(0x7f31f4100190 msgr2=0x7f31f41009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.137+0000 7f31fb800700 1 -- 172.21.15.79:0/992715363 shutdown_connections 2024-06-03T02:31:59.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:31:59.137+0000 7f31fb800700 1 -- 172.21.15.79:0/992715363 wait complete. 2024-06-03T02:32:00.354 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:32:00.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:00 smithi079 ceph-mon[31819]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 512 op/s 2024-06-03T02:32:00.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:00 smithi135 ceph-mon[36653]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 512 op/s 2024-06-03T02:32:00.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:00 smithi123 ceph-mon[36640]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 512 op/s 2024-06-03T02:32:01.146 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:32:02.804 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:02 smithi079 ceph-mon[31819]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 429 op/s 2024-06-03T02:32:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:02 smithi135 ceph-mon[36653]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 429 op/s 2024-06-03T02:32:02.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:02 smithi123 ceph-mon[36640]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 429 op/s 2024-06-03T02:32:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 715.0 s, 548.6 tps, lat 180.680 ms stddev 236.460 2024-06-03T02:32:04.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:04 smithi079 ceph-mon[31819]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.6 MiB/s wr, 483 op/s 2024-06-03T02:32:04.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:04 smithi135 ceph-mon[36653]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.6 MiB/s wr, 483 op/s 2024-06-03T02:32:04.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.929+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1528747630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e9c06ab30 msgr2=0x7f5e9c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:04.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.929+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1528747630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e9c06ab30 0x7f5e9c10f320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009990 tx=0x7f5e88009ca0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.933+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1528747630 shutdown_connections 2024-06-03T02:32:04.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.933+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1528747630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e9c06ab30 0x7f5e9c10f320 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.933+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1528747630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9c108500 0x7f5e9c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.933+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1528747630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e9c06a170 0x7f5e9c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.933+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1528747630 >> 172.21.15.79:0/1528747630 conn(0x7f5e9c078340 msgr2=0x7f5e9c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:04.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.933+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1528747630 shutdown_connections 2024-06-03T02:32:04.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.933+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1528747630 wait complete. 2024-06-03T02:32:04.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.935+0000 7f5ea0e00700 1 Processor -- start 2024-06-03T02:32:04.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.935+0000 7f5ea0e00700 1 -- start start 2024-06-03T02:32:04.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5ea0e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e9c06a170 0x7f5e9c109310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5ea0e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e9c06ab30 0x7f5e9c109850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5ea0e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9c108500 0x7f5e9c109d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5ea0e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c112760 con 0x7f5e9c06ab30 2024-06-03T02:32:04.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5ea0e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c1125e0 con 0x7f5e9c108500 2024-06-03T02:32:04.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5ea0e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c1128e0 con 0x7f5e9c06a170 2024-06-03T02:32:04.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e9c06a170 0x7f5e9c109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e9c06a170 0x7f5e9c109310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37090/0 (socket says 172.21.15.79:37090) 2024-06-03T02:32:04.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5e99e00700 1 -- 172.21.15.79:0/1643765860 learned_addr learned my addr 172.21.15.79:0/1643765860 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:04.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.936+0000 7f5e9a800700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9c108500 0x7f5e9c109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.937+0000 7f5e99e00700 1 -- 172.21.15.79:0/1643765860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9c108500 msgr2=0x7f5e9c109d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:04.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.937+0000 7f5e99e00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9c108500 0x7f5e9c109d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.937+0000 7f5e99e00700 1 -- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e9c06ab30 msgr2=0x7f5e9c109850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T02:32:04.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.938+0000 7f5e99400700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e9c06ab30 0x7f5e9c109850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.939+0000 7f5e99e00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e9c06ab30 0x7f5e9c109850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.939+0000 7f5e99e00700 1 -- 172.21.15.79:0/1643765860 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c1ab4c0 con 0x7f5e9c06a170 2024-06-03T02:32:04.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.940+0000 7f5e99e00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e9c06a170 0x7f5e9c109310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c3a0 tx=0x7f5e9400c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:04.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.940+0000 7f5e92a00700 1 -- 172.21.15.79:0/1643765860 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940090d0 con 0x7f5e9c06a170 2024-06-03T02:32:04.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.940+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1643765860 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9c1ab750 con 0x7f5e9c06a170 2024-06-03T02:32:04.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.940+0000 7f5e92a00700 1 -- 172.21.15.79:0/1643765860 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e94009270 con 0x7f5e9c06a170 2024-06-03T02:32:04.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.940+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1643765860 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9c1abce0 con 0x7f5e9c06a170 2024-06-03T02:32:04.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.940+0000 7f5e92a00700 1 -- 172.21.15.79:0/1643765860 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940090d0 con 0x7f5e9c06a170 2024-06-03T02:32:04.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.956+0000 7f5e92a00700 1 -- 172.21.15.79:0/1643765860 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5e9401a020 con 0x7f5e9c06a170 2024-06-03T02:32:04.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.958+0000 7f5e92a00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5e800a3090 0x7f5e800a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.958+0000 7f5e92a00700 1 -- 172.21.15.79:0/1643765860 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5e94018020 con 0x7f5e9c06a170 2024-06-03T02:32:04.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.958+0000 7f5e99400700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5e800a3090 0x7f5e800a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.960+0000 7f5eab83eb80 1 Processor -- start 2024-06-03T02:32:04.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.960+0000 7f5eab83eb80 1 -- start start 2024-06-03T02:32:04.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df485b2e40 0x55df48636070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486365b0 0x55df48636a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48654700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df48545a10 con 0x55df48652310 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df48588b00 con 0x55df485b2e40 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55df485438f0 con 0x55df486365b0 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df485b2e40 0x55df48636070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df485b2e40 0x55df48636070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34836/0 (socket says 172.21.15.79:34836) 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5e99e00700 1 -- 172.21.15.79:0/2603131018 learned_addr learned my addr 172.21.15.79:0/2603131018 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.961+0000 7f5e99400700 1 --2- 172.21.15.79:0/2603131018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486365b0 0x55df48636a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.962+0000 7f5e99e00700 1 -- 172.21.15.79:0/2603131018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486365b0 msgr2=0x55df48636a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.962+0000 7f5e99e00700 1 --2- 172.21.15.79:0/2603131018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486365b0 0x55df48636a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.962+0000 7f5e9a800700 1 --2- 172.21.15.79:0/2603131018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48654700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.962+0000 7f5e99e00700 1 -- 172.21.15.79:0/2603131018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 msgr2=0x55df48654700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:04.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.962+0000 7f5e99e00700 1 --2- 172.21.15.79:0/2603131018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48654700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.962+0000 7f5e99e00700 1 -- 172.21.15.79:0/2603131018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df48530f90 con 0x55df485b2e40 2024-06-03T02:32:04.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.962+0000 7f5e99400700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5e800a3090 0x7f5e800a5550 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c10aad0 tx=0x7f5e8c0147a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:04.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.963+0000 7f5e99e00700 1 --2- 172.21.15.79:0/2603131018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df485b2e40 0x55df48636070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5e9409abd0 tx=0x7f5e9409aae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3da91a9e5b818fdd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:04.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.964+0000 7f5e87400700 1 -- 172.21.15.79:0/2603131018 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940090b0 con 0x55df485b2e40 2024-06-03T02:32:04.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.964+0000 7f5e87400700 1 -- 172.21.15.79:0/2603131018 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e9409b340 con 0x55df485b2e40 2024-06-03T02:32:04.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.964+0000 7f5e87400700 1 -- 172.21.15.79:0/2603131018 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940cbff0 con 0x55df485b2e40 2024-06-03T02:32:04.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.966+0000 7f5eab83eb80 1 -- 172.21.15.79:0/2603131018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df485b2e40 msgr2=0x55df48636070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:04.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.966+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/2603131018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df485b2e40 0x55df48636070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5e9409abd0 tx=0x7f5e9409aae0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.966+0000 7f5eab83eb80 1 -- 172.21.15.79:0/2603131018 shutdown_connections 2024-06-03T02:32:04.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.966+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/2603131018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48654700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.966+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/2603131018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df485b2e40 0x55df48636070 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.966+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/2603131018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486365b0 0x55df48636a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.966+0000 7f5eab83eb80 1 -- 172.21.15.79:0/2603131018 >> 172.21.15.79:0/2603131018 conn(0x55df48563170 msgr2=0x55df486324b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:04.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.970+0000 7f5eab83eb80 1 -- 172.21.15.79:0/2603131018 shutdown_connections 2024-06-03T02:32:04.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.970+0000 7f5eab83eb80 1 -- 172.21.15.79:0/2603131018 wait complete. 2024-06-03T02:32:04.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:04 smithi123 ceph-mon[36640]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.6 MiB/s wr, 483 op/s 2024-06-03T02:32:04.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.971+0000 7f5eab83eb80 1 -- start start 2024-06-03T02:32:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.972+0000 7f5eab83eb80 1 Processor -- start 2024-06-03T02:32:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.973+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df485b2e40 0x55df48651bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.973+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.973+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df48652310 0x55df4864f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.973+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df48524f60 con 0x55df485b2e40 2024-06-03T02:32:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.973+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df48584ac0 con 0x55df486365b0 2024-06-03T02:32:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.973+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55df48654fd0 con 0x55df48652310 2024-06-03T02:32:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.974+0000 7f5eab83eb80 10 client.0 _ll_register_callbacks cb 0x55df48522720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:04.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.974+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df485b2e40 0x55df48651bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.974+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df485b2e40 0x55df48651bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58512/0 (socket says 172.21.15.79:58512) 2024-06-03T02:32:04.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.975+0000 7f5e99e00700 1 -- 172.21.15.79:0/1038278370 learned_addr learned my addr 172.21.15.79:0/1038278370 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.975+0000 7f5e9a800700 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df48652310 0x55df4864f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.975+0000 7f5e99400700 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.975+0000 7f5e99e00700 1 -- 172.21.15.79:0/1038278370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df48652310 msgr2=0x55df4864f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.975+0000 7f5e99e00700 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df48652310 0x55df4864f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.975+0000 7f5e99e00700 1 -- 172.21.15.79:0/1038278370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 msgr2=0x55df4864ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.975+0000 7f5e99e00700 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.975+0000 7f5e99e00700 1 -- 172.21.15.79:0/1038278370 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df484d78d0 con 0x55df485b2e40 2024-06-03T02:32:04.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.976+0000 7f5e99400700 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:32:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.976+0000 7f5e99e00700 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df485b2e40 0x55df48651bc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c370 tx=0x7f5e9409a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05eac000d6697c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.976+0000 7f5e7be00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:32:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.977+0000 7f5e7be00700 1 -- 172.21.15.79:0/1038278370 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94004840 con 0x55df485b2e40 2024-06-03T02:32:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.977+0000 7f5e7be00700 1 -- 172.21.15.79:0/1038278370 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5e9409aea0 con 0x55df485b2e40 2024-06-03T02:32:04.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.977+0000 7f5eab83eb80 1 -- 172.21.15.79:0/1038278370 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e9409b340 con 0x55df485b2e40 2024-06-03T02:32:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.978+0000 7f5e7be00700 1 -- 172.21.15.79:0/1038278370 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e9407f050 con 0x55df485b2e40 2024-06-03T02:32:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.978+0000 7f5eab83eb80 10 client.16164 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.978+0000 7f5eab83eb80 1 -- 172.21.15.79:0/1038278370 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df48670790 con 0x55df485b2e40 2024-06-03T02:32:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.978+0000 7f5e7be00700 1 -- 172.21.15.79:0/1038278370 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e9409b340 con 0x55df485b2e40 2024-06-03T02:32:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.978+0000 7f5e7be00700 1 -- 172.21.15.79:0/1038278370 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5e940a3420 con 0x55df485b2e40 2024-06-03T02:32:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.979+0000 7f5eab83eb80 10 client.16164 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:04.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.979+0000 7f5eab83eb80 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:32:04.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.979+0000 7f5eab83eb80 20 client.16164 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:04.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.979+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55df48672c60 0x55df48675050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:04.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.979+0000 7f5eab83eb80 4 client.16164 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:32:04.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.979+0000 7f5eab83eb80 1 -- 172.21.15.79:0/1038278370 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df48596000 con 0x55df48672c60 2024-06-03T02:32:04.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.982+0000 7f5e99400700 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55df48672c60 0x55df48675050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:04.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.985+0000 7f5e7be00700 10 client.16164 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:32:04.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.985+0000 7f5e99400700 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55df48672c60 0x55df48675050 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=945c403e1790052 server_cookie=34eb86f8d227288f in_seq=0 out_seq=0 2024-06-03T02:32:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.987+0000 7f5e7be00700 1 -- 172.21.15.79:0/1038278370 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55df48596000 con 0x55df48672c60 2024-06-03T02:32:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.987+0000 7f5e7be00700 10 client.16164 handle_command_reply: tid=0 2024-06-03T02:32:04.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.987+0000 7f5eab83eb80 1 client.16164 shutdown 2024-06-03T02:32:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.988+0000 7f5eab83eb80 1 -- 172.21.15.79:0/1038278370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df485b2e40 msgr2=0x55df48651bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.988+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df485b2e40 0x55df48651bc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c370 tx=0x7f5e9409a880 comp rx=0 tx=0).stop 2024-06-03T02:32:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.988+0000 7f5eab83eb80 1 -- 172.21.15.79:0/1038278370 shutdown_connections 2024-06-03T02:32:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.988+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55df48672c60 0x55df48675050 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.988+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df48652310 0x55df4864f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.988+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df485b2e40 0x55df48651bc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.988+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/1038278370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:04.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.988+0000 7f5eab83eb80 1 -- 172.21.15.79:0/1038278370 >> 172.21.15.79:0/1038278370 conn(0x55df48563170 msgr2=0x55df48631740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.989+0000 7f5e7be00700 0 client.16164 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:32:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.989+0000 7f5eab83eb80 1 -- 172.21.15.79:0/1038278370 shutdown_connections 2024-06-03T02:32:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.989+0000 7f5eab83eb80 1 -- 172.21.15.79:0/1038278370 wait complete. 2024-06-03T02:32:04.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:04.989+0000 7f5eab83eb80 20 client.16164 trim_cache size 0 max 16384 2024-06-03T02:32:05.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.018+0000 7f5eab83eb80 1 Processor -- start 2024-06-03T02:32:05.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.018+0000 7f5eab83eb80 1 -- start start 2024-06-03T02:32:05.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.018+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df485b2e40 0x55df48654760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.018+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.018+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48653700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.018+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c19af30 con 0x55df48652310 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.018+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c19ca70 con 0x55df486365b0 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.018+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c19b600 con 0x55df485b2e40 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df485b2e40 0x55df48654760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df485b2e40 0x55df48654760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37118/0 (socket says 172.21.15.79:37118) 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34850/0 (socket says 172.21.15.79:34850) 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e9a800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48653700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99e00700 1 -- 172.21.15.79:0/4131191592 learned_addr learned my addr 172.21.15.79:0/4131191592 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:05.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99e00700 1 -- 172.21.15.79:0/4131191592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 msgr2=0x55df4864de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:05.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99e00700 1 --2- 172.21.15.79:0/4131191592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99e00700 1 -- 172.21.15.79:0/4131191592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 msgr2=0x55df48653700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:05.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99e00700 1 --2- 172.21.15.79:0/4131191592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48653700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99e00700 1 -- 172.21.15.79:0/4131191592 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df48637bd0 con 0x55df485b2e40 2024-06-03T02:32:05.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99400700 1 --2- 172.21.15.79:0/4131191592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:32:05.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e99e00700 1 --2- 172.21.15.79:0/4131191592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df485b2e40 0x55df48654760 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400cc90 tx=0x7f5e94004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffac438a0e5f3dee server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.019+0000 7f5e86a00700 1 -- 172.21.15.79:0/4131191592 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940b7050 con 0x55df485b2e40 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5e86a00700 1 -- 172.21.15.79:0/4131191592 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e940090a0 con 0x55df485b2e40 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5e86a00700 1 -- 172.21.15.79:0/4131191592 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9409eff0 con 0x55df485b2e40 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5eab83eb80 1 -- 172.21.15.79:0/4131191592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df485b2e40 msgr2=0x55df48654760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/4131191592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df485b2e40 0x55df48654760 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400cc90 tx=0x7f5e94004340 comp rx=0 tx=0).stop 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5eab83eb80 1 -- 172.21.15.79:0/4131191592 shutdown_connections 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/4131191592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48653700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/4131191592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df485b2e40 0x55df48654760 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/4131191592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df4864de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5eab83eb80 1 -- 172.21.15.79:0/4131191592 >> 172.21.15.79:0/4131191592 conn(0x55df48563170 msgr2=0x55df4864d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:05.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.020+0000 7f5eab83eb80 1 -- 172.21.15.79:0/4131191592 shutdown_connections 2024-06-03T02:32:05.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.021+0000 7f5eab83eb80 1 -- 172.21.15.79:0/4131191592 wait complete. 2024-06-03T02:32:05.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.021+0000 7f5eab83eb80 1 -- start start 2024-06-03T02:32:05.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.021+0000 7f5eab83eb80 1 Processor -- start 2024-06-03T02:32:05.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df486784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:05.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48672c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:05.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5eab83eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486731a0 0x55df486735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c19c3a0 con 0x55df48652310 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c19bcd0 con 0x55df486365b0 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5eab83eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c072a40 con 0x55df486731a0 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5eab83eb80 10 client.0 _ll_register_callbacks cb 0x55df48522720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5e99400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48672c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df486784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5e99e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df486784a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34862/0 (socket says 172.21.15.79:34862) 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5e99e00700 1 -- 172.21.15.79:0/3984244426 learned_addr learned my addr 172.21.15.79:0/3984244426 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.022+0000 7f5e9a800700 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486731a0 0x55df486735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5e99e00700 1 -- 172.21.15.79:0/3984244426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486731a0 msgr2=0x55df486735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5e99e00700 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486731a0 0x55df486735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5e99e00700 1 -- 172.21.15.79:0/3984244426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 msgr2=0x55df48672c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5e99e00700 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48672c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5e99e00700 1 -- 172.21.15.79:0/3984244426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df486db730 con 0x55df486365b0 2024-06-03T02:32:05.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5e99e00700 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df486784a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5e9409a940 tx=0x7f5e9409ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd78c8541370fc10 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:05.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5e7be00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:32:05.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5e7be00700 1 -- 172.21.15.79:0/3984244426 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940b7070 con 0x55df486365b0 2024-06-03T02:32:05.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5eab83eb80 1 -- 172.21.15.79:0/3984244426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e940090a0 con 0x55df486365b0 2024-06-03T02:32:05.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.023+0000 7f5e7be00700 1 -- 172.21.15.79:0/3984244426 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5e94085070 con 0x55df486365b0 2024-06-03T02:32:05.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5e7be00700 1 -- 172.21.15.79:0/3984244426 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e9409a780 con 0x55df486365b0 2024-06-03T02:32:05.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5eab83eb80 10 client.26696 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:05.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5eab83eb80 1 -- 172.21.15.79:0/3984244426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df486db730 con 0x55df486365b0 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5e7be00700 1 -- 172.21.15.79:0/3984244426 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e940090a0 con 0x55df486365b0 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5e7be00700 1 -- 172.21.15.79:0/3984244426 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5e94004640 con 0x55df486365b0 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5eab83eb80 10 client.26696 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5eab83eb80 10 client.26696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5eab83eb80 20 client.26696 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55df486dcc50 0x55df486df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5eab83eb80 4 client.26696 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.024+0000 7f5eab83eb80 1 -- 172.21.15.79:0/3984244426 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df48590580 con 0x55df486dcc50 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.025+0000 7f5e99400700 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55df486dcc50 0x55df486df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.025+0000 7f5e99400700 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55df486dcc50 0x55df486df040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15d11e3e50ed1199 server_cookie=ff07f74f4bdf0337 in_seq=0 out_seq=0 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.025+0000 7f5e7be00700 10 client.26696 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:32:05.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5e7be00700 1 -- 172.21.15.79:0/3984244426 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df48590580 con 0x55df486dcc50 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5e7be00700 10 client.26696 handle_command_reply: tid=0 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 client.26696 shutdown 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 -- 172.21.15.79:0/3984244426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 msgr2=0x55df486784a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df486784a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5e9409a940 tx=0x7f5e9409ac40 comp rx=0 tx=0).stop 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 -- 172.21.15.79:0/3984244426 shutdown_connections 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55df486dcc50 0x55df486df040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df486731a0 0x55df486735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df486365b0 0x55df486784a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 --2- 172.21.15.79:0/3984244426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df48652310 0x55df48672c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5e7be00700 0 client.26696 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 -- 172.21.15.79:0/3984244426 >> 172.21.15.79:0/3984244426 conn(0x55df48563170 msgr2=0x55df48631740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 -- 172.21.15.79:0/3984244426 shutdown_connections 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.026+0000 7f5eab83eb80 1 -- 172.21.15.79:0/3984244426 wait complete. 2024-06-03T02:32:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5eab83eb80 20 client.26696 trim_cache size 0 max 16384 2024-06-03T02:32:05.030 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:32:05.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5e800a3090 msgr2=0x7f5e800a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:05.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5e800a3090 0x7f5e800a5550 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c10aad0 tx=0x7f5e8c0147a0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1643765860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e9c06a170 msgr2=0x7f5e9c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:05.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e9c06a170 0x7f5e9c109310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c3a0 tx=0x7f5e9400c6b0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1643765860 shutdown_connections 2024-06-03T02:32:05.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e9c108500 0x7f5e9c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e9c06a170 0x7f5e9c109310 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5e800a3090 0x7f5e800a5550 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 --2- 172.21.15.79:0/1643765860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e9c06ab30 0x7f5e9c109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:05.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.027+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1643765860 >> 172.21.15.79:0/1643765860 conn(0x7f5e9c078340 msgr2=0x7f5e9c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:05.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.028+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1643765860 shutdown_connections 2024-06-03T02:32:05.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:05.028+0000 7f5ea0e00700 1 -- 172.21.15.79:0/1643765860 wait complete. 2024-06-03T02:32:06.005 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:32:06.652 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:32:06.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:06 smithi079 ceph-mon[31819]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-06-03T02:32:06.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:06 smithi135 ceph-mon[36653]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-06-03T02:32:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:06 smithi123 ceph-mon[36640]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-06-03T02:32:08.313 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 720.0 s, 484.6 tps, lat 206.042 ms stddev 252.812 2024-06-03T02:32:08.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:08 smithi135 ceph-mon[36653]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 452 op/s 2024-06-03T02:32:08.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:08 smithi123 ceph-mon[36640]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 452 op/s 2024-06-03T02:32:09.014 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:08 smithi079 ceph-mon[31819]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 452 op/s 2024-06-03T02:32:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.605+0000 7f556e600700 1 -- 172.21.15.79:0/3137360132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5568107840 msgr2=0x7f556810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.605+0000 7f556e600700 1 --2- 172.21.15.79:0/3137360132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5568107840 0x7f556810f3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f556000b0d0 tx=0x7f556000b3e0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.605+0000 7f556e600700 1 -- 172.21.15.79:0/3137360132 shutdown_connections 2024-06-03T02:32:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.605+0000 7f556e600700 1 --2- 172.21.15.79:0/3137360132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5568107840 0x7f556810f3c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.605+0000 7f556e600700 1 --2- 172.21.15.79:0/3137360132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5568102740 0x7f5568102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.605+0000 7f556e600700 1 --2- 172.21.15.79:0/3137360132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55681030f0 0x7f5568107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.605+0000 7f556e600700 1 -- 172.21.15.79:0/3137360132 >> 172.21.15.79:0/3137360132 conn(0x7f55680795f0 msgr2=0x7f5568079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:09.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.605+0000 7f556e600700 1 -- 172.21.15.79:0/3137360132 shutdown_connections 2024-06-03T02:32:09.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.605+0000 7f556e600700 1 -- 172.21.15.79:0/3137360132 wait complete. 2024-06-03T02:32:09.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.607+0000 7f556e600700 1 Processor -- start 2024-06-03T02:32:09.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.607+0000 7f556e600700 1 -- start start 2024-06-03T02:32:09.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f556e600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5568102740 0x7f556819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f556e600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55681030f0 0x7f556819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f556e600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5568107840 0x7f556819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f556e600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5568074b90 con 0x7f5568102740 2024-06-03T02:32:09.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f556e600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5568074a10 con 0x7f55681030f0 2024-06-03T02:32:09.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f556e600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5568074d10 con 0x7f5568107840 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f5566a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55681030f0 0x7f556819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f5566a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55681030f0 0x7f556819fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34890/0 (socket says 172.21.15.79:34890) 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f5566a00700 1 -- 172.21.15.79:0/2635357767 learned_addr learned my addr 172.21.15.79:0/2635357767 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f5567400700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5568102740 0x7f556819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.608+0000 7f5567e00700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5568107840 0x7f556819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.609+0000 7f5566a00700 1 -- 172.21.15.79:0/2635357767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5568107840 msgr2=0x7f556819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.609+0000 7f5566a00700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5568107840 0x7f556819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.609+0000 7f5566a00700 1 -- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5568102740 msgr2=0x7f556819f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.609+0000 7f5566a00700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5568102740 0x7f556819f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.609+0000 7f5566a00700 1 -- 172.21.15.79:0/2635357767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556819bb90 con 0x7f55681030f0 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.609+0000 7f5566a00700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55681030f0 0x7f556819fef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f555400c920 tx=0x7f555400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.609+0000 7f555fe00700 1 -- 172.21.15.79:0/2635357767 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554004a30 con 0x7f55681030f0 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.609+0000 7f555fe00700 1 -- 172.21.15.79:0/2635357767 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5554005500 con 0x7f55681030f0 2024-06-03T02:32:09.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.609+0000 7f555fe00700 1 -- 172.21.15.79:0/2635357767 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554007d40 con 0x7f55681030f0 2024-06-03T02:32:09.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.610+0000 7f556e600700 1 -- 172.21.15.79:0/2635357767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556819be80 con 0x7f55681030f0 2024-06-03T02:32:09.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.610+0000 7f556e600700 1 -- 172.21.15.79:0/2635357767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556806beb0 con 0x7f55681030f0 2024-06-03T02:32:09.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.617+0000 7f5578f2cb80 1 Processor -- start 2024-06-03T02:32:09.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.618+0000 7f5578f2cb80 1 -- start start 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b21070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b21a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2b2c300 0x5579b2b2e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f5578f2cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5579b2a30a10 con 0x5579b2a9de40 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f5578f2cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5579b2a73b00 con 0x5579b2b2c300 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f5578f2cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5579b2a2e8f0 con 0x5579b2b215b0 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f5567e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b21070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f5567e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b21070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58560/0 (socket says 172.21.15.79:58560) 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f5567e00700 1 -- 172.21.15.79:0/1441136458 learned_addr learned my addr 172.21.15.79:0/1441136458 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:09.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.619+0000 7f555fe00700 1 -- 172.21.15.79:0/2635357767 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5554004bd0 con 0x7f55681030f0 2024-06-03T02:32:09.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.620+0000 7f5567400700 1 --2- 172.21.15.79:0/1441136458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b21a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.620+0000 7f5567e00700 1 -- 172.21.15.79:0/1441136458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 msgr2=0x5579b2b21a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:09.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.620+0000 7f5567e00700 1 --2- 172.21.15.79:0/1441136458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b21a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.620+0000 7f5567e00700 1 -- 172.21.15.79:0/1441136458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2b2c300 msgr2=0x5579b2b2e6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:32:09.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.620+0000 7f5567e00700 1 --2- 172.21.15.79:0/1441136458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2b2c300 0x5579b2b2e6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.620+0000 7f5567e00700 1 -- 172.21.15.79:0/1441136458 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b2a1bf90 con 0x5579b2a9de40 2024-06-03T02:32:09.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.620+0000 7f5567400700 1 --2- 172.21.15.79:0/1441136458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b21a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:32:09.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.621+0000 7f5567e00700 1 --2- 172.21.15.79:0/1441136458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b21070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f556000cf00 tx=0x7f5560006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba7d726819fa1e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:09.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.621+0000 7f555cc00700 1 -- 172.21.15.79:0/1441136458 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560042070 con 0x5579b2a9de40 2024-06-03T02:32:09.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.621+0000 7f555cc00700 1 -- 172.21.15.79:0/1441136458 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5560009450 con 0x5579b2a9de40 2024-06-03T02:32:09.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.621+0000 7f555cc00700 1 -- 172.21.15.79:0/1441136458 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560047460 con 0x5579b2a9de40 2024-06-03T02:32:09.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.621+0000 7f555fe00700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f554c0a2d20 0x7f554c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.622+0000 7f555fe00700 1 -- 172.21.15.79:0/2635357767 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f55540cee80 con 0x7f55681030f0 2024-06-03T02:32:09.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.622+0000 7f5567400700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f554c0a2d20 0x7f554c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.622+0000 7f5578f2cb80 1 -- 172.21.15.79:0/1441136458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 msgr2=0x5579b2b21070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:09.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.622+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/1441136458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b21070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f556000cf00 tx=0x7f5560006210 comp rx=0 tx=0).stop 2024-06-03T02:32:09.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.623+0000 7f5578f2cb80 1 -- 172.21.15.79:0/1441136458 shutdown_connections 2024-06-03T02:32:09.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.623+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/1441136458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2b2c300 0x5579b2b2e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.623+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/1441136458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b21070 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.623+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/1441136458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b21a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.623+0000 7f5578f2cb80 1 -- 172.21.15.79:0/1441136458 >> 172.21.15.79:0/1441136458 conn(0x5579b2a4e170 msgr2=0x5579b2b1d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:09.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.623+0000 7f5567400700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f554c0a2d20 0x7f554c0a51e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5579b2b1e920 tx=0x7f555800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:09.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.623+0000 7f5578f2cb80 1 -- 172.21.15.79:0/1441136458 shutdown_connections 2024-06-03T02:32:09.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.623+0000 7f5578f2cb80 1 -- 172.21.15.79:0/1441136458 wait complete. 2024-06-03T02:32:09.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.624+0000 7f5578f2cb80 1 -- start start 2024-06-03T02:32:09.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.624+0000 7f5578f2cb80 1 Processor -- start 2024-06-03T02:32:09.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b2bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b28dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2b2c300 0x5579b2b29300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5578f2cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5579b2a0ff60 con 0x5579b2a9de40 2024-06-03T02:32:09.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5578f2cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5579b2a6fac0 con 0x5579b2b2c300 2024-06-03T02:32:09.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5578f2cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5579b2b2ef60 con 0x5579b2b215b0 2024-06-03T02:32:09.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5578f2cb80 10 client.0 _ll_register_callbacks cb 0x5579b2a0d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:09.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5567400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b2bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5567400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b2bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58572/0 (socket says 172.21.15.79:58572) 2024-06-03T02:32:09.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.625+0000 7f5567400700 1 -- 172.21.15.79:0/730185551 learned_addr learned my addr 172.21.15.79:0/730185551 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:09.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.626+0000 7f5567400700 1 -- 172.21.15.79:0/730185551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 msgr2=0x5579b2b28dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:32:09.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.626+0000 7f5567400700 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b28dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.627+0000 7f5567400700 1 -- 172.21.15.79:0/730185551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2b2c300 msgr2=0x5579b2b29300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:32:09.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.627+0000 7f5567400700 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2b2c300 0x5579b2b29300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.627+0000 7f5567400700 1 -- 172.21.15.79:0/730185551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b29c28d0 con 0x5579b2a9de40 2024-06-03T02:32:09.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.668+0000 7f5567400700 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b2bac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f555800bb70 tx=0x7f5558004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9effbc865367c104 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:09.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.668+0000 7f5551600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:32:09.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.673+0000 7f5551600700 1 -- 172.21.15.79:0/730185551 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555801f830 con 0x5579b2a9de40 2024-06-03T02:32:09.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.673+0000 7f5551600700 1 -- 172.21.15.79:0/730185551 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f555801f9d0 con 0x5579b2a9de40 2024-06-03T02:32:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.676+0000 7f5578f2cb80 1 -- 172.21.15.79:0/730185551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5560009450 con 0x5579b2a9de40 2024-06-03T02:32:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.676+0000 7f5551600700 1 -- 172.21.15.79:0/730185551 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5558011e60 con 0x5579b2a9de40 2024-06-03T02:32:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.676+0000 7f5578f2cb80 10 client.16182 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.676+0000 7f5578f2cb80 1 -- 172.21.15.79:0/730185551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579b2b39730 con 0x5579b2a9de40 2024-06-03T02:32:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.677+0000 7f5551600700 1 -- 172.21.15.79:0/730185551 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5560009450 con 0x5579b2a9de40 2024-06-03T02:32:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.677+0000 7f5551600700 1 -- 172.21.15.79:0/730185551 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f55580273f0 con 0x5579b2a9de40 2024-06-03T02:32:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.677+0000 7f5578f2cb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.677+0000 7f5578f2cb80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:32:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.677+0000 7f5578f2cb80 20 client.16182 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.677+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579b2b3bdc0 0x5579b2b3e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.677+0000 7f5578f2cb80 4 client.16182 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:32:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.677+0000 7f5578f2cb80 1 -- 172.21.15.79:0/730185551 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579b2a81000 con 0x5579b2b3bdc0 2024-06-03T02:32:09.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.682+0000 7f5566a00700 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579b2b3bdc0 0x5579b2b3e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.683+0000 7f5566a00700 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579b2b3bdc0 0x5579b2b3e240 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebcdb1c76b723e2e server_cookie=969bd5a74762def3 in_seq=0 out_seq=0 2024-06-03T02:32:09.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.683+0000 7f5551600700 10 client.16182 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:09.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.685+0000 7f5551600700 1 -- 172.21.15.79:0/730185551 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5579b2a81000 con 0x5579b2b3bdc0 2024-06-03T02:32:09.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.685+0000 7f5551600700 10 client.16182 handle_command_reply: tid=0 2024-06-03T02:32:09.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.685+0000 7f5578f2cb80 1 client.16182 shutdown 2024-06-03T02:32:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.685+0000 7f5578f2cb80 1 -- 172.21.15.79:0/730185551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 msgr2=0x5579b2b2bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.685+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b2bac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f555800bb70 tx=0x7f5558004750 comp rx=0 tx=0).stop 2024-06-03T02:32:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5578f2cb80 1 -- 172.21.15.79:0/730185551 shutdown_connections 2024-06-03T02:32:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2b2c300 0x5579b2b29300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579b2b3bdc0 0x5579b2b3e240 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2a9de40 0x5579b2b2bac0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/730185551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b28dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5551600700 0 client.16182 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:09.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5578f2cb80 1 -- 172.21.15.79:0/730185551 >> 172.21.15.79:0/730185551 conn(0x5579b2a4e170 msgr2=0x5579b2b1c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:09.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5578f2cb80 1 -- 172.21.15.79:0/730185551 shutdown_connections 2024-06-03T02:32:09.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5578f2cb80 1 -- 172.21.15.79:0/730185551 wait complete. 2024-06-03T02:32:09.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.686+0000 7f5578f2cb80 20 client.16182 trim_cache size 0 max 16384 2024-06-03T02:32:09.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.719+0000 7f5578f2cb80 1 Processor -- start 2024-06-03T02:32:09.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.720+0000 7f5578f2cb80 1 -- start start 2024-06-03T02:32:09.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.720+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b27d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.720+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b2b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.720+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b2c300 0x5579b2b24010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.720+0000 7f5578f2cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55681a7d70 con 0x5579b2b2c300 2024-06-03T02:32:09.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.720+0000 7f5578f2cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55681a98b0 con 0x5579b2a9de40 2024-06-03T02:32:09.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.720+0000 7f5578f2cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55681a8440 con 0x5579b2b215b0 2024-06-03T02:32:09.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.720+0000 7f5566a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b2b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.721+0000 7f5566a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b2b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37194/0 (socket says 172.21.15.79:37194) 2024-06-03T02:32:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.721+0000 7f5566a00700 1 -- 172.21.15.79:0/2033749389 learned_addr learned my addr 172.21.15.79:0/2033749389 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.721+0000 7f5567400700 1 --2- 172.21.15.79:0/2033749389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b27d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.721+0000 7f5567400700 1 -- 172.21.15.79:0/2033749389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 msgr2=0x5579b2b2b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.721+0000 7f5567400700 1 --2- 172.21.15.79:0/2033749389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b2b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.721+0000 7f5567400700 1 -- 172.21.15.79:0/2033749389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b2c300 msgr2=0x5579b2b24010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.721+0000 7f5567400700 1 --2- 172.21.15.79:0/2033749389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b2c300 0x5579b2b24010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.721+0000 7f5567400700 1 -- 172.21.15.79:0/2033749389 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b2b2ea60 con 0x5579b2a9de40 2024-06-03T02:32:09.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.722+0000 7f5567400700 1 --2- 172.21.15.79:0/2033749389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b27d20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5558000c00 tx=0x7f5558027c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46544ca577bf635 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:09.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.725+0000 7f5553e00700 1 -- 172.21.15.79:0/2033749389 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558036070 con 0x5579b2a9de40 2024-06-03T02:32:09.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.725+0000 7f5553e00700 1 -- 172.21.15.79:0/2033749389 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55580275f0 con 0x5579b2a9de40 2024-06-03T02:32:09.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.725+0000 7f5553e00700 1 -- 172.21.15.79:0/2033749389 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558004c00 con 0x5579b2a9de40 2024-06-03T02:32:09.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.726+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2033749389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 msgr2=0x5579b2b27d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:09.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.726+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2033749389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b27d20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5558000c00 tx=0x7f5558027c90 comp rx=0 tx=0).stop 2024-06-03T02:32:09.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.727+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2033749389 shutdown_connections 2024-06-03T02:32:09.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.727+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2033749389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b2c300 0x5579b2b24010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.727+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2033749389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b27d20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.727+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2033749389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b215b0 0x5579b2b2b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.727+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2033749389 >> 172.21.15.79:0/2033749389 conn(0x5579b2a4e170 msgr2=0x5579b2b1c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:09.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.727+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2033749389 shutdown_connections 2024-06-03T02:32:09.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.728+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2033749389 wait complete. 2024-06-03T02:32:09.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.728+0000 7f5578f2cb80 1 -- start start 2024-06-03T02:32:09.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.729+0000 7f5578f2cb80 1 Processor -- start 2024-06-03T02:32:09.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.730+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b2f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.730+0000 7f5578f2cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b215b0 0x5579b2b2f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.730+0000 7f5567400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b2f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.730+0000 7f5567400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b2f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34918/0 (socket says 172.21.15.79:34918) 2024-06-03T02:32:09.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.730+0000 7f5567400700 1 -- 172.21.15.79:0/2254986060 learned_addr learned my addr 172.21.15.79:0/2254986060 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:09.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.730+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b2c300 0x5579b2ba4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.731+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55681a91e0 con 0x5579b2b215b0 2024-06-03T02:32:09.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.731+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55681a8b10 con 0x5579b2a9de40 2024-06-03T02:32:09.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.731+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5568072af0 con 0x5579b2b2c300 2024-06-03T02:32:09.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.731+0000 7f5578f2cb80 10 client.0 _ll_register_callbacks cb 0x5579b2a0d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:09.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.732+0000 7f5567e00700 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b2c300 0x5579b2ba4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.733+0000 7f5566a00700 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b215b0 0x5579b2b2f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.734+0000 7f5567e00700 1 -- 172.21.15.79:0/2254986060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 msgr2=0x5579b2b2f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:09.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.734+0000 7f5567e00700 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b2f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.734+0000 7f5567e00700 1 -- 172.21.15.79:0/2254986060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b215b0 msgr2=0x5579b2b2f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.735+0000 7f5567e00700 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b215b0 0x5579b2b2f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.735+0000 7f5567e00700 1 -- 172.21.15.79:0/2254986060 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b2b30060 con 0x5579b2b2c300 2024-06-03T02:32:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.736+0000 7f5566a00700 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b215b0 0x5579b2b2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:32:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.736+0000 7f5567400700 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b2f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:32:09.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.742+0000 7f5567e00700 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b2c300 0x5579b2ba4ad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x5579b2b2a060 tx=0x7f5560003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba466e2be2f68d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:09.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.742+0000 7f5551600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:32:09.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.742+0000 7f5551600700 1 -- 172.21.15.79:0/2254986060 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556004b070 con 0x5579b2b2c300 2024-06-03T02:32:09.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.743+0000 7f5551600700 1 -- 172.21.15.79:0/2254986060 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f55600078b0 con 0x5579b2b2c300 2024-06-03T02:32:09.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.743+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55580275f0 con 0x5579b2b2c300 2024-06-03T02:32:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.745+0000 7f5551600700 1 -- 172.21.15.79:0/2254986060 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5560059050 con 0x5579b2b2c300 2024-06-03T02:32:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.745+0000 7f5578f2cb80 10 client.26695 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:09.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.746+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579b2ba60e0 con 0x5579b2b2c300 2024-06-03T02:32:09.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.746+0000 7f5551600700 1 -- 172.21.15.79:0/2254986060 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55580275f0 con 0x5579b2b2c300 2024-06-03T02:32:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.747+0000 7f5551600700 1 -- 172.21.15.79:0/2254986060 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5560011070 con 0x5579b2b2c300 2024-06-03T02:32:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.747+0000 7f5578f2cb80 10 client.26695 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.747+0000 7f5578f2cb80 10 client.26695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:32:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.747+0000 7f5578f2cb80 20 client.26695 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.747+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579b2ba6310 0x5579b2ba8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.747+0000 7f5578f2cb80 4 client.26695 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:32:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.747+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579b2a7b580 con 0x5579b2ba6310 2024-06-03T02:32:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.748+0000 7f5567400700 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579b2ba6310 0x5579b2ba8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.748+0000 7f5567400700 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579b2ba6310 0x5579b2ba8700 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8b94f9eb9863a5 server_cookie=42d806eaddfd4a72 in_seq=0 out_seq=0 2024-06-03T02:32:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.748+0000 7f5551600700 10 client.26695 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:09.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.749+0000 7f5551600700 1 -- 172.21.15.79:0/2254986060 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579b2a7b580 con 0x5579b2ba6310 2024-06-03T02:32:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.749+0000 7f5551600700 10 client.26695 handle_command_reply: tid=0 2024-06-03T02:32:09.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.749+0000 7f5578f2cb80 1 client.26695 shutdown 2024-06-03T02:32:09.755 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:32:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.752+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b2c300 msgr2=0x5579b2ba4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.752+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b2c300 0x5579b2ba4ad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x5579b2b2a060 tx=0x7f5560003ce0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.752+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 shutdown_connections 2024-06-03T02:32:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.752+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579b2b2c300 0x5579b2ba4ad0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.752+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579b2a9de40 0x5579b2b2f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.752+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5579b2ba6310 0x5579b2ba8700 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.752+0000 7f5551600700 0 client.26695 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.752+0000 7f5578f2cb80 1 --2- 172.21.15.79:0/2254986060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5579b2b215b0 0x5579b2b2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.752+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 >> 172.21.15.79:0/2254986060 conn(0x5579b2a4e170 msgr2=0x5579b2b28670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.753+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 shutdown_connections 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.753+0000 7f5578f2cb80 1 -- 172.21.15.79:0/2254986060 wait complete. 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.753+0000 7f5578f2cb80 20 client.26695 trim_cache size 0 max 16384 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.754+0000 7f556e600700 1 -- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f554c0a2d20 msgr2=0x7f554c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.754+0000 7f556e600700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f554c0a2d20 0x7f554c0a51e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5579b2b1e920 tx=0x7f555800b410 comp rx=0 tx=0).stop 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.754+0000 7f556e600700 1 -- 172.21.15.79:0/2635357767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55681030f0 msgr2=0x7f556819fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.754+0000 7f556e600700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55681030f0 0x7f556819fef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f555400c920 tx=0x7f555400cc30 comp rx=0 tx=0).stop 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.755+0000 7f556e600700 1 -- 172.21.15.79:0/2635357767 shutdown_connections 2024-06-03T02:32:09.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.755+0000 7f556e600700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5568107840 0x7f556819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.755+0000 7f556e600700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f554c0a2d20 0x7f554c0a51e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.755+0000 7f556e600700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5568102740 0x7f556819f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.755+0000 7f556e600700 1 --2- 172.21.15.79:0/2635357767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55681030f0 0x7f556819fef0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:09.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.755+0000 7f556e600700 1 -- 172.21.15.79:0/2635357767 >> 172.21.15.79:0/2635357767 conn(0x7f55680795f0 msgr2=0x7f5568106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.757+0000 7f556e600700 1 -- 172.21.15.79:0/2635357767 shutdown_connections 2024-06-03T02:32:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:09.757+0000 7f556e600700 1 -- 172.21.15.79:0/2635357767 wait complete. 2024-06-03T02:32:10.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:10 smithi079 ceph-mon[31819]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2024-06-03T02:32:10.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:10 smithi123 ceph-mon[36640]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2024-06-03T02:32:11.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:10 smithi135 ceph-mon[36653]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2024-06-03T02:32:12.678 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:32:12.679 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:32:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:12 smithi123 ceph-mon[36640]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 472 op/s 2024-06-03T02:32:13.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:12 smithi079 ceph-mon[31819]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 472 op/s 2024-06-03T02:32:13.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:12 smithi135 ceph-mon[36653]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 472 op/s 2024-06-03T02:32:13.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 725.0 s, 493.8 tps, lat 207.262 ms stddev 286.611 2024-06-03T02:32:13.668 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:32:14.897 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:14 smithi135 ceph-mon[36653]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-06-03T02:32:14.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:14 smithi123 ceph-mon[36640]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-06-03T02:32:15.090 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:14 smithi079 ceph-mon[31819]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-06-03T02:32:15.702 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:15 smithi079 ceph-mon[31819]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 504 op/s 2024-06-03T02:32:15.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:15 smithi123 ceph-mon[36640]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 504 op/s 2024-06-03T02:32:16.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:15 smithi135 ceph-mon[36653]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 504 op/s 2024-06-03T02:32:18.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:17 smithi079 ceph-mon[31819]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.4 MiB/s wr, 409 op/s 2024-06-03T02:32:18.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:17 smithi135 ceph-mon[36653]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.4 MiB/s wr, 409 op/s 2024-06-03T02:32:18.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:17 smithi123 ceph-mon[36640]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.4 MiB/s wr, 409 op/s 2024-06-03T02:32:18.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 730.0 s, 449.8 tps, lat 219.743 ms stddev 258.904 2024-06-03T02:32:19.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.982+0000 7f7c2ec00700 1 -- 172.21.15.79:0/610767701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c2806a170 msgr2=0x7f7c2806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:19.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.982+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/610767701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c2806a170 0x7f7c2806a5f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009990 tx=0x7f7c14009ca0 comp rx=0 tx=0).stop 2024-06-03T02:32:19.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.983+0000 7f7c2ec00700 1 -- 172.21.15.79:0/610767701 shutdown_connections 2024-06-03T02:32:19.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.983+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/610767701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2806ab30 0x7f7c2810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:19.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.983+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/610767701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c28108500 0x7f7c281088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:19.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.983+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/610767701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c2806a170 0x7f7c2806a5f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:19.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.983+0000 7f7c2ec00700 1 -- 172.21.15.79:0/610767701 >> 172.21.15.79:0/610767701 conn(0x7f7c28078340 msgr2=0x7f7c28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:19.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.983+0000 7f7c2ec00700 1 -- 172.21.15.79:0/610767701 shutdown_connections 2024-06-03T02:32:19.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.983+0000 7f7c2ec00700 1 -- 172.21.15.79:0/610767701 wait complete. 2024-06-03T02:32:19.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.985+0000 7f7c2ec00700 1 Processor -- start 2024-06-03T02:32:19.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.986+0000 7f7c2ec00700 1 -- start start 2024-06-03T02:32:19.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.988+0000 7f7c2ec00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2806a170 0x7f7c28073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:19.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.988+0000 7f7c2ec00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c2806ab30 0x7f7c28074080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:19.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.988+0000 7f7c2ec00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c28108500 0x7f7c2806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:19.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.988+0000 7f7c2ec00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28112800 con 0x7f7c2806a170 2024-06-03T02:32:19.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.988+0000 7f7c2ec00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28112680 con 0x7f7c2806ab30 2024-06-03T02:32:19.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.988+0000 7f7c2ec00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28112980 con 0x7f7c28108500 2024-06-03T02:32:19.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.988+0000 7f7c27400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c2806ab30 0x7f7c28074080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:19.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.988+0000 7f7c27400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c2806ab30 0x7f7c28074080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33406/0 (socket says 172.21.15.79:33406) 2024-06-03T02:32:19.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.988+0000 7f7c27400700 1 -- 172.21.15.79:0/1599487474 learned_addr learned my addr 172.21.15.79:0/1599487474 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:19.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.989+0000 7f7c27e00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2806a170 0x7f7c28073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:19.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.989+0000 7f7c2cc00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c28108500 0x7f7c2806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:19.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.989+0000 7f7c27400700 1 -- 172.21.15.79:0/1599487474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c28108500 msgr2=0x7f7c2806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:19.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.989+0000 7f7c27400700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c28108500 0x7f7c2806cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:19.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.990+0000 7f7c27400700 1 -- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2806a170 msgr2=0x7f7c28073b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:19.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.990+0000 7f7c27400700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2806a170 0x7f7c28073b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:19.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.990+0000 7f7c27400700 1 -- 172.21.15.79:0/1599487474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2806d5d0 con 0x7f7c2806ab30 2024-06-03T02:32:19.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.990+0000 7f7c27400700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c2806ab30 0x7f7c28074080 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7c140098e0 tx=0x7f7c140057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:19.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.990+0000 7f7c24c00700 1 -- 172.21.15.79:0/1599487474 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1400bb50 con 0x7f7c2806ab30 2024-06-03T02:32:19.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.991+0000 7f7c24c00700 1 -- 172.21.15.79:0/1599487474 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c14023db0 con 0x7f7c2806ab30 2024-06-03T02:32:19.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.991+0000 7f7c24c00700 1 -- 172.21.15.79:0/1599487474 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1400fbf0 con 0x7f7c2806ab30 2024-06-03T02:32:19.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.991+0000 7f7c2ec00700 1 -- 172.21.15.79:0/1599487474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2806d860 con 0x7f7c2806ab30 2024-06-03T02:32:19.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.991+0000 7f7c2ec00700 1 -- 172.21.15.79:0/1599487474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c28071840 con 0x7f7c2806ab30 2024-06-03T02:32:20.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:19.999+0000 7f7c24c00700 1 -- 172.21.15.79:0/1599487474 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7c14003ac0 con 0x7f7c2806ab30 2024-06-03T02:32:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.006+0000 7f7c24c00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c0c0a2d60 0x7f7c0c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.006+0000 7f7c24c00700 1 -- 172.21.15.79:0/1599487474 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7c140d90c0 con 0x7f7c2806ab30 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.007+0000 7f7c39506b80 1 Processor -- start 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.008+0000 7f7c39506b80 1 -- start start 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.008+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a662ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.008+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a65cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.008+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.008+0000 7f7c39506b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56143a56ba90 con 0x56143a663530 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.008+0000 7f7c39506b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56143a5aeb80 con 0x56143a662c10 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.008+0000 7f7c39506b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56143a569970 con 0x56143a65d2c0 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.008+0000 7f7c27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a662ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.009+0000 7f7c27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a662ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33408/0 (socket says 172.21.15.79:33408) 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.009+0000 7f7c27e00700 1 -- 172.21.15.79:0/3621526761 learned_addr learned my addr 172.21.15.79:0/3621526761 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.009+0000 7f7c2cc00700 1 --2- 172.21.15.79:0/3621526761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.009+0000 7f7c2cc00700 1 -- 172.21.15.79:0/3621526761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 msgr2=0x56143a662ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.009+0000 7f7c2cc00700 1 --2- 172.21.15.79:0/3621526761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a662ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.009+0000 7f7c2cc00700 1 -- 172.21.15.79:0/3621526761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 msgr2=0x56143a65cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.009+0000 7f7c2cc00700 1 --2- 172.21.15.79:0/3621526761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a65cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.009+0000 7f7c2cc00700 1 -- 172.21.15.79:0/3621526761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143a557010 con 0x56143a65d2c0 2024-06-03T02:32:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.009+0000 7f7c27e00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c0c0a2d60 0x7f7c0c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.010+0000 7f7c2cc00700 1 --2- 172.21.15.79:0/3621526761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65d700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7c18004630 tx=0x7f7c1800b970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2b25cd20d11a2e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.010+0000 7f7c1d600700 1 -- 172.21.15.79:0/3621526761 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18004a30 con 0x56143a65d2c0 2024-06-03T02:32:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.010+0000 7f7c1d600700 1 -- 172.21.15.79:0/3621526761 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c18015070 con 0x56143a65d2c0 2024-06-03T02:32:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.011+0000 7f7c39506b80 1 -- 172.21.15.79:0/3621526761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 msgr2=0x56143a65d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.011+0000 7f7c39506b80 1 --2- 172.21.15.79:0/3621526761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65d700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7c18004630 tx=0x7f7c1800b970 comp rx=0 tx=0).stop 2024-06-03T02:32:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.011+0000 7f7c39506b80 1 -- 172.21.15.79:0/3621526761 shutdown_connections 2024-06-03T02:32:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.011+0000 7f7c39506b80 1 --2- 172.21.15.79:0/3621526761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65d700 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.011+0000 7f7c39506b80 1 --2- 172.21.15.79:0/3621526761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a662ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.011+0000 7f7c39506b80 1 --2- 172.21.15.79:0/3621526761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a65cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.011+0000 7f7c39506b80 1 -- 172.21.15.79:0/3621526761 >> 172.21.15.79:0/3621526761 conn(0x56143a5891f0 msgr2=0x56143a658070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.012+0000 7f7c39506b80 1 -- 172.21.15.79:0/3621526761 shutdown_connections 2024-06-03T02:32:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.012+0000 7f7c39506b80 1 -- 172.21.15.79:0/3621526761 wait complete. 2024-06-03T02:32:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.012+0000 7f7c27e00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c0c0a2d60 0x7f7c0c0a5220 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c200060b0 tx=0x7f7c200096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.015+0000 7f7c39506b80 1 -- start start 2024-06-03T02:32:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.016+0000 7f7c39506b80 1 Processor -- start 2024-06-03T02:32:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a65d2c0 0x56143a669240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a662c10 0x56143a666590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a666ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c39506b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56143a54afe0 con 0x56143a663530 2024-06-03T02:32:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c39506b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56143a5aab40 con 0x56143a65d2c0 2024-06-03T02:32:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c39506b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56143a669fb0 con 0x56143a662c10 2024-06-03T02:32:20.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c39506b80 10 client.0 _ll_register_callbacks cb 0x56143a544400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:20.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a65d2c0 0x56143a669240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a65d2c0 0x56143a669240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33422/0 (socket says 172.21.15.79:33422) 2024-06-03T02:32:20.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.017+0000 7f7c27e00700 1 -- 172.21.15.79:0/2182007092 learned_addr learned my addr 172.21.15.79:0/2182007092 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.018+0000 7f7c27e00700 1 -- 172.21.15.79:0/2182007092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a662c10 msgr2=0x56143a666590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.018+0000 7f7c27e00700 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a662c10 0x56143a666590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.018+0000 7f7c27e00700 1 -- 172.21.15.79:0/2182007092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 msgr2=0x56143a666ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.018+0000 7f7c27e00700 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a666ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.018+0000 7f7c27e00700 1 -- 172.21.15.79:0/2182007092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143a4fd950 con 0x56143a65d2c0 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.018+0000 7f7c27e00700 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a65d2c0 0x56143a669240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7c20019b50 tx=0x7f7c20004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56cb7ed538bb49d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.018+0000 7f7c12000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.019+0000 7f7c12000700 1 -- 172.21.15.79:0/2182007092 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20024070 con 0x56143a65d2c0 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.019+0000 7f7c12000700 1 -- 172.21.15.79:0/2182007092 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7c2001b8a0 con 0x56143a65d2c0 2024-06-03T02:32:20.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.019+0000 7f7c39506b80 1 -- 172.21.15.79:0/2182007092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c18015070 con 0x56143a65d2c0 2024-06-03T02:32:20.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.021+0000 7f7c12000700 1 -- 172.21.15.79:0/2182007092 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c2002e050 con 0x56143a65d2c0 2024-06-03T02:32:20.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.021+0000 7f7c39506b80 10 client.26720 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:20.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.021+0000 7f7c39506b80 1 -- 172.21.15.79:0/2182007092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56143a6747d0 con 0x56143a65d2c0 2024-06-03T02:32:20.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.022+0000 7f7c12000700 1 -- 172.21.15.79:0/2182007092 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c18015070 con 0x56143a65d2c0 2024-06-03T02:32:20.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.023+0000 7f7c12000700 1 -- 172.21.15.79:0/2182007092 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7c2001ece0 con 0x56143a65d2c0 2024-06-03T02:32:20.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.023+0000 7f7c39506b80 10 client.26720 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:20.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.023+0000 7f7c39506b80 10 client.26720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:32:20.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.023+0000 7f7c39506b80 20 client.26720 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:20.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.024+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56143a676e60 0x56143a6792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.024+0000 7f7c39506b80 4 client.26720 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:32:20.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.024+0000 7f7c39506b80 1 -- 172.21.15.79:0/2182007092 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56143a5bc080 con 0x56143a676e60 2024-06-03T02:32:20.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.024+0000 7f7c27400700 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56143a676e60 0x56143a6792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.025+0000 7f7c27400700 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56143a676e60 0x56143a6792f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441b2f0118ea65bf server_cookie=e787cdc6eb921027 in_seq=0 out_seq=0 2024-06-03T02:32:20.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.025+0000 7f7c12000700 10 client.26720 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:20.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.026+0000 7f7c12000700 1 -- 172.21.15.79:0/2182007092 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56143a5bc080 con 0x56143a676e60 2024-06-03T02:32:20.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.026+0000 7f7c12000700 10 client.26720 handle_command_reply: tid=0 2024-06-03T02:32:20.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.026+0000 7f7c39506b80 1 client.26720 shutdown 2024-06-03T02:32:20.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.026+0000 7f7c39506b80 1 -- 172.21.15.79:0/2182007092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a65d2c0 msgr2=0x56143a669240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:20.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.026+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a65d2c0 0x56143a669240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7c20019b50 tx=0x7f7c20004b40 comp rx=0 tx=0).stop 2024-06-03T02:32:20.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.027+0000 7f7c39506b80 1 -- 172.21.15.79:0/2182007092 shutdown_connections 2024-06-03T02:32:20.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.027+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56143a676e60 0x56143a6792f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.027+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a666ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.027+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a65d2c0 0x56143a669240 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.027+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2182007092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a662c10 0x56143a666590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.027+0000 7f7c39506b80 1 -- 172.21.15.79:0/2182007092 >> 172.21.15.79:0/2182007092 conn(0x56143a5891f0 msgr2=0x56143a651810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:20.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.027+0000 7f7c12000700 0 client.26720 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:20.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.027+0000 7f7c39506b80 1 -- 172.21.15.79:0/2182007092 shutdown_connections 2024-06-03T02:32:20.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.027+0000 7f7c39506b80 1 -- 172.21.15.79:0/2182007092 wait complete. 2024-06-03T02:32:20.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.029+0000 7f7c39506b80 20 client.26720 trim_cache size 0 max 16384 2024-06-03T02:32:20.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.071+0000 7f7c39506b80 1 Processor -- start 2024-06-03T02:32:20.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.071+0000 7f7c39506b80 1 -- start start 2024-06-03T02:32:20.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.071+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:19 smithi079 ceph-mon[31819]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 502 op/s 2024-06-03T02:32:20.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.072+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a6521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.072+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a665710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.072+0000 7f7c39506b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281b4860 con 0x56143a663530 2024-06-03T02:32:20.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.072+0000 7f7c39506b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281b63a0 con 0x56143a662c10 2024-06-03T02:32:20.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.072+0000 7f7c39506b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281b4f30 con 0x56143a65d2c0 2024-06-03T02:32:20.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.072+0000 7f7c27400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a6521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.072+0000 7f7c27400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a6521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33438/0 (socket says 172.21.15.79:33438) 2024-06-03T02:32:20.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.072+0000 7f7c27400700 1 -- 172.21.15.79:0/2342751314 learned_addr learned my addr 172.21.15.79:0/2342751314 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.073+0000 7f7c27e00700 1 --2- 172.21.15.79:0/2342751314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.073+0000 7f7c2cc00700 1 --2- 172.21.15.79:0/2342751314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a665710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.073+0000 7f7c27e00700 1 -- 172.21.15.79:0/2342751314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 msgr2=0x56143a6521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.073+0000 7f7c27e00700 1 --2- 172.21.15.79:0/2342751314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a6521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.073+0000 7f7c27e00700 1 -- 172.21.15.79:0/2342751314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 msgr2=0x56143a665710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.073+0000 7f7c27e00700 1 --2- 172.21.15.79:0/2342751314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a665710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.073+0000 7f7c27e00700 1 -- 172.21.15.79:0/2342751314 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143a664b00 con 0x56143a65d2c0 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.074+0000 7f7c27e00700 1 --2- 172.21.15.79:0/2342751314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65cd80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009d10 tx=0x7f7c2001b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc0ef55ac3e3ef85 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.074+0000 7f7c1cc00700 1 -- 172.21.15.79:0/2342751314 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20024040 con 0x56143a65d2c0 2024-06-03T02:32:20.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.074+0000 7f7c1cc00700 1 -- 172.21.15.79:0/2342751314 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2001bd80 con 0x56143a65d2c0 2024-06-03T02:32:20.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.074+0000 7f7c1cc00700 1 -- 172.21.15.79:0/2342751314 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2001e6e0 con 0x56143a65d2c0 2024-06-03T02:32:20.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.075+0000 7f7c39506b80 1 -- 172.21.15.79:0/2342751314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 msgr2=0x56143a65cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:20.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.076+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2342751314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65cd80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009d10 tx=0x7f7c2001b5e0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.076+0000 7f7c39506b80 1 -- 172.21.15.79:0/2342751314 shutdown_connections 2024-06-03T02:32:20.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.076+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2342751314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a665710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.076+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2342751314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a65cd80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.076+0000 7f7c39506b80 1 --2- 172.21.15.79:0/2342751314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a6521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.076+0000 7f7c39506b80 1 -- 172.21.15.79:0/2342751314 >> 172.21.15.79:0/2342751314 conn(0x56143a5891f0 msgr2=0x56143a663fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:20.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.077+0000 7f7c39506b80 1 -- 172.21.15.79:0/2342751314 shutdown_connections 2024-06-03T02:32:20.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.077+0000 7f7c39506b80 1 -- 172.21.15.79:0/2342751314 wait complete. 2024-06-03T02:32:20.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.078+0000 7f7c39506b80 1 -- start start 2024-06-03T02:32:20.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.078+0000 7f7c39506b80 1 Processor -- start 2024-06-03T02:32:20.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.079+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a67e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.079+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a67ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.079+0000 7f7c39506b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a67f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.079+0000 7f7c27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a67e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.079+0000 7f7c27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a67e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:47534/0 (socket says 172.21.15.79:47534) 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.079+0000 7f7c27400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a67ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c27e00700 1 -- 172.21.15.79:0/1989851509 learned_addr learned my addr 172.21.15.79:0/1989851509 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281b5cd0 con 0x56143a663530 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281b5600 con 0x56143a662c10 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281b2800 con 0x56143a65d2c0 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c27400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a67ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:33448/0 (socket says 172.21.15.79:33448) 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c39506b80 10 client.0 _ll_register_callbacks cb 0x56143a544400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c27400700 1 -- 172.21.15.79:0/1989851509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 msgr2=0x56143a67e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c27400700 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a67e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c27400700 1 -- 172.21.15.79:0/1989851509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 msgr2=0x56143a67f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c27400700 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a67f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.080+0000 7f7c27400700 1 -- 172.21.15.79:0/1989851509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143a6e0cc0 con 0x56143a662c10 2024-06-03T02:32:20.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.082+0000 7f7c27400700 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a67ee50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7c140a1f70 tx=0x7f7c14009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb6e1f9e28065ed4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:20.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.082+0000 7f7c12000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:32:20.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.083+0000 7f7c12000700 1 -- 172.21.15.79:0/1989851509 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140a5370 con 0x56143a662c10 2024-06-03T02:32:20.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.083+0000 7f7c12000700 1 -- 172.21.15.79:0/1989851509 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7c140d68c0 con 0x56143a662c10 2024-06-03T02:32:20.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.083+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c2001bd80 con 0x56143a662c10 2024-06-03T02:32:20.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.084+0000 7f7c12000700 1 -- 172.21.15.79:0/1989851509 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c140a4f00 con 0x56143a662c10 2024-06-03T02:32:20.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.085+0000 7f7c39506b80 10 client.26726 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:20.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.085+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56143a6e2040 con 0x56143a662c10 2024-06-03T02:32:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.085+0000 7f7c12000700 1 -- 172.21.15.79:0/1989851509 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2001bd80 con 0x56143a662c10 2024-06-03T02:32:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.085+0000 7f7c12000700 1 -- 172.21.15.79:0/1989851509 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7c140a5800 con 0x56143a662c10 2024-06-03T02:32:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.085+0000 7f7c39506b80 10 client.26726 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.085+0000 7f7c39506b80 10 client.26726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:32:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.085+0000 7f7c39506b80 20 client.26726 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.086+0000 7f7c39506b80 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56143a6e2730 0x56143a6e4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.086+0000 7f7c39506b80 4 client.26726 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:32:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.086+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56143a5b6600 con 0x56143a6e2730 2024-06-03T02:32:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.086+0000 7f7c27e00700 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56143a6e2730 0x56143a6e4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.087+0000 7f7c27e00700 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56143a6e2730 0x56143a6e4b50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9912817d379f0f56 server_cookie=fc846f7ac357ffb6 in_seq=0 out_seq=0 2024-06-03T02:32:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.087+0000 7f7c12000700 10 client.26726 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.089+0000 7f7c12000700 1 -- 172.21.15.79:0/1989851509 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56143a5b6600 con 0x56143a6e2730 2024-06-03T02:32:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.089+0000 7f7c12000700 10 client.26726 handle_command_reply: tid=0 2024-06-03T02:32:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.089+0000 7f7c39506b80 1 client.26726 shutdown 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.089+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 msgr2=0x56143a67ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.089+0000 7f7c39506b80 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a67ee50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7c140a1f70 tx=0x7f7c14009310 comp rx=0 tx=0).stop 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.090+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 shutdown_connections 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.090+0000 7f7c39506b80 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x56143a6e2730 0x56143a6e4b50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.090+0000 7f7c39506b80 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56143a663530 0x56143a67f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.090+0000 7f7c39506b80 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56143a65d2c0 0x56143a67e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.090+0000 7f7c39506b80 1 --2- 172.21.15.79:0/1989851509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56143a662c10 0x56143a67ee50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.090+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 >> 172.21.15.79:0/1989851509 conn(0x56143a5891f0 msgr2=0x56143a665710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.090+0000 7f7c12000700 0 client.26726 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.090+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 shutdown_connections 2024-06-03T02:32:20.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.090+0000 7f7c39506b80 1 -- 172.21.15.79:0/1989851509 wait complete. 2024-06-03T02:32:20.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.091+0000 7f7c39506b80 20 client.26726 trim_cache size 0 max 16384 2024-06-03T02:32:20.092 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:32:20.092 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:32:20.092 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "52a6b2c0-68e5-4c4c-b475-1be1930ce768", 2024-06-03T02:32:20.092 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:32:20.093 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:32:20.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 -- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c0c0a2d60 msgr2=0x7f7c0c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:20.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c0c0a2d60 0x7f7c0c0a5220 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c200060b0 tx=0x7f7c200096a0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 -- 172.21.15.79:0/1599487474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c2806ab30 msgr2=0x7f7c28074080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:20.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c2806ab30 0x7f7c28074080 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7c140098e0 tx=0x7f7c140057c0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 -- 172.21.15.79:0/1599487474 shutdown_connections 2024-06-03T02:32:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c28108500 0x7f7c2806cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7c0c0a2d60 0x7f7c0c0a5220 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7c2806a170 0x7f7c28073b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 --2- 172.21.15.79:0/1599487474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c2806ab30 0x7f7c28074080 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.092+0000 7f7c2ec00700 1 -- 172.21.15.79:0/1599487474 >> 172.21.15.79:0/1599487474 conn(0x7f7c28078340 msgr2=0x7f7c280fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:20.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.093+0000 7f7c2ec00700 1 -- 172.21.15.79:0/1599487474 shutdown_connections 2024-06-03T02:32:20.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:20.094+0000 7f7c2ec00700 1 -- 172.21.15.79:0/1599487474 wait complete. 2024-06-03T02:32:20.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:19 smithi135 ceph-mon[36653]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 502 op/s 2024-06-03T02:32:20.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:19 smithi123 ceph-mon[36640]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 502 op/s 2024-06-03T02:32:21.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:20 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:32:21.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:20 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:32:21.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:20 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:32:21.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:20 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:32:21.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:20 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:32:21.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:20 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:32:21.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:20 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:32:21.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:20 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:32:21.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:20 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:32:21.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:32:22.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:21 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:32:22.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:21 smithi123 ceph-mon[36640]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 486 op/s 2024-06-03T02:32:22.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:21 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:32:22.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:21 smithi079 ceph-mon[31819]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 486 op/s 2024-06-03T02:32:22.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:21 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:32:22.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:21 smithi135 ceph-mon[36653]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 486 op/s 2024-06-03T02:32:22.796 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:32:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 735.0 s, 451.8 tps, lat 221.563 ms stddev 255.175 2024-06-03T02:32:24.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:23 smithi123 ceph-mon[36640]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.0 MiB/s wr, 482 op/s 2024-06-03T02:32:24.255 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:23 smithi079 ceph-mon[31819]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.0 MiB/s wr, 482 op/s 2024-06-03T02:32:24.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:23 smithi135 ceph-mon[36653]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.0 MiB/s wr, 482 op/s 2024-06-03T02:32:25.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:24 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:32:25.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:24 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:32:25.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:24 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:32:26.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:25 smithi123 ceph-mon[36640]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.3 MiB/s wr, 508 op/s 2024-06-03T02:32:26.238 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:25 smithi079 ceph-mon[31819]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.3 MiB/s wr, 508 op/s 2024-06-03T02:32:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:25 smithi135 ceph-mon[36653]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.3 MiB/s wr, 508 op/s 2024-06-03T02:32:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.793+0000 7f5b6d200700 1 -- 172.21.15.79:0/959344677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b6811bb40 msgr2=0x7f5b6811df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.793+0000 7f5b6d200700 1 --2- 172.21.15.79:0/959344677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b6811bb40 0x7f5b6811df30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009990 tx=0x7f5b60009ca0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.793+0000 7f5b6d200700 1 -- 172.21.15.79:0/959344677 shutdown_connections 2024-06-03T02:32:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.793+0000 7f5b6d200700 1 --2- 172.21.15.79:0/959344677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b6811bb40 0x7f5b6811df30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.793+0000 7f5b6d200700 1 --2- 172.21.15.79:0/959344677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b6806dfd0 0x7f5b6806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.793+0000 7f5b6d200700 1 --2- 172.21.15.79:0/959344677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6806e8f0 0x7f5b6811b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.793+0000 7f5b6d200700 1 -- 172.21.15.79:0/959344677 >> 172.21.15.79:0/959344677 conn(0x7f5b6806d6b0 msgr2=0x7f5b6810b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.793+0000 7f5b6d200700 1 -- 172.21.15.79:0/959344677 shutdown_connections 2024-06-03T02:32:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.793+0000 7f5b6d200700 1 -- 172.21.15.79:0/959344677 wait complete. 2024-06-03T02:32:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.794+0000 7f5b6d200700 1 Processor -- start 2024-06-03T02:32:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.795+0000 7f5b6d200700 1 -- start start 2024-06-03T02:32:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.795+0000 7f5b6d200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b6806dfd0 0x7f5b681ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.795+0000 7f5b6d200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6806e8f0 0x7f5b681adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.795+0000 7f5b6d200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b6811bb40 0x7f5b681a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.795+0000 7f5b6d200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68121610 con 0x7f5b6806dfd0 2024-06-03T02:32:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.795+0000 7f5b6d200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68121490 con 0x7f5b6806e8f0 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.795+0000 7f5b6d200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68121790 con 0x7f5b6811bb40 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.796+0000 7f5b5fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6806e8f0 0x7f5b681adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.796+0000 7f5b5fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6806e8f0 0x7f5b681adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34704/0 (socket says 172.21.15.79:34704) 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.796+0000 7f5b5fe00700 1 -- 172.21.15.79:0/3063589980 learned_addr learned my addr 172.21.15.79:0/3063589980 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.796+0000 7f5b66800700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b6811bb40 0x7f5b681a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.797+0000 7f5b5fe00700 1 -- 172.21.15.79:0/3063589980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b6811bb40 msgr2=0x7f5b681a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.797+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b6811bb40 0x7f5b681a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.797+0000 7f5b5fe00700 1 -- 172.21.15.79:0/3063589980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b6806dfd0 msgr2=0x7f5b681ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.797+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b6806dfd0 0x7f5b681ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.797+0000 7f5b5fe00700 1 -- 172.21.15.79:0/3063589980 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b681a7fd0 con 0x7f5b6806e8f0 2024-06-03T02:32:27.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.798+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6806e8f0 0x7f5b681adbf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000e9a0 tx=0x7f5b5000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:27.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.798+0000 7f5b5ea00700 1 -- 172.21.15.79:0/3063589980 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5000cbd0 con 0x7f5b6806e8f0 2024-06-03T02:32:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.798+0000 7f5b5ea00700 1 -- 172.21.15.79:0/3063589980 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5000cd70 con 0x7f5b6806e8f0 2024-06-03T02:32:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.798+0000 7f5b5ea00700 1 -- 172.21.15.79:0/3063589980 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50018840 con 0x7f5b6806e8f0 2024-06-03T02:32:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.798+0000 7f5b66800700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b6811bb40 0x7f5b681a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:32:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.799+0000 7f5b6d200700 1 -- 172.21.15.79:0/3063589980 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b681a82c0 con 0x7f5b6806e8f0 2024-06-03T02:32:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.800+0000 7f5b6d200700 1 -- 172.21.15.79:0/3063589980 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681b4580 con 0x7f5b6806e8f0 2024-06-03T02:32:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.805+0000 7f5b5ea00700 1 -- 172.21.15.79:0/3063589980 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5b500189e0 con 0x7f5b6806e8f0 2024-06-03T02:32:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.806+0000 7f5b77c3db80 1 Processor -- start 2024-06-03T02:32:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.806+0000 7f5b77c3db80 1 -- start start 2024-06-03T02:32:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.806+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 0x55ed76d4ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.806+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d49d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.806+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4a240 0x55ed76d4a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.806+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ed76c58a10 con 0x55ed76d4fb90 2024-06-03T02:32:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.806+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed76c9bb00 con 0x55ed76d4a240 2024-06-03T02:32:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.806+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed76c568f0 con 0x55ed76d504b0 2024-06-03T02:32:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.807+0000 7f5b5fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4a240 0x55ed76d4a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.807+0000 7f5b5fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4a240 0x55ed76d4a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34720/0 (socket says 172.21.15.79:34720) 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.807+0000 7f5b5fe00700 1 -- 172.21.15.79:0/298885558 learned_addr learned my addr 172.21.15.79:0/298885558 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.808+0000 7f5b5ea00700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b4c0a2ca0 0x7f5b4c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.808+0000 7f5b66800700 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 0x55ed76d4ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.809+0000 7f5b65e00700 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d49d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.810+0000 7f5b66800700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b4c0a2ca0 0x7f5b4c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.810+0000 7f5b5fe00700 1 -- 172.21.15.79:0/298885558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 msgr2=0x55ed76d49d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.810+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d49d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.810+0000 7f5b5fe00700 1 -- 172.21.15.79:0/298885558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 msgr2=0x55ed76d4ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.810+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 0x55ed76d4ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.810+0000 7f5b5fe00700 1 -- 172.21.15.79:0/298885558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed76c43f90 con 0x55ed76d4a240 2024-06-03T02:32:27.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.810+0000 7f5b65e00700 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d49d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:32:27.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.810+0000 7f5b5ea00700 1 -- 172.21.15.79:0/3063589980 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5b500cf770 con 0x7f5b6806e8f0 2024-06-03T02:32:27.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.810+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4a240 0x55ed76d4a680 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5b500cce00 tx=0x7f5b50022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=598c04cb08eea621 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:27.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.811+0000 7f5b57400700 1 -- 172.21.15.79:0/298885558 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5009b760 con 0x55ed76d4a240 2024-06-03T02:32:27.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.811+0000 7f5b57400700 1 -- 172.21.15.79:0/298885558 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5009c320 con 0x55ed76d4a240 2024-06-03T02:32:27.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.811+0000 7f5b57400700 1 -- 172.21.15.79:0/298885558 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50053bf0 con 0x55ed76d4a240 2024-06-03T02:32:27.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.812+0000 7f5b77c3db80 1 -- 172.21.15.79:0/298885558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4a240 msgr2=0x55ed76d4a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.812+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4a240 0x55ed76d4a680 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5b500cce00 tx=0x7f5b50022000 comp rx=0 tx=0).stop 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.812+0000 7f5b66800700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b4c0a2ca0 0x7f5b4c0a5160 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5b60004800 tx=0x7f5b60019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.813+0000 7f5b77c3db80 1 -- 172.21.15.79:0/298885558 shutdown_connections 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.813+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4a240 0x55ed76d4a680 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.813+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 0x55ed76d4ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.813+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/298885558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d49d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.813+0000 7f5b77c3db80 1 -- 172.21.15.79:0/298885558 >> 172.21.15.79:0/298885558 conn(0x55ed76c76170 msgr2=0x55ed76d44ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.814+0000 7f5b77c3db80 1 -- 172.21.15.79:0/298885558 shutdown_connections 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.814+0000 7f5b77c3db80 1 -- 172.21.15.79:0/298885558 wait complete. 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.815+0000 7f5b77c3db80 1 -- start start 2024-06-03T02:32:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.815+0000 7f5b77c3db80 1 Processor -- start 2024-06-03T02:32:27.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 0x55ed76d534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d504b0 0x55ed76d53a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ed76c37f60 con 0x55ed76d4fb90 2024-06-03T02:32:27.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed76c97ac0 con 0x55ed76d504b0 2024-06-03T02:32:27.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed76d56e30 con 0x55ed76d4a240 2024-06-03T02:32:27.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b77c3db80 10 client.0 _ll_register_callbacks cb 0x55ed76c31380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:27.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b66800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d504b0 0x55ed76d53a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b65e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b66800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d504b0 0x55ed76d53a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:34736/0 (socket says 172.21.15.79:34736) 2024-06-03T02:32:27.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.816+0000 7f5b66800700 1 -- 172.21.15.79:0/3428300821 learned_addr learned my addr 172.21.15.79:0/3428300821 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:27.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.817+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 0x55ed76d534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.817+0000 7f5b65e00700 1 -- 172.21.15.79:0/3428300821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d504b0 msgr2=0x55ed76d53a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:27.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.817+0000 7f5b65e00700 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d504b0 0x55ed76d53a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.817+0000 7f5b65e00700 1 -- 172.21.15.79:0/3428300821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 msgr2=0x55ed76d534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:27.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.817+0000 7f5b65e00700 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 0x55ed76d534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.817+0000 7f5b65e00700 1 -- 172.21.15.79:0/3428300821 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed76bea8d0 con 0x55ed76d4a240 2024-06-03T02:32:27.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.818+0000 7f5b65e00700 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d561f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5b58007aa0 tx=0x7f5b58007db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f082ac3163e6e3a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:27.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.818+0000 7f5b4be00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:32:27.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.818+0000 7f5b4be00700 1 -- 172.21.15.79:0/3428300821 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58004bb0 con 0x55ed76d4a240 2024-06-03T02:32:27.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.818+0000 7f5b4be00700 1 -- 172.21.15.79:0/3428300821 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5b58004d50 con 0x55ed76d4a240 2024-06-03T02:32:27.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.819+0000 7f5b77c3db80 1 -- 172.21.15.79:0/3428300821 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b5009c320 con 0x55ed76d4a240 2024-06-03T02:32:27.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.819+0000 7f5b4be00700 1 -- 172.21.15.79:0/3428300821 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b5800f890 con 0x55ed76d4a240 2024-06-03T02:32:27.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.819+0000 7f5b77c3db80 10 client.26716 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:27.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.819+0000 7f5b77c3db80 1 -- 172.21.15.79:0/3428300821 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed76d615b0 con 0x55ed76d4a240 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.819+0000 7f5b4be00700 1 -- 172.21.15.79:0/3428300821 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5009c320 con 0x55ed76d4a240 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.819+0000 7f5b4be00700 1 -- 172.21.15.79:0/3428300821 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5b580143d0 con 0x55ed76d4a240 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.820+0000 7f5b77c3db80 10 client.26716 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.820+0000 7f5b77c3db80 10 client.26716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.820+0000 7f5b77c3db80 20 client.26716 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.820+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed76d63bf0 0x55ed76d65fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.820+0000 7f5b77c3db80 4 client.26716 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.820+0000 7f5b77c3db80 1 -- 172.21.15.79:0/3428300821 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed76ca9000 con 0x55ed76d63bf0 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.821+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed76d63bf0 0x55ed76d65fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.821+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed76d63bf0 0x55ed76d65fe0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1e50754df1bec3 server_cookie=6ad607e067e7de3a in_seq=0 out_seq=0 2024-06-03T02:32:27.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.821+0000 7f5b4be00700 10 client.26716 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:27.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.822+0000 7f5b4be00700 1 -- 172.21.15.79:0/3428300821 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ed76ca9000 con 0x55ed76d63bf0 2024-06-03T02:32:27.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.822+0000 7f5b4be00700 10 client.26716 handle_command_reply: tid=0 2024-06-03T02:32:27.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.822+0000 7f5b77c3db80 1 client.26716 shutdown 2024-06-03T02:32:27.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.822+0000 7f5b77c3db80 1 -- 172.21.15.79:0/3428300821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 msgr2=0x55ed76d561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:27.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.822+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d561f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5b58007aa0 tx=0x7f5b58007db0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b77c3db80 1 -- 172.21.15.79:0/3428300821 shutdown_connections 2024-06-03T02:32:27.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d504b0 0x55ed76d53a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d561f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed76d63bf0 0x55ed76d65fe0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/3428300821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4fb90 0x55ed76d534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b4be00700 0 client.26716 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b77c3db80 1 -- 172.21.15.79:0/3428300821 >> 172.21.15.79:0/3428300821 conn(0x55ed76c76170 msgr2=0x55ed76d3e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b77c3db80 1 -- 172.21.15.79:0/3428300821 shutdown_connections 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b77c3db80 1 -- 172.21.15.79:0/3428300821 wait complete. 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.823+0000 7f5b77c3db80 20 client.26716 trim_cache size 0 max 16384 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.854+0000 7f5b77c3db80 1 Processor -- start 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.855+0000 7f5b77c3db80 1 -- start start 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.855+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4a240 0x55ed76d49d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.855+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 0x55ed76d511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.855+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d3f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.855+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681bd2e0 con 0x55ed76d4a240 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.855+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681bee20 con 0x55ed76d4fb90 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.855+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681bd9b0 con 0x55ed76d504b0 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.855+0000 7f5b66800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d3f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.856+0000 7f5b66800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d3f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:49590/0 (socket says 172.21.15.79:49590) 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.856+0000 7f5b66800700 1 -- 172.21.15.79:0/785846020 learned_addr learned my addr 172.21.15.79:0/785846020 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.856+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 0x55ed76d511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.856+0000 7f5b66800700 1 -- 172.21.15.79:0/785846020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 msgr2=0x55ed76d511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:27.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.856+0000 7f5b66800700 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 0x55ed76d511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.856+0000 7f5b66800700 1 -- 172.21.15.79:0/785846020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4a240 msgr2=0x55ed76d49d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:32:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.856+0000 7f5b65e00700 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4a240 0x55ed76d49d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.857+0000 7f5b66800700 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4a240 0x55ed76d49d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.857+0000 7f5b66800700 1 -- 172.21.15.79:0/785846020 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed76d4dfe0 con 0x55ed76d504b0 2024-06-03T02:32:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.857+0000 7f5b65e00700 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4a240 0x55ed76d49d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:32:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.857+0000 7f5b66800700 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d3f130 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005b40 tx=0x7f5b60021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6191cbd74f759d79 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.857+0000 7f5b56a00700 1 -- 172.21.15.79:0/785846020 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60020040 con 0x55ed76d504b0 2024-06-03T02:32:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.857+0000 7f5b56a00700 1 -- 172.21.15.79:0/785846020 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60035070 con 0x55ed76d504b0 2024-06-03T02:32:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.857+0000 7f5b56a00700 1 -- 172.21.15.79:0/785846020 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60030d10 con 0x55ed76d504b0 2024-06-03T02:32:27.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.858+0000 7f5b77c3db80 1 -- 172.21.15.79:0/785846020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 msgr2=0x55ed76d3f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:27.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.858+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d3f130 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005b40 tx=0x7f5b60021000 comp rx=0 tx=0).stop 2024-06-03T02:32:27.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.858+0000 7f5b77c3db80 1 -- 172.21.15.79:0/785846020 shutdown_connections 2024-06-03T02:32:27.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.858+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d504b0 0x55ed76d3f130 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.858+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d4a240 0x55ed76d49d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.858+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/785846020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 0x55ed76d511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.858+0000 7f5b77c3db80 1 -- 172.21.15.79:0/785846020 >> 172.21.15.79:0/785846020 conn(0x55ed76c76170 msgr2=0x55ed76d4e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:27.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.859+0000 7f5b77c3db80 1 -- 172.21.15.79:0/785846020 shutdown_connections 2024-06-03T02:32:27.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.859+0000 7f5b77c3db80 1 -- 172.21.15.79:0/785846020 wait complete. 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.859+0000 7f5b77c3db80 1 -- start start 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.859+0000 7f5b77c3db80 1 Processor -- start 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d5c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 0x55ed76d6b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b77c3db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d504b0 0x55ed76d6b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681be750 con 0x55ed76d504b0 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681be080 con 0x55ed76d4fb90 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b77c3db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681bb280 con 0x55ed76d4a240 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b77c3db80 10 client.0 _ll_register_callbacks cb 0x55ed76c31380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:27.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b65e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d5c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b66800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d504b0 0x55ed76d6b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b65e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d5c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:49606/0 (socket says 172.21.15.79:49606) 2024-06-03T02:32:27.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b65e00700 1 -- 172.21.15.79:0/2235989625 learned_addr learned my addr 172.21.15.79:0/2235989625 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:27.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b5fe00700 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 0x55ed76d6b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b66800700 1 -- 172.21.15.79:0/2235989625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 msgr2=0x55ed76d5c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:27.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.860+0000 7f5b66800700 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d5c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b66800700 1 -- 172.21.15.79:0/2235989625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 msgr2=0x55ed76d6b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:27.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b66800700 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 0x55ed76d6b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b66800700 1 -- 172.21.15.79:0/2235989625 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed76d6bff0 con 0x55ed76d504b0 2024-06-03T02:32:27.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b66800700 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d504b0 0x55ed76d6b780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005b40 tx=0x7f5b60030440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4971d760810d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:27.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b4be00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:32:27.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b4be00700 1 -- 172.21.15.79:0/2235989625 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60020040 con 0x55ed76d504b0 2024-06-03T02:32:27.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b4be00700 1 -- 172.21.15.79:0/2235989625 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5b60030da0 con 0x55ed76d504b0 2024-06-03T02:32:27.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b77c3db80 1 -- 172.21.15.79:0/2235989625 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b60035070 con 0x55ed76d504b0 2024-06-03T02:32:27.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b4be00700 1 -- 172.21.15.79:0/2235989625 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b60047050 con 0x55ed76d504b0 2024-06-03T02:32:27.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b77c3db80 10 client.16206 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:27.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.861+0000 7f5b77c3db80 1 -- 172.21.15.79:0/2235989625 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed76dce820 con 0x55ed76d504b0 2024-06-03T02:32:27.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.862+0000 7f5b4be00700 1 -- 172.21.15.79:0/2235989625 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60035070 con 0x55ed76d504b0 2024-06-03T02:32:27.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.862+0000 7f5b4be00700 1 -- 172.21.15.79:0/2235989625 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5b60003680 con 0x55ed76d504b0 2024-06-03T02:32:27.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.862+0000 7f5b77c3db80 10 client.16206 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:27.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.862+0000 7f5b77c3db80 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:32:27.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.862+0000 7f5b77c3db80 20 client.16206 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:27.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.862+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed76dcee30 0x55ed76dd1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:27.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.862+0000 7f5b77c3db80 4 client.16206 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:32:27.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.862+0000 7f5b77c3db80 1 -- 172.21.15.79:0/2235989625 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed76ca3580 con 0x55ed76dcee30 2024-06-03T02:32:27.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.862+0000 7f5b65e00700 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed76dcee30 0x55ed76dd1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:27.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.863+0000 7f5b65e00700 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed76dcee30 0x55ed76dd1250 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb0d7d57d936498 server_cookie=f0b807924115f0e7 in_seq=0 out_seq=0 2024-06-03T02:32:27.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.863+0000 7f5b4be00700 10 client.16206 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:27.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.863+0000 7f5b4be00700 1 -- 172.21.15.79:0/2235989625 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed76ca3580 con 0x55ed76dcee30 2024-06-03T02:32:27.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.863+0000 7f5b4be00700 10 client.16206 handle_command_reply: tid=0 2024-06-03T02:32:27.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.863+0000 7f5b77c3db80 1 client.16206 shutdown 2024-06-03T02:32:27.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 -- 172.21.15.79:0/2235989625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d504b0 msgr2=0x55ed76d6b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:27.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d504b0 0x55ed76d6b780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005b40 tx=0x7f5b60030440 comp rx=0 tx=0).stop 2024-06-03T02:32:27.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 -- 172.21.15.79:0/2235989625 shutdown_connections 2024-06-03T02:32:27.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55ed76dcee30 0x55ed76dd1250 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ed76d504b0 0x55ed76d6b780 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed76d4a240 0x55ed76d5c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 --2- 172.21.15.79:0/2235989625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76d4fb90 0x55ed76d6b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b4be00700 0 client.16206 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:32:27.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 -- 172.21.15.79:0/2235989625 >> 172.21.15.79:0/2235989625 conn(0x55ed76c76170 msgr2=0x55ed76d3e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:27.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 -- 172.21.15.79:0/2235989625 shutdown_connections 2024-06-03T02:32:27.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 1 -- 172.21.15.79:0/2235989625 wait complete. 2024-06-03T02:32:27.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.864+0000 7f5b77c3db80 20 client.16206 trim_cache size 0 max 16384 2024-06-03T02:32:27.109 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:32:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:32:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:32:27.109 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:32:27.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 -- 172.21.15.79:0/3063589980 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b4c0a2ca0 msgr2=0x7f5b4c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:27.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b4c0a2ca0 0x7f5b4c0a5160 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5b60004800 tx=0x7f5b60019040 comp rx=0 tx=0).stop 2024-06-03T02:32:27.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 -- 172.21.15.79:0/3063589980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6806e8f0 msgr2=0x7f5b681adbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:27.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6806e8f0 0x7f5b681adbf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000e9a0 tx=0x7f5b5000ed60 comp rx=0 tx=0).stop 2024-06-03T02:32:27.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 -- 172.21.15.79:0/3063589980 shutdown_connections 2024-06-03T02:32:27.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b6811bb40 0x7f5b681a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5b4c0a2ca0 0x7f5b4c0a5160 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b6806dfd0 0x7f5b681ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 --2- 172.21.15.79:0/3063589980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6806e8f0 0x7f5b681adbf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:27.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 -- 172.21.15.79:0/3063589980 >> 172.21.15.79:0/3063589980 conn(0x7f5b6806d6b0 msgr2=0x7f5b6811c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:27.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 -- 172.21.15.79:0/3063589980 shutdown_connections 2024-06-03T02:32:27.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:26.865+0000 7f5b6d200700 1 -- 172.21.15.79:0/3063589980 wait complete. 2024-06-03T02:32:28.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:27 smithi123 ceph-mon[36640]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.1 MiB/s wr, 418 op/s 2024-06-03T02:32:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 740.0 s, 428.2 tps, lat 220.976 ms stddev 282.796 2024-06-03T02:32:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:27 smithi079 ceph-mon[31819]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.1 MiB/s wr, 418 op/s 2024-06-03T02:32:28.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:27 smithi135 ceph-mon[36653]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.1 MiB/s wr, 418 op/s 2024-06-03T02:32:28.994 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:32:28.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:32:29.685 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:32:30.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:29 smithi079 ceph-mon[31819]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.5 MiB/s wr, 467 op/s 2024-06-03T02:32:30.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:29 smithi123 ceph-mon[36640]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.5 MiB/s wr, 467 op/s 2024-06-03T02:32:30.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:29 smithi135 ceph-mon[36653]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.5 MiB/s wr, 467 op/s 2024-06-03T02:32:32.225 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:31 smithi123 ceph-mon[36640]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.1 MiB/s wr, 474 op/s 2024-06-03T02:32:32.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:31 smithi079 ceph-mon[31819]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.1 MiB/s wr, 474 op/s 2024-06-03T02:32:32.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:31 smithi135 ceph-mon[36653]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.1 MiB/s wr, 474 op/s 2024-06-03T02:32:32.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.990+0000 7fd5d0a00700 1 -- 172.21.15.79:0/2342744355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5cc1155e0 msgr2=0x7fd5cc1179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:32.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.990+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/2342744355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5cc1155e0 0x7fd5cc1179d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009990 tx=0x7fd5c4009ca0 comp rx=0 tx=0).stop 2024-06-03T02:32:32.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.993+0000 7fd5d0a00700 1 -- 172.21.15.79:0/2342744355 shutdown_connections 2024-06-03T02:32:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.993+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/2342744355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5cc1155e0 0x7fd5cc1179d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.993+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/2342744355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc109490 0x7fd5cc109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.993+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/2342744355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5cc109db0 0x7fd5cc1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.993+0000 7fd5d0a00700 1 -- 172.21.15.79:0/2342744355 >> 172.21.15.79:0/2342744355 conn(0x7fd5cc06d600 msgr2=0x7fd5cc06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.993+0000 7fd5d0a00700 1 -- 172.21.15.79:0/2342744355 shutdown_connections 2024-06-03T02:32:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.993+0000 7fd5d0a00700 1 -- 172.21.15.79:0/2342744355 wait complete. 2024-06-03T02:32:32.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.994+0000 7fd5d0a00700 1 Processor -- start 2024-06-03T02:32:32.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.995+0000 7fd5d0a00700 1 -- start start 2024-06-03T02:32:32.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.995+0000 7fd5d0a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5cc109490 0x7fd5cc114ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.995+0000 7fd5d0a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5cc109db0 0x7fd5cc1120a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.995+0000 7fd5d0a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1155e0 0x7fd5cc1126b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.995+0000 7fd5d0a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc10ca30 con 0x7fd5cc109490 2024-06-03T02:32:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.995+0000 7fd5d0a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc10c8b0 con 0x7fd5cc109db0 2024-06-03T02:32:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.995+0000 7fd5d0a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc10cbb0 con 0x7fd5cc1155e0 2024-06-03T02:32:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.995+0000 7fd5ca800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1155e0 0x7fd5cc1126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:32.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.996+0000 7fd5ca800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1155e0 0x7fd5cc1126b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:49620/0 (socket says 172.21.15.79:49620) 2024-06-03T02:32:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.996+0000 7fd5ca800700 1 -- 172.21.15.79:0/1437009425 learned_addr learned my addr 172.21.15.79:0/1437009425 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.996+0000 7fd5c9e00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5cc109490 0x7fd5cc114ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.997+0000 7fd5c9e00700 1 -- 172.21.15.79:0/1437009425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1155e0 msgr2=0x7fd5cc1126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.997+0000 7fd5c9e00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1155e0 0x7fd5cc1126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.997+0000 7fd5c9e00700 1 -- 172.21.15.79:0/1437009425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5cc109db0 msgr2=0x7fd5cc1120a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:32:33.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.997+0000 7fd5c9e00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5cc109db0 0x7fd5cc1120a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.997+0000 7fd5c9e00700 1 -- 172.21.15.79:0/1437009425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc1b3e40 con 0x7fd5cc109490 2024-06-03T02:32:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.997+0000 7fd5ca800700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1155e0 0x7fd5cc1126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:32:33.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.997+0000 7fd5c9e00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5cc109490 0x7fd5cc114ff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00efc0 tx=0x7fd5bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.999+0000 7fd5c2a00700 1 -- 172.21.15.79:0/1437009425 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0092e0 con 0x7fd5cc109490 2024-06-03T02:32:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.999+0000 7fd5c2a00700 1 -- 172.21.15.79:0/1437009425 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc004bb0 con 0x7fd5cc109490 2024-06-03T02:32:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:32.999+0000 7fd5c2a00700 1 -- 172.21.15.79:0/1437009425 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc004280 con 0x7fd5cc109490 2024-06-03T02:32:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.000+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc1b4010 con 0x7fd5cc109490 2024-06-03T02:32:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.000+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc1b44d0 con 0x7fd5cc109490 2024-06-03T02:32:33.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.002+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc06eb90 con 0x7fd5cc109490 2024-06-03T02:32:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.005+0000 7fd5c2a00700 1 -- 172.21.15.79:0/1437009425 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd5bc044d10 con 0x7fd5cc109490 2024-06-03T02:32:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.008+0000 7fd5c2a00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd5b00a2db0 0x7fd5b00a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.008+0000 7fd5c2a00700 1 -- 172.21.15.79:0/1437009425 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fd5bc0cf300 con 0x7fd5cc109490 2024-06-03T02:32:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.009+0000 7fd5c3e00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd5b00a2db0 0x7fd5b00a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:33.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.010+0000 7fd5c3e00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd5b00a2db0 0x7fd5b00a5270 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc113860 tx=0x7fd5b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:33.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.015+0000 7fd5c2a00700 1 -- 172.21.15.79:0/1437009425 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5bc022020 con 0x7fd5cc109490 2024-06-03T02:32:33.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 745.0 s, 469.8 tps, lat 222.084 ms stddev 302.229 2024-06-03T02:32:33.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.456+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5cc1b47f0 con 0x7fd5cc109490 2024-06-03T02:32:33.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.459+0000 7fd5c2a00700 1 -- 172.21.15.79:0/1437009425 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7fd5bc010040 con 0x7fd5cc109490 2024-06-03T02:32:33.469 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:32:33.470 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:32:33.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.470+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd5b00a2db0 msgr2=0x7fd5b00a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:33.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.470+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd5b00a2db0 0x7fd5b00a5270 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc113860 tx=0x7fd5b4009260 comp rx=0 tx=0).stop 2024-06-03T02:32:33.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.470+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5cc109490 msgr2=0x7fd5cc114ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:33.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.470+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5cc109490 0x7fd5cc114ff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00efc0 tx=0x7fd5bc00c3a0 comp rx=0 tx=0).stop 2024-06-03T02:32:33.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.470+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 shutdown_connections 2024-06-03T02:32:33.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.470+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1155e0 0x7fd5cc1126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:33.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.470+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fd5b00a2db0 0x7fd5b00a5270 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:33.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.471+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5cc109490 0x7fd5cc114ff0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:33.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.471+0000 7fd5d0a00700 1 --2- 172.21.15.79:0/1437009425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5cc109db0 0x7fd5cc1120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:33.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.471+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 >> 172.21.15.79:0/1437009425 conn(0x7fd5cc06d600 msgr2=0x7fd5cc071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:33.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.471+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 shutdown_connections 2024-06-03T02:32:33.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:33.471+0000 7fd5d0a00700 1 -- 172.21.15.79:0/1437009425 wait complete. 2024-06-03T02:32:33.475 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:32:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:33 smithi123 ceph-mon[36640]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.3 MiB/s wr, 423 op/s 2024-06-03T02:32:34.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:33 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1437009425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:32:34.255 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:33 smithi079 ceph-mon[31819]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.3 MiB/s wr, 423 op/s 2024-06-03T02:32:34.255 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:33 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1437009425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:32:34.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:33 smithi135 ceph-mon[36653]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.3 MiB/s wr, 423 op/s 2024-06-03T02:32:34.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:33 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1437009425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:32:35.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:32:36.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:35 smithi079 ceph-mon[31819]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.2 MiB/s wr, 476 op/s 2024-06-03T02:32:36.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:35 smithi123 ceph-mon[36640]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.2 MiB/s wr, 476 op/s 2024-06-03T02:32:36.242 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:32:36.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:35 smithi135 ceph-mon[36653]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.2 MiB/s wr, 476 op/s 2024-06-03T02:32:38.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:37 smithi079 ceph-mon[31819]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.1 MiB/s wr, 393 op/s 2024-06-03T02:32:38.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:37 smithi123 ceph-mon[36640]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.1 MiB/s wr, 393 op/s 2024-06-03T02:32:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 750.0 s, 471.0 tps, lat 208.207 ms stddev 286.936 2024-06-03T02:32:38.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:37 smithi135 ceph-mon[36653]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.1 MiB/s wr, 393 op/s 2024-06-03T02:32:39.946 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:39 smithi079 ceph-mon[31819]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.5 MiB/s wr, 445 op/s 2024-06-03T02:32:40.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:39 smithi123 ceph-mon[36640]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.5 MiB/s wr, 445 op/s 2024-06-03T02:32:40.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:39 smithi135 ceph-mon[36653]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.5 MiB/s wr, 445 op/s 2024-06-03T02:32:41.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.102+0000 7f568ce00700 1 -- 172.21.15.79:0/4183412124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810cc10 msgr2=0x7f5688109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.102+0000 7f568ce00700 1 --2- 172.21.15.79:0/4183412124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810cc10 0x7f5688109490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5678009990 tx=0x7f5678009ca0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.102+0000 7f568ce00700 1 -- 172.21.15.79:0/4183412124 shutdown_connections 2024-06-03T02:32:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.102+0000 7f568ce00700 1 --2- 172.21.15.79:0/4183412124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56881175c0 0x7f56881199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.102+0000 7f568ce00700 1 --2- 172.21.15.79:0/4183412124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810cc10 0x7f5688109490 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.102+0000 7f568ce00700 1 --2- 172.21.15.79:0/4183412124 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5688109a60 0x7f5688109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.103+0000 7f568ce00700 1 -- 172.21.15.79:0/4183412124 >> 172.21.15.79:0/4183412124 conn(0x7f568806d600 msgr2=0x7f568806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.103+0000 7f568ce00700 1 -- 172.21.15.79:0/4183412124 shutdown_connections 2024-06-03T02:32:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.103+0000 7f568ce00700 1 -- 172.21.15.79:0/4183412124 wait complete. 2024-06-03T02:32:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.104+0000 7f568ce00700 1 Processor -- start 2024-06-03T02:32:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.105+0000 7f568ce00700 1 -- start start 2024-06-03T02:32:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.106+0000 7f568ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5688109a60 0x7f56881a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.106+0000 7f568ce00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568810cc10 0x7f56881a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.106+0000 7f568ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56881175c0 0x7f56881a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f568ce00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f568811d300 con 0x7f56881175c0 2024-06-03T02:32:41.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f568ce00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f568811d180 con 0x7f5688109a60 2024-06-03T02:32:41.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f568ce00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f568811d480 con 0x7f568810cc10 2024-06-03T02:32:41.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f5685400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568810cc10 0x7f56881a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f5685400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568810cc10 0x7f56881a3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40140/0 (socket says 172.21.15.79:40140) 2024-06-03T02:32:41.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f5685400700 1 -- 172.21.15.79:0/1064182220 learned_addr learned my addr 172.21.15.79:0/1064182220 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:41.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f5686800700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56881175c0 0x7f56881a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f5685400700 1 -- 172.21.15.79:0/1064182220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5688109a60 msgr2=0x7f56881a3730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:32:41.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f5685400700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5688109a60 0x7f56881a3730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f5685400700 1 -- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56881175c0 msgr2=0x7f56881a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:41.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f5685400700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56881175c0 0x7f56881a8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.107+0000 7f5685400700 1 -- 172.21.15.79:0/1064182220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56881a8740 con 0x7f568810cc10 2024-06-03T02:32:41.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.108+0000 7f5685400700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568810cc10 0x7f56881a3c70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f567400d950 tx=0x7f567400dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:41.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.109+0000 7f567ea00700 1 -- 172.21.15.79:0/1064182220 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674016070 con 0x7f568810cc10 2024-06-03T02:32:41.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.109+0000 7f567ea00700 1 -- 172.21.15.79:0/1064182220 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5674009b40 con 0x7f568810cc10 2024-06-03T02:32:41.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.109+0000 7f567ea00700 1 -- 172.21.15.79:0/1064182220 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567400f620 con 0x7f568810cc10 2024-06-03T02:32:41.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.109+0000 7f568ce00700 1 -- 172.21.15.79:0/1064182220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56881a8a30 con 0x7f568810cc10 2024-06-03T02:32:41.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.109+0000 7f568ce00700 1 -- 172.21.15.79:0/1064182220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56881b02b0 con 0x7f568810cc10 2024-06-03T02:32:41.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.123+0000 7f567ea00700 1 -- 172.21.15.79:0/1064182220 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5674019070 con 0x7f568810cc10 2024-06-03T02:32:41.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.125+0000 7f567ea00700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f566c0a2e20 0x7f566c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.126+0000 7f567ea00700 1 -- 172.21.15.79:0/1064182220 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f56740cefb0 con 0x7f568810cc10 2024-06-03T02:32:41.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.126+0000 7f5697843b80 1 Processor -- start 2024-06-03T02:32:41.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.126+0000 7f5697843b80 1 -- start start 2024-06-03T02:32:41.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5697843b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e84070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5697843b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e84a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5697843b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e99f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5697843b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0d93a10 con 0x55f2b0e97b10 2024-06-03T02:32:41.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5697843b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0dd6b00 con 0x55f2b0e00e40 2024-06-03T02:32:41.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5697843b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0d918f0 con 0x55f2b0e845b0 2024-06-03T02:32:41.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5685e00700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f566c0a2e20 0x7f566c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5685400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e84a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5685400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e84a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40154/0 (socket says 172.21.15.79:40154) 2024-06-03T02:32:41.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5685400700 1 -- 172.21.15.79:0/2919176180 learned_addr learned my addr 172.21.15.79:0/2919176180 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:41.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.127+0000 7f5686800700 1 --2- 172.21.15.79:0/2919176180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e99f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.128+0000 7f5685400700 1 -- 172.21.15.79:0/2919176180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 msgr2=0x55f2b0e84070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:41.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.128+0000 7f5685400700 1 --2- 172.21.15.79:0/2919176180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e84070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.128+0000 7f5685400700 1 -- 172.21.15.79:0/2919176180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 msgr2=0x55f2b0e99f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:41.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.128+0000 7f5685400700 1 --2- 172.21.15.79:0/2919176180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e99f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.128+0000 7f5685400700 1 -- 172.21.15.79:0/2919176180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b0d7ef90 con 0x55f2b0e845b0 2024-06-03T02:32:41.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.128+0000 7f5686800700 1 --2- 172.21.15.79:0/2919176180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e99f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:32:41.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.129+0000 7f5685400700 1 --2- 172.21.15.79:0/2919176180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e84a30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f567409b100 tx=0x7f5674010bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b7b9038cc865053 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:41.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.129+0000 7f5673400700 1 -- 172.21.15.79:0/2919176180 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674020040 con 0x55f2b0e845b0 2024-06-03T02:32:41.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.130+0000 7f5673400700 1 -- 172.21.15.79:0/2919176180 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567409b400 con 0x55f2b0e845b0 2024-06-03T02:32:41.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5697843b80 1 -- 172.21.15.79:0/2919176180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 msgr2=0x55f2b0e84a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:41.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5685e00700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f566c0a2e20 0x7f566c0a52e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5678000c00 tx=0x7f5678009710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:41.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5697843b80 1 --2- 172.21.15.79:0/2919176180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e84a30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f567409b100 tx=0x7f5674010bf0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5697843b80 1 -- 172.21.15.79:0/2919176180 shutdown_connections 2024-06-03T02:32:41.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5697843b80 1 --2- 172.21.15.79:0/2919176180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e99f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5697843b80 1 --2- 172.21.15.79:0/2919176180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e84070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5697843b80 1 --2- 172.21.15.79:0/2919176180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e84a30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5697843b80 1 -- 172.21.15.79:0/2919176180 >> 172.21.15.79:0/2919176180 conn(0x55f2b0db1170 msgr2=0x55f2b0e804b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:41.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5697843b80 1 -- 172.21.15.79:0/2919176180 shutdown_connections 2024-06-03T02:32:41.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.131+0000 7f5697843b80 1 -- 172.21.15.79:0/2919176180 wait complete. 2024-06-03T02:32:41.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.132+0000 7f5697843b80 1 -- start start 2024-06-03T02:32:41.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.132+0000 7f5697843b80 1 Processor -- start 2024-06-03T02:32:41.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.133+0000 7f5697843b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e971b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.133+0000 7f5697843b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.133+0000 7f5697843b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.133+0000 7f5697843b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0d72f60 con 0x55f2b0e97b10 2024-06-03T02:32:41.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.133+0000 7f5697843b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0dd2ac0 con 0x55f2b0e00e40 2024-06-03T02:32:41.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.133+0000 7f5697843b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0e9a870 con 0x55f2b0e845b0 2024-06-03T02:32:41.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.133+0000 7f5697843b80 10 client.0 _ll_register_callbacks cb 0x55f2b0d70720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:41.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.133+0000 7f5685e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.134+0000 7f5686800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.134+0000 7f5685400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.134+0000 7f5686800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e949f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36658/0 (socket says 172.21.15.79:36658) 2024-06-03T02:32:41.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.134+0000 7f5686800700 1 -- 172.21.15.79:0/3792778987 learned_addr learned my addr 172.21.15.79:0/3792778987 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:41.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.135+0000 7f5686800700 1 -- 172.21.15.79:0/3792778987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 msgr2=0x55f2b0e944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:41.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.135+0000 7f5686800700 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.135+0000 7f5686800700 1 -- 172.21.15.79:0/3792778987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 msgr2=0x55f2b0e971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:41.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.135+0000 7f5686800700 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.135+0000 7f5686800700 1 -- 172.21.15.79:0/3792778987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b0d258d0 con 0x55f2b0e97b10 2024-06-03T02:32:41.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.138+0000 7f5686800700 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e949f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f568000e8b0 tx=0x7f568000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f923ac97c5a57da1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:41.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.138+0000 7f5667e00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:32:41.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.138+0000 7f5667e00700 1 -- 172.21.15.79:0/3792778987 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680018020 con 0x55f2b0e97b10 2024-06-03T02:32:41.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.138+0000 7f5667e00700 1 -- 172.21.15.79:0/3792778987 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f568000c880 con 0x55f2b0e97b10 2024-06-03T02:32:41.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.139+0000 7f5697843b80 1 -- 172.21.15.79:0/3792778987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567409b400 con 0x55f2b0e97b10 2024-06-03T02:32:41.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.139+0000 7f5667e00700 1 -- 172.21.15.79:0/3792778987 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f568000ca80 con 0x55f2b0e97b10 2024-06-03T02:32:41.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.139+0000 7f5697843b80 10 client.16221 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:41.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.139+0000 7f5697843b80 1 -- 172.21.15.79:0/3792778987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f2b0e8f670 con 0x55f2b0e97b10 2024-06-03T02:32:41.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.141+0000 7f5667e00700 1 -- 172.21.15.79:0/3792778987 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567409b400 con 0x55f2b0e97b10 2024-06-03T02:32:41.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.141+0000 7f5667e00700 1 -- 172.21.15.79:0/3792778987 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5680025020 con 0x55f2b0e97b10 2024-06-03T02:32:41.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.141+0000 7f5697843b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:41.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.141+0000 7f5697843b80 10 client.16221 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:32:41.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.141+0000 7f5697843b80 20 client.16221 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:41.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.141+0000 7f5697843b80 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55f2b0eafeb0 0x55f2b0eb22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.141+0000 7f5697843b80 4 client.16221 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:32:41.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.141+0000 7f5697843b80 1 -- 172.21.15.79:0/3792778987 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b0de4000 con 0x55f2b0eafeb0 2024-06-03T02:32:41.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.142+0000 7f5685e00700 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55f2b0eafeb0 0x55f2b0eb22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.143+0000 7f5685e00700 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55f2b0eafeb0 0x55f2b0eb22a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1fb3f3acbfb6592e server_cookie=eafc52010ae9fa76 in_seq=0 out_seq=0 2024-06-03T02:32:41.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.143+0000 7f5667e00700 10 client.16221 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:32:41.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.144+0000 7f5667e00700 1 -- 172.21.15.79:0/3792778987 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f2b0de4000 con 0x55f2b0eafeb0 2024-06-03T02:32:41.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.144+0000 7f5667e00700 10 client.16221 handle_command_reply: tid=0 2024-06-03T02:32:41.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.145+0000 7f5697843b80 1 client.16221 shutdown 2024-06-03T02:32:41.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.145+0000 7f5697843b80 1 -- 172.21.15.79:0/3792778987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 msgr2=0x55f2b0e949f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:41.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.145+0000 7f5697843b80 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e949f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f568000e8b0 tx=0x7f568000ec70 comp rx=0 tx=0).stop 2024-06-03T02:32:41.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.146+0000 7f5697843b80 1 -- 172.21.15.79:0/3792778987 shutdown_connections 2024-06-03T02:32:41.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.146+0000 7f5697843b80 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55f2b0eafeb0 0x55f2b0eb22a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.146+0000 7f5697843b80 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0e949f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.147+0000 7f5697843b80 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.147+0000 7f5697843b80 1 --2- 172.21.15.79:0/3792778987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.147+0000 7f5697843b80 1 -- 172.21.15.79:0/3792778987 >> 172.21.15.79:0/3792778987 conn(0x55f2b0db1170 msgr2=0x55f2b0e7f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:41.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.147+0000 7f5667e00700 0 client.16221 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:32:41.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.147+0000 7f5697843b80 1 -- 172.21.15.79:0/3792778987 shutdown_connections 2024-06-03T02:32:41.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.147+0000 7f5697843b80 1 -- 172.21.15.79:0/3792778987 wait complete. 2024-06-03T02:32:41.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.147+0000 7f5697843b80 20 client.16221 trim_cache size 0 max 16384 2024-06-03T02:32:41.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.177+0000 7f5697843b80 1 Processor -- start 2024-06-03T02:32:41.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.177+0000 7f5697843b80 1 -- start start 2024-06-03T02:32:41.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5697843b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e98f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5697843b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e845b0 0x55f2b0e92da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5697843b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e97b10 0x55f2b0e932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5697843b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b4ae0 con 0x55f2b0e845b0 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5697843b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b6620 con 0x55f2b0e00e40 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5697843b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b51b0 con 0x55f2b0e97b10 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5685e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e98f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5685e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e98f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:57228/0 (socket says 172.21.15.79:57228) 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5685400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e845b0 0x55f2b0e92da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5685e00700 1 -- 172.21.15.79:0/1906636341 learned_addr learned my addr 172.21.15.79:0/1906636341 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.178+0000 7f5686800700 1 --2- 172.21.15.79:0/1906636341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e97b10 0x55f2b0e932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.179+0000 7f5686800700 1 -- 172.21.15.79:0/1906636341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 msgr2=0x55f2b0e98f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.179+0000 7f5686800700 1 --2- 172.21.15.79:0/1906636341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e98f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.179+0000 7f5686800700 1 -- 172.21.15.79:0/1906636341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e845b0 msgr2=0x55f2b0e92da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:41.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.179+0000 7f5686800700 1 --2- 172.21.15.79:0/1906636341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e845b0 0x55f2b0e92da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.179+0000 7f5686800700 1 -- 172.21.15.79:0/1906636341 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b0eb28d0 con 0x55f2b0e97b10 2024-06-03T02:32:41.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.179+0000 7f5685400700 1 --2- 172.21.15.79:0/1906636341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e845b0 0x55f2b0e92da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:32:41.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.179+0000 7f5686800700 1 --2- 172.21.15.79:0/1906636341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e97b10 0x55f2b0e932e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f568000e8b0 tx=0x7f568000c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c11b1f4ab2dd819a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:41.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.180+0000 7f5672a00700 1 -- 172.21.15.79:0/1906636341 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680018040 con 0x55f2b0e97b10 2024-06-03T02:32:41.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.180+0000 7f5672a00700 1 -- 172.21.15.79:0/1906636341 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56800078f0 con 0x55f2b0e97b10 2024-06-03T02:32:41.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.180+0000 7f5672a00700 1 -- 172.21.15.79:0/1906636341 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680014440 con 0x55f2b0e97b10 2024-06-03T02:32:41.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.181+0000 7f5697843b80 1 -- 172.21.15.79:0/1906636341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e97b10 msgr2=0x55f2b0e932e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:41.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.181+0000 7f5697843b80 1 --2- 172.21.15.79:0/1906636341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e97b10 0x55f2b0e932e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f568000e8b0 tx=0x7f568000c520 comp rx=0 tx=0).stop 2024-06-03T02:32:41.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.181+0000 7f5697843b80 1 -- 172.21.15.79:0/1906636341 shutdown_connections 2024-06-03T02:32:41.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.181+0000 7f5697843b80 1 --2- 172.21.15.79:0/1906636341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e97b10 0x55f2b0e932e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.181+0000 7f5697843b80 1 --2- 172.21.15.79:0/1906636341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e98f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.181+0000 7f5697843b80 1 --2- 172.21.15.79:0/1906636341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e845b0 0x55f2b0e92da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.181+0000 7f5697843b80 1 -- 172.21.15.79:0/1906636341 >> 172.21.15.79:0/1906636341 conn(0x55f2b0db1170 msgr2=0x55f2b0e7ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:41.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.181+0000 7f5697843b80 1 -- 172.21.15.79:0/1906636341 shutdown_connections 2024-06-03T02:32:41.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.181+0000 7f5697843b80 1 -- 172.21.15.79:0/1906636341 wait complete. 2024-06-03T02:32:41.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.182+0000 7f5697843b80 1 -- start start 2024-06-03T02:32:41.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.182+0000 7f5697843b80 1 Processor -- start 2024-06-03T02:32:41.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.182+0000 7f5697843b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e8c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.182+0000 7f5697843b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e8caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.182+0000 7f5697843b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0f18ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.182+0000 7f5697843b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b5f50 con 0x55f2b0e97b10 2024-06-03T02:32:41.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.182+0000 7f5697843b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b5880 con 0x55f2b0e00e40 2024-06-03T02:32:41.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.182+0000 7f5697843b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b2aa0 con 0x55f2b0e845b0 2024-06-03T02:32:41.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.182+0000 7f5697843b80 10 client.0 _ll_register_callbacks cb 0x55f2b0d70720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:41.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.183+0000 7f5685e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e8c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.183+0000 7f5685400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e8caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.183+0000 7f5685400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e8caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40176/0 (socket says 172.21.15.79:40176) 2024-06-03T02:32:41.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.183+0000 7f5685400700 1 -- 172.21.15.79:0/3088952727 learned_addr learned my addr 172.21.15.79:0/3088952727 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.184+0000 7f5685400700 1 -- 172.21.15.79:0/3088952727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 msgr2=0x55f2b0e8c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.184+0000 7f5685400700 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e8c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.184+0000 7f5685400700 1 -- 172.21.15.79:0/3088952727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 msgr2=0x55f2b0f18ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.184+0000 7f5685400700 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0f18ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.184+0000 7f5685400700 1 -- 172.21.15.79:0/3088952727 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b0f19080 con 0x55f2b0e845b0 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.185+0000 7f5685400700 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e8caa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5674097860 tx=0x7f567409b400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ade7c029de4434 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.185+0000 7f5667e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.185+0000 7f5667e00700 1 -- 172.21.15.79:0/3088952727 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674020040 con 0x55f2b0e845b0 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.185+0000 7f5667e00700 1 -- 172.21.15.79:0/3088952727 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f56740ccb60 con 0x55f2b0e845b0 2024-06-03T02:32:41.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.185+0000 7f5697843b80 1 -- 172.21.15.79:0/3088952727 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56800078f0 con 0x55f2b0e845b0 2024-06-03T02:32:41.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.186+0000 7f5667e00700 1 -- 172.21.15.79:0/3088952727 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f567409f7b0 con 0x55f2b0e845b0 2024-06-03T02:32:41.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.186+0000 7f5697843b80 10 client.26743 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:41.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.186+0000 7f5697843b80 1 -- 172.21.15.79:0/3088952727 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f2b0f1a360 con 0x55f2b0e845b0 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.186+0000 7f5667e00700 1 -- 172.21.15.79:0/3088952727 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56800078f0 con 0x55f2b0e845b0 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.186+0000 7f5667e00700 1 -- 172.21.15.79:0/3088952727 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f567409a2c0 con 0x55f2b0e845b0 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.187+0000 7f5697843b80 10 client.26743 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.187+0000 7f5697843b80 10 client.26743 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.187+0000 7f5697843b80 20 client.26743 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.187+0000 7f5697843b80 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55f2b0f1a5e0 0x55f2b0f1c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.187+0000 7f5697843b80 4 client.26743 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.187+0000 7f5697843b80 1 -- 172.21.15.79:0/3088952727 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2b0dde580 con 0x55f2b0f1a5e0 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.187+0000 7f5685e00700 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55f2b0f1a5e0 0x55f2b0f1c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.188+0000 7f5667e00700 10 client.26743 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.188+0000 7f5685e00700 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55f2b0f1a5e0 0x55f2b0f1c9d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=820f43f26830cae7 server_cookie=1d341ddb1c58538f in_seq=0 out_seq=0 2024-06-03T02:32:41.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.188+0000 7f5667e00700 1 -- 172.21.15.79:0/3088952727 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2b0dde580 con 0x55f2b0f1a5e0 2024-06-03T02:32:41.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.188+0000 7f5667e00700 10 client.26743 handle_command_reply: tid=0 2024-06-03T02:32:41.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.188+0000 7f5697843b80 1 client.26743 shutdown 2024-06-03T02:32:41.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5697843b80 1 -- 172.21.15.79:0/3088952727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 msgr2=0x55f2b0e8caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:41.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5697843b80 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e8caa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5674097860 tx=0x7f567409b400 comp rx=0 tx=0).stop 2024-06-03T02:32:41.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5697843b80 1 -- 172.21.15.79:0/3088952727 shutdown_connections 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5697843b80 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x55f2b0f1a5e0 0x55f2b0f1c9d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5697843b80 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f2b0e97b10 0x55f2b0f18ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5697843b80 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2b0e00e40 0x55f2b0e8c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5697843b80 1 --2- 172.21.15.79:0/3088952727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f2b0e845b0 0x55f2b0e8caa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5697843b80 1 -- 172.21.15.79:0/3088952727 >> 172.21.15.79:0/3088952727 conn(0x55f2b0db1170 msgr2=0x55f2b0e94080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5667e00700 0 client.26743 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.189+0000 7f5697843b80 1 -- 172.21.15.79:0/3088952727 shutdown_connections 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.190+0000 7f5697843b80 1 -- 172.21.15.79:0/3088952727 wait complete. 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.190+0000 7f5697843b80 20 client.26743 trim_cache size 0 max 16384 2024-06-03T02:32:41.191 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:32:41.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 -- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f566c0a2e20 msgr2=0x7f566c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:41.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f566c0a2e20 0x7f566c0a52e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5678000c00 tx=0x7f5678009710 comp rx=0 tx=0).stop 2024-06-03T02:32:41.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 -- 172.21.15.79:0/1064182220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568810cc10 msgr2=0x7f56881a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:41.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568810cc10 0x7f56881a3c70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f567400d950 tx=0x7f567400dd10 comp rx=0 tx=0).stop 2024-06-03T02:32:41.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 -- 172.21.15.79:0/1064182220 shutdown_connections 2024-06-03T02:32:41.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f566c0a2e20 0x7f566c0a52e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56881175c0 0x7f56881a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5688109a60 0x7f56881a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 --2- 172.21.15.79:0/1064182220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568810cc10 0x7f56881a3c70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:41.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 -- 172.21.15.79:0/1064182220 >> 172.21.15.79:0/1064182220 conn(0x7f568806d600 msgr2=0x7f5688118c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:41.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 -- 172.21.15.79:0/1064182220 shutdown_connections 2024-06-03T02:32:41.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:41.191+0000 7f568ce00700 1 -- 172.21.15.79:0/1064182220 wait complete. 2024-06-03T02:32:42.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:41 smithi123 ceph-mon[36640]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-06-03T02:32:42.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:41 smithi079 ceph-mon[31819]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-06-03T02:32:42.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:41 smithi135 ceph-mon[36653]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-06-03T02:32:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 755.0 s, 469.0 tps, lat 220.616 ms stddev 283.153 2024-06-03T02:32:43.508 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:32:44.198 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:32:44.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:43 smithi123 ceph-mon[36640]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.2 MiB/s wr, 414 op/s 2024-06-03T02:32:44.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:43 smithi079 ceph-mon[31819]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.2 MiB/s wr, 414 op/s 2024-06-03T02:32:44.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:43 smithi135 ceph-mon[36653]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.2 MiB/s wr, 414 op/s 2024-06-03T02:32:46.710 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:46 smithi079 ceph-mon[31819]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-06-03T02:32:46.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:46 smithi123 ceph-mon[36640]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-06-03T02:32:46.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.790+0000 7ffb99e00700 1 -- 172.21.15.79:0/822219666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9411ba10 msgr2=0x7ffb9411de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:46.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.790+0000 7ffb99e00700 1 --2- 172.21.15.79:0/822219666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9411ba10 0x7ffb9411de00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00b0a0 tx=0x7ffb8c00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.791+0000 7ffb99e00700 1 -- 172.21.15.79:0/822219666 shutdown_connections 2024-06-03T02:32:46.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.791+0000 7ffb99e00700 1 --2- 172.21.15.79:0/822219666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9411ba10 0x7ffb9411de00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.791+0000 7ffb99e00700 1 --2- 172.21.15.79:0/822219666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb94082fb0 0x7ffb9407adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.792+0000 7ffb99e00700 1 --2- 172.21.15.79:0/822219666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb9407b330 0x7ffb9407b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.792+0000 7ffb99e00700 1 -- 172.21.15.79:0/822219666 >> 172.21.15.79:0/822219666 conn(0x7ffb94076a50 msgr2=0x7ffb94078e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:46.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.792+0000 7ffb99e00700 1 -- 172.21.15.79:0/822219666 shutdown_connections 2024-06-03T02:32:46.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.792+0000 7ffb99e00700 1 -- 172.21.15.79:0/822219666 wait complete. 2024-06-03T02:32:46.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.794+0000 7ffb99e00700 1 Processor -- start 2024-06-03T02:32:46.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.794+0000 7ffb99e00700 1 -- start start 2024-06-03T02:32:46.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.795+0000 7ffb99e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb9407b330 0x7ffb941ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.795+0000 7ffb99e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb94082fb0 0x7ffb941adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.795+0000 7ffb99e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9411ba10 0x7ffb941a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.795+0000 7ffb99e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94121640 con 0x7ffb9407b330 2024-06-03T02:32:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.795+0000 7ffb92000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb94082fb0 0x7ffb941adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.796+0000 7ffb93400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9411ba10 0x7ffb941a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.796+0000 7ffb92a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb9407b330 0x7ffb941ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.797+0000 7ffb92a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb9407b330 0x7ffb941ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37286/0 (socket says 172.21.15.79:37286) 2024-06-03T02:32:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.797+0000 7ffb92a00700 1 -- 172.21.15.79:0/2430547687 learned_addr learned my addr 172.21.15.79:0/2430547687 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.797+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb941214c0 con 0x7ffb9411ba10 2024-06-03T02:32:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.797+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb941217c0 con 0x7ffb94082fb0 2024-06-03T02:32:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.798+0000 7ffb93400700 1 -- 172.21.15.79:0/2430547687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb94082fb0 msgr2=0x7ffb941adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.799+0000 7ffb93400700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb94082fb0 0x7ffb941adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.801+0000 7ffb93400700 1 -- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb9407b330 msgr2=0x7ffb941ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.801+0000 7ffb93400700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb9407b330 0x7ffb941ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.801+0000 7ffb93400700 1 -- 172.21.15.79:0/2430547687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb941a8040 con 0x7ffb9411ba10 2024-06-03T02:32:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.801+0000 7ffb92a00700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb9407b330 0x7ffb941ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:32:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.801+0000 7ffb93400700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9411ba10 0x7ffb941a78e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009750 tx=0x7ffb8c004cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.802+0000 7ffb98a00700 1 -- 172.21.15.79:0/2430547687 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c021070 con 0x7ffb9411ba10 2024-06-03T02:32:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.802+0000 7ffb98a00700 1 -- 172.21.15.79:0/2430547687 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb8c007b80 con 0x7ffb9411ba10 2024-06-03T02:32:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.802+0000 7ffb98a00700 1 -- 172.21.15.79:0/2430547687 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0052a0 con 0x7ffb9411ba10 2024-06-03T02:32:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.802+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb941a82d0 con 0x7ffb9411ba10 2024-06-03T02:32:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.802+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9406beb0 con 0x7ffb9411ba10 2024-06-03T02:32:46.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.808+0000 7ffb98a00700 1 -- 172.21.15.79:0/2430547687 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ffb8c024070 con 0x7ffb9411ba10 2024-06-03T02:32:46.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.813+0000 7ffba47f8b80 1 Processor -- start 2024-06-03T02:32:46.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.814+0000 7ffba47f8b80 1 -- start start 2024-06-03T02:32:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.820+0000 7ffb98a00700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb780a2d20 0x7ffb780a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.820+0000 7ffb98a00700 1 -- 172.21.15.79:0/2430547687 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ffb8c00f020 con 0x7ffb9411ba10 2024-06-03T02:32:46.825 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.821+0000 7ffb92000700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb780a2d20 0x7ffb780a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.829+0000 7ffb92000700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb780a2d20 0x7ffb780a51e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffb80006fd0 tx=0x7ffb80008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:46.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.832+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde56f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.832+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde50d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.832+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 0x55aefde51680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.832+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55aefdd5fa10 con 0x55aefde51240 2024-06-03T02:32:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.832+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aefdda2b00 con 0x55aefde574b0 2024-06-03T02:32:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.832+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55aefdd5d8f0 con 0x55aefde56b90 2024-06-03T02:32:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.836+0000 7ffb92a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde56f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.836+0000 7ffb93400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 0x55aefde51680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.836+0000 7ffb92a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde56f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40516/0 (socket says 172.21.15.79:40516) 2024-06-03T02:32:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.836+0000 7ffb92a00700 1 -- 172.21.15.79:0/751166844 learned_addr learned my addr 172.21.15.79:0/751166844 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.837+0000 7ffb92000700 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde50d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.837+0000 7ffb92a00700 1 -- 172.21.15.79:0/751166844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 msgr2=0x55aefde50d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.837+0000 7ffb92a00700 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde50d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.837+0000 7ffb92a00700 1 -- 172.21.15.79:0/751166844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 msgr2=0x55aefde51680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.837+0000 7ffb92a00700 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 0x55aefde51680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.837+0000 7ffb92a00700 1 -- 172.21.15.79:0/751166844 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefdd4af90 con 0x55aefde56b90 2024-06-03T02:32:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.837+0000 7ffb92000700 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:32:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.837+0000 7ffb93400700 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 0x55aefde51680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:32:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.838+0000 7ffb92a00700 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde56f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffb84011680 tx=0x7ffb84011990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8917219077134dc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.838+0000 7ffb88c00700 1 -- 172.21.15.79:0/751166844 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84004030 con 0x55aefde56b90 2024-06-03T02:32:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.839+0000 7ffb88c00700 1 -- 172.21.15.79:0/751166844 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb84003680 con 0x55aefde56b90 2024-06-03T02:32:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.839+0000 7ffb88c00700 1 -- 172.21.15.79:0/751166844 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84019910 con 0x55aefde56b90 2024-06-03T02:32:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.840+0000 7ffba47f8b80 1 -- 172.21.15.79:0/751166844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 msgr2=0x55aefde56f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.840+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde56f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffb84011680 tx=0x7ffb84011990 comp rx=0 tx=0).stop 2024-06-03T02:32:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.840+0000 7ffba47f8b80 1 -- 172.21.15.79:0/751166844 shutdown_connections 2024-06-03T02:32:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.840+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 0x55aefde51680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.840+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde56f70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.840+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/751166844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.840+0000 7ffba47f8b80 1 -- 172.21.15.79:0/751166844 >> 172.21.15.79:0/751166844 conn(0x55aefdd7d170 msgr2=0x55aefde4bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:46.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.841+0000 7ffba47f8b80 1 -- 172.21.15.79:0/751166844 shutdown_connections 2024-06-03T02:32:46.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:46 smithi135 ceph-mon[36653]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-06-03T02:32:46.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.843+0000 7ffba47f8b80 1 -- 172.21.15.79:0/751166844 wait complete. 2024-06-03T02:32:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.844+0000 7ffba47f8b80 1 -- start start 2024-06-03T02:32:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.844+0000 7ffba47f8b80 1 Processor -- start 2024-06-03T02:32:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.845+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde51240 0x55aefde5d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.846+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.846+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 0x55aefde5a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.846+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55aefdd3ef60 con 0x55aefde574b0 2024-06-03T02:32:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.846+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aefdd9eac0 con 0x55aefde51240 2024-06-03T02:32:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.846+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55aefde5dee0 con 0x55aefde56b90 2024-06-03T02:32:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.846+0000 7ffba47f8b80 10 client.0 _ll_register_callbacks cb 0x55aefdd38380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.847+0000 7ffb92a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde51240 0x55aefde5d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.847+0000 7ffb92a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde51240 0x55aefde5d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:55538/0 (socket says 172.21.15.79:55538) 2024-06-03T02:32:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.847+0000 7ffb92a00700 1 -- 172.21.15.79:0/2920528539 learned_addr learned my addr 172.21.15.79:0/2920528539 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.847+0000 7ffb92000700 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde5a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.847+0000 7ffb92a00700 1 -- 172.21.15.79:0/2920528539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 msgr2=0x55aefde5a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.847+0000 7ffb92a00700 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde5a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.847+0000 7ffb93400700 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 0x55aefde5a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.848+0000 7ffb92a00700 1 -- 172.21.15.79:0/2920528539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 msgr2=0x55aefde5a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.848+0000 7ffb92a00700 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 0x55aefde5a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.848+0000 7ffb92a00700 1 -- 172.21.15.79:0/2920528539 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefdcf18d0 con 0x55aefde51240 2024-06-03T02:32:46.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.848+0000 7ffb93400700 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 0x55aefde5a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:32:46.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.849+0000 7ffb92a00700 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde51240 0x55aefde5d170 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009880 tx=0x7ffb840053a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=812c84bc1ecce263 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:46.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.849+0000 7ffb7d600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:32:46.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.849+0000 7ffb7d600700 1 -- 172.21.15.79:0/2920528539 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84027030 con 0x55aefde51240 2024-06-03T02:32:46.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.849+0000 7ffb7d600700 1 -- 172.21.15.79:0/2920528539 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ffb84016050 con 0x55aefde51240 2024-06-03T02:32:46.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.850+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2920528539 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb84003680 con 0x55aefde51240 2024-06-03T02:32:46.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.850+0000 7ffb7d600700 1 -- 172.21.15.79:0/2920528539 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb84005ce0 con 0x55aefde51240 2024-06-03T02:32:46.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.850+0000 7ffba47f8b80 10 client.26768 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:46.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.850+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2920528539 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aefde68580 con 0x55aefde51240 2024-06-03T02:32:46.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.851+0000 7ffb7d600700 1 -- 172.21.15.79:0/2920528539 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb84003680 con 0x55aefde51240 2024-06-03T02:32:46.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.851+0000 7ffb7d600700 1 -- 172.21.15.79:0/2920528539 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffb84019590 con 0x55aefde51240 2024-06-03T02:32:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.852+0000 7ffba47f8b80 10 client.26768 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.852+0000 7ffba47f8b80 10 client.26768 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:32:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.852+0000 7ffba47f8b80 20 client.26768 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.852+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55aefde6ab60 0x55aefde6cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.852+0000 7ffba47f8b80 4 client.26768 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:32:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.852+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2920528539 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aefddb0000 con 0x55aefde6ab60 2024-06-03T02:32:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.852+0000 7ffb92000700 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55aefde6ab60 0x55aefde6cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.853+0000 7ffb92000700 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55aefde6ab60 0x55aefde6cf50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=179ededd9ff81e68 server_cookie=be0d74acd29b1125 in_seq=0 out_seq=0 2024-06-03T02:32:46.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.853+0000 7ffb7d600700 10 client.26768 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:32:46.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.854+0000 7ffb7d600700 1 -- 172.21.15.79:0/2920528539 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aefddb0000 con 0x55aefde6ab60 2024-06-03T02:32:46.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.854+0000 7ffb7d600700 10 client.26768 handle_command_reply: tid=0 2024-06-03T02:32:46.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.854+0000 7ffba47f8b80 1 client.26768 shutdown 2024-06-03T02:32:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.855+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2920528539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde51240 msgr2=0x55aefde5d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.855+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde51240 0x55aefde5d170 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009880 tx=0x7ffb840053a0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.855+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2920528539 shutdown_connections 2024-06-03T02:32:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.855+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55aefde6ab60 0x55aefde6cf50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.855+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 0x55aefde5a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.855+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde51240 0x55aefde5d170 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.855+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2920528539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde56b90 0x55aefde5a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.855+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2920528539 >> 172.21.15.79:0/2920528539 conn(0x55aefdd7d170 msgr2=0x55aefde45790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:46.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.855+0000 7ffb7d600700 0 client.26768 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:32:46.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.856+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2920528539 shutdown_connections 2024-06-03T02:32:46.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.856+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2920528539 wait complete. 2024-06-03T02:32:46.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.856+0000 7ffba47f8b80 20 client.26768 trim_cache size 0 max 16384 2024-06-03T02:32:46.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.886+0000 7ffba47f8b80 1 Processor -- start 2024-06-03T02:32:46.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.886+0000 7ffba47f8b80 1 -- start start 2024-06-03T02:32:46.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.887+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde51240 0x55aefde50d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.887+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde56b90 0x55aefde46130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.887+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 0x55aefde5ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.887+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7400eb50 con 0x55aefde574b0 2024-06-03T02:32:46.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.887+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74010690 con 0x55aefde56b90 2024-06-03T02:32:46.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.887+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7400f220 con 0x55aefde51240 2024-06-03T02:32:46.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.887+0000 7ffb92000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde56b90 0x55aefde46130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.887+0000 7ffb92000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde56b90 0x55aefde46130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:55540/0 (socket says 172.21.15.79:55540) 2024-06-03T02:32:46.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.887+0000 7ffb92000700 1 -- 172.21.15.79:0/102407906 learned_addr learned my addr 172.21.15.79:0/102407906 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:46.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.888+0000 7ffb92a00700 1 --2- 172.21.15.79:0/102407906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde51240 0x55aefde50d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.888+0000 7ffb92000700 1 -- 172.21.15.79:0/102407906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde51240 msgr2=0x55aefde50d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:46.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.888+0000 7ffb92000700 1 --2- 172.21.15.79:0/102407906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde51240 0x55aefde50d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.888+0000 7ffb92000700 1 -- 172.21.15.79:0/102407906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 msgr2=0x55aefde5ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:46.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.888+0000 7ffb92000700 1 --2- 172.21.15.79:0/102407906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 0x55aefde5ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.888+0000 7ffb92000700 1 -- 172.21.15.79:0/102407906 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefde58a00 con 0x55aefde56b90 2024-06-03T02:32:46.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.891+0000 7ffb92000700 1 --2- 172.21.15.79:0/102407906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde56b90 0x55aefde46130 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55aefde4cd20 tx=0x7ffb80013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8615f00f7c31a810 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:46.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.891+0000 7ffb7fe00700 1 -- 172.21.15.79:0/102407906 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8000de40 con 0x55aefde56b90 2024-06-03T02:32:46.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.891+0000 7ffb7fe00700 1 -- 172.21.15.79:0/102407906 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb80014910 con 0x55aefde56b90 2024-06-03T02:32:46.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.891+0000 7ffb7fe00700 1 -- 172.21.15.79:0/102407906 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb800228b0 con 0x55aefde56b90 2024-06-03T02:32:46.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.893+0000 7ffba47f8b80 1 -- 172.21.15.79:0/102407906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde56b90 msgr2=0x55aefde46130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:46.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.893+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/102407906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde56b90 0x55aefde46130 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55aefde4cd20 tx=0x7ffb80013000 comp rx=0 tx=0).stop 2024-06-03T02:32:46.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.893+0000 7ffba47f8b80 1 -- 172.21.15.79:0/102407906 shutdown_connections 2024-06-03T02:32:46.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.893+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/102407906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde574b0 0x55aefde5ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.893+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/102407906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde51240 0x55aefde50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.893+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/102407906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde56b90 0x55aefde46130 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.893+0000 7ffba47f8b80 1 -- 172.21.15.79:0/102407906 >> 172.21.15.79:0/102407906 conn(0x55aefdd7d170 msgr2=0x55aefde58d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:46.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.894+0000 7ffba47f8b80 1 -- 172.21.15.79:0/102407906 shutdown_connections 2024-06-03T02:32:46.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.894+0000 7ffba47f8b80 1 -- 172.21.15.79:0/102407906 wait complete. 2024-06-03T02:32:46.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.894+0000 7ffba47f8b80 1 -- start start 2024-06-03T02:32:46.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.895+0000 7ffba47f8b80 1 Processor -- start 2024-06-03T02:32:46.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.896+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 0x55aefde63200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.897+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde63740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.897+0000 7ffba47f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde63c80 0x55aefde71720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.897+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7400ffc0 con 0x55aefde51240 2024-06-03T02:32:46.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.897+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7400f8f0 con 0x55aefde574b0 2024-06-03T02:32:46.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.897+0000 7ffba47f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7400caf0 con 0x55aefde63c80 2024-06-03T02:32:46.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb93400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde63c80 0x55aefde71720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb92000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde63740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb93400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde63c80 0x55aefde71720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:40538/0 (socket says 172.21.15.79:40538) 2024-06-03T02:32:46.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb93400700 1 -- 172.21.15.79:0/2641349410 learned_addr learned my addr 172.21.15.79:0/2641349410 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:46.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffba47f8b80 10 client.0 _ll_register_callbacks cb 0x55aefdd38380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:46.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb92000700 1 -- 172.21.15.79:0/2641349410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde63c80 msgr2=0x55aefde71720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:46.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb92000700 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde63c80 0x55aefde71720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb92000700 1 -- 172.21.15.79:0/2641349410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 msgr2=0x55aefde63200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:46.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb92000700 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 0x55aefde63200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb92000700 1 -- 172.21.15.79:0/2641349410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefde71e40 con 0x55aefde574b0 2024-06-03T02:32:46.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.898+0000 7ffb92000700 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde63740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55aefde67ac0 tx=0x7ffb80004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4161b790672f0949 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:46.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.899+0000 7ffb7d600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:32:46.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.900+0000 7ffb7d600700 1 -- 172.21.15.79:0/2641349410 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80038060 con 0x55aefde574b0 2024-06-03T02:32:46.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.900+0000 7ffb7d600700 1 -- 172.21.15.79:0/2641349410 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ffb80014cc0 con 0x55aefde574b0 2024-06-03T02:32:46.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.900+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2641349410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb80014910 con 0x55aefde574b0 2024-06-03T02:32:46.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.901+0000 7ffb7d600700 1 -- 172.21.15.79:0/2641349410 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb800224c0 con 0x55aefde574b0 2024-06-03T02:32:46.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.902+0000 7ffba47f8b80 10 client.26774 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:46.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.902+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2641349410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aefded50e0 con 0x55aefde574b0 2024-06-03T02:32:46.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.902+0000 7ffb7d600700 1 -- 172.21.15.79:0/2641349410 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb80014910 con 0x55aefde574b0 2024-06-03T02:32:46.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.903+0000 7ffb7d600700 1 -- 172.21.15.79:0/2641349410 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffb80013070 con 0x55aefde574b0 2024-06-03T02:32:46.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.903+0000 7ffba47f8b80 10 client.26774 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:46.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.903+0000 7ffba47f8b80 10 client.26774 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:32:46.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.903+0000 7ffba47f8b80 20 client.26774 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:46.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.903+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55aefded5620 0x55aefded7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:46.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.903+0000 7ffba47f8b80 4 client.26774 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:32:46.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.903+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2641349410 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aefddaa580 con 0x55aefded5620 2024-06-03T02:32:46.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.904+0000 7ffb92a00700 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55aefded5620 0x55aefded7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:46.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.905+0000 7ffb92a00700 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55aefded5620 0x55aefded7a40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe6c6ff088a485c7 server_cookie=fdf0a5680c382053 in_seq=0 out_seq=0 2024-06-03T02:32:46.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.905+0000 7ffb7d600700 10 client.26774 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:32:46.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.905+0000 7ffb7d600700 1 -- 172.21.15.79:0/2641349410 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aefddaa580 con 0x55aefded5620 2024-06-03T02:32:46.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.905+0000 7ffb7d600700 10 client.26774 handle_command_reply: tid=0 2024-06-03T02:32:46.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.905+0000 7ffba47f8b80 1 client.26774 shutdown 2024-06-03T02:32:46.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.907+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2641349410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 msgr2=0x55aefde63740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:46.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.907+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde63740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55aefde67ac0 tx=0x7ffb80004860 comp rx=0 tx=0).stop 2024-06-03T02:32:46.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.907+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2641349410 shutdown_connections 2024-06-03T02:32:46.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.907+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aefde63c80 0x55aefde71720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.907+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55aefded5620 0x55aefded7a40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.907+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aefde51240 0x55aefde63200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.907+0000 7ffba47f8b80 1 --2- 172.21.15.79:0/2641349410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aefde574b0 0x55aefde63740 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.908+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2641349410 >> 172.21.15.79:0/2641349410 conn(0x55aefdd7d170 msgr2=0x55aefde57890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:46.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.909+0000 7ffb7d600700 0 client.26774 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:32:46.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.910+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2641349410 shutdown_connections 2024-06-03T02:32:46.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.910+0000 7ffba47f8b80 1 -- 172.21.15.79:0/2641349410 wait complete. 2024-06-03T02:32:46.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.910+0000 7ffba47f8b80 20 client.26774 trim_cache size 0 max 16384 2024-06-03T02:32:46.915 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:32:46.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb780a2d20 msgr2=0x7ffb780a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:46.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb780a2d20 0x7ffb780a51e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffb80006fd0 tx=0x7ffb80008040 comp rx=0 tx=0).stop 2024-06-03T02:32:46.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9411ba10 msgr2=0x7ffb941a78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:46.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9411ba10 0x7ffb941a78e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009750 tx=0x7ffb8c004cc0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 shutdown_connections 2024-06-03T02:32:46.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9411ba10 0x7ffb941a78e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ffb780a2d20 0x7ffb780a51e0 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffb80006fd0 tx=0x7ffb80008040 comp rx=0 tx=0).stop 2024-06-03T02:32:46.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb9407b330 0x7ffb941ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 --2- 172.21.15.79:0/2430547687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb94082fb0 0x7ffb941adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:46.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.911+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 >> 172.21.15.79:0/2430547687 conn(0x7ffb94076a50 msgr2=0x7ffb94082090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:46.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.913+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 shutdown_connections 2024-06-03T02:32:46.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:46.913+0000 7ffb99e00700 1 -- 172.21.15.79:0/2430547687 wait complete. 2024-06-03T02:32:48.313 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 760.0 s, 414.8 tps, lat 238.900 ms stddev 258.275 2024-06-03T02:32:48.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:48 smithi123 ceph-mon[36640]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 3.9 MiB/s wr, 397 op/s 2024-06-03T02:32:48.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:48 smithi079 ceph-mon[31819]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 3.9 MiB/s wr, 397 op/s 2024-06-03T02:32:48.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:48 smithi135 ceph-mon[36653]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 3.9 MiB/s wr, 397 op/s 2024-06-03T02:32:50.702 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:50 smithi079 ceph-mon[31819]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.2 MiB/s wr, 445 op/s 2024-06-03T02:32:50.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:50 smithi123 ceph-mon[36640]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.2 MiB/s wr, 445 op/s 2024-06-03T02:32:50.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:32:50.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:50 smithi135 ceph-mon[36653]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.2 MiB/s wr, 445 op/s 2024-06-03T02:32:51.457 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:32:52.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:52 smithi123 ceph-mon[36640]: pgmap v900: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-06-03T02:32:52.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:52 smithi079 ceph-mon[31819]: pgmap v900: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-06-03T02:32:52.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:52 smithi135 ceph-mon[36653]: pgmap v900: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-06-03T02:32:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 765.0 s, 403.4 tps, lat 238.711 ms stddev 294.409 2024-06-03T02:32:54.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:54 smithi123 ceph-mon[36640]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 3.8 MiB/s wr, 391 op/s 2024-06-03T02:32:54.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:54 smithi079 ceph-mon[31819]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 3.8 MiB/s wr, 391 op/s 2024-06-03T02:32:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:54 smithi135 ceph-mon[36653]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 3.8 MiB/s wr, 391 op/s 2024-06-03T02:32:56.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.513+0000 7fcd2fc00700 1 -- 172.21.15.79:0/1215629187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd280746c0 msgr2=0x7fcd28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:56.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.513+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/1215629187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd280746c0 0x7fcd28074aa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009990 tx=0x7fcd1c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.513+0000 7fcd2fc00700 1 -- 172.21.15.79:0/1215629187 shutdown_connections 2024-06-03T02:32:56.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.513+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/1215629187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd28113660 0x7fcd28115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.513+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/1215629187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd280746c0 0x7fcd28074aa0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.513+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/1215629187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd28074fe0 0x7fcd28113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.513+0000 7fcd2fc00700 1 -- 172.21.15.79:0/1215629187 >> 172.21.15.79:0/1215629187 conn(0x7fcd280795f0 msgr2=0x7fcd28079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:56.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.513+0000 7fcd2fc00700 1 -- 172.21.15.79:0/1215629187 shutdown_connections 2024-06-03T02:32:56.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.513+0000 7fcd2fc00700 1 -- 172.21.15.79:0/1215629187 wait complete. 2024-06-03T02:32:56.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.516+0000 7fcd2fc00700 1 Processor -- start 2024-06-03T02:32:56.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.516+0000 7fcd2fc00700 1 -- start start 2024-06-03T02:32:56.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.516+0000 7fcd2fc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280746c0 0x7fcd2819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.516+0000 7fcd2fc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd28074fe0 0x7fcd2819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.516+0000 7fcd2fc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd28113660 0x7fcd281a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.516+0000 7fcd2fc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcd28118e00 con 0x7fcd28074fe0 2024-06-03T02:32:56.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.516+0000 7fcd2fc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd28118c80 con 0x7fcd28113660 2024-06-03T02:32:56.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.516+0000 7fcd2fc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd28118f80 con 0x7fcd280746c0 2024-06-03T02:32:56.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd2d200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280746c0 0x7fcd2819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd28074fe0 0x7fcd2819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd2dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd28113660 0x7fcd281a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd28074fe0 0x7fcd2819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40880/0 (socket says 172.21.15.79:40880) 2024-06-03T02:32:56.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd27e00700 1 -- 172.21.15.79:0/599738690 learned_addr learned my addr 172.21.15.79:0/599738690 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:56.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd2dc00700 1 -- 172.21.15.79:0/599738690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280746c0 msgr2=0x7fcd2819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:56.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd2dc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280746c0 0x7fcd2819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd2dc00700 1 -- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd28074fe0 msgr2=0x7fcd2819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:56.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd2dc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd28074fe0 0x7fcd2819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.517+0000 7fcd2dc00700 1 -- 172.21.15.79:0/599738690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd281a42a0 con 0x7fcd28113660 2024-06-03T02:32:56.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.518+0000 7fcd27e00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd28074fe0 0x7fcd2819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:32:56.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.518+0000 7fcd2d200700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280746c0 0x7fcd2819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:32:56.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.518+0000 7fcd2dc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd28113660 0x7fcd281a3b60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000ef90 tx=0x7fcd2000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:56.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.518+0000 7fcd25600700 1 -- 172.21.15.79:0/599738690 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200092e0 con 0x7fcd28113660 2024-06-03T02:32:56.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.518+0000 7fcd25600700 1 -- 172.21.15.79:0/599738690 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd20004bb0 con 0x7fcd28113660 2024-06-03T02:32:56.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.518+0000 7fcd25600700 1 -- 172.21.15.79:0/599738690 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20004350 con 0x7fcd28113660 2024-06-03T02:32:56.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.518+0000 7fcd2fc00700 1 -- 172.21.15.79:0/599738690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd281a4530 con 0x7fcd28113660 2024-06-03T02:32:56.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.518+0000 7fcd2fc00700 1 -- 172.21.15.79:0/599738690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd281068f0 con 0x7fcd28113660 2024-06-03T02:32:56.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.522+0000 7fcd25600700 1 -- 172.21.15.79:0/599738690 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcd2001b030 con 0x7fcd28113660 2024-06-03T02:32:56.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.524+0000 7fcd3a4cbb80 1 Processor -- start 2024-06-03T02:32:56.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.524+0000 7fcd3a4cbb80 1 -- start start 2024-06-03T02:32:56.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.524+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640992c7e40 0x56409934b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.524+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56409934b5b0 0x56409934ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.524+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564099356300 0x5640993586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.525+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56409925aa10 con 0x564099356300 2024-06-03T02:32:56.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.525+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56409929db00 con 0x56409934b5b0 2024-06-03T02:32:56.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.525+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5640992588f0 con 0x5640992c7e40 2024-06-03T02:32:56.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.525+0000 7fcd25600700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcd100a2ca0 0x7fcd100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.525+0000 7fcd25600700 1 -- 172.21.15.79:0/599738690 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fcd200cee90 con 0x7fcd28113660 2024-06-03T02:32:56.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.526+0000 7fcd2d200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56409934b5b0 0x56409934ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.526+0000 7fcd2dc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcd100a2ca0 0x7fcd100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.527+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640992c7e40 0x56409934b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.527+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640992c7e40 0x56409934b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56508/0 (socket says 172.21.15.79:56508) 2024-06-03T02:32:56.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.527+0000 7fcd27e00700 1 -- 172.21.15.79:0/2845360983 learned_addr learned my addr 172.21.15.79:0/2845360983 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:56.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.527+0000 7fcd27e00700 1 -- 172.21.15.79:0/2845360983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56409934b5b0 msgr2=0x56409934ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:56.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.527+0000 7fcd27e00700 1 --2- 172.21.15.79:0/2845360983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56409934b5b0 0x56409934ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.527+0000 7fcd27e00700 1 -- 172.21.15.79:0/2845360983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564099356300 msgr2=0x5640993586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:56.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.527+0000 7fcd27e00700 1 --2- 172.21.15.79:0/2845360983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564099356300 0x5640993586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.527+0000 7fcd27e00700 1 -- 172.21.15.79:0/2845360983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564099245f90 con 0x5640992c7e40 2024-06-03T02:32:56.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.528+0000 7fcd27e00700 1 --2- 172.21.15.79:0/2845360983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640992c7e40 0x56409934b070 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009ea0 tx=0x7fcd14012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=977bc79fa0f0b988 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:56.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.529+0000 7fcd2dc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcd100a2ca0 0x7fcd100a5160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcd200090a0 tx=0x7fcd20009480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:56.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.529+0000 7fcd1a000700 1 -- 172.21.15.79:0/2845360983 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1401b440 con 0x5640992c7e40 2024-06-03T02:32:56.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.529+0000 7fcd1a000700 1 -- 172.21.15.79:0/2845360983 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd1400f910 con 0x5640992c7e40 2024-06-03T02:32:56.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.529+0000 7fcd1a000700 1 -- 172.21.15.79:0/2845360983 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1401aa40 con 0x5640992c7e40 2024-06-03T02:32:56.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.530+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2845360983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640992c7e40 msgr2=0x56409934b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:56.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.530+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2845360983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640992c7e40 0x56409934b070 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009ea0 tx=0x7fcd14012730 comp rx=0 tx=0).stop 2024-06-03T02:32:56.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.531+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2845360983 shutdown_connections 2024-06-03T02:32:56.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.531+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2845360983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564099356300 0x5640993586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.531+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2845360983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640992c7e40 0x56409934b070 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.531+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2845360983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56409934b5b0 0x56409934ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.531+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2845360983 >> 172.21.15.79:0/2845360983 conn(0x564099278170 msgr2=0x5640993474b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.531+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2845360983 shutdown_connections 2024-06-03T02:32:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.531+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2845360983 wait complete. 2024-06-03T02:32:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.532+0000 7fcd3a4cbb80 1 -- start start 2024-06-03T02:32:56.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.532+0000 7fcd3a4cbb80 1 Processor -- start 2024-06-03T02:32:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.535+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099355af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.535+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x564099352df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.535+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x564099353330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.535+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x564099239f60 con 0x56409934b5b0 2024-06-03T02:32:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.535+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564099299ac0 con 0x5640992c7e40 2024-06-03T02:32:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.535+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564099358fc0 con 0x564099356300 2024-06-03T02:32:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.535+0000 7fcd3a4cbb80 10 client.0 _ll_register_callbacks cb 0x564099237720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:56.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.537+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x564099353330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.537+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x564099353330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56516/0 (socket says 172.21.15.79:56516) 2024-06-03T02:32:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.537+0000 7fcd27e00700 1 -- 172.21.15.79:0/2263609890 learned_addr learned my addr 172.21.15.79:0/2263609890 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.537+0000 7fcd2d200700 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x564099352df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.537+0000 7fcd27e00700 1 -- 172.21.15.79:0/2263609890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 msgr2=0x564099355af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:56.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.537+0000 7fcd27e00700 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099355af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.537+0000 7fcd27e00700 1 -- 172.21.15.79:0/2263609890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 msgr2=0x564099352df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:56.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.538+0000 7fcd27e00700 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x564099352df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.538+0000 7fcd27e00700 1 -- 172.21.15.79:0/2263609890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640991ec8d0 con 0x564099356300 2024-06-03T02:32:56.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.538+0000 7fcd2d200700 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x564099352df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:32:56.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.540+0000 7fcd27e00700 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x564099353330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcd14012f80 tx=0x7fcd14005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365cc74eb0cfc4e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:56.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.540+0000 7fcd0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:32:56.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.541+0000 7fcd0ea00700 1 -- 172.21.15.79:0/2263609890 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1401a6c0 con 0x564099356300 2024-06-03T02:32:56.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.541+0000 7fcd0ea00700 1 -- 172.21.15.79:0/2263609890 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fcd1401a860 con 0x564099356300 2024-06-03T02:32:56.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.541+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2263609890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd1400f910 con 0x564099356300 2024-06-03T02:32:56.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.541+0000 7fcd0ea00700 1 -- 172.21.15.79:0/2263609890 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd1401aa60 con 0x564099356300 2024-06-03T02:32:56.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.541+0000 7fcd3a4cbb80 10 client.26767 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:56.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.543+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2263609890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564099363770 con 0x564099356300 2024-06-03T02:32:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.543+0000 7fcd0ea00700 1 -- 172.21.15.79:0/2263609890 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd1400f910 con 0x564099356300 2024-06-03T02:32:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.543+0000 7fcd0ea00700 1 -- 172.21.15.79:0/2263609890 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcd14019070 con 0x564099356300 2024-06-03T02:32:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.544+0000 7fcd3a4cbb80 10 client.26767 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.544+0000 7fcd3a4cbb80 10 client.26767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:32:56.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.544+0000 7fcd3a4cbb80 20 client.26767 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.544+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x564099365cf0 0x5640993680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.544+0000 7fcd3a4cbb80 4 client.26767 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:32:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.544+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2263609890 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640992ab000 con 0x564099365cf0 2024-06-03T02:32:56.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.544+0000 7fcd2d200700 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x564099365cf0 0x5640993680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.548+0000 7fcd0ea00700 10 client.26767 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:32:56.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.548+0000 7fcd2d200700 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x564099365cf0 0x5640993680e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1db662056ae18638 server_cookie=cda32e5571e488a3 in_seq=0 out_seq=0 2024-06-03T02:32:56.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.549+0000 7fcd0ea00700 1 -- 172.21.15.79:0/2263609890 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5640992ab000 con 0x564099365cf0 2024-06-03T02:32:56.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.549+0000 7fcd0ea00700 10 client.26767 handle_command_reply: tid=0 2024-06-03T02:32:56.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.549+0000 7fcd3a4cbb80 1 client.26767 shutdown 2024-06-03T02:32:56.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.549+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2263609890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 msgr2=0x564099353330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:56.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.549+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x564099353330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcd14012f80 tx=0x7fcd14005760 comp rx=0 tx=0).stop 2024-06-03T02:32:56.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.550+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2263609890 shutdown_connections 2024-06-03T02:32:56.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.550+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x564099353330 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.550+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x564099365cf0 0x5640993680e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.550+0000 7fcd0ea00700 0 client.26767 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:32:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.550+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099355af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.550+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2263609890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x564099352df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.550+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2263609890 >> 172.21.15.79:0/2263609890 conn(0x564099278170 msgr2=0x564099346740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:56.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.554+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2263609890 shutdown_connections 2024-06-03T02:32:56.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.554+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2263609890 wait complete. 2024-06-03T02:32:56.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.554+0000 7fcd3a4cbb80 20 client.26767 trim_cache size 0 max 16384 2024-06-03T02:32:56.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.597+0000 7fcd3a4cbb80 1 Processor -- start 2024-06-03T02:32:56.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.601+0000 7fcd3a4cbb80 1 -- start start 2024-06-03T02:32:56.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.601+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099355630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.601+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x5640993576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.601+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x5640993516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.601+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281bd3d0 con 0x56409934b5b0 2024-06-03T02:32:56.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.601+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281bef10 con 0x5640992c7e40 2024-06-03T02:32:56.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.601+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281bdaa0 con 0x564099356300 2024-06-03T02:32:56.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.602+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099355630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.602+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099355630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43428/0 (socket says 172.21.15.79:43428) 2024-06-03T02:32:56.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.602+0000 7fcd27e00700 1 -- 172.21.15.79:0/2127077746 learned_addr learned my addr 172.21.15.79:0/2127077746 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:56.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.602+0000 7fcd27e00700 1 --2- 172.21.15.79:0/2127077746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x5640993516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.603+0000 7fcd2d200700 1 --2- 172.21.15.79:0/2127077746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x5640993576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.603+0000 7fcd2d200700 1 -- 172.21.15.79:0/2127077746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 msgr2=0x5640993516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:56.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.603+0000 7fcd2d200700 1 --2- 172.21.15.79:0/2127077746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x5640993516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.603+0000 7fcd2d200700 1 -- 172.21.15.79:0/2127077746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 msgr2=0x564099355630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:56.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.603+0000 7fcd2d200700 1 --2- 172.21.15.79:0/2127077746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099355630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.603+0000 7fcd2d200700 1 -- 172.21.15.79:0/2127077746 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564099348210 con 0x56409934b5b0 2024-06-03T02:32:56.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.605+0000 7fcd2d200700 1 --2- 172.21.15.79:0/2127077746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x5640993576f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c006010 tx=0x7fcd1c00b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809b2d72951f0a0f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:56.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.605+0000 7fcd19600700 1 -- 172.21.15.79:0/2127077746 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0050b0 con 0x56409934b5b0 2024-06-03T02:32:56.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.606+0000 7fcd19600700 1 -- 172.21.15.79:0/2127077746 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd1c003680 con 0x56409934b5b0 2024-06-03T02:32:56.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.606+0000 7fcd19600700 1 -- 172.21.15.79:0/2127077746 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c00f6c0 con 0x56409934b5b0 2024-06-03T02:32:56.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.607+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2127077746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 msgr2=0x5640993576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:56.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.607+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2127077746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x5640993576f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c006010 tx=0x7fcd1c00b280 comp rx=0 tx=0).stop 2024-06-03T02:32:56.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.607+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2127077746 shutdown_connections 2024-06-03T02:32:56.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.607+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2127077746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x5640993516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.607+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2127077746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099355630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.607+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/2127077746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56409934b5b0 0x5640993576f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.607+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2127077746 >> 172.21.15.79:0/2127077746 conn(0x564099278170 msgr2=0x564099352db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:56.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.607+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2127077746 shutdown_connections 2024-06-03T02:32:56.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.607+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/2127077746 wait complete. 2024-06-03T02:32:56.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.608+0000 7fcd3a4cbb80 1 -- start start 2024-06-03T02:32:56.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.608+0000 7fcd3a4cbb80 1 Processor -- start 2024-06-03T02:32:56.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.609+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099356cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.609+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x564099357200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.609+0000 7fcd3a4cbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564099365cf0 0x564099357740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.609+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281be840 con 0x564099365cf0 2024-06-03T02:32:56.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.609+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281be170 con 0x5640992c7e40 2024-06-03T02:32:56.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.609+0000 7fcd3a4cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2810d610 con 0x564099356300 2024-06-03T02:32:56.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.609+0000 7fcd3a4cbb80 10 client.0 _ll_register_callbacks cb 0x564099237720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:32:56.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.610+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099356cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.610+0000 7fcd27e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099356cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43434/0 (socket says 172.21.15.79:43434) 2024-06-03T02:32:56.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.610+0000 7fcd27e00700 1 -- 172.21.15.79:0/46687460 learned_addr learned my addr 172.21.15.79:0/46687460 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:32:56.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.610+0000 7fcd27e00700 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564099365cf0 0x564099357740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.610+0000 7fcd27e00700 1 -- 172.21.15.79:0/46687460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 msgr2=0x564099357200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:56.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.610+0000 7fcd27e00700 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x564099357200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.611+0000 7fcd27e00700 1 -- 172.21.15.79:0/46687460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564099365cf0 msgr2=0x564099357740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:32:56.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.611+0000 7fcd27e00700 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564099365cf0 0x564099357740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.611+0000 7fcd27e00700 1 -- 172.21.15.79:0/46687460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56409935d280 con 0x5640992c7e40 2024-06-03T02:32:56.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.613+0000 7fcd27e00700 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099356cc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcd14012770 tx=0x7fcd14000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928187d1a1f3d8e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:32:56.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.613+0000 7fcd0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:32:56.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.615+0000 7fcd0ea00700 1 -- 172.21.15.79:0/46687460 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1401aa60 con 0x5640992c7e40 2024-06-03T02:32:56.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.616+0000 7fcd0ea00700 1 -- 172.21.15.79:0/46687460 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fcd1400f6c0 con 0x5640992c7e40 2024-06-03T02:32:56.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.616+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/46687460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd1c003680 con 0x5640992c7e40 2024-06-03T02:32:56.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.616+0000 7fcd0ea00700 1 -- 172.21.15.79:0/46687460 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd1400d540 con 0x5640992c7e40 2024-06-03T02:32:56.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.616+0000 7fcd3a4cbb80 10 client.26789 fetch_fsmap learned FSMap version 30 2024-06-03T02:32:56.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.616+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/46687460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56409935e580 con 0x5640992c7e40 2024-06-03T02:32:56.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.617+0000 7fcd0ea00700 1 -- 172.21.15.79:0/46687460 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd1c003680 con 0x5640992c7e40 2024-06-03T02:32:56.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.617+0000 7fcd0ea00700 1 -- 172.21.15.79:0/46687460 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcd14019070 con 0x5640992c7e40 2024-06-03T02:32:56.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.617+0000 7fcd3a4cbb80 10 client.26789 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:32:56.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.617+0000 7fcd3a4cbb80 10 client.26789 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:32:56.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.617+0000 7fcd3a4cbb80 20 client.26789 populate_metadata read hostname 'smithi079' 2024-06-03T02:32:56.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.617+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56409935ebc0 0x5640993d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:32:56.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.617+0000 7fcd3a4cbb80 4 client.26789 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:32:56.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.617+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/46687460 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640992a5580 con 0x56409935ebc0 2024-06-03T02:32:56.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.618+0000 7fcd2d200700 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56409935ebc0 0x5640993d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:32:56.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.618+0000 7fcd0ea00700 10 client.26789 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:32:56.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.618+0000 7fcd2d200700 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56409935ebc0 0x5640993d2800 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e69bbe9ddb9370b5 server_cookie=608a10748e560a72 in_seq=0 out_seq=0 2024-06-03T02:32:56.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd0ea00700 1 -- 172.21.15.79:0/46687460 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640992a5580 con 0x56409935ebc0 2024-06-03T02:32:56.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd0ea00700 10 client.26789 handle_command_reply: tid=0 2024-06-03T02:32:56.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd3a4cbb80 1 client.26789 shutdown 2024-06-03T02:32:56.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/46687460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 msgr2=0x564099356cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:56.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099356cc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcd14012770 tx=0x7fcd14000bc0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/46687460 shutdown_connections 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564099365cf0 0x564099357740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x56409935ebc0 0x5640993d2800 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5640992c7e40 0x564099356cc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd3a4cbb80 1 --2- 172.21.15.79:0/46687460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564099356300 0x564099357200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.619+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/46687460 >> 172.21.15.79:0/46687460 conn(0x564099278170 msgr2=0x56409934cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.620+0000 7fcd0ea00700 0 client.26789 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.620+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/46687460 shutdown_connections 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.620+0000 7fcd3a4cbb80 1 -- 172.21.15.79:0/46687460 wait complete. 2024-06-03T02:32:56.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.620+0000 7fcd3a4cbb80 20 client.26789 trim_cache size 0 max 16384 2024-06-03T02:32:56.622 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:32:56.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 -- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcd100a2ca0 msgr2=0x7fcd100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:56.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcd100a2ca0 0x7fcd100a5160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcd200090a0 tx=0x7fcd20009480 comp rx=0 tx=0).stop 2024-06-03T02:32:56.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 -- 172.21.15.79:0/599738690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd28113660 msgr2=0x7fcd281a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:32:56.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd28113660 0x7fcd281a3b60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000ef90 tx=0x7fcd2000c3a0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 -- 172.21.15.79:0/599738690 shutdown_connections 2024-06-03T02:32:56.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd28113660 0x7fcd281a3b60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280746c0 0x7fcd2819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fcd100a2ca0 0x7fcd100a5160 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 --2- 172.21.15.79:0/599738690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd28074fe0 0x7fcd2819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:32:56.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 -- 172.21.15.79:0/599738690 >> 172.21.15.79:0/599738690 conn(0x7fcd280795f0 msgr2=0x7fcd28112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:32:56.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 -- 172.21.15.79:0/599738690 shutdown_connections 2024-06-03T02:32:56.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:32:56.622+0000 7fcd2fc00700 1 -- 172.21.15.79:0/599738690 wait complete. 2024-06-03T02:32:56.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:56 smithi123 ceph-mon[36640]: pgmap v902: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.5 MiB/s wr, 466 op/s 2024-06-03T02:32:56.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:56 smithi079 ceph-mon[31819]: pgmap v902: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.5 MiB/s wr, 466 op/s 2024-06-03T02:32:56.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:56 smithi135 ceph-mon[36653]: pgmap v902: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.5 MiB/s wr, 466 op/s 2024-06-03T02:32:58.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:32:58.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 770.0 s, 592.8 tps, lat 176.228 ms stddev 259.953 2024-06-03T02:32:58.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:58 smithi079 ceph-mon[31819]: pgmap v903: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 3.5 MiB/s wr, 364 op/s 2024-06-03T02:32:58.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:32:58 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:32:58.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:58 smithi135 ceph-mon[36653]: pgmap v903: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 3.5 MiB/s wr, 364 op/s 2024-06-03T02:32:58.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:32:58 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:32:58.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:58 smithi123 ceph-mon[36640]: pgmap v903: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 3.5 MiB/s wr, 364 op/s 2024-06-03T02:32:58.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:32:58 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:32:59.159 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:33:00.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:00 smithi079 ceph-mon[31819]: pgmap v904: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.2 MiB/s wr, 424 op/s 2024-06-03T02:33:00.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:00 smithi135 ceph-mon[36653]: pgmap v904: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.2 MiB/s wr, 424 op/s 2024-06-03T02:33:00.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:00 smithi123 ceph-mon[36640]: pgmap v904: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.2 MiB/s wr, 424 op/s 2024-06-03T02:33:01.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:01 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi079 to 176.4M 2024-06-03T02:33:01.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:01 smithi079 ceph-mon[31819]: Unable to set osd_memory_target on smithi079 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-06-03T02:33:01.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:01 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi079 to 176.4M 2024-06-03T02:33:01.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:01 smithi135 ceph-mon[36653]: Unable to set osd_memory_target on smithi079 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-06-03T02:33:01.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:33:01.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:01 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi079 to 176.4M 2024-06-03T02:33:01.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:01 smithi123 ceph-mon[36640]: Unable to set osd_memory_target on smithi079 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-06-03T02:33:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:02 smithi079 ceph-mon[31819]: pgmap v905: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.0 MiB/s wr, 487 op/s 2024-06-03T02:33:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:33:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:33:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:33:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:02 smithi135 ceph-mon[36653]: pgmap v905: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.0 MiB/s wr, 487 op/s 2024-06-03T02:33:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:33:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:33:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:33:02.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:02 smithi123 ceph-mon[36640]: pgmap v905: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.0 MiB/s wr, 487 op/s 2024-06-03T02:33:02.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:33:02.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:33:02.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:33:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 775.0 s, 371.0 tps, lat 262.739 ms stddev 374.536 2024-06-03T02:33:03.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.509+0000 7f1e0de00700 1 -- 172.21.15.79:0/383978702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08107840 msgr2=0x7f1e0810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:03.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.509+0000 7f1e0de00700 1 --2- 172.21.15.79:0/383978702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08107840 0x7f1e0810f3c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1df00099c0 tx=0x7f1df0009cd0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.509+0000 7f1e0de00700 1 -- 172.21.15.79:0/383978702 shutdown_connections 2024-06-03T02:33:03.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.509+0000 7f1e0de00700 1 --2- 172.21.15.79:0/383978702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08107840 0x7f1e0810f3c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.509+0000 7f1e0de00700 1 --2- 172.21.15.79:0/383978702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e08102740 0x7f1e08102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.509+0000 7f1e0de00700 1 --2- 172.21.15.79:0/383978702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e081030f0 0x7f1e08107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.509+0000 7f1e0de00700 1 -- 172.21.15.79:0/383978702 >> 172.21.15.79:0/383978702 conn(0x7f1e080795f0 msgr2=0x7f1e08079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:03.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.509+0000 7f1e0de00700 1 -- 172.21.15.79:0/383978702 shutdown_connections 2024-06-03T02:33:03.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.509+0000 7f1e0de00700 1 -- 172.21.15.79:0/383978702 wait complete. 2024-06-03T02:33:03.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.511+0000 7f1e0de00700 1 Processor -- start 2024-06-03T02:33:03.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.511+0000 7f1e0de00700 1 -- start start 2024-06-03T02:33:03.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.512+0000 7f1e0de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08102740 0x7f1e08076380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.512+0000 7f1e0de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e081030f0 0x7f1e08074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.512+0000 7f1e0de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e08107840 0x7f1e08074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.512+0000 7f1e0de00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08112850 con 0x7f1e081030f0 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.512+0000 7f1e0de00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081126d0 con 0x7f1e08102740 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.512+0000 7f1e0de00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081129d0 con 0x7f1e08107840 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.512+0000 7f1e06a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08102740 0x7f1e08076380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.513+0000 7f1e06a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08102740 0x7f1e08076380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43444/0 (socket says 172.21.15.79:43444) 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.513+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e081030f0 0x7f1e08074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.513+0000 7f1e06a00700 1 -- 172.21.15.79:0/3491409822 learned_addr learned my addr 172.21.15.79:0/3491409822 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.513+0000 7f1e06a00700 1 -- 172.21.15.79:0/3491409822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e08107840 msgr2=0x7f1e08074df0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.513+0000 7f1e06a00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e08107840 0x7f1e08074df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.513+0000 7f1e06a00700 1 -- 172.21.15.79:0/3491409822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e081030f0 msgr2=0x7f1e08074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.513+0000 7f1e06a00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e081030f0 0x7f1e08074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.513+0000 7f1e06a00700 1 -- 172.21.15.79:0/3491409822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e081ab650 con 0x7f1e08102740 2024-06-03T02:33:03.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.520+0000 7f1e06a00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08102740 0x7f1e08076380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000ba50 tx=0x7f1e0000be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:03.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.520+0000 7f1e0ca00700 1 -- 172.21.15.79:0/3491409822 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00019070 con 0x7f1e08102740 2024-06-03T02:33:03.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.520+0000 7f1e0de00700 1 -- 172.21.15.79:0/3491409822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e081ab8e0 con 0x7f1e08102740 2024-06-03T02:33:03.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.520+0000 7f1e0de00700 1 -- 172.21.15.79:0/3491409822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e081abe20 con 0x7f1e08102740 2024-06-03T02:33:03.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.521+0000 7f1e0ca00700 1 -- 172.21.15.79:0/3491409822 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e00014440 con 0x7f1e08102740 2024-06-03T02:33:03.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.521+0000 7f1e0ca00700 1 -- 172.21.15.79:0/3491409822 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000075d0 con 0x7f1e08102740 2024-06-03T02:33:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.534+0000 7f1e0ca00700 1 -- 172.21.15.79:0/3491409822 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1e0001c070 con 0x7f1e08102740 2024-06-03T02:33:03.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.536+0000 7f1e0ca00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1dec0a2d20 0x7f1dec0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.536+0000 7f1e0ca00700 1 -- 172.21.15.79:0/3491409822 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1e000ce770 con 0x7f1e08102740 2024-06-03T02:33:03.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.537+0000 7f1dfe000700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1dec0a2d20 0x7f1dec0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.538+0000 7f1e18761b80 1 Processor -- start 2024-06-03T02:33:03.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.538+0000 7f1e18761b80 1 -- start start 2024-06-03T02:33:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.539+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.539+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9edd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.539+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.539+0000 7f1e06a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.540+0000 7f1e18761b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee8fca10 con 0x55b9ee9f44b0 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.540+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.540+0000 7f1e06a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56554/0 (socket says 172.21.15.79:56554) 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.540+0000 7f1e07400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.540+0000 7f1e07400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9ee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43460/0 (socket says 172.21.15.79:43460) 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.540+0000 7f1e07400700 1 -- 172.21.15.79:0/547869771 learned_addr learned my addr 172.21.15.79:0/547869771 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.540+0000 7f1e18761b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee93fb00 con 0x55b9ee9ee240 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.540+0000 7f1e18761b80 1 -- 172.21.15.79:0/547869771 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee8fa8f0 con 0x55b9ee9f3b90 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.541+0000 7f1e06a00700 1 -- 172.21.15.79:0/547869771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 msgr2=0x55b9ee9ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.541+0000 7f1e06a00700 1 --2- 172.21.15.79:0/547869771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.541+0000 7f1e06a00700 1 -- 172.21.15.79:0/547869771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 msgr2=0x55b9ee9edd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.541+0000 7f1e06a00700 1 --2- 172.21.15.79:0/547869771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9edd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.541+0000 7f1e06a00700 1 -- 172.21.15.79:0/547869771 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ee8e7f90 con 0x55b9ee9f3b90 2024-06-03T02:33:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.542+0000 7f1e07400700 1 --2- 172.21.15.79:0/547869771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:33:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.542+0000 7f1dfe000700 1 --2- 172.21.15.79:0/547869771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9edd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:33:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.542+0000 7f1e06a00700 1 --2- 172.21.15.79:0/547869771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f3f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e0009a8c0 tx=0x7f1e0009b340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0b4ba0c8f74ef8f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.542+0000 7f1dfcc00700 1 -- 172.21.15.79:0/547869771 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0009ef70 con 0x55b9ee9f3b90 2024-06-03T02:33:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.542+0000 7f1dfcc00700 1 -- 172.21.15.79:0/547869771 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e000a3830 con 0x55b9ee9f3b90 2024-06-03T02:33:03.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.542+0000 7f1dfcc00700 1 -- 172.21.15.79:0/547869771 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0009a270 con 0x55b9ee9f3b90 2024-06-03T02:33:03.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.543+0000 7f1e18761b80 1 -- 172.21.15.79:0/547869771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 msgr2=0x55b9ee9f3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:03.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.543+0000 7f1e18761b80 1 --2- 172.21.15.79:0/547869771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f3f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e0009a8c0 tx=0x7f1e0009b340 comp rx=0 tx=0).stop 2024-06-03T02:33:03.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.543+0000 7f1dfe000700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1dec0a2d20 0x7f1dec0a51e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1e08075f00 tx=0x7f1df800a710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:03.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.544+0000 7f1e18761b80 1 -- 172.21.15.79:0/547869771 shutdown_connections 2024-06-03T02:33:03.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.544+0000 7f1e18761b80 1 --2- 172.21.15.79:0/547869771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.544+0000 7f1e18761b80 1 --2- 172.21.15.79:0/547869771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f3f70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.544+0000 7f1e18761b80 1 --2- 172.21.15.79:0/547869771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9edd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.544+0000 7f1e18761b80 1 -- 172.21.15.79:0/547869771 >> 172.21.15.79:0/547869771 conn(0x55b9ee91a170 msgr2=0x55b9ee9e8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:03.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.544+0000 7f1e18761b80 1 -- 172.21.15.79:0/547869771 shutdown_connections 2024-06-03T02:33:03.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.547+0000 7f1e18761b80 1 -- 172.21.15.79:0/547869771 wait complete. 2024-06-03T02:33:03.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.547+0000 7f1e18761b80 1 -- start start 2024-06-03T02:33:03.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.548+0000 7f1e18761b80 1 Processor -- start 2024-06-03T02:33:03.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.549+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9ee9fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.549+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.549+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.549+0000 7f1e18761b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee8dbf60 con 0x55b9ee9f3b90 2024-06-03T02:33:03.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.549+0000 7f1e18761b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee93bac0 con 0x55b9ee9f44b0 2024-06-03T02:33:03.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.549+0000 7f1e18761b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee9fae90 con 0x55b9ee9ee240 2024-06-03T02:33:03.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.549+0000 7f1e18761b80 10 client.0 _ll_register_callbacks cb 0x55b9ee8d5380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:03.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.550+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.550+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40940/0 (socket says 172.21.15.79:40940) 2024-06-03T02:33:03.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.550+0000 7f1dfe000700 1 -- 172.21.15.79:0/3777436776 learned_addr learned my addr 172.21.15.79:0/3777436776 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:03.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.551+0000 7f1e06a00700 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9ee9fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.552+0000 7f1e06a00700 1 -- 172.21.15.79:0/3777436776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9f44b0 msgr2=0x55b9ee9f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:03.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.552+0000 7f1e06a00700 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.552+0000 7f1e06a00700 1 -- 172.21.15.79:0/3777436776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f3b90 msgr2=0x55b9ee9f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:03.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.552+0000 7f1e06a00700 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.552+0000 7f1e06a00700 1 -- 172.21.15.79:0/3777436776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9ee88e8d0 con 0x55b9ee9ee240 2024-06-03T02:33:03.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.552+0000 7f1e06a00700 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9ee9fa180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000bfd0 tx=0x7f1e000a3730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1934efe16038259 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:03.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.552+0000 7f1df5600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:03.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.552+0000 7f1df5600700 1 -- 172.21.15.79:0/3777436776 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000b7070 con 0x55b9ee9ee240 2024-06-03T02:33:03.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.553+0000 7f1df5600700 1 -- 172.21.15.79:0/3777436776 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1e000b94c0 con 0x55b9ee9ee240 2024-06-03T02:33:03.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.554+0000 7f1e18761b80 1 -- 172.21.15.79:0/3777436776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e000a3830 con 0x55b9ee9ee240 2024-06-03T02:33:03.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.554+0000 7f1df5600700 1 -- 172.21.15.79:0/3777436776 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e0009a010 con 0x55b9ee9ee240 2024-06-03T02:33:03.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.554+0000 7f1e18761b80 10 client.26773 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:03.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.554+0000 7f1e18761b80 1 -- 172.21.15.79:0/3777436776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b9eea165a0 con 0x55b9ee9ee240 2024-06-03T02:33:03.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.555+0000 7f1df5600700 1 -- 172.21.15.79:0/3777436776 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e000a3830 con 0x55b9ee9ee240 2024-06-03T02:33:03.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.555+0000 7f1df5600700 1 -- 172.21.15.79:0/3777436776 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1e0000cab0 con 0x55b9ee9ee240 2024-06-03T02:33:03.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.556+0000 7f1e18761b80 10 client.26773 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:03.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.556+0000 7f1e18761b80 10 client.26773 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:33:03.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.556+0000 7f1e18761b80 20 client.26773 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:03.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.556+0000 7f1e18761b80 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b9eea18a70 0x55b9eea1ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.556+0000 7f1e18761b80 4 client.26773 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:33:03.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.556+0000 7f1e18761b80 1 -- 172.21.15.79:0/3777436776 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9ee94d000 con 0x55b9eea18a70 2024-06-03T02:33:03.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.558+0000 7f1dfe000700 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b9eea18a70 0x55b9eea1ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.559+0000 7f1dfe000700 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b9eea18a70 0x55b9eea1ae60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae7c32407da04850 server_cookie=7664b251405d01bf in_seq=0 out_seq=0 2024-06-03T02:33:03.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.559+0000 7f1df5600700 10 client.26773 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:33:03.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.561+0000 7f1df5600700 1 -- 172.21.15.79:0/3777436776 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b9ee94d000 con 0x55b9eea18a70 2024-06-03T02:33:03.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.561+0000 7f1df5600700 10 client.26773 handle_command_reply: tid=0 2024-06-03T02:33:03.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.561+0000 7f1e18761b80 1 client.26773 shutdown 2024-06-03T02:33:03.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.562+0000 7f1e18761b80 1 -- 172.21.15.79:0/3777436776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 msgr2=0x55b9ee9fa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:03.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.562+0000 7f1e18761b80 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9ee9fa180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000bfd0 tx=0x7f1e000a3730 comp rx=0 tx=0).stop 2024-06-03T02:33:03.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.562+0000 7f1e18761b80 1 -- 172.21.15.79:0/3777436776 shutdown_connections 2024-06-03T02:33:03.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.562+0000 7f1e18761b80 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.562+0000 7f1e18761b80 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b9eea18a70 0x55b9eea1ae60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.562+0000 7f1e18761b80 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9ee9fa180 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.562+0000 7f1e18761b80 1 --2- 172.21.15.79:0/3777436776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.562+0000 7f1e18761b80 1 -- 172.21.15.79:0/3777436776 >> 172.21.15.79:0/3777436776 conn(0x55b9ee91a170 msgr2=0x55b9ee9e2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:03.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.563+0000 7f1df5600700 0 client.26773 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:33:03.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.563+0000 7f1e18761b80 1 -- 172.21.15.79:0/3777436776 shutdown_connections 2024-06-03T02:33:03.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.563+0000 7f1e18761b80 1 -- 172.21.15.79:0/3777436776 wait complete. 2024-06-03T02:33:03.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.563+0000 7f1e18761b80 20 client.26773 trim_cache size 0 max 16384 2024-06-03T02:33:03.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.600+0000 7f1e18761b80 1 Processor -- start 2024-06-03T02:33:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.601+0000 7f1e18761b80 1 -- start start 2024-06-03T02:33:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9edd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9e3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e18761b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1de400eb90 con 0x55b9ee9f44b0 2024-06-03T02:33:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e18761b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1de40106d0 con 0x55b9ee9ee240 2024-06-03T02:33:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e18761b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1de400f260 con 0x55b9ee9f3b90 2024-06-03T02:33:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e06a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e06a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9edd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:43476/0 (socket says 172.21.15.79:43476) 2024-06-03T02:33:03.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e06a00700 1 -- 172.21.15.79:0/108124458 learned_addr learned my addr 172.21.15.79:0/108124458 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:03.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9e91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56576/0 (socket says 172.21.15.79:56576) 2024-06-03T02:33:03.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e06a00700 1 -- 172.21.15.79:0/108124458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 msgr2=0x55b9ee9e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.602+0000 7f1e06a00700 1 --2- 172.21.15.79:0/108124458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.603+0000 7f1e06a00700 1 -- 172.21.15.79:0/108124458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 msgr2=0x55b9ee9e3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.603+0000 7f1e06a00700 1 --2- 172.21.15.79:0/108124458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.603+0000 7f1e06a00700 1 -- 172.21.15.79:0/108124458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ee9fa980 con 0x55b9ee9ee240 2024-06-03T02:33:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.604+0000 7f1e06a00700 1 --2- 172.21.15.79:0/108124458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9edd00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e00098e70 tx=0x7f1e0009b340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4698d3449c7c6fd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.604+0000 7f1df7e00700 1 -- 172.21.15.79:0/108124458 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000b7070 con 0x55b9ee9ee240 2024-06-03T02:33:03.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.604+0000 7f1df7e00700 1 -- 172.21.15.79:0/108124458 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e000b1070 con 0x55b9ee9ee240 2024-06-03T02:33:03.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.604+0000 7f1df7e00700 1 -- 172.21.15.79:0/108124458 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0009a1b0 con 0x55b9ee9ee240 2024-06-03T02:33:03.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.605+0000 7f1e18761b80 1 -- 172.21.15.79:0/108124458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 msgr2=0x55b9ee9edd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:03.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.605+0000 7f1e18761b80 1 --2- 172.21.15.79:0/108124458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9edd00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e00098e70 tx=0x7f1e0009b340 comp rx=0 tx=0).stop 2024-06-03T02:33:03.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.605+0000 7f1e18761b80 1 -- 172.21.15.79:0/108124458 shutdown_connections 2024-06-03T02:33:03.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.605+0000 7f1e18761b80 1 --2- 172.21.15.79:0/108124458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9ee9e3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.605+0000 7f1e18761b80 1 --2- 172.21.15.79:0/108124458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9ee240 0x55b9ee9edd00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.605+0000 7f1e18761b80 1 --2- 172.21.15.79:0/108124458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9f3b90 0x55b9ee9e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.605+0000 7f1e18761b80 1 -- 172.21.15.79:0/108124458 >> 172.21.15.79:0/108124458 conn(0x55b9ee91a170 msgr2=0x55b9ee9e8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:03.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.606+0000 7f1e18761b80 1 -- 172.21.15.79:0/108124458 shutdown_connections 2024-06-03T02:33:03.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.606+0000 7f1e18761b80 1 -- 172.21.15.79:0/108124458 wait complete. 2024-06-03T02:33:03.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.606+0000 7f1e18761b80 1 -- start start 2024-06-03T02:33:03.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.607+0000 7f1e18761b80 1 Processor -- start 2024-06-03T02:33:03.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.610+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9eea00180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.610+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9f3b90 0x55b9eea006c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.610+0000 7f1e18761b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9eea1f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9f3b90 0x55b9eea006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e18761b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1de4010000 con 0x55b9ee9f44b0 2024-06-03T02:33:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e18761b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1de400f930 con 0x55b9ee9f3b90 2024-06-03T02:33:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e18761b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1de400cb30 con 0x55b9ee9ee240 2024-06-03T02:33:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e18761b80 10 client.0 _ll_register_callbacks cb 0x55b9ee8d5380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:03.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e06a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9eea00180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e06a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9eea00180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:56590/0 (socket says 172.21.15.79:56590) 2024-06-03T02:33:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e06a00700 1 -- 172.21.15.79:0/2986336729 learned_addr learned my addr 172.21.15.79:0/2986336729 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:03.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e07400700 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9eea1f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e06a00700 1 -- 172.21.15.79:0/2986336729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9f3b90 msgr2=0x55b9eea006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:03.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e06a00700 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9f3b90 0x55b9eea006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e06a00700 1 -- 172.21.15.79:0/2986336729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 msgr2=0x55b9eea1f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:03.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e06a00700 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9eea1f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.611+0000 7f1e06a00700 1 -- 172.21.15.79:0/2986336729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9eea1fc00 con 0x55b9ee9ee240 2024-06-03T02:33:03.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.614+0000 7f1e06a00700 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9eea00180 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1e0009ad80 tx=0x7f1e000a3760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b107a15a13c97b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:03.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.614+0000 7f1df5600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:03.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.614+0000 7f1df5600700 1 -- 172.21.15.79:0/2986336729 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000b7070 con 0x55b9ee9ee240 2024-06-03T02:33:03.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.614+0000 7f1e18761b80 1 -- 172.21.15.79:0/2986336729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e000b1070 con 0x55b9ee9ee240 2024-06-03T02:33:03.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.614+0000 7f1df5600700 1 -- 172.21.15.79:0/2986336729 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f1e00099a80 con 0x55b9ee9ee240 2024-06-03T02:33:03.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.614+0000 7f1df5600700 1 -- 172.21.15.79:0/2986336729 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e00096e60 con 0x55b9ee9ee240 2024-06-03T02:33:03.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.615+0000 7f1e18761b80 10 client.26779 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:03.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.615+0000 7f1e18761b80 1 -- 172.21.15.79:0/2986336729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b9eea1fc00 con 0x55b9ee9ee240 2024-06-03T02:33:03.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.615+0000 7f1df5600700 1 -- 172.21.15.79:0/2986336729 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e000b1070 con 0x55b9ee9ee240 2024-06-03T02:33:03.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.617+0000 7f1df5600700 1 -- 172.21.15.79:0/2986336729 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1e000b92c0 con 0x55b9ee9ee240 2024-06-03T02:33:03.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.617+0000 7f1e18761b80 10 client.26779 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:03.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.617+0000 7f1e18761b80 10 client.26779 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:33:03.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.617+0000 7f1e18761b80 20 client.26779 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:03.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.617+0000 7f1e18761b80 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b9eea82f40 0x55b9eea85360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:03.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.617+0000 7f1e18761b80 4 client.26779 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:33:03.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.617+0000 7f1e18761b80 1 -- 172.21.15.79:0/2986336729 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9ee947580 con 0x55b9eea82f40 2024-06-03T02:33:03.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.619+0000 7f1dfe000700 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b9eea82f40 0x55b9eea85360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:03.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.620+0000 7f1dfe000700 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b9eea82f40 0x55b9eea85360 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8482d619419c315 server_cookie=d6ad02c416708559 in_seq=0 out_seq=0 2024-06-03T02:33:03.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.620+0000 7f1df5600700 10 client.26779 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:33:03.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.621+0000 7f1df5600700 1 -- 172.21.15.79:0/2986336729 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9ee947580 con 0x55b9eea82f40 2024-06-03T02:33:03.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.621+0000 7f1df5600700 10 client.26779 handle_command_reply: tid=0 2024-06-03T02:33:03.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.621+0000 7f1e18761b80 1 client.26779 shutdown 2024-06-03T02:33:03.624 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:33:03.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1e18761b80 1 -- 172.21.15.79:0/2986336729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 msgr2=0x55b9eea00180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:03.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1e18761b80 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9eea00180 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1e0009ad80 tx=0x7f1e000a3760 comp rx=0 tx=0).stop 2024-06-03T02:33:03.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1e18761b80 1 -- 172.21.15.79:0/2986336729 shutdown_connections 2024-06-03T02:33:03.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1e18761b80 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b9ee9f44b0 0x55b9eea1f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1e18761b80 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55b9eea82f40 0x55b9eea85360 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1e18761b80 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ee9ee240 0x55b9eea00180 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1e0009ad80 tx=0x7f1e000a3760 comp rx=0 tx=0).stop 2024-06-03T02:33:03.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1e18761b80 1 --2- 172.21.15.79:0/2986336729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9ee9f3b90 0x55b9eea006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1df5600700 0 client.26779 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:33:03.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1e18761b80 1 -- 172.21.15.79:0/2986336729 >> 172.21.15.79:0/2986336729 conn(0x55b9ee91a170 msgr2=0x55b9ee9f4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:03.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.622+0000 7f1e18761b80 1 -- 172.21.15.79:0/2986336729 shutdown_connections 2024-06-03T02:33:03.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.623+0000 7f1e18761b80 1 -- 172.21.15.79:0/2986336729 wait complete. 2024-06-03T02:33:03.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.623+0000 7f1e18761b80 20 client.26779 trim_cache size 0 max 16384 2024-06-03T02:33:03.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.624+0000 7f1e0de00700 1 -- 172.21.15.79:0/3491409822 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1dec0a2d20 msgr2=0x7f1dec0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:03.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.624+0000 7f1e0de00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1dec0a2d20 0x7f1dec0a51e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1e08075f00 tx=0x7f1df800a710 comp rx=0 tx=0).stop 2024-06-03T02:33:03.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.624+0000 7f1e0de00700 1 -- 172.21.15.79:0/3491409822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08102740 msgr2=0x7f1e08076380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:03.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.624+0000 7f1e0de00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08102740 0x7f1e08076380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000ba50 tx=0x7f1e0000be10 comp rx=0 tx=0).stop 2024-06-03T02:33:03.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.625+0000 7f1e0de00700 1 -- 172.21.15.79:0/3491409822 shutdown_connections 2024-06-03T02:33:03.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.625+0000 7f1e0de00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e08107840 0x7f1e08074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.625+0000 7f1e0de00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e08102740 0x7f1e08076380 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.625+0000 7f1e0de00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f1dec0a2d20 0x7f1dec0a51e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.625+0000 7f1e0de00700 1 --2- 172.21.15.79:0/3491409822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e081030f0 0x7f1e08074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:03.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.625+0000 7f1e0de00700 1 -- 172.21.15.79:0/3491409822 >> 172.21.15.79:0/3491409822 conn(0x7f1e080795f0 msgr2=0x7f1e08100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:03.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.626+0000 7f1e0de00700 1 -- 172.21.15.79:0/3491409822 shutdown_connections 2024-06-03T02:33:03.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:03.626+0000 7f1e0de00700 1 -- 172.21.15.79:0/3491409822 wait complete. 2024-06-03T02:33:04.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:04 smithi079 ceph-mon[31819]: pgmap v906: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-06-03T02:33:04.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:04 smithi135 ceph-mon[36653]: pgmap v906: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-06-03T02:33:04.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:04 smithi123 ceph-mon[36640]: pgmap v906: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-06-03T02:33:06.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:33:06.755 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:33:06.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:06 smithi079 ceph-mon[31819]: pgmap v907: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.0 MiB/s wr, 533 op/s 2024-06-03T02:33:06.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:06 smithi135 ceph-mon[36653]: pgmap v907: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.0 MiB/s wr, 533 op/s 2024-06-03T02:33:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:06 smithi123 ceph-mon[36640]: pgmap v907: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.0 MiB/s wr, 533 op/s 2024-06-03T02:33:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 780.0 s, 490.8 tps, lat 210.034 ms stddev 268.038 2024-06-03T02:33:08.760 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:08 smithi079 ceph-mon[31819]: pgmap v908: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.5 MiB/s wr, 458 op/s 2024-06-03T02:33:08.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:08 smithi135 ceph-mon[36653]: pgmap v908: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.5 MiB/s wr, 458 op/s 2024-06-03T02:33:08.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:08 smithi123 ceph-mon[36640]: pgmap v908: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.5 MiB/s wr, 458 op/s 2024-06-03T02:33:09.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.673+0000 7ff0b4a00700 1 -- 172.21.15.79:0/2336875815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b0074fe0 msgr2=0x7ff0b0113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:09.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.673+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/2336875815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b0074fe0 0x7ff0b0113120 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0098f0 tx=0x7ff09c009c00 comp rx=0 tx=0).stop 2024-06-03T02:33:09.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.673+0000 7ff0b4a00700 1 -- 172.21.15.79:0/2336875815 shutdown_connections 2024-06-03T02:33:09.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.673+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/2336875815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0b0113660 0x7ff0b0115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.673+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/2336875815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00746c0 0x7ff0b0074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.673+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/2336875815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b0074fe0 0x7ff0b0113120 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.673+0000 7ff0b4a00700 1 -- 172.21.15.79:0/2336875815 >> 172.21.15.79:0/2336875815 conn(0x7ff0b00795f0 msgr2=0x7ff0b0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:09.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.673+0000 7ff0b4a00700 1 -- 172.21.15.79:0/2336875815 shutdown_connections 2024-06-03T02:33:09.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.674+0000 7ff0b4a00700 1 -- 172.21.15.79:0/2336875815 wait complete. 2024-06-03T02:33:09.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.675+0000 7ff0b4a00700 1 Processor -- start 2024-06-03T02:33:09.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.675+0000 7ff0b4a00700 1 -- start start 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.675+0000 7ff0b4a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00746c0 0x7ff0b019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.675+0000 7ff0b4a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0b0074fe0 0x7ff0b019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.675+0000 7ff0b4a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b0113660 0x7ff0b01a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.676+0000 7ff0b4a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0118df0 con 0x7ff0b0074fe0 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.676+0000 7ff0b4a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0118c70 con 0x7ff0b00746c0 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.676+0000 7ff0b4a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0118f70 con 0x7ff0b0113660 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.676+0000 7ff0ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00746c0 0x7ff0b019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.676+0000 7ff0ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00746c0 0x7ff0b019f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38898/0 (socket says 172.21.15.79:38898) 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.676+0000 7ff0ade00700 1 -- 172.21.15.79:0/1169050063 learned_addr learned my addr 172.21.15.79:0/1169050063 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:09.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.676+0000 7ff0ae800700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b0113660 0x7ff0b01a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.676+0000 7ff0ad400700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0b0074fe0 0x7ff0b019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.676+0000 7ff0ade00700 1 -- 172.21.15.79:0/1169050063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b0113660 msgr2=0x7ff0b01a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:09.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.677+0000 7ff0ade00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b0113660 0x7ff0b01a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.677+0000 7ff0ade00700 1 -- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0b0074fe0 msgr2=0x7ff0b019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:09.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.677+0000 7ff0ade00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0b0074fe0 0x7ff0b019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.677+0000 7ff0ade00700 1 -- 172.21.15.79:0/1169050063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b01a4310 con 0x7ff0b00746c0 2024-06-03T02:33:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.677+0000 7ff0ad400700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0b0074fe0 0x7ff0b019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:33:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.677+0000 7ff0ae800700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b0113660 0x7ff0b01a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:33:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.677+0000 7ff0ade00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00746c0 0x7ff0b019f300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000eb30 tx=0x7ff0a000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.678+0000 7ff0a6a00700 1 -- 172.21.15.79:0/1169050063 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000cc90 con 0x7ff0b00746c0 2024-06-03T02:33:09.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.678+0000 7ff0a6a00700 1 -- 172.21.15.79:0/1169050063 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0004500 con 0x7ff0b00746c0 2024-06-03T02:33:09.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.678+0000 7ff0a6a00700 1 -- 172.21.15.79:0/1169050063 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0010430 con 0x7ff0b00746c0 2024-06-03T02:33:09.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.679+0000 7ff0b4a00700 1 -- 172.21.15.79:0/1169050063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b01a4600 con 0x7ff0b00746c0 2024-06-03T02:33:09.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.679+0000 7ff0b4a00700 1 -- 172.21.15.79:0/1169050063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b01abe80 con 0x7ff0b00746c0 2024-06-03T02:33:09.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.685+0000 7ff0a6a00700 1 -- 172.21.15.79:0/1169050063 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff0a0003f70 con 0x7ff0b00746c0 2024-06-03T02:33:09.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.685+0000 7ff0bf3c2b80 1 Processor -- start 2024-06-03T02:33:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.686+0000 7ff0bf3c2b80 1 -- start start 2024-06-03T02:33:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.686+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b79b90 0x560be4b79f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.686+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b7a4b0 0x560be4b73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.686+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.686+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560be4a82a10 con 0x560be4b74240 2024-06-03T02:33:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.686+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560be4ac5b00 con 0x560be4b7a4b0 2024-06-03T02:33:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.686+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560be4a808f0 con 0x560be4b79b90 2024-06-03T02:33:09.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.686+0000 7ff0a6a00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0940a2d70 0x7ff0940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.686+0000 7ff0a6a00700 1 -- 172.21.15.79:0/1169050063 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff0a00cf720 con 0x7ff0b00746c0 2024-06-03T02:33:09.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.687+0000 7ff0ae800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b79b90 0x560be4b79f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.687+0000 7ff0ae800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b79b90 0x560be4b79f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:38414/0 (socket says 172.21.15.79:38414) 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.687+0000 7ff0ae800700 1 -- 172.21.15.79:0/4117976851 learned_addr learned my addr 172.21.15.79:0/4117976851 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.687+0000 7ff0ae800700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0940a2d70 0x7ff0940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.687+0000 7ff0ade00700 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b7a4b0 0x560be4b73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.687+0000 7ff0ad400700 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.687+0000 7ff0ade00700 1 -- 172.21.15.79:0/4117976851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b79b90 msgr2=0x560be4b79f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.687+0000 7ff0ade00700 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b79b90 0x560be4b79f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.687+0000 7ff0ade00700 1 -- 172.21.15.79:0/4117976851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 msgr2=0x560be4b74680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.688+0000 7ff0ade00700 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b74680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.688+0000 7ff0ade00700 1 -- 172.21.15.79:0/4117976851 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be4a6df90 con 0x560be4b7a4b0 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.688+0000 7ff0ae800700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0940a2d70 0x7ff0940a5230 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009fd0 tx=0x7ff0a80099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.688+0000 7ff0ad400700 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:33:09.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.688+0000 7ff0ade00700 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b7a4b0 0x560be4b73d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff0a009bb00 tx=0x7ff0a0097f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9258581782d4b29 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:09.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.688+0000 7ff09b400700 1 -- 172.21.15.79:0/4117976851 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a009b4d0 con 0x560be4b7a4b0 2024-06-03T02:33:09.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.688+0000 7ff09b400700 1 -- 172.21.15.79:0/4117976851 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0009500 con 0x560be4b7a4b0 2024-06-03T02:33:09.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.688+0000 7ff09b400700 1 -- 172.21.15.79:0/4117976851 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0099e70 con 0x560be4b7a4b0 2024-06-03T02:33:09.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.689+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/4117976851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b7a4b0 msgr2=0x560be4b73d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:09.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.689+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b7a4b0 0x560be4b73d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff0a009bb00 tx=0x7ff0a0097f30 comp rx=0 tx=0).stop 2024-06-03T02:33:09.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.689+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/4117976851 shutdown_connections 2024-06-03T02:33:09.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.689+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.690+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b79b90 0x560be4b79f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.690+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/4117976851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b7a4b0 0x560be4b73d00 secure :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff0a009bb00 tx=0x7ff0a0097f30 comp rx=0 tx=0).stop 2024-06-03T02:33:09.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.690+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/4117976851 >> 172.21.15.79:0/4117976851 conn(0x560be4aa0170 msgr2=0x560be4b6eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:09.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.690+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/4117976851 shutdown_connections 2024-06-03T02:33:09.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.690+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/4117976851 wait complete. 2024-06-03T02:33:09.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.691+0000 7ff0bf3c2b80 1 -- start start 2024-06-03T02:33:09.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.691+0000 7ff0bf3c2b80 1 Processor -- start 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b74240 0x560be4b804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 0x560be4b7d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 0x560be4b9b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560be4a61f60 con 0x560be4b7de50 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560be4ac1ac0 con 0x560be4b79b90 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560be4b80e30 con 0x560be4b74240 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0bf3c2b80 10 client.0 _ll_register_callbacks cb 0x560be4a5b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0ade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b74240 0x560be4b804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0ade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b74240 0x560be4b804b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:38424/0 (socket says 172.21.15.79:38424) 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0ade00700 1 -- 172.21.15.79:0/2152435925 learned_addr learned my addr 172.21.15.79:0/2152435925 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0ad400700 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 0x560be4b7d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.692+0000 7ff0ade00700 1 -- 172.21.15.79:0/2152435925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 msgr2=0x560be4b7d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:09.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff0ade00700 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 0x560be4b7d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff0ade00700 1 -- 172.21.15.79:0/2152435925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 msgr2=0x560be4b9b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff0ade00700 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 0x560be4b9b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff0ade00700 1 -- 172.21.15.79:0/2152435925 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be4a148d0 con 0x560be4b74240 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff0ad400700 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 0x560be4b7d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff0ade00700 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b74240 0x560be4b804b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000eb00 tx=0x7ff0a009bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fb7b0302a43284d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff08fe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff08fe00700 1 -- 172.21.15.79:0/2152435925 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00aa070 con 0x560be4b74240 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff08fe00700 1 -- 172.21.15.79:0/2152435925 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff0a0099e70 con 0x560be4b74240 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/2152435925 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a0009500 con 0x560be4b74240 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff08fe00700 1 -- 172.21.15.79:0/2152435925 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff0a0081050 con 0x560be4b74240 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff0bf3c2b80 10 client.26788 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.693+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/2152435925 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560be4b9cbb0 con 0x560be4b74240 2024-06-03T02:33:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.694+0000 7ff08fe00700 1 -- 172.21.15.79:0/2152435925 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0009500 con 0x560be4b74240 2024-06-03T02:33:09.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.694+0000 7ff08fe00700 1 -- 172.21.15.79:0/2152435925 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff0a00a4350 con 0x560be4b74240 2024-06-03T02:33:09.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.694+0000 7ff0bf3c2b80 10 client.26788 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:09.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.694+0000 7ff0bf3c2b80 10 client.26788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:33:09.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.694+0000 7ff0bf3c2b80 20 client.26788 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:09.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.694+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560be4b9f080 0x560be4ba1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.694+0000 7ff0bf3c2b80 4 client.26788 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:33:09.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.694+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/2152435925 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560be4ad3000 con 0x560be4b9f080 2024-06-03T02:33:09.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.695+0000 7ff0ad400700 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560be4b9f080 0x560be4ba1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.696+0000 7ff08fe00700 10 client.26788 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:09.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.697+0000 7ff0ad400700 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560be4b9f080 0x560be4ba1470 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801567effa64793 server_cookie=14a401b4e4487425 in_seq=0 out_seq=0 2024-06-03T02:33:09.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.698+0000 7ff08fe00700 1 -- 172.21.15.79:0/2152435925 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560be4ad3000 con 0x560be4b9f080 2024-06-03T02:33:09.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.698+0000 7ff08fe00700 10 client.26788 handle_command_reply: tid=0 2024-06-03T02:33:09.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.698+0000 7ff0bf3c2b80 1 client.26788 shutdown 2024-06-03T02:33:09.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/2152435925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b74240 msgr2=0x560be4b804b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:09.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b74240 0x560be4b804b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000eb00 tx=0x7ff0a009bc20 comp rx=0 tx=0).stop 2024-06-03T02:33:09.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/2152435925 shutdown_connections 2024-06-03T02:33:09.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560be4b9f080 0x560be4ba1470 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 0x560be4b9b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b74240 0x560be4b804b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/2152435925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 0x560be4b7d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/2152435925 >> 172.21.15.79:0/2152435925 conn(0x560be4aa0170 msgr2=0x560be4b68790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:09.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff08fe00700 0 client.26788 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:09.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/2152435925 shutdown_connections 2024-06-03T02:33:09.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/2152435925 wait complete. 2024-06-03T02:33:09.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.699+0000 7ff0bf3c2b80 20 client.26788 trim_cache size 0 max 16384 2024-06-03T02:33:09.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.721+0000 7ff0bf3c2b80 1 Processor -- start 2024-06-03T02:33:09.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.721+0000 7ff0bf3c2b80 1 -- start start 2024-06-03T02:33:09.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.721+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.721+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 0x560be4b79180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.721+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b7de50 0x560be4b77d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ade00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01ac280 con 0x560be4b74240 2024-06-03T02:33:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ade00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b69130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57440/0 (socket says 172.21.15.79:57440) 2024-06-03T02:33:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ade00700 1 -- 172.21.15.79:0/3372247890 learned_addr learned my addr 172.21.15.79:0/3372247890 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ad400700 1 --2- 172.21.15.79:0/3372247890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 0x560be4b79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/3372247890 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01addc0 con 0x560be4b79b90 2024-06-03T02:33:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/3372247890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01ac950 con 0x560be4b7de50 2024-06-03T02:33:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ae800700 1 --2- 172.21.15.79:0/3372247890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b7de50 0x560be4b77d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ae800700 1 -- 172.21.15.79:0/3372247890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 msgr2=0x560be4b79180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ae800700 1 --2- 172.21.15.79:0/3372247890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 0x560be4b79180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ae800700 1 -- 172.21.15.79:0/3372247890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 msgr2=0x560be4b69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ae800700 1 --2- 172.21.15.79:0/3372247890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.722+0000 7ff0ae800700 1 -- 172.21.15.79:0/3372247890 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be4b7e390 con 0x560be4b7de50 2024-06-03T02:33:09.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.723+0000 7ff0ae800700 1 --2- 172.21.15.79:0/3372247890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b7de50 0x560be4b77d50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009750 tx=0x7ff0a801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=212fbac884613ccc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:09.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.723+0000 7ff09aa00700 1 -- 172.21.15.79:0/3372247890 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8009470 con 0x560be4b7de50 2024-06-03T02:33:09.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.723+0000 7ff09aa00700 1 -- 172.21.15.79:0/3372247890 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a8021070 con 0x560be4b7de50 2024-06-03T02:33:09.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.723+0000 7ff09aa00700 1 -- 172.21.15.79:0/3372247890 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8007d20 con 0x560be4b7de50 2024-06-03T02:33:09.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.724+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/3372247890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b7de50 msgr2=0x560be4b77d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:09.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.724+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/3372247890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b7de50 0x560be4b77d50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009750 tx=0x7ff0a801b000 comp rx=0 tx=0).stop 2024-06-03T02:33:09.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.724+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/3372247890 shutdown_connections 2024-06-03T02:33:09.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.724+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/3372247890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4b7de50 0x560be4b77d50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.724+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/3372247890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b74240 0x560be4b69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.724+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/3372247890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b79b90 0x560be4b79180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.724+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/3372247890 >> 172.21.15.79:0/3372247890 conn(0x560be4aa0170 msgr2=0x560be4b7cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:09.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.727+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/3372247890 shutdown_connections 2024-06-03T02:33:09.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.727+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/3372247890 wait complete. 2024-06-03T02:33:09.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.727+0000 7ff0bf3c2b80 1 -- start start 2024-06-03T02:33:09.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.728+0000 7ff0bf3c2b80 1 Processor -- start 2024-06-03T02:33:09.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.733+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b74240 0x560be4ba1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.733+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 0x560be4ba1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.733+0000 7ff0bf3c2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4c059a0 0x560be4c07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.734+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01ad6f0 con 0x560be4b7de50 2024-06-03T02:33:09.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.734+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01ad020 con 0x560be4b74240 2024-06-03T02:33:09.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.734+0000 7ff0bf3c2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0072880 con 0x560be4c059a0 2024-06-03T02:33:09.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.734+0000 7ff0bf3c2b80 10 client.0 _ll_register_callbacks cb 0x560be4a5b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:09.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.735+0000 7ff0ad400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 0x560be4ba1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.735+0000 7ff0ae800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4c059a0 0x560be4c07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.735+0000 7ff0ae800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4c059a0 0x560be4c07d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:38434/0 (socket says 172.21.15.79:38434) 2024-06-03T02:33:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.735+0000 7ff0ae800700 1 -- 172.21.15.79:0/334435608 learned_addr learned my addr 172.21.15.79:0/334435608 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.735+0000 7ff0ad400700 1 -- 172.21.15.79:0/334435608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4c059a0 msgr2=0x560be4c07d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.735+0000 7ff0ad400700 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4c059a0 0x560be4c07d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.735+0000 7ff0ad400700 1 -- 172.21.15.79:0/334435608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b74240 msgr2=0x560be4ba1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.735+0000 7ff0ad400700 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b74240 0x560be4ba1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.735+0000 7ff0ad400700 1 -- 172.21.15.79:0/334435608 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be4c083a0 con 0x560be4b7de50 2024-06-03T02:33:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.736+0000 7ff0ad400700 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 0x560be4ba1fb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009fd0 tx=0x7ff09c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7db071bb9f73278 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:09.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.736+0000 7ff08fe00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:33:09.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.740+0000 7ff08fe00700 1 -- 172.21.15.79:0/334435608 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c032060 con 0x560be4b7de50 2024-06-03T02:33:09.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.740+0000 7ff08fe00700 1 -- 172.21.15.79:0/334435608 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff09c005040 con 0x560be4b7de50 2024-06-03T02:33:09.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.740+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/334435608 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a8021070 con 0x560be4b7de50 2024-06-03T02:33:09.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.741+0000 7ff08fe00700 1 -- 172.21.15.79:0/334435608 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff09c005240 con 0x560be4b7de50 2024-06-03T02:33:09.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.742+0000 7ff0bf3c2b80 10 client.16257 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:09.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.742+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/334435608 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560be4c096e0 con 0x560be4b7de50 2024-06-03T02:33:09.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.743+0000 7ff08fe00700 1 -- 172.21.15.79:0/334435608 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a8021070 con 0x560be4b7de50 2024-06-03T02:33:09.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.743+0000 7ff08fe00700 1 -- 172.21.15.79:0/334435608 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff09c00b560 con 0x560be4b7de50 2024-06-03T02:33:09.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.743+0000 7ff0bf3c2b80 10 client.16257 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:09.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.743+0000 7ff0bf3c2b80 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:33:09.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.743+0000 7ff0bf3c2b80 20 client.16257 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.743+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560be4c09d70 0x560be4c0c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.743+0000 7ff0bf3c2b80 4 client.16257 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:33:09.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.743+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/334435608 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560be4acd580 con 0x560be4c09d70 2024-06-03T02:33:09.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.747+0000 7ff0ade00700 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560be4c09d70 0x560be4c0c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.748+0000 7ff0ade00700 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560be4c09d70 0x560be4c0c1b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=299c917725b29f7 server_cookie=888284773f8586f7 in_seq=0 out_seq=0 2024-06-03T02:33:09.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.748+0000 7ff08fe00700 10 client.16257 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:09.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.752+0000 7ff08fe00700 1 -- 172.21.15.79:0/334435608 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560be4acd580 con 0x560be4c09d70 2024-06-03T02:33:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.752+0000 7ff08fe00700 10 client.16257 handle_command_reply: tid=0 2024-06-03T02:33:09.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.752+0000 7ff0bf3c2b80 1 client.16257 shutdown 2024-06-03T02:33:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.753+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/334435608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 msgr2=0x560be4ba1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.753+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 0x560be4ba1fb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009fd0 tx=0x7ff09c009560 comp rx=0 tx=0).stop 2024-06-03T02:33:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.753+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/334435608 shutdown_connections 2024-06-03T02:33:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.753+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560be4c059a0 0x560be4c07d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.753+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560be4b74240 0x560be4ba1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.753+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x560be4c09d70 0x560be4c0c1b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.754+0000 7ff0bf3c2b80 1 --2- 172.21.15.79:0/334435608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560be4b7de50 0x560be4ba1fb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.754+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/334435608 >> 172.21.15.79:0/334435608 conn(0x560be4aa0170 msgr2=0x560be4b7d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.754+0000 7ff08fe00700 0 client.16257 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.754+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/334435608 shutdown_connections 2024-06-03T02:33:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.754+0000 7ff0bf3c2b80 1 -- 172.21.15.79:0/334435608 wait complete. 2024-06-03T02:33:09.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.756+0000 7ff0bf3c2b80 20 client.16257 trim_cache size 0 max 16384 2024-06-03T02:33:09.757 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 -- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0940a2d70 msgr2=0x7ff0940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0940a2d70 0x7ff0940a5230 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009fd0 tx=0x7ff0a80099e0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 -- 172.21.15.79:0/1169050063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00746c0 msgr2=0x7ff0b019f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00746c0 0x7ff0b019f300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000eb30 tx=0x7ff0a000ee40 comp rx=0 tx=0).stop 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 -- 172.21.15.79:0/1169050063 shutdown_connections 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b0113660 0x7ff0b01a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0b00746c0 0x7ff0b019f300 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff0940a2d70 0x7ff0940a5230 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 --2- 172.21.15.79:0/1169050063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0b0074fe0 0x7ff0b019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.758+0000 7ff0b4a00700 1 -- 172.21.15.79:0/1169050063 >> 172.21.15.79:0/1169050063 conn(0x7ff0b00795f0 msgr2=0x7ff0b0112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:09.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.762+0000 7ff0b4a00700 1 -- 172.21.15.79:0/1169050063 shutdown_connections 2024-06-03T02:33:09.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:09.762+0000 7ff0b4a00700 1 -- 172.21.15.79:0/1169050063 wait complete. 2024-06-03T02:33:10.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:10 smithi079 ceph-mon[31819]: pgmap v909: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.0 MiB/s wr, 512 op/s 2024-06-03T02:33:10.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:10 smithi135 ceph-mon[36653]: pgmap v909: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.0 MiB/s wr, 512 op/s 2024-06-03T02:33:10.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:10 smithi123 ceph-mon[36640]: pgmap v909: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.0 MiB/s wr, 512 op/s 2024-06-03T02:33:11.695 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:33:11.696 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:33:12.343 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:33:12.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:12 smithi079 ceph-mon[31819]: pgmap v910: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 4.9 MiB/s wr, 522 op/s 2024-06-03T02:33:12.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:12 smithi135 ceph-mon[36653]: pgmap v910: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 4.9 MiB/s wr, 522 op/s 2024-06-03T02:33:12.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:12 smithi123 ceph-mon[36640]: pgmap v910: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 4.9 MiB/s wr, 522 op/s 2024-06-03T02:33:13.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 785.0 s, 448.0 tps, lat 223.620 ms stddev 234.636 2024-06-03T02:33:14.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:14 smithi079 ceph-mon[31819]: pgmap v911: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-06-03T02:33:14.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:14 smithi135 ceph-mon[36653]: pgmap v911: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-06-03T02:33:14.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:14 smithi123 ceph-mon[36640]: pgmap v911: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-06-03T02:33:16.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:16 smithi079 ceph-mon[31819]: pgmap v912: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.1 MiB/s wr, 536 op/s 2024-06-03T02:33:16.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:16 smithi135 ceph-mon[36653]: pgmap v912: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.1 MiB/s wr, 536 op/s 2024-06-03T02:33:16.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:16 smithi123 ceph-mon[36640]: pgmap v912: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.1 MiB/s wr, 536 op/s 2024-06-03T02:33:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 790.0 s, 499.4 tps, lat 199.088 ms stddev 227.799 2024-06-03T02:33:18.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:18 smithi079 ceph-mon[31819]: pgmap v913: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-06-03T02:33:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:18 smithi135 ceph-mon[36653]: pgmap v913: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-06-03T02:33:18.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:18 smithi123 ceph-mon[36640]: pgmap v913: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-06-03T02:33:19.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.566+0000 7ff7fe200700 1 -- 172.21.15.79:0/414431134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8107450 msgr2=0x7ff7f8107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:19.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.566+0000 7ff7fe200700 1 --2- 172.21.15.79:0/414431134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8107450 0x7ff7f8107830 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000b0a0 tx=0x7ff7f000b3b0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.568+0000 7ff7fe200700 1 -- 172.21.15.79:0/414431134 shutdown_connections 2024-06-03T02:33:19.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.568+0000 7ff7fe200700 1 --2- 172.21.15.79:0/414431134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7f8102ee0 0x7ff7f810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.568+0000 7ff7fe200700 1 --2- 172.21.15.79:0/414431134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8107450 0x7ff7f8107830 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.568+0000 7ff7fe200700 1 --2- 172.21.15.79:0/414431134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f8102540 0x7ff7f81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.568+0000 7ff7fe200700 1 -- 172.21.15.79:0/414431134 >> 172.21.15.79:0/414431134 conn(0x7ff7f80795f0 msgr2=0x7ff7f8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:19.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.569+0000 7ff7fe200700 1 -- 172.21.15.79:0/414431134 shutdown_connections 2024-06-03T02:33:19.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.569+0000 7ff7fe200700 1 -- 172.21.15.79:0/414431134 wait complete. 2024-06-03T02:33:19.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.570+0000 7ff7fe200700 1 Processor -- start 2024-06-03T02:33:19.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.571+0000 7ff7fe200700 1 -- start start 2024-06-03T02:33:19.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.572+0000 7ff7fe200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8102540 0x7ff7f8073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.572+0000 7ff7fe200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7f8102ee0 0x7ff7f8074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.572+0000 7ff7fe200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f8107450 0x7ff7f806cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.572+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8102540 0x7ff7f8073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.572+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8102540 0x7ff7f8073be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36522/0 (socket says 172.21.15.79:36522) 2024-06-03T02:33:19.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.573+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f8107450 0x7ff7f806cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.573+0000 7ff7f6a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7f8102ee0 0x7ff7f8074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.573+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f8107450 0x7ff7f806cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36622/0 (socket says 172.21.15.79:36622) 2024-06-03T02:33:19.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.573+0000 7ff7f7400700 1 -- 172.21.15.79:0/1622845622 learned_addr learned my addr 172.21.15.79:0/1622845622 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:19.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.573+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8112870 con 0x7ff7f8102ee0 2024-06-03T02:33:19.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.574+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81126f0 con 0x7ff7f8102540 2024-06-03T02:33:19.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.574+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81129f0 con 0x7ff7f8107450 2024-06-03T02:33:19.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.574+0000 7ff7f7400700 1 -- 172.21.15.79:0/1622845622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f8107450 msgr2=0x7ff7f806cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:19.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.575+0000 7ff7f7400700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f8107450 0x7ff7f806cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.575+0000 7ff7f7400700 1 -- 172.21.15.79:0/1622845622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7f8102ee0 msgr2=0x7ff7f8074120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:19.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.575+0000 7ff7f7400700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7f8102ee0 0x7ff7f8074120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.575+0000 7ff7f7400700 1 -- 172.21.15.79:0/1622845622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f806d810 con 0x7ff7f8102540 2024-06-03T02:33:19.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.579+0000 7ff7f7400700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8102540 0x7ff7f8073be0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000b780 tx=0x7ff7f0004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:19.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.579+0000 7ff7efe00700 1 -- 172.21.15.79:0/1622845622 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0021070 con 0x7ff7f8102540 2024-06-03T02:33:19.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.579+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f806da40 con 0x7ff7f8102540 2024-06-03T02:33:19.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.580+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f8071990 con 0x7ff7f8102540 2024-06-03T02:33:19.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.580+0000 7ff7efe00700 1 -- 172.21.15.79:0/1622845622 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f0007b80 con 0x7ff7f8102540 2024-06-03T02:33:19.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.580+0000 7ff7efe00700 1 -- 172.21.15.79:0/1622845622 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0021070 con 0x7ff7f8102540 2024-06-03T02:33:19.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.593+0000 7ff7efe00700 1 -- 172.21.15.79:0/1622845622 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff7f0024070 con 0x7ff7f8102540 2024-06-03T02:33:19.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.593+0000 7ff808b45b80 1 Processor -- start 2024-06-03T02:33:19.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.595+0000 7ff808b45b80 1 -- start start 2024-06-03T02:33:19.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 0x5562cfeb10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfeb1630 0x5562cfeb1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 0x5562cfec6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff808b45b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5562cfdc0a90 con 0x5562cfe2dec0 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff808b45b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5562cfe03b80 con 0x5562cfeb1630 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff808b45b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5562cfdbe970 con 0x5562cfec4b90 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfeb1630 0x5562cfeb1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfeb1630 0x5562cfeb1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36524/0 (socket says 172.21.15.79:36524) 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff7f6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 0x5562cfec6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.596+0000 7ff7f7400700 1 -- 172.21.15.79:0/213832882 learned_addr learned my addr 172.21.15.79:0/213832882 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:19.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.597+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/213832882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 0x5562cfeb10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.597+0000 7ff7f7400700 1 -- 172.21.15.79:0/213832882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 msgr2=0x5562cfec6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:19.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.597+0000 7ff7f7400700 1 --2- 172.21.15.79:0/213832882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 0x5562cfec6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.597+0000 7ff7f7400700 1 -- 172.21.15.79:0/213832882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 msgr2=0x5562cfeb10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:19.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.598+0000 7ff7f7400700 1 --2- 172.21.15.79:0/213832882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 0x5562cfeb10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.598+0000 7ff7f7400700 1 -- 172.21.15.79:0/213832882 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562cfdac010 con 0x5562cfeb1630 2024-06-03T02:33:19.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.598+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/213832882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 0x5562cfeb10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:33:19.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.598+0000 7ff7efe00700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7dc0a2ea0 0x7ff7dc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.599+0000 7ff7efe00700 1 -- 172.21.15.79:0/1622845622 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff7f000f020 con 0x7ff7f8102540 2024-06-03T02:33:19.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.600+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7dc0a2ea0 0x7ff7dc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.601+0000 7ff7f7400700 1 --2- 172.21.15.79:0/213832882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfeb1630 0x5562cfeb1ab0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00d8520 tx=0x7ff7f00d8550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed91c220427e91f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:19.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.601+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7dc0a2ea0 0x7ff7dc0a5360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x5562cfeb2470 tx=0x7ff7e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:19.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.601+0000 7ff7ecc00700 1 -- 172.21.15.79:0/213832882 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0098d70 con 0x5562cfeb1630 2024-06-03T02:33:19.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.602+0000 7ff7ecc00700 1 -- 172.21.15.79:0/213832882 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f009d1d0 con 0x5562cfeb1630 2024-06-03T02:33:19.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.602+0000 7ff7ecc00700 1 -- 172.21.15.79:0/213832882 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f009c430 con 0x5562cfeb1630 2024-06-03T02:33:19.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.603+0000 7ff808b45b80 1 -- 172.21.15.79:0/213832882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfeb1630 msgr2=0x5562cfeb1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:19.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.603+0000 7ff808b45b80 1 --2- 172.21.15.79:0/213832882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfeb1630 0x5562cfeb1ab0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00d8520 tx=0x7ff7f00d8550 comp rx=0 tx=0).stop 2024-06-03T02:33:19.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.603+0000 7ff808b45b80 1 -- 172.21.15.79:0/213832882 shutdown_connections 2024-06-03T02:33:19.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.603+0000 7ff808b45b80 1 --2- 172.21.15.79:0/213832882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 0x5562cfec6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.603+0000 7ff808b45b80 1 --2- 172.21.15.79:0/213832882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 0x5562cfeb10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.603+0000 7ff808b45b80 1 --2- 172.21.15.79:0/213832882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfeb1630 0x5562cfeb1ab0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.603+0000 7ff808b45b80 1 -- 172.21.15.79:0/213832882 >> 172.21.15.79:0/213832882 conn(0x5562cfdde1f0 msgr2=0x5562cfead530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:19.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.605+0000 7ff808b45b80 1 -- 172.21.15.79:0/213832882 shutdown_connections 2024-06-03T02:33:19.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.605+0000 7ff808b45b80 1 -- 172.21.15.79:0/213832882 wait complete. 2024-06-03T02:33:19.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.606+0000 7ff808b45b80 1 -- start start 2024-06-03T02:33:19.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.606+0000 7ff808b45b80 1 Processor -- start 2024-06-03T02:33:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 0x5562cfebde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfeb1630 0x5562cfebe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 0x5562cfeb9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff808b45b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5562cfd9ffe0 con 0x5562cfe2dec0 2024-06-03T02:33:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff808b45b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5562cfdffb40 con 0x5562cfec4b90 2024-06-03T02:33:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff808b45b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5562cfeb4460 con 0x5562cfeb1630 2024-06-03T02:33:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff808b45b80 10 client.0 _ll_register_callbacks cb 0x5562cfd9d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:19.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff7f6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfeb1630 0x5562cfebe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 0x5562cfebde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff7f6a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfeb1630 0x5562cfebe360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36650/0 (socket says 172.21.15.79:36650) 2024-06-03T02:33:19.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff7f6a00700 1 -- 172.21.15.79:0/1900626714 learned_addr learned my addr 172.21.15.79:0/1900626714 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:19.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.607+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 0x5562cfeb9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.608+0000 7ff7f6a00700 1 -- 172.21.15.79:0/1900626714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 msgr2=0x5562cfeb9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:19.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.608+0000 7ff7f6a00700 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 0x5562cfeb9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.608+0000 7ff7f6a00700 1 -- 172.21.15.79:0/1900626714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 msgr2=0x5562cfebde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:19.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.608+0000 7ff7f6a00700 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 0x5562cfebde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.608+0000 7ff7f6a00700 1 -- 172.21.15.79:0/1900626714 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562cfd52950 con 0x5562cfeb1630 2024-06-03T02:33:19.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.609+0000 7ff7e1600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:19.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.609+0000 7ff7f6a00700 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfeb1630 0x5562cfebe360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4010700 tx=0x7ff7e4010a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f7dc314752e4361 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:19.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.611+0000 7ff7e1600700 1 -- 172.21.15.79:0/1900626714 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4011970 con 0x5562cfeb1630 2024-06-03T02:33:19.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.611+0000 7ff808b45b80 1 -- 172.21.15.79:0/1900626714 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f009d1d0 con 0x5562cfeb1630 2024-06-03T02:33:19.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.611+0000 7ff7e1600700 1 -- 172.21.15.79:0/1900626714 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff7e4011770 con 0x5562cfeb1630 2024-06-03T02:33:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.614+0000 7ff7e1600700 1 -- 172.21.15.79:0/1900626714 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7e4025050 con 0x5562cfeb1630 2024-06-03T02:33:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.614+0000 7ff808b45b80 10 client.26803 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.614+0000 7ff808b45b80 1 -- 172.21.15.79:0/1900626714 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562cfec4970 con 0x5562cfeb1630 2024-06-03T02:33:19.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.617+0000 7ff7e1600700 1 -- 172.21.15.79:0/1900626714 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f009d1d0 con 0x5562cfeb1630 2024-06-03T02:33:19.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.617+0000 7ff7e1600700 1 -- 172.21.15.79:0/1900626714 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff7e401a380 con 0x5562cfeb1630 2024-06-03T02:33:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.617+0000 7ff808b45b80 10 client.26803 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.617+0000 7ff808b45b80 10 client.26803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:33:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.617+0000 7ff808b45b80 20 client.26803 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.617+0000 7ff808b45b80 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5562cfedc8f0 0x5562cfeded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.617+0000 7ff808b45b80 4 client.26803 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:33:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.617+0000 7ff808b45b80 1 -- 172.21.15.79:0/1900626714 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562cfe11080 con 0x5562cfedc8f0 2024-06-03T02:33:19.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.617+0000 7ff7f7400700 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5562cfedc8f0 0x5562cfeded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.621+0000 7ff7e1600700 10 client.26803 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:19.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.621+0000 7ff7f7400700 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5562cfedc8f0 0x5562cfeded40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e8a05f5d52fb26 server_cookie=725be0201240e6cc in_seq=0 out_seq=0 2024-06-03T02:33:19.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.622+0000 7ff7e1600700 1 -- 172.21.15.79:0/1900626714 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5562cfe11080 con 0x5562cfedc8f0 2024-06-03T02:33:19.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff7e1600700 10 client.26803 handle_command_reply: tid=0 2024-06-03T02:33:19.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff808b45b80 1 client.26803 shutdown 2024-06-03T02:33:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff808b45b80 1 -- 172.21.15.79:0/1900626714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfeb1630 msgr2=0x5562cfebe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:19.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff808b45b80 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfeb1630 0x5562cfebe360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4010700 tx=0x7ff7e4010a10 comp rx=0 tx=0).stop 2024-06-03T02:33:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff808b45b80 1 -- 172.21.15.79:0/1900626714 shutdown_connections 2024-06-03T02:33:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff808b45b80 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 0x5562cfeb9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff808b45b80 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5562cfedc8f0 0x5562cfeded40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff808b45b80 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfe2dec0 0x5562cfebde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff7e1600700 0 client.26803 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff808b45b80 1 --2- 172.21.15.79:0/1900626714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfeb1630 0x5562cfebe360 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.623+0000 7ff808b45b80 1 -- 172.21.15.79:0/1900626714 >> 172.21.15.79:0/1900626714 conn(0x5562cfdde1f0 msgr2=0x5562cfeac7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:19.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.624+0000 7ff808b45b80 1 -- 172.21.15.79:0/1900626714 shutdown_connections 2024-06-03T02:33:19.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.625+0000 7ff808b45b80 1 -- 172.21.15.79:0/1900626714 wait complete. 2024-06-03T02:33:19.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.625+0000 7ff808b45b80 20 client.26803 trim_cache size 0 max 16384 2024-06-03T02:33:19.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.671+0000 7ff808b45b80 1 Processor -- start 2024-06-03T02:33:19.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.671+0000 7ff808b45b80 1 -- start start 2024-06-03T02:33:19.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfe2dec0 0x5562cfead6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 0x5562cfeac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 0x5562cfec5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff808b45b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b49c0 con 0x5562cfeb1630 2024-06-03T02:33:19.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff808b45b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b6500 con 0x5562cfec4b90 2024-06-03T02:33:19.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff808b45b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b5090 con 0x5562cfe2dec0 2024-06-03T02:33:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfe2dec0 0x5562cfead6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfe2dec0 0x5562cfead6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:36660/0 (socket says 172.21.15.79:36660) 2024-06-03T02:33:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff7f7400700 1 -- 172.21.15.79:0/2299579906 learned_addr learned my addr 172.21.15.79:0/2299579906 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff7f6a00700 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 0x5562cfeac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.672+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 0x5562cfec5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.673+0000 7ff7f7e00700 1 -- 172.21.15.79:0/2299579906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfe2dec0 msgr2=0x5562cfead6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:19.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.673+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfe2dec0 0x5562cfead6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.673+0000 7ff7f7e00700 1 -- 172.21.15.79:0/2299579906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 msgr2=0x5562cfeac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:19.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.673+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 0x5562cfeac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.673+0000 7ff7f7e00700 1 -- 172.21.15.79:0/2299579906 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562cfedf3c0 con 0x5562cfec4b90 2024-06-03T02:33:19.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.673+0000 7ff7f7400700 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfe2dec0 0x5562cfead6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:33:19.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.675+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 0x5562cfec5f80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80098d0 tx=0x7ff7e800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f96d4fd30bfde4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:19.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.675+0000 7ff7e3e00700 1 -- 172.21.15.79:0/2299579906 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e802a070 con 0x5562cfec4b90 2024-06-03T02:33:19.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.675+0000 7ff7e3e00700 1 -- 172.21.15.79:0/2299579906 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e801ab40 con 0x5562cfec4b90 2024-06-03T02:33:19.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.675+0000 7ff7e3e00700 1 -- 172.21.15.79:0/2299579906 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8025b90 con 0x5562cfec4b90 2024-06-03T02:33:19.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.676+0000 7ff808b45b80 1 -- 172.21.15.79:0/2299579906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 msgr2=0x5562cfec5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:19.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.677+0000 7ff808b45b80 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 0x5562cfec5f80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80098d0 tx=0x7ff7e800f000 comp rx=0 tx=0).stop 2024-06-03T02:33:19.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.677+0000 7ff808b45b80 1 -- 172.21.15.79:0/2299579906 shutdown_connections 2024-06-03T02:33:19.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.677+0000 7ff808b45b80 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfec4b90 0x5562cfec5f80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.677+0000 7ff808b45b80 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfe2dec0 0x5562cfead6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.677+0000 7ff808b45b80 1 --2- 172.21.15.79:0/2299579906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 0x5562cfeac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.677+0000 7ff808b45b80 1 -- 172.21.15.79:0/2299579906 >> 172.21.15.79:0/2299579906 conn(0x5562cfdde1f0 msgr2=0x5562cfeabf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:19.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.678+0000 7ff808b45b80 1 -- 172.21.15.79:0/2299579906 shutdown_connections 2024-06-03T02:33:19.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.678+0000 7ff808b45b80 1 -- 172.21.15.79:0/2299579906 wait complete. 2024-06-03T02:33:19.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.679+0000 7ff808b45b80 1 -- start start 2024-06-03T02:33:19.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.679+0000 7ff808b45b80 1 Processor -- start 2024-06-03T02:33:19.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.680+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfe2dec0 0x5562cfef0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.680+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 0x5562cfef0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.680+0000 7ff808b45b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 0x5562cfeeb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.680+0000 7ff808b45b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b5e30 con 0x5562cfeb1630 2024-06-03T02:33:19.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.680+0000 7ff808b45b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b5760 con 0x5562cfe2dec0 2024-06-03T02:33:19.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.680+0000 7ff808b45b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b2960 con 0x5562cfec4b90 2024-06-03T02:33:19.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.680+0000 7ff808b45b80 10 client.0 _ll_register_callbacks cb 0x5562cfd9d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:19.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.680+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfe2dec0 0x5562cfef0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.680+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfe2dec0 0x5562cfef0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:36552/0 (socket says 172.21.15.79:36552) 2024-06-03T02:33:19.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.681+0000 7ff7f6a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 0x5562cfef0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.681+0000 7ff7f6a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 0x5562cfef0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38942/0 (socket says 172.21.15.79:38942) 2024-06-03T02:33:19.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.681+0000 7ff7f6a00700 1 -- 172.21.15.79:0/3215034215 learned_addr learned my addr 172.21.15.79:0/3215034215 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:19.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.681+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 0x5562cfeeb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.681+0000 7ff7f7e00700 1 -- 172.21.15.79:0/3215034215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfe2dec0 msgr2=0x5562cfef0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:19.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.681+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfe2dec0 0x5562cfef0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.682+0000 7ff7f7e00700 1 -- 172.21.15.79:0/3215034215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 msgr2=0x5562cfef0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:19.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.682+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 0x5562cfef0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.682+0000 7ff7f7e00700 1 -- 172.21.15.79:0/3215034215 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562cfeeba50 con 0x5562cfec4b90 2024-06-03T02:33:19.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.682+0000 7ff7f7400700 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfe2dec0 0x5562cfef0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:33:19.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.682+0000 7ff7f7e00700 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 0x5562cfeeb300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8028040 tx=0x7ff7e8025650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=200ae9d158ce2d94 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:19.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.682+0000 7ff7e1600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:19.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.682+0000 7ff7e1600700 1 -- 172.21.15.79:0/3215034215 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e802a040 con 0x5562cfec4b90 2024-06-03T02:33:19.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.682+0000 7ff7e1600700 1 -- 172.21.15.79:0/3215034215 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff7e801a660 con 0x5562cfec4b90 2024-06-03T02:33:19.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.683+0000 7ff808b45b80 1 -- 172.21.15.79:0/3215034215 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e801ab40 con 0x5562cfec4b90 2024-06-03T02:33:19.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.683+0000 7ff7e1600700 1 -- 172.21.15.79:0/3215034215 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7e801a860 con 0x5562cfec4b90 2024-06-03T02:33:19.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.683+0000 7ff808b45b80 10 client.26809 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.683+0000 7ff808b45b80 1 -- 172.21.15.79:0/3215034215 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562cff47ee0 con 0x5562cfec4b90 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.683+0000 7ff7e1600700 1 -- 172.21.15.79:0/3215034215 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e801ab40 con 0x5562cfec4b90 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.683+0000 7ff7e1600700 1 -- 172.21.15.79:0/3215034215 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff7e8036030 con 0x5562cfec4b90 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.683+0000 7ff808b45b80 10 client.26809 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.684+0000 7ff808b45b80 10 client.26809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.684+0000 7ff808b45b80 20 client.26809 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.684+0000 7ff808b45b80 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5562cff48500 0x5562cff4a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.684+0000 7ff808b45b80 4 client.26809 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.684+0000 7ff808b45b80 1 -- 172.21.15.79:0/3215034215 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5562cfe0b600 con 0x5562cff48500 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.684+0000 7ff7f7400700 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5562cff48500 0x5562cff4a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:19.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.684+0000 7ff7f7400700 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5562cff48500 0x5562cff4a920 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae8bf564131a44b server_cookie=698aad1d78add788 in_seq=0 out_seq=0 2024-06-03T02:33:19.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.684+0000 7ff7e1600700 10 client.26809 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.685+0000 7ff7e1600700 1 -- 172.21.15.79:0/3215034215 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5562cfe0b600 con 0x5562cff48500 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.685+0000 7ff7e1600700 10 client.26809 handle_command_reply: tid=0 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.686+0000 7ff808b45b80 1 client.26809 shutdown 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.686+0000 7ff808b45b80 1 -- 172.21.15.79:0/3215034215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 msgr2=0x5562cfeeb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.686+0000 7ff808b45b80 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 0x5562cfeeb300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8028040 tx=0x7ff7e8025650 comp rx=0 tx=0).stop 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff808b45b80 1 -- 172.21.15.79:0/3215034215 shutdown_connections 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff808b45b80 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562cfec4b90 0x5562cfeeb300 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff808b45b80 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562cfe2dec0 0x5562cfef0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff808b45b80 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5562cff48500 0x5562cff4a920 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff808b45b80 1 --2- 172.21.15.79:0/3215034215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562cfeb1630 0x5562cfef0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff808b45b80 1 -- 172.21.15.79:0/3215034215 >> 172.21.15.79:0/3215034215 conn(0x5562cfdde1f0 msgr2=0x5562cfeb38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:19.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff7e1600700 0 client.26809 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:19.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff808b45b80 1 -- 172.21.15.79:0/3215034215 shutdown_connections 2024-06-03T02:33:19.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff808b45b80 1 -- 172.21.15.79:0/3215034215 wait complete. 2024-06-03T02:33:19.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.687+0000 7ff808b45b80 20 client.26809 trim_cache size 0 max 16384 2024-06-03T02:33:19.690 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:33:19.690 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:33:19.690 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "a0005db6-4f93-4465-946e-c3c4dee82996", 2024-06-03T02:33:19.690 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:33:19.690 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:33:19.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.688+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7dc0a2ea0 msgr2=0x7ff7dc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:19.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.688+0000 7ff7fe200700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7dc0a2ea0 0x7ff7dc0a5360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x5562cfeb2470 tx=0x7ff7e8009260 comp rx=0 tx=0).stop 2024-06-03T02:33:19.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8102540 msgr2=0x7ff7f8073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:19.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8102540 0x7ff7f8073be0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000b780 tx=0x7ff7f0004d00 comp rx=0 tx=0).stop 2024-06-03T02:33:19.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 shutdown_connections 2024-06-03T02:33:19.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f8107450 0x7ff7f806cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f8102540 0x7ff7f8073be0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff7dc0a2ea0 0x7ff7dc0a5360 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 --2- 172.21.15.79:0/1622845622 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff7f8102ee0 0x7ff7f8074120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:19.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 >> 172.21.15.79:0/1622845622 conn(0x7ff7f80795f0 msgr2=0x7ff7f8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:19.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 shutdown_connections 2024-06-03T02:33:19.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:19.689+0000 7ff7fe200700 1 -- 172.21.15.79:0/1622845622 wait complete. 2024-06-03T02:33:20.641 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:20 smithi079 ceph-mon[31819]: pgmap v914: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.6 MiB/s wr, 470 op/s 2024-06-03T02:33:20.641 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:20 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:33:20.642 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:20 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:33:20.642 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:20 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:33:20.642 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:20 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:33:20.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:33:20.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:20 smithi135 ceph-mon[36653]: pgmap v914: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.6 MiB/s wr, 470 op/s 2024-06-03T02:33:20.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:20 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:33:20.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:20 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:33:20.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:20 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:33:20.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:20 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:33:20.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:20 smithi123 ceph-mon[36640]: pgmap v914: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.6 MiB/s wr, 470 op/s 2024-06-03T02:33:20.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:20 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:33:20.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:20 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:33:20.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:20 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:33:20.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:20 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:33:21.421 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:33:22.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:22 smithi079 ceph-mon[31819]: pgmap v915: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.4 MiB/s wr, 476 op/s 2024-06-03T02:33:22.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:22 smithi135 ceph-mon[36653]: pgmap v915: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.4 MiB/s wr, 476 op/s 2024-06-03T02:33:22.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:22 smithi123 ceph-mon[36640]: pgmap v915: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.4 MiB/s wr, 476 op/s 2024-06-03T02:33:23.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 795.0 s, 433.4 tps, lat 232.246 ms stddev 268.077 2024-06-03T02:33:24.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:24 smithi123 ceph-mon[36640]: pgmap v916: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.3 MiB/s wr, 440 op/s 2024-06-03T02:33:24.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:24 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:33:24.998 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:24 smithi079 ceph-mon[31819]: pgmap v916: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.3 MiB/s wr, 440 op/s 2024-06-03T02:33:24.998 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:24 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:33:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:24 smithi135 ceph-mon[36653]: pgmap v916: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.3 MiB/s wr, 440 op/s 2024-06-03T02:33:25.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:24 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:33:26.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:26 smithi079 ceph-mon[31819]: pgmap v917: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.7 MiB/s wr, 503 op/s 2024-06-03T02:33:26.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:26 smithi123 ceph-mon[36640]: pgmap v917: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.7 MiB/s wr, 503 op/s 2024-06-03T02:33:27.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.011+0000 7feb2ee00700 1 -- 172.21.15.79:0/383627728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb20096210 msgr2=0x7feb200965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:27.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.011+0000 7feb2ee00700 1 --2- 172.21.15.79:0/383627728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb20096210 0x7feb200965f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feb14009990 tx=0x7feb14009ca0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.014+0000 7feb2ee00700 1 -- 172.21.15.79:0/383627728 shutdown_connections 2024-06-03T02:33:27.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.014+0000 7feb2ee00700 1 --2- 172.21.15.79:0/383627728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb2009b0f0 0x7feb200a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.014+0000 7feb2ee00700 1 --2- 172.21.15.79:0/383627728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb20096210 0x7feb200965f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.014+0000 7feb2ee00700 1 --2- 172.21.15.79:0/383627728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb20096bc0 0x7feb2009a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.014+0000 7feb2ee00700 1 -- 172.21.15.79:0/383627728 >> 172.21.15.79:0/383627728 conn(0x7feb2000b8e0 msgr2=0x7feb2000bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:27.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.014+0000 7feb2ee00700 1 -- 172.21.15.79:0/383627728 shutdown_connections 2024-06-03T02:33:27.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.014+0000 7feb2ee00700 1 -- 172.21.15.79:0/383627728 wait complete. 2024-06-03T02:33:27.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.016+0000 7feb2ee00700 1 Processor -- start 2024-06-03T02:33:27.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.017+0000 7feb2ee00700 1 -- start start 2024-06-03T02:33:27.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.019+0000 7feb2ee00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20096210 0x7feb20133080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.019+0000 7feb2ee00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb20096bc0 0x7feb201335c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.019+0000 7feb2da00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20096210 0x7feb20133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.020+0000 7feb2da00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20096210 0x7feb20133080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:37456/0 (socket says 172.21.15.79:37456) 2024-06-03T02:33:27.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.020+0000 7feb2da00700 1 -- 172.21.15.79:0/99561520 learned_addr learned my addr 172.21.15.79:0/99561520 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:27.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.020+0000 7feb2ee00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb2009b0f0 0x7feb200a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.020+0000 7feb2d000700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb20096bc0 0x7feb201335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.020+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feb200a60d0 con 0x7feb2009b0f0 2024-06-03T02:33:27.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.020+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb200a5f50 con 0x7feb20096210 2024-06-03T02:33:27.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.020+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb200a6250 con 0x7feb20096bc0 2024-06-03T02:33:27.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.020+0000 7feb2e400700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb2009b0f0 0x7feb200a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.021+0000 7feb2da00700 1 -- 172.21.15.79:0/99561520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb20096bc0 msgr2=0x7feb201335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:27.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.021+0000 7feb2da00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb20096bc0 0x7feb201335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.021+0000 7feb2da00700 1 -- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb2009b0f0 msgr2=0x7feb200a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:27.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.021+0000 7feb2da00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb2009b0f0 0x7feb200a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.021+0000 7feb2da00700 1 -- 172.21.15.79:0/99561520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb200a25e0 con 0x7feb20096210 2024-06-03T02:33:27.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.022+0000 7feb2e400700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb2009b0f0 0x7feb200a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:33:27.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.023+0000 7feb2da00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20096210 0x7feb20133080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feb14005910 tx=0x7feb140047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:27.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.023+0000 7feb26000700 1 -- 172.21.15.79:0/99561520 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14027070 con 0x7feb20096210 2024-06-03T02:33:27.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.023+0000 7feb26000700 1 -- 172.21.15.79:0/99561520 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb14022470 con 0x7feb20096210 2024-06-03T02:33:27.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.023+0000 7feb26000700 1 -- 172.21.15.79:0/99561520 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14017650 con 0x7feb20096210 2024-06-03T02:33:27.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.024+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb200a2870 con 0x7feb20096210 2024-06-03T02:33:27.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.024+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2012ebc0 con 0x7feb20096210 2024-06-03T02:33:27.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.031+0000 7feb3ad48b80 1 Processor -- start 2024-06-03T02:33:27.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.032+0000 7feb3ad48b80 1 -- start start 2024-06-03T02:33:27.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.032+0000 7feb26000700 1 -- 172.21.15.79:0/99561520 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7feb14009d10 con 0x7feb20096210 2024-06-03T02:33:27.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.035+0000 7feb26000700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7feb100a2ca0 0x7feb100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.035+0000 7feb26000700 1 -- 172.21.15.79:0/99561520 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7feb140cfeb0 con 0x7feb20096210 2024-06-03T02:33:27.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.035+0000 7feb2e400700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7feb100a2ca0 0x7feb100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.035+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb43d60 0x55fe1cb44140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.035+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb44680 0x55fe1cb4eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.036+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb4f530 0x55fe1cb51920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.036+0000 7feb3ad48b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fe1ca53a10 con 0x55fe1cb4f530 2024-06-03T02:33:27.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.036+0000 7feb3ad48b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fe1ca96b00 con 0x55fe1cb44680 2024-06-03T02:33:27.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.036+0000 7feb3ad48b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fe1ca518f0 con 0x55fe1cb43d60 2024-06-03T02:33:27.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.036+0000 7feb2d000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb44680 0x55fe1cb4eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.037+0000 7feb2da00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb43d60 0x55fe1cb44140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.037+0000 7feb2da00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb43d60 0x55fe1cb44140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44524/0 (socket says 172.21.15.79:44524) 2024-06-03T02:33:27.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.037+0000 7feb2da00700 1 -- 172.21.15.79:0/2579086870 learned_addr learned my addr 172.21.15.79:0/2579086870 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:27.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.038+0000 7feb2da00700 1 -- 172.21.15.79:0/2579086870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb44680 msgr2=0x55fe1cb4eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:27.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.038+0000 7feb2da00700 1 --2- 172.21.15.79:0/2579086870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb44680 0x55fe1cb4eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.038+0000 7feb2da00700 1 -- 172.21.15.79:0/2579086870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb4f530 msgr2=0x55fe1cb51920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:33:27.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.039+0000 7feb2e400700 1 --2- 172.21.15.79:0/2579086870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb4f530 0x55fe1cb51920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.039+0000 7feb2da00700 1 --2- 172.21.15.79:0/2579086870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb4f530 0x55fe1cb51920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.039+0000 7feb2da00700 1 -- 172.21.15.79:0/2579086870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe1ca3ef90 con 0x55fe1cb43d60 2024-06-03T02:33:27.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.039+0000 7feb2d000700 1 --2- 172.21.15.79:0/2579086870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb44680 0x55fe1cb4eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:33:27.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.039+0000 7feb2da00700 1 --2- 172.21.15.79:0/2579086870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb43d60 0x55fe1cb44140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feb140a4a40 tx=0x7feb14022d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4d1640beabcdcba server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:27.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.040+0000 7feb2e400700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7feb100a2ca0 0x7feb100a5160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feb280517a0 tx=0x7feb280507d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:27.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.040+0000 7feb1aa00700 1 -- 172.21.15.79:0/2579086870 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1409c390 con 0x55fe1cb43d60 2024-06-03T02:33:27.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.040+0000 7feb1aa00700 1 -- 172.21.15.79:0/2579086870 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb140a0740 con 0x55fe1cb43d60 2024-06-03T02:33:27.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.040+0000 7feb1aa00700 1 -- 172.21.15.79:0/2579086870 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140a4ae0 con 0x55fe1cb43d60 2024-06-03T02:33:27.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.041+0000 7feb3ad48b80 1 -- 172.21.15.79:0/2579086870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb43d60 msgr2=0x55fe1cb44140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:27.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.041+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/2579086870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb43d60 0x55fe1cb44140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feb140a4a40 tx=0x7feb14022d60 comp rx=0 tx=0).stop 2024-06-03T02:33:27.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.042+0000 7feb3ad48b80 1 -- 172.21.15.79:0/2579086870 shutdown_connections 2024-06-03T02:33:27.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.042+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/2579086870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb4f530 0x55fe1cb51920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.042+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/2579086870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb43d60 0x55fe1cb44140 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.042+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/2579086870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb44680 0x55fe1cb4eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.042+0000 7feb3ad48b80 1 -- 172.21.15.79:0/2579086870 >> 172.21.15.79:0/2579086870 conn(0x55fe1ca71170 msgr2=0x55fe1cb40280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:27.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.044+0000 7feb3ad48b80 1 -- 172.21.15.79:0/2579086870 shutdown_connections 2024-06-03T02:33:27.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.044+0000 7feb3ad48b80 1 -- 172.21.15.79:0/2579086870 wait complete. 2024-06-03T02:33:27.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.045+0000 7feb3ad48b80 1 -- start start 2024-06-03T02:33:27.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.046+0000 7feb3ad48b80 1 Processor -- start 2024-06-03T02:33:27.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.046+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.047+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb4bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.047+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 0x55fe1cb4c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.047+0000 7feb2d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb4bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.047+0000 7feb2da00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.047+0000 7feb2d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb4bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44528/0 (socket says 172.21.15.79:44528) 2024-06-03T02:33:27.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.047+0000 7feb2da00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56938/0 (socket says 172.21.15.79:56938) 2024-06-03T02:33:27.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.050+0000 7feb2e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 0x55fe1cb4c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.050+0000 7feb2e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 0x55fe1cb4c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:37476/0 (socket says 172.21.15.79:37476) 2024-06-03T02:33:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.050+0000 7feb2d000700 1 -- 172.21.15.79:0/3858717304 learned_addr learned my addr 172.21.15.79:0/3858717304 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.050+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55fe1ca32f60 con 0x55fe1cb43d60 2024-06-03T02:33:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.050+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fe1ca92ac0 con 0x55fe1cb4f530 2024-06-03T02:33:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.050+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fe1ca518f0 con 0x55fe1cb44680 2024-06-03T02:33:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.051+0000 7feb3ad48b80 10 client.0 _ll_register_callbacks cb 0x55fe1ca2c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.052+0000 7feb2d000700 1 -- 172.21.15.79:0/3858717304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 msgr2=0x55fe1cb4c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.052+0000 7feb2d000700 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 0x55fe1cb4c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.052+0000 7feb2d000700 1 -- 172.21.15.79:0/3858717304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 msgr2=0x55fe1cb4ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.052+0000 7feb2d000700 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.052+0000 7feb2d000700 1 -- 172.21.15.79:0/3858717304 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe1c9e58d0 con 0x55fe1cb44680 2024-06-03T02:33:27.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.055+0000 7feb0f400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:27.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.055+0000 7feb2d000700 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb4bfd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00f1e0 tx=0x7feb1c00f4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57fddebed85f8111 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:27.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.055+0000 7feb0f400700 1 -- 172.21.15.79:0/3858717304 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c018020 con 0x55fe1cb44680 2024-06-03T02:33:27.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.055+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb140a0740 con 0x55fe1cb44680 2024-06-03T02:33:27.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.056+0000 7feb0f400700 1 -- 172.21.15.79:0/3858717304 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7feb1c014460 con 0x55fe1cb44680 2024-06-03T02:33:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.058+0000 7feb0f400700 1 -- 172.21.15.79:0/3858717304 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb1c023050 con 0x55fe1cb44680 2024-06-03T02:33:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.058+0000 7feb3ad48b80 10 client.26821 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.058+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fe1cb6d4e0 con 0x55fe1cb44680 2024-06-03T02:33:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.058+0000 7feb0f400700 1 -- 172.21.15.79:0/3858717304 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb140a0740 con 0x55fe1cb44680 2024-06-03T02:33:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.059+0000 7feb0f400700 1 -- 172.21.15.79:0/3858717304 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feb1c013850 con 0x55fe1cb44680 2024-06-03T02:33:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.059+0000 7feb3ad48b80 10 client.26821 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.059+0000 7feb3ad48b80 10 client.26821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:33:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.059+0000 7feb3ad48b80 20 client.26821 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.059+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fe1cb6fac0 0x55fe1cb71f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.059+0000 7feb3ad48b80 4 client.26821 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:33:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.059+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe1caa4000 con 0x55fe1cb6fac0 2024-06-03T02:33:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.059+0000 7feb2da00700 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fe1cb6fac0 0x55fe1cb71f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.060+0000 7feb0f400700 10 client.26821 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:27.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.062+0000 7feb2da00700 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fe1cb6fac0 0x55fe1cb71f80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad0fed192d03066 server_cookie=747db4bdffd538a8 in_seq=0 out_seq=0 2024-06-03T02:33:27.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.063+0000 7feb0f400700 1 -- 172.21.15.79:0/3858717304 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe1caa4000 con 0x55fe1cb6fac0 2024-06-03T02:33:27.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.063+0000 7feb0f400700 10 client.26821 handle_command_reply: tid=0 2024-06-03T02:33:27.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.063+0000 7feb3ad48b80 1 client.26821 shutdown 2024-06-03T02:33:27.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.064+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 msgr2=0x55fe1cb4bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:27.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.064+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb4bfd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00f1e0 tx=0x7feb1c00f4f0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.064+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 shutdown_connections 2024-06-03T02:33:27.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.064+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 0x55fe1cb4c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.064+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fe1cb6fac0 0x55fe1cb71f80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.064+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.064+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3858717304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb4bfd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.064+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 >> 172.21.15.79:0/3858717304 conn(0x55fe1ca71170 msgr2=0x55fe1cb3fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:27.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.064+0000 7feb0f400700 0 client.26821 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:27.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.068+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 shutdown_connections 2024-06-03T02:33:27.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.068+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3858717304 wait complete. 2024-06-03T02:33:27.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.068+0000 7feb3ad48b80 20 client.26821 trim_cache size 0 max 16384 2024-06-03T02:33:27.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:26 smithi135 ceph-mon[36653]: pgmap v917: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.7 MiB/s wr, 503 op/s 2024-06-03T02:33:27.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.119+0000 7feb3ad48b80 1 Processor -- start 2024-06-03T02:33:27.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.120+0000 7feb3ad48b80 1 -- start start 2024-06-03T02:33:27.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.120+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.120+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb44680 0x55fe1cb403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.120+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb4f530 0x55fe1cb77890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.120+0000 7feb3ad48b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feb20150a40 con 0x55fe1cb44680 2024-06-03T02:33:27.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.120+0000 7feb3ad48b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb20152580 con 0x55fe1cb43d60 2024-06-03T02:33:27.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.120+0000 7feb3ad48b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb20151110 con 0x55fe1cb4f530 2024-06-03T02:33:27.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.121+0000 7feb2da00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.121+0000 7feb2da00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:37478/0 (socket says 172.21.15.79:37478) 2024-06-03T02:33:27.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.121+0000 7feb2da00700 1 -- 172.21.15.79:0/3133678409 learned_addr learned my addr 172.21.15.79:0/3133678409 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:27.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.121+0000 7feb2e400700 1 --2- 172.21.15.79:0/3133678409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb4f530 0x55fe1cb77890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.122+0000 7feb2e400700 1 -- 172.21.15.79:0/3133678409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb43d60 msgr2=0x55fe1cb4c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:27.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.122+0000 7feb2e400700 1 --2- 172.21.15.79:0/3133678409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.122+0000 7feb2e400700 1 -- 172.21.15.79:0/3133678409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb44680 msgr2=0x55fe1cb403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:27.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.122+0000 7feb2e400700 1 --2- 172.21.15.79:0/3133678409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb44680 0x55fe1cb403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.122+0000 7feb2e400700 1 -- 172.21.15.79:0/3133678409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe1cb45bb0 con 0x55fe1cb4f530 2024-06-03T02:33:27.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.123+0000 7feb2da00700 1 --2- 172.21.15.79:0/3133678409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:33:27.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.123+0000 7feb2e400700 1 --2- 172.21.15.79:0/3133678409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb4f530 0x55fe1cb77890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb28068f20 tx=0x7feb28081000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d169a9111a11f935 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:27.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.127+0000 7feb1a000700 1 -- 172.21.15.79:0/3133678409 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2808f070 con 0x55fe1cb4f530 2024-06-03T02:33:27.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.127+0000 7feb1a000700 1 -- 172.21.15.79:0/3133678409 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb2806f070 con 0x55fe1cb4f530 2024-06-03T02:33:27.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.127+0000 7feb1a000700 1 -- 172.21.15.79:0/3133678409 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28071070 con 0x55fe1cb4f530 2024-06-03T02:33:27.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.129+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3133678409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb4f530 msgr2=0x55fe1cb77890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:27.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.129+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3133678409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb4f530 0x55fe1cb77890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb28068f20 tx=0x7feb28081000 comp rx=0 tx=0).stop 2024-06-03T02:33:27.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.129+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3133678409 shutdown_connections 2024-06-03T02:33:27.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.129+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3133678409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb4f530 0x55fe1cb77890 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.129+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3133678409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb43d60 0x55fe1cb4c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.129+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3133678409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb44680 0x55fe1cb403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.129+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3133678409 >> 172.21.15.79:0/3133678409 conn(0x55fe1ca71170 msgr2=0x55fe1cb4bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:27.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.129+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3133678409 shutdown_connections 2024-06-03T02:33:27.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.130+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3133678409 wait complete. 2024-06-03T02:33:27.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.131+0000 7feb3ad48b80 1 -- start start 2024-06-03T02:33:27.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.131+0000 7feb3ad48b80 1 Processor -- start 2024-06-03T02:33:27.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.133+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 0x55fe1cb83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.133+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb81120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb3ad48b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 0x55fe1cb81660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb3ad48b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feb20151eb0 con 0x55fe1cb43d60 2024-06-03T02:33:27.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb2d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb81120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb2d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb81120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44558/0 (socket says 172.21.15.79:44558) 2024-06-03T02:33:27.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb2d000700 1 -- 172.21.15.79:0/3481603893 learned_addr learned my addr 172.21.15.79:0/3481603893 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:27.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb2da00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 0x55fe1cb83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb201517e0 con 0x55fe1cb4f530 2024-06-03T02:33:27.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb2014e9e0 con 0x55fe1cb44680 2024-06-03T02:33:27.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb2e400700 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 0x55fe1cb81660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.134+0000 7feb3ad48b80 10 client.0 _ll_register_callbacks cb 0x55fe1ca2c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:27.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.135+0000 7feb2d000700 1 -- 172.21.15.79:0/3481603893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 msgr2=0x55fe1cb81660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:27.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.135+0000 7feb2d000700 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 0x55fe1cb81660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.135+0000 7feb2d000700 1 -- 172.21.15.79:0/3481603893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 msgr2=0x55fe1cb83d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:27.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.135+0000 7feb2d000700 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 0x55fe1cb83d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.135+0000 7feb2d000700 1 -- 172.21.15.79:0/3481603893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe1cb81e70 con 0x55fe1cb44680 2024-06-03T02:33:27.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.135+0000 7feb2d000700 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb81120 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7feb1c001eb0 tx=0x7feb1c006e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb9c95a35bfff5a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:27.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.135+0000 7feb0f400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:27.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.136+0000 7feb0f400700 1 -- 172.21.15.79:0/3481603893 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c02b070 con 0x55fe1cb44680 2024-06-03T02:33:27.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.136+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2806f070 con 0x55fe1cb44680 2024-06-03T02:33:27.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.136+0000 7feb0f400700 1 -- 172.21.15.79:0/3481603893 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7feb1c013e10 con 0x55fe1cb44680 2024-06-03T02:33:27.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.136+0000 7feb0f400700 1 -- 172.21.15.79:0/3481603893 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb1c037050 con 0x55fe1cb44680 2024-06-03T02:33:27.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.136+0000 7feb3ad48b80 10 client.26827 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:27.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.136+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fe1cbda750 con 0x55fe1cb44680 2024-06-03T02:33:27.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.137+0000 7feb0f400700 1 -- 172.21.15.79:0/3481603893 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb2806f070 con 0x55fe1cb44680 2024-06-03T02:33:27.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.138+0000 7feb0f400700 1 -- 172.21.15.79:0/3481603893 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feb1c01cc70 con 0x55fe1cb44680 2024-06-03T02:33:27.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.139+0000 7feb3ad48b80 10 client.26827 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:27.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.139+0000 7feb3ad48b80 10 client.26827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:33:27.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.139+0000 7feb3ad48b80 20 client.26827 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:27.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.139+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fe1cbdae60 0x55fe1cbdd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:27.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.139+0000 7feb3ad48b80 4 client.26827 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:33:27.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.139+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe1cb3efa0 con 0x55fe1cbdae60 2024-06-03T02:33:27.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.140+0000 7feb2da00700 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fe1cbdae60 0x55fe1cbdd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:27.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.140+0000 7feb2da00700 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fe1cbdae60 0x55fe1cbdd280 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=179de9a622361782 server_cookie=71a07dc0f8bfd996 in_seq=0 out_seq=0 2024-06-03T02:33:27.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.140+0000 7feb0f400700 10 client.26827 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:27.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.141+0000 7feb0f400700 1 -- 172.21.15.79:0/3481603893 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fe1cb3efa0 con 0x55fe1cbdae60 2024-06-03T02:33:27.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.141+0000 7feb0f400700 10 client.26827 handle_command_reply: tid=0 2024-06-03T02:33:27.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb3ad48b80 1 client.26827 shutdown 2024-06-03T02:33:27.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 msgr2=0x55fe1cb81120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:27.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb81120 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7feb1c001eb0 tx=0x7feb1c006e60 comp rx=0 tx=0).stop 2024-06-03T02:33:27.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 shutdown_connections 2024-06-03T02:33:27.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe1cb4f530 0x55fe1cb81660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x55fe1cbdae60 0x55fe1cbdd280 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55fe1cb43d60 0x55fe1cb83d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb3ad48b80 1 --2- 172.21.15.79:0/3481603893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe1cb44680 0x55fe1cb81120 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb0f400700 0 client.26827 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:33:27.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.144+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 >> 172.21.15.79:0/3481603893 conn(0x55fe1ca71170 msgr2=0x55fe1cb55f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:27.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.145+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 shutdown_connections 2024-06-03T02:33:27.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.145+0000 7feb3ad48b80 1 -- 172.21.15.79:0/3481603893 wait complete. 2024-06-03T02:33:27.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.145+0000 7feb3ad48b80 20 client.26827 trim_cache size 0 max 16384 2024-06-03T02:33:27.145 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:33:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:33:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:33:27.146 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:33:27.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.146+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7feb100a2ca0 msgr2=0x7feb100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.146+0000 7feb2ee00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7feb100a2ca0 0x7feb100a5160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feb280517a0 tx=0x7feb280507d0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.146+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20096210 msgr2=0x7feb20133080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.146+0000 7feb2ee00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20096210 0x7feb20133080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feb14005910 tx=0x7feb140047b0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.147+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 shutdown_connections 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.147+0000 7feb2ee00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7feb100a2ca0 0x7feb100a5160 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.147+0000 7feb2ee00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb2009b0f0 0x7feb200a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.147+0000 7feb2ee00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb20096210 0x7feb20133080 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.147+0000 7feb2ee00700 1 --2- 172.21.15.79:0/99561520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb20096bc0 0x7feb201335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.147+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 >> 172.21.15.79:0/99561520 conn(0x7feb2000b8e0 msgr2=0x7feb20094340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.147+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 shutdown_connections 2024-06-03T02:33:27.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:27.147+0000 7feb2ee00700 1 -- 172.21.15.79:0/99561520 wait complete. 2024-06-03T02:33:28.300 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:33:28.300 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:33:28.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 800.0 s, 479.7 tps, lat 208.162 ms stddev 286.394 2024-06-03T02:33:28.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:28 smithi123 ceph-mon[36640]: pgmap v918: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 3.9 MiB/s wr, 412 op/s 2024-06-03T02:33:29.030 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:33:29.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:28 smithi079 ceph-mon[31819]: pgmap v918: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 3.9 MiB/s wr, 412 op/s 2024-06-03T02:33:29.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:28 smithi135 ceph-mon[36653]: pgmap v918: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 3.9 MiB/s wr, 412 op/s 2024-06-03T02:33:30.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:30 smithi123 ceph-mon[36640]: pgmap v919: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.1 MiB/s wr, 437 op/s 2024-06-03T02:33:31.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:30 smithi079 ceph-mon[31819]: pgmap v919: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.1 MiB/s wr, 437 op/s 2024-06-03T02:33:31.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:30 smithi135 ceph-mon[36653]: pgmap v919: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.1 MiB/s wr, 437 op/s 2024-06-03T02:33:32.850 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:32 smithi079 ceph-mon[31819]: pgmap v920: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.5 MiB/s wr, 482 op/s 2024-06-03T02:33:32.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:32 smithi123 ceph-mon[36640]: pgmap v920: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.5 MiB/s wr, 482 op/s 2024-06-03T02:33:33.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:32 smithi135 ceph-mon[36653]: pgmap v920: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.5 MiB/s wr, 482 op/s 2024-06-03T02:33:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 805.0 s, 603.9 tps, lat 160.221 ms stddev 190.218 2024-06-03T02:33:33.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.562+0000 7fb788800700 1 -- 172.21.15.79:0/10298094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7801155e0 msgr2=0x7fb7801179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:33.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.562+0000 7fb788800700 1 --2- 172.21.15.79:0/10298094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7801155e0 0x7fb7801179d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00f490 tx=0x7fb77c00f7a0 comp rx=0 tx=0).stop 2024-06-03T02:33:33.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.565+0000 7fb788800700 1 -- 172.21.15.79:0/10298094 shutdown_connections 2024-06-03T02:33:33.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.565+0000 7fb788800700 1 --2- 172.21.15.79:0/10298094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7801155e0 0x7fb7801179d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:33.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.565+0000 7fb788800700 1 --2- 172.21.15.79:0/10298094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb780109490 0x7fb780109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:33.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.565+0000 7fb788800700 1 --2- 172.21.15.79:0/10298094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb780109db0 0x7fb7801150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:33.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.565+0000 7fb788800700 1 -- 172.21.15.79:0/10298094 >> 172.21.15.79:0/10298094 conn(0x7fb78006d600 msgr2=0x7fb78006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:33.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.566+0000 7fb788800700 1 -- 172.21.15.79:0/10298094 shutdown_connections 2024-06-03T02:33:33.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.566+0000 7fb788800700 1 -- 172.21.15.79:0/10298094 wait complete. 2024-06-03T02:33:33.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.567+0000 7fb788800700 1 Processor -- start 2024-06-03T02:33:33.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.568+0000 7fb788800700 1 -- start start 2024-06-03T02:33:33.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.570+0000 7fb788800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb780109490 0x7fb7801ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:33.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.570+0000 7fb788800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb780109db0 0x7fb7801adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:33.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.570+0000 7fb788800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7801155e0 0x7fb7801a7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:33.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.570+0000 7fb788800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb780121680 con 0x7fb780109db0 2024-06-03T02:33:33.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.570+0000 7fb788800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb780121500 con 0x7fb780109490 2024-06-03T02:33:33.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.570+0000 7fb788800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb780121800 con 0x7fb7801155e0 2024-06-03T02:33:33.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.571+0000 7fb786800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7801155e0 0x7fb7801a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:33.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.571+0000 7fb786800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7801155e0 0x7fb7801a7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44570/0 (socket says 172.21.15.79:44570) 2024-06-03T02:33:33.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.571+0000 7fb786800700 1 -- 172.21.15.79:0/3987529521 learned_addr learned my addr 172.21.15.79:0/3987529521 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:33.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.571+0000 7fb785e00700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb780109490 0x7fb7801ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:33.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.572+0000 7fb785e00700 1 -- 172.21.15.79:0/3987529521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7801155e0 msgr2=0x7fb7801a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:33.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.572+0000 7fb785e00700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7801155e0 0x7fb7801a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:33.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.572+0000 7fb785e00700 1 -- 172.21.15.79:0/3987529521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb780109db0 msgr2=0x7fb7801adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:33.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.572+0000 7fb785e00700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb780109db0 0x7fb7801adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:33.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.572+0000 7fb785e00700 1 -- 172.21.15.79:0/3987529521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7801a8080 con 0x7fb780109490 2024-06-03T02:33:33.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.572+0000 7fb785e00700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb780109490 0x7fb7801ad710 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb77400d950 tx=0x7fb77400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:33.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.575+0000 7fb77aa00700 1 -- 172.21.15.79:0/3987529521 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774017070 con 0x7fb780109490 2024-06-03T02:33:33.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.575+0000 7fb77aa00700 1 -- 172.21.15.79:0/3987529521 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb774009e10 con 0x7fb780109490 2024-06-03T02:33:33.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.575+0000 7fb77aa00700 1 -- 172.21.15.79:0/3987529521 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77400f3c0 con 0x7fb780109490 2024-06-03T02:33:33.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.575+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7801a8370 con 0x7fb780109490 2024-06-03T02:33:33.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.575+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7801b45e0 con 0x7fb780109490 2024-06-03T02:33:33.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.580+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb780069180 con 0x7fb780109490 2024-06-03T02:33:33.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.584+0000 7fb77aa00700 1 -- 172.21.15.79:0/3987529521 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb77401a070 con 0x7fb780109490 2024-06-03T02:33:33.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.592+0000 7fb77aa00700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7680a2da0 0x7fb7680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:33.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.592+0000 7fb77aa00700 1 -- 172.21.15.79:0/3987529521 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb7740ceca0 con 0x7fb780109490 2024-06-03T02:33:33.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.593+0000 7fb77aa00700 1 -- 172.21.15.79:0/3987529521 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb7740d3370 con 0x7fb780109490 2024-06-03T02:33:33.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.601+0000 7fb785400700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7680a2da0 0x7fb7680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:33.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:33.625+0000 7fb785400700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7680a2da0 0x7fb7680a5260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb780073be0 tx=0x7fb770009690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:34.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.053+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb78002dca0 con 0x7fb780109490 2024-06-03T02:33:34.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.054+0000 7fb77aa00700 1 -- 172.21.15.79:0/3987529521 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7fb77409b8d0 con 0x7fb780109490 2024-06-03T02:33:34.055 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:33:34.055 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:33:34.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7680a2da0 msgr2=0x7fb7680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:34.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7680a2da0 0x7fb7680a5260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb780073be0 tx=0x7fb770009690 comp rx=0 tx=0).stop 2024-06-03T02:33:34.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb780109490 msgr2=0x7fb7801ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:34.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb780109490 0x7fb7801ad710 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb77400d950 tx=0x7fb77400dc60 comp rx=0 tx=0).stop 2024-06-03T02:33:34.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 shutdown_connections 2024-06-03T02:33:34.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7801155e0 0x7fb7801a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:34.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb780109490 0x7fb7801ad710 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:34.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb7680a2da0 0x7fb7680a5260 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:34.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 --2- 172.21.15.79:0/3987529521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb780109db0 0x7fb7801adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:34.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 >> 172.21.15.79:0/3987529521 conn(0x7fb78006d600 msgr2=0x7fb780071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:34.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 shutdown_connections 2024-06-03T02:33:34.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:34.056+0000 7fb788800700 1 -- 172.21.15.79:0/3987529521 wait complete. 2024-06-03T02:33:34.058 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:33:34.633 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:34 smithi079 ceph-mon[31819]: pgmap v921: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-06-03T02:33:34.948 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:34 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3987529521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:33:34.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:34 smithi123 ceph-mon[36640]: pgmap v921: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-06-03T02:33:34.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:34 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3987529521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:33:34.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:33:35.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:34 smithi135 ceph-mon[36653]: pgmap v921: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-06-03T02:33:35.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:34 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3987529521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:33:35.736 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:33:36.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:36 smithi123 ceph-mon[36640]: pgmap v922: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 5.5 MiB/s wr, 599 op/s 2024-06-03T02:33:37.011 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:36 smithi079 ceph-mon[31819]: pgmap v922: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 5.5 MiB/s wr, 599 op/s 2024-06-03T02:33:37.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:36 smithi135 ceph-mon[36653]: pgmap v922: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 5.5 MiB/s wr, 599 op/s 2024-06-03T02:33:38.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 810.0 s, 519.4 tps, lat 188.624 ms stddev 244.114 2024-06-03T02:33:38.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:38 smithi123 ceph-mon[36640]: pgmap v923: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 4.8 MiB/s wr, 513 op/s 2024-06-03T02:33:39.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:38 smithi079 ceph-mon[31819]: pgmap v923: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 4.8 MiB/s wr, 513 op/s 2024-06-03T02:33:39.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:38 smithi135 ceph-mon[36653]: pgmap v923: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 4.8 MiB/s wr, 513 op/s 2024-06-03T02:33:40.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:40 smithi079 ceph-mon[31819]: pgmap v924: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2024-06-03T02:33:40.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:40 smithi123 ceph-mon[36640]: pgmap v924: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2024-06-03T02:33:41.097 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:40 smithi135 ceph-mon[36653]: pgmap v924: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2024-06-03T02:33:41.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.864+0000 7f81a4e00700 1 -- 172.21.15.79:0/1445972910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81a0101d50 msgr2=0x7f81a010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:41.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.864+0000 7f81a4e00700 1 --2- 172.21.15.79:0/1445972910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81a0101d50 0x7f81a010e240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8198009990 tx=0x7f8198009ca0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.865+0000 7f81a4e00700 1 -- 172.21.15.79:0/1445972910 shutdown_connections 2024-06-03T02:33:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.865+0000 7f81a4e00700 1 --2- 172.21.15.79:0/1445972910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81a0101d50 0x7f81a010e240 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.865+0000 7f81a4e00700 1 --2- 172.21.15.79:0/1445972910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81a0107400 0x7f81a01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.865+0000 7f81a4e00700 1 --2- 172.21.15.79:0/1445972910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a0101390 0x7f81a0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.865+0000 7f81a4e00700 1 -- 172.21.15.79:0/1445972910 >> 172.21.15.79:0/1445972910 conn(0x7f81a0078340 msgr2=0x7f81a0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:41.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.865+0000 7f81a4e00700 1 -- 172.21.15.79:0/1445972910 shutdown_connections 2024-06-03T02:33:41.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.865+0000 7f81a4e00700 1 -- 172.21.15.79:0/1445972910 wait complete. 2024-06-03T02:33:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.868+0000 7f81a4e00700 1 Processor -- start 2024-06-03T02:33:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.868+0000 7f81a4e00700 1 -- start start 2024-06-03T02:33:41.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.869+0000 7f81a4e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81a0101390 0x7f81a0198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.869+0000 7f81a4e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a0101d50 0x7f81a01994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.869+0000 7f81a4e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81a0107400 0x7f81a019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.870+0000 7f8195400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a0101d50 0x7f81a01994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.870+0000 7f8195400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a0101d50 0x7f81a01994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:37942/0 (socket says 172.21.15.79:37942) 2024-06-03T02:33:41.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.870+0000 7f819de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81a0101390 0x7f81a0198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.870+0000 7f819de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81a0101390 0x7f81a0198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47042/0 (socket says 172.21.15.79:47042) 2024-06-03T02:33:41.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.872+0000 7f8195400700 1 -- 172.21.15.79:0/551049741 learned_addr learned my addr 172.21.15.79:0/551049741 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:41.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.872+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0111800 con 0x7f81a0101390 2024-06-03T02:33:41.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.872+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0111680 con 0x7f81a0101d50 2024-06-03T02:33:41.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.872+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0111980 con 0x7f81a0107400 2024-06-03T02:33:41.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.872+0000 7f819e800700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81a0107400 0x7f81a019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.873+0000 7f819e800700 1 -- 172.21.15.79:0/551049741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a0101d50 msgr2=0x7f81a01994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:41.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.873+0000 7f819e800700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a0101d50 0x7f81a01994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.873+0000 7f819e800700 1 -- 172.21.15.79:0/551049741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81a0101390 msgr2=0x7f81a0198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:41.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.873+0000 7f819e800700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81a0101390 0x7f81a0198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.873+0000 7f819e800700 1 -- 172.21.15.79:0/551049741 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a019df20 con 0x7f81a0107400 2024-06-03T02:33:41.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.874+0000 7f819e800700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81a0107400 0x7f81a019d7e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8198005e50 tx=0x7f8198003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:41.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.875+0000 7f8197400700 1 -- 172.21.15.79:0/551049741 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198026070 con 0x7f81a0107400 2024-06-03T02:33:41.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.875+0000 7f8197400700 1 -- 172.21.15.79:0/551049741 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81980040f0 con 0x7f81a0107400 2024-06-03T02:33:41.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.875+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a019e1b0 con 0x7f81a0107400 2024-06-03T02:33:41.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.875+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a0108210 con 0x7f81a0107400 2024-06-03T02:33:41.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.875+0000 7f8197400700 1 -- 172.21.15.79:0/551049741 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198021410 con 0x7f81a0107400 2024-06-03T02:33:41.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.881+0000 7f8197400700 1 -- 172.21.15.79:0/551049741 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8198003c10 con 0x7f81a0107400 2024-06-03T02:33:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.884+0000 7f8197400700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81840a2d70 0x7f81840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.884+0000 7f8197400700 1 -- 172.21.15.79:0/551049741 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f81980d01a0 con 0x7f81a0107400 2024-06-03T02:33:41.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.885+0000 7f81af6abb80 1 Processor -- start 2024-06-03T02:33:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.885+0000 7f819de00700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81840a2d70 0x7f81840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.886+0000 7f81af6abb80 1 -- start start 2024-06-03T02:33:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.888+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a550dee40 0x563a55162070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.888+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55162a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.888+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.889+0000 7f81af6abb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563a55071a10 con 0x563a550dee40 2024-06-03T02:33:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.889+0000 7f81af6abb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a550b4b00 con 0x563a551625b0 2024-06-03T02:33:41.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.889+0000 7f81af6abb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563a5506f8f0 con 0x563a5516d300 2024-06-03T02:33:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.889+0000 7f819e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.889+0000 7f819e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46242/0 (socket says 172.21.15.79:46242) 2024-06-03T02:33:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.889+0000 7f819e800700 1 -- 172.21.15.79:0/2634430061 learned_addr learned my addr 172.21.15.79:0/2634430061 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.889+0000 7f819de00700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81840a2d70 0x7f81840a5230 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8188009910 tx=0x7f8188008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.889+0000 7f819de00700 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a550dee40 0x563a55162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.889+0000 7f8195400700 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55162a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.890+0000 7f8195400700 1 -- 172.21.15.79:0/2634430061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 msgr2=0x563a5516f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.890+0000 7f8195400700 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.890+0000 7f8195400700 1 -- 172.21.15.79:0/2634430061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a550dee40 msgr2=0x563a55162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.890+0000 7f8195400700 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a550dee40 0x563a55162070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.890+0000 7f8195400700 1 -- 172.21.15.79:0/2634430061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5505cf90 con 0x563a551625b0 2024-06-03T02:33:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.890+0000 7f819e800700 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:33:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.890+0000 7f8195400700 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55162a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8190009f80 tx=0x7f8190007630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9592017d72fa0bbe server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.891+0000 7f818f400700 1 -- 172.21.15.79:0/2634430061 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190019070 con 0x563a551625b0 2024-06-03T02:33:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.892+0000 7f818f400700 1 -- 172.21.15.79:0/2634430061 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8190014ae0 con 0x563a551625b0 2024-06-03T02:33:41.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.892+0000 7f818f400700 1 -- 172.21.15.79:0/2634430061 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190013b80 con 0x563a551625b0 2024-06-03T02:33:41.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.893+0000 7f81af6abb80 1 -- 172.21.15.79:0/2634430061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 msgr2=0x563a55162a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:41.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.893+0000 7f81af6abb80 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55162a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8190009f80 tx=0x7f8190007630 comp rx=0 tx=0).stop 2024-06-03T02:33:41.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.893+0000 7f81af6abb80 1 -- 172.21.15.79:0/2634430061 shutdown_connections 2024-06-03T02:33:41.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.893+0000 7f81af6abb80 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.893+0000 7f81af6abb80 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a550dee40 0x563a55162070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.893+0000 7f81af6abb80 1 --2- 172.21.15.79:0/2634430061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55162a30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.893+0000 7f81af6abb80 1 -- 172.21.15.79:0/2634430061 >> 172.21.15.79:0/2634430061 conn(0x563a5508f170 msgr2=0x563a5515e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:41.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.893+0000 7f81af6abb80 1 -- 172.21.15.79:0/2634430061 shutdown_connections 2024-06-03T02:33:41.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.893+0000 7f81af6abb80 1 -- 172.21.15.79:0/2634430061 wait complete. 2024-06-03T02:33:41.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.894+0000 7f81af6abb80 1 -- start start 2024-06-03T02:33:41.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.894+0000 7f81af6abb80 1 Processor -- start 2024-06-03T02:33:41.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a5516c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55169c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a5516a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f81af6abb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563a55050f60 con 0x563a5516d300 2024-06-03T02:33:41.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f81af6abb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a550b0ac0 con 0x563a551625b0 2024-06-03T02:33:41.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f81af6abb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563a5516ffc0 con 0x563a550dee40 2024-06-03T02:33:41.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f81af6abb80 10 client.0 _ll_register_callbacks cb 0x563a5504e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:41.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f819de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a5516c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f819de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a5516c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46256/0 (socket says 172.21.15.79:46256) 2024-06-03T02:33:41.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.895+0000 7f819de00700 1 -- 172.21.15.79:0/1568795854 learned_addr learned my addr 172.21.15.79:0/1568795854 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:41.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.896+0000 7f819e800700 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a5516a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.896+0000 7f819de00700 1 -- 172.21.15.79:0/1568795854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 msgr2=0x563a55169c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:33:41.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.896+0000 7f819de00700 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55169c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.896+0000 7f819de00700 1 -- 172.21.15.79:0/1568795854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 msgr2=0x563a5516a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:41.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.896+0000 7f819de00700 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a5516a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.896+0000 7f819de00700 1 -- 172.21.15.79:0/1568795854 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a550038d0 con 0x563a550dee40 2024-06-03T02:33:41.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.899+0000 7f8183e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:41.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.899+0000 7f819de00700 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a5516c980 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f818800dd60 tx=0x7f8188004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d240aadfde256ce8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:41.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.899+0000 7f8183e00700 1 -- 172.21.15.79:0/1568795854 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188014e60 con 0x563a550dee40 2024-06-03T02:33:41.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.899+0000 7f81af6abb80 1 -- 172.21.15.79:0/1568795854 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8190014ae0 con 0x563a550dee40 2024-06-03T02:33:41.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.900+0000 7f8183e00700 1 -- 172.21.15.79:0/1568795854 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f818800f040 con 0x563a550dee40 2024-06-03T02:33:41.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.900+0000 7f8183e00700 1 -- 172.21.15.79:0/1568795854 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8188014e60 con 0x563a550dee40 2024-06-03T02:33:41.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.900+0000 7f81af6abb80 10 client.26842 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:41.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.901+0000 7f81af6abb80 1 -- 172.21.15.79:0/1568795854 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a55178760 con 0x563a550dee40 2024-06-03T02:33:41.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.902+0000 7f8183e00700 1 -- 172.21.15.79:0/1568795854 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8190014ae0 con 0x563a550dee40 2024-06-03T02:33:41.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.902+0000 7f8183e00700 1 -- 172.21.15.79:0/1568795854 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8188013070 con 0x563a550dee40 2024-06-03T02:33:41.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.903+0000 7f81af6abb80 10 client.26842 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:41.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.903+0000 7f81af6abb80 10 client.26842 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:33:41.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.903+0000 7f81af6abb80 20 client.26842 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:41.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.903+0000 7f81af6abb80 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x563a5518de50 0x563a55190240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.903+0000 7f81af6abb80 4 client.26842 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:33:41.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.903+0000 7f81af6abb80 1 -- 172.21.15.79:0/1568795854 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a550c2000 con 0x563a5518de50 2024-06-03T02:33:41.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.904+0000 7f8195400700 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x563a5518de50 0x563a55190240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.905+0000 7f8195400700 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x563a5518de50 0x563a55190240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=601143249a5b443 server_cookie=33921dd22d7fcce6 in_seq=0 out_seq=0 2024-06-03T02:33:41.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.905+0000 7f8183e00700 10 client.26842 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:33:41.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.906+0000 7f8183e00700 1 -- 172.21.15.79:0/1568795854 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563a550c2000 con 0x563a5518de50 2024-06-03T02:33:41.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.906+0000 7f8183e00700 10 client.26842 handle_command_reply: tid=0 2024-06-03T02:33:41.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.906+0000 7f81af6abb80 1 client.26842 shutdown 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.908+0000 7f81af6abb80 1 -- 172.21.15.79:0/1568795854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 msgr2=0x563a5516c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.908+0000 7f81af6abb80 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a5516c980 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f818800dd60 tx=0x7f8188004770 comp rx=0 tx=0).stop 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.911+0000 7f81af6abb80 1 -- 172.21.15.79:0/1568795854 shutdown_connections 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.911+0000 7f81af6abb80 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x563a5518de50 0x563a55190240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.911+0000 7f8183e00700 0 client.26842 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.911+0000 7f81af6abb80 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a5516a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.911+0000 7f81af6abb80 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a5516c980 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.911+0000 7f81af6abb80 1 --2- 172.21.15.79:0/1568795854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55169c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.911+0000 7f81af6abb80 1 -- 172.21.15.79:0/1568795854 >> 172.21.15.79:0/1568795854 conn(0x563a5508f170 msgr2=0x563a5515d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.911+0000 7f81af6abb80 1 -- 172.21.15.79:0/1568795854 shutdown_connections 2024-06-03T02:33:41.914 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.912+0000 7f81af6abb80 1 -- 172.21.15.79:0/1568795854 wait complete. 2024-06-03T02:33:41.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.912+0000 7f81af6abb80 20 client.26842 trim_cache size 0 max 16384 2024-06-03T02:33:41.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.946+0000 7f81af6abb80 1 Processor -- start 2024-06-03T02:33:41.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.946+0000 7f81af6abb80 1 -- start start 2024-06-03T02:33:41.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.947+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a550dee40 0x563a5516f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.947+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a551625b0 0x563a5516e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.947+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.947+0000 7f81af6abb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f817c00eca0 con 0x563a551625b0 2024-06-03T02:33:41.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.947+0000 7f81af6abb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f817c0107e0 con 0x563a550dee40 2024-06-03T02:33:41.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.947+0000 7f81af6abb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f817c00f370 con 0x563a5516d300 2024-06-03T02:33:41.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.947+0000 7f819e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.947+0000 7f819e800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:46272/0 (socket says 172.21.15.79:46272) 2024-06-03T02:33:41.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.947+0000 7f819e800700 1 -- 172.21.15.79:0/400012586 learned_addr learned my addr 172.21.15.79:0/400012586 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:41.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.950+0000 7f8195400700 1 --2- 172.21.15.79:0/400012586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a551625b0 0x563a5516e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.951+0000 7f819de00700 1 --2- 172.21.15.79:0/400012586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a550dee40 0x563a5516f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.951+0000 7f819e800700 1 -- 172.21.15.79:0/400012586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a550dee40 msgr2=0x563a5516f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:41.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.951+0000 7f819e800700 1 --2- 172.21.15.79:0/400012586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a550dee40 0x563a5516f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.951+0000 7f819e800700 1 -- 172.21.15.79:0/400012586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a551625b0 msgr2=0x563a5516e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:41.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.951+0000 7f819e800700 1 --2- 172.21.15.79:0/400012586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a551625b0 0x563a5516e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.951+0000 7f819e800700 1 -- 172.21.15.79:0/400012586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5516a540 con 0x563a5516d300 2024-06-03T02:33:41.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.952+0000 7f819e800700 1 --2- 172.21.15.79:0/400012586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516c4c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f81980a3100 tx=0x7f8198087040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e57e47adf6c8f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:41.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.952+0000 7f818ea00700 1 -- 172.21.15.79:0/400012586 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81980053d0 con 0x563a5516d300 2024-06-03T02:33:41.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.953+0000 7f818ea00700 1 -- 172.21.15.79:0/400012586 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81980870b0 con 0x563a5516d300 2024-06-03T02:33:41.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.953+0000 7f818ea00700 1 -- 172.21.15.79:0/400012586 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198003f00 con 0x563a5516d300 2024-06-03T02:33:41.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.954+0000 7f81af6abb80 1 -- 172.21.15.79:0/400012586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 msgr2=0x563a5516c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.954+0000 7f81af6abb80 1 --2- 172.21.15.79:0/400012586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516c4c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f81980a3100 tx=0x7f8198087040 comp rx=0 tx=0).stop 2024-06-03T02:33:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.956+0000 7f81af6abb80 1 -- 172.21.15.79:0/400012586 shutdown_connections 2024-06-03T02:33:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.956+0000 7f81af6abb80 1 --2- 172.21.15.79:0/400012586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a5516d300 0x563a5516c4c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.956+0000 7f81af6abb80 1 --2- 172.21.15.79:0/400012586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a550dee40 0x563a5516f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.956+0000 7f81af6abb80 1 --2- 172.21.15.79:0/400012586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a551625b0 0x563a5516e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.956+0000 7f81af6abb80 1 -- 172.21.15.79:0/400012586 >> 172.21.15.79:0/400012586 conn(0x563a5508f170 msgr2=0x563a55168570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:41.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.957+0000 7f81af6abb80 1 -- 172.21.15.79:0/400012586 shutdown_connections 2024-06-03T02:33:41.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.957+0000 7f81af6abb80 1 -- 172.21.15.79:0/400012586 wait complete. 2024-06-03T02:33:41.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.958+0000 7f81af6abb80 1 -- start start 2024-06-03T02:33:41.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.959+0000 7f81af6abb80 1 Processor -- start 2024-06-03T02:33:41.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.960+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a551676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.961+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55165f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.961+0000 7f81af6abb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a55166450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.961+0000 7f81af6abb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f817c010110 con 0x563a5516d300 2024-06-03T02:33:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.961+0000 7f81af6abb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f817c00fa40 con 0x563a551625b0 2024-06-03T02:33:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.961+0000 7f81af6abb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f817c00cbe0 con 0x563a550dee40 2024-06-03T02:33:41.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.961+0000 7f81af6abb80 10 client.0 _ll_register_callbacks cb 0x563a5504e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.962+0000 7f8195400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55165f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.962+0000 7f819e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a55166450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.962+0000 7f819de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a551676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.962+0000 7f819e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a55166450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47078/0 (socket says 172.21.15.79:47078) 2024-06-03T02:33:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.962+0000 7f819e800700 1 -- 172.21.15.79:0/379491156 learned_addr learned my addr 172.21.15.79:0/379491156 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.963+0000 7f819e800700 1 -- 172.21.15.79:0/379491156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 msgr2=0x563a551676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.963+0000 7f819e800700 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a551676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.963+0000 7f819e800700 1 -- 172.21.15.79:0/379491156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 msgr2=0x563a55165f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.963+0000 7f819e800700 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55165f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.963+0000 7f819e800700 1 -- 172.21.15.79:0/379491156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a551669c0 con 0x563a5516d300 2024-06-03T02:33:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.963+0000 7f819e800700 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a55166450 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f81980a4d30 tx=0x7f8198005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7519640e578c180d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.963+0000 7f8183e00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:33:41.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.967+0000 7f8183e00700 1 -- 172.21.15.79:0/379491156 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81980b9070 con 0x563a5516d300 2024-06-03T02:33:41.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.967+0000 7f8183e00700 1 -- 172.21.15.79:0/379491156 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f81980a1dc0 con 0x563a5516d300 2024-06-03T02:33:41.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.967+0000 7f81af6abb80 1 -- 172.21.15.79:0/379491156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563a550bc580 con 0x563a5516d300 2024-06-03T02:33:41.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.968+0000 7f8183e00700 1 -- 172.21.15.79:0/379491156 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f819809cd80 con 0x563a5516d300 2024-06-03T02:33:41.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.968+0000 7f81af6abb80 10 client.16299 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:41.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.968+0000 7f81af6abb80 1 -- 172.21.15.79:0/379491156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a551770e0 con 0x563a5516d300 2024-06-03T02:33:41.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.968+0000 7f8183e00700 1 -- 172.21.15.79:0/379491156 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563a550bc580 con 0x563a5516d300 2024-06-03T02:33:41.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.969+0000 7f8183e00700 1 -- 172.21.15.79:0/379491156 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f819809bad0 con 0x563a5516d300 2024-06-03T02:33:41.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.969+0000 7f81af6abb80 10 client.16299 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:41.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.969+0000 7f81af6abb80 10 client.16299 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:33:41.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.969+0000 7f81af6abb80 20 client.16299 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:41.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.970+0000 7f81af6abb80 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x563a5518de50 0x563a551fae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:41.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.970+0000 7f81af6abb80 4 client.16299 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:33:41.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.970+0000 7f81af6abb80 1 -- 172.21.15.79:0/379491156 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a54f36250 con 0x563a5518de50 2024-06-03T02:33:41.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.973+0000 7f819de00700 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x563a5518de50 0x563a551fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:41.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.976+0000 7f8183e00700 10 client.16299 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:33:41.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.976+0000 7f819de00700 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x563a5518de50 0x563a551fae50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9433eca4e91d3a42 server_cookie=3bb1a2c227a35441 in_seq=0 out_seq=0 2024-06-03T02:33:41.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.976+0000 7f8183e00700 1 -- 172.21.15.79:0/379491156 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a54f36250 con 0x563a5518de50 2024-06-03T02:33:41.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.976+0000 7f8183e00700 10 client.16299 handle_command_reply: tid=0 2024-06-03T02:33:41.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f81af6abb80 1 client.16299 shutdown 2024-06-03T02:33:41.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f81af6abb80 1 -- 172.21.15.79:0/379491156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 msgr2=0x563a55166450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:41.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f81af6abb80 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a55166450 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f81980a4d30 tx=0x7f8198005500 comp rx=0 tx=0).stop 2024-06-03T02:33:41.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f81af6abb80 1 -- 172.21.15.79:0/379491156 shutdown_connections 2024-06-03T02:33:41.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f81af6abb80 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x563a5518de50 0x563a551fae50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f81af6abb80 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563a5516d300 0x563a55166450 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f8183e00700 0 client.16299 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:33:41.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f81af6abb80 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a550dee40 0x563a551676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f81af6abb80 1 --2- 172.21.15.79:0/379491156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a551625b0 0x563a55165f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.977+0000 7f81af6abb80 1 -- 172.21.15.79:0/379491156 >> 172.21.15.79:0/379491156 conn(0x563a5508f170 msgr2=0x563a55163fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:41.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.978+0000 7f81af6abb80 1 -- 172.21.15.79:0/379491156 shutdown_connections 2024-06-03T02:33:41.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.978+0000 7f81af6abb80 1 -- 172.21.15.79:0/379491156 wait complete. 2024-06-03T02:33:41.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.978+0000 7f81af6abb80 20 client.16299 trim_cache size 0 max 16384 2024-06-03T02:33:41.981 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81840a2d70 msgr2=0x7f81840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81840a2d70 0x7f81840a5230 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8188009910 tx=0x7f8188008040 comp rx=0 tx=0).stop 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81a0107400 msgr2=0x7f81a019d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81a0107400 0x7f81a019d7e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8198005e50 tx=0x7f8198003730 comp rx=0 tx=0).stop 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 shutdown_connections 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81a0107400 0x7f81a019d7e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f81840a2d70 0x7f81840a5230 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81a0101390 0x7f81a0198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 --2- 172.21.15.79:0/551049741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a0101d50 0x7f81a01994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:41.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.979+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 >> 172.21.15.79:0/551049741 conn(0x7f81a0078340 msgr2=0x7f81a00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:41.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.980+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 shutdown_connections 2024-06-03T02:33:41.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:41.983+0000 7f81a4e00700 1 -- 172.21.15.79:0/551049741 wait complete. 2024-06-03T02:33:42.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:42 smithi123 ceph-mon[36640]: pgmap v925: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.5 MiB/s wr, 605 op/s 2024-06-03T02:33:42.975 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:42 smithi079 ceph-mon[31819]: pgmap v925: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.5 MiB/s wr, 605 op/s 2024-06-03T02:33:43.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:42 smithi135 ceph-mon[36653]: pgmap v925: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.5 MiB/s wr, 605 op/s 2024-06-03T02:33:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 815.0 s, 502.2 tps, lat 200.329 ms stddev 231.714 2024-06-03T02:33:44.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:33:44.841 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:44 smithi079 ceph-mon[31819]: pgmap v926: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.0 MiB/s wr, 543 op/s 2024-06-03T02:33:44.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:44 smithi123 ceph-mon[36640]: pgmap v926: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.0 MiB/s wr, 543 op/s 2024-06-03T02:33:45.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:44 smithi135 ceph-mon[36653]: pgmap v926: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.0 MiB/s wr, 543 op/s 2024-06-03T02:33:45.190 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:33:46.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:46 smithi123 ceph-mon[36640]: pgmap v927: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 5.4 MiB/s wr, 596 op/s 2024-06-03T02:33:47.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:46 smithi079 ceph-mon[31819]: pgmap v927: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 5.4 MiB/s wr, 596 op/s 2024-06-03T02:33:47.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:46 smithi135 ceph-mon[36653]: pgmap v927: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 5.4 MiB/s wr, 596 op/s 2024-06-03T02:33:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 820.0 s, 466.8 tps, lat 221.891 ms stddev 247.824 2024-06-03T02:33:48.948 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:48 smithi079 ceph-mon[31819]: pgmap v928: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-06-03T02:33:48.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:48 smithi123 ceph-mon[36640]: pgmap v928: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-06-03T02:33:49.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:48 smithi135 ceph-mon[36653]: pgmap v928: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-06-03T02:33:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.818+0000 7fb2d0400700 1 -- 172.21.15.79:0/1951474768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2c8109db0 msgr2=0x7fb2c81150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.818+0000 7fb2d0400700 1 --2- 172.21.15.79:0/1951474768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2c8109db0 0x7fb2c81150a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0098f0 tx=0x7fb2bc009c00 comp rx=0 tx=0).stop 2024-06-03T02:33:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.819+0000 7fb2d0400700 1 -- 172.21.15.79:0/1951474768 shutdown_connections 2024-06-03T02:33:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.819+0000 7fb2d0400700 1 --2- 172.21.15.79:0/1951474768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2c81155e0 0x7fb2c81179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.819+0000 7fb2d0400700 1 --2- 172.21.15.79:0/1951474768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c8109490 0x7fb2c8109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.819+0000 7fb2d0400700 1 --2- 172.21.15.79:0/1951474768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2c8109db0 0x7fb2c81150a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.819+0000 7fb2d0400700 1 -- 172.21.15.79:0/1951474768 >> 172.21.15.79:0/1951474768 conn(0x7fb2c806d600 msgr2=0x7fb2c806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.820+0000 7fb2d0400700 1 -- 172.21.15.79:0/1951474768 shutdown_connections 2024-06-03T02:33:50.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.820+0000 7fb2d0400700 1 -- 172.21.15.79:0/1951474768 wait complete. 2024-06-03T02:33:50.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.822+0000 7fb2d0400700 1 Processor -- start 2024-06-03T02:33:50.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.822+0000 7fb2d0400700 1 -- start start 2024-06-03T02:33:50.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.825+0000 7fb2d0400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c8109490 0x7fb2c8114980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.825+0000 7fb2d0400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2c8109db0 0x7fb2c8114ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.826+0000 7fb2cda00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c8109490 0x7fb2c8114980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.826+0000 7fb2cda00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c8109490 0x7fb2c8114980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48182/0 (socket says 172.21.15.79:48182) 2024-06-03T02:33:50.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.826+0000 7fb2cda00700 1 -- 172.21.15.79:0/3357164962 learned_addr learned my addr 172.21.15.79:0/3357164962 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:50.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.826+0000 7fb2d0400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2c81155e0 0x7fb2c810c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.826+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c811b0c0 con 0x7fb2c8109db0 2024-06-03T02:33:50.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.827+0000 7fb2ce400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2c81155e0 0x7fb2c810c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.827+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c811af40 con 0x7fb2c8109490 2024-06-03T02:33:50.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.827+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c811b240 con 0x7fb2c81155e0 2024-06-03T02:33:50.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.827+0000 7fb2cd000700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2c8109db0 0x7fb2c8114ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.828+0000 7fb2ce400700 1 -- 172.21.15.79:0/3357164962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c8109490 msgr2=0x7fb2c8114980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.828+0000 7fb2ce400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c8109490 0x7fb2c8114980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.828+0000 7fb2ce400700 1 -- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2c8109db0 msgr2=0x7fb2c8114ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.828+0000 7fb2ce400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2c8109db0 0x7fb2c8114ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.828+0000 7fb2ce400700 1 -- 172.21.15.79:0/3357164962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c810c9e0 con 0x7fb2c81155e0 2024-06-03T02:33:50.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.829+0000 7fb2ce400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2c81155e0 0x7fb2c810c160 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000be20 tx=0x7fb2c000bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:50.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.829+0000 7fb2c6000700 1 -- 172.21.15.79:0/3357164962 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0019070 con 0x7fb2c81155e0 2024-06-03T02:33:50.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.829+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c810ccd0 con 0x7fb2c81155e0 2024-06-03T02:33:50.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.829+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c8110690 con 0x7fb2c81155e0 2024-06-03T02:33:50.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.829+0000 7fb2c6000700 1 -- 172.21.15.79:0/3357164962 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c000cd10 con 0x7fb2c81155e0 2024-06-03T02:33:50.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.831+0000 7fb2c6000700 1 -- 172.21.15.79:0/3357164962 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c001d770 con 0x7fb2c81155e0 2024-06-03T02:33:50.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.845+0000 7fb2c6000700 1 -- 172.21.15.79:0/3357164962 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb2c001c070 con 0x7fb2c81155e0 2024-06-03T02:33:50.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.846+0000 7fb2dad15b80 1 Processor -- start 2024-06-03T02:33:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.846+0000 7fb2dad15b80 1 -- start start 2024-06-03T02:33:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.847+0000 7fb2c6000700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2b00a2ca0 0x7fb2b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.847+0000 7fb2c6000700 1 -- 172.21.15.79:0/3357164962 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb2c00ceb10 con 0x7fb2c81155e0 2024-06-03T02:33:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.848+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a5df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.848+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a57d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.848+0000 7fb2cd000700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2b00a2ca0 0x7fb2b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.848+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a58240 0x55c2c1a58680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.848+0000 7fb2cda00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a5df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.848+0000 7fb2cda00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a5df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48192/0 (socket says 172.21.15.79:48192) 2024-06-03T02:33:50.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.849+0000 7fb2cda00700 1 -- 172.21.15.79:0/1675498788 learned_addr learned my addr 172.21.15.79:0/1675498788 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:50.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.850+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1675498788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1966a10 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.849+0000 7fb2ce400700 1 --2- 172.21.15.79:0/1675498788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a58240 0x55c2c1a58680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.850+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1675498788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c2c19a9b00 con 0x55c2c1a5db90 2024-06-03T02:33:50.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.850+0000 7fb2cd000700 1 --2- 172.21.15.79:0/1675498788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a57d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.850+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1675498788 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c2c19648f0 con 0x55c2c1a58240 2024-06-03T02:33:50.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.851+0000 7fb2cda00700 1 -- 172.21.15.79:0/1675498788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a58240 msgr2=0x55c2c1a58680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:50.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.851+0000 7fb2cda00700 1 --2- 172.21.15.79:0/1675498788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a58240 0x55c2c1a58680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.851+0000 7fb2cda00700 1 -- 172.21.15.79:0/1675498788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a5e4b0 msgr2=0x55c2c1a57d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:50.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.851+0000 7fb2cda00700 1 --2- 172.21.15.79:0/1675498788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a57d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.851+0000 7fb2cda00700 1 -- 172.21.15.79:0/1675498788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c1951f90 con 0x55c2c1a5db90 2024-06-03T02:33:50.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.852+0000 7fb2cda00700 1 --2- 172.21.15.79:0/1675498788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a5df70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009e70 tx=0x7fb2b40116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64a9d1617f3d1a2f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:50.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.852+0000 7fb2cd000700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2b00a2ca0 0x7fb2b00a5160 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009680 tx=0x7fb2bc005b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:50.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.852+0000 7fb2baa00700 1 -- 172.21.15.79:0/1675498788 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4011e20 con 0x55c2c1a5db90 2024-06-03T02:33:50.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.852+0000 7fb2baa00700 1 -- 172.21.15.79:0/1675498788 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b4012930 con 0x55c2c1a5db90 2024-06-03T02:33:50.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.852+0000 7fb2baa00700 1 -- 172.21.15.79:0/1675498788 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400f9a0 con 0x55c2c1a5db90 2024-06-03T02:33:50.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.853+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1675498788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 msgr2=0x55c2c1a5df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:50.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.853+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1675498788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a5df70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009e70 tx=0x7fb2b40116c0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.855+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1675498788 shutdown_connections 2024-06-03T02:33:50.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.855+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1675498788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a58240 0x55c2c1a58680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.855+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1675498788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a5df70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.855+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1675498788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a57d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.855+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1675498788 >> 172.21.15.79:0/1675498788 conn(0x55c2c1984170 msgr2=0x55c2c1a52ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:50.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.855+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1675498788 shutdown_connections 2024-06-03T02:33:50.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.855+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1675498788 wait complete. 2024-06-03T02:33:50.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.855+0000 7fb2dad15b80 1 -- start start 2024-06-03T02:33:50.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.856+0000 7fb2dad15b80 1 Processor -- start 2024-06-03T02:33:50.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.858+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a58240 0x55c2c1a641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.858+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a5db90 0x55c2c1a614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.858+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.858+0000 7fb2dad15b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1945f60 con 0x55c2c1a5db90 2024-06-03T02:33:50.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.858+0000 7fb2dad15b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c2c19a5ac0 con 0x55c2c1a58240 2024-06-03T02:33:50.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.858+0000 7fb2dad15b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1a64e90 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.858+0000 7fb2dad15b80 10 client.0 _ll_register_callbacks cb 0x55c2c193f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:50.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.859+0000 7fb2ce400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.859+0000 7fb2ce400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:43072/0 (socket says 172.21.15.79:43072) 2024-06-03T02:33:50.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.859+0000 7fb2ce400700 1 -- 172.21.15.79:0/451074309 learned_addr learned my addr 172.21.15.79:0/451074309 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:50.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.859+0000 7fb2ce400700 1 -- 172.21.15.79:0/451074309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a58240 msgr2=0x55c2c1a641a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:33:50.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.859+0000 7fb2ce400700 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a58240 0x55c2c1a641a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.859+0000 7fb2ce400700 1 -- 172.21.15.79:0/451074309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a5db90 msgr2=0x55c2c1a614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:50.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.859+0000 7fb2ce400700 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a5db90 0x55c2c1a614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.859+0000 7fb2ce400700 1 -- 172.21.15.79:0/451074309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c18f88d0 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.860+0000 7fb2ce400700 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a619e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000bfd0 tx=0x7fb2c009b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813070d1a1be211e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:50.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.860+0000 7fb2af400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:50.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.860+0000 7fb2af400700 1 -- 172.21.15.79:0/451074309 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00a3940 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.860+0000 7fb2af400700 1 -- 172.21.15.79:0/451074309 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb2c00b9ba0 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.860+0000 7fb2dad15b80 1 -- 172.21.15.79:0/451074309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b4012930 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.860+0000 7fb2af400700 1 -- 172.21.15.79:0/451074309 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2c00a3bd0 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.862+0000 7fb2dad15b80 10 client.26860 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:50.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.862+0000 7fb2dad15b80 1 -- 172.21.15.79:0/451074309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2c1a6f660 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.863+0000 7fb2af400700 1 -- 172.21.15.79:0/451074309 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b4012930 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.863+0000 7fb2af400700 1 -- 172.21.15.79:0/451074309 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb2c0095ad0 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.864+0000 7fb2dad15b80 10 client.26860 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:50.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.864+0000 7fb2dad15b80 10 client.26860 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:33:50.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.864+0000 7fb2dad15b80 20 client.26860 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:50.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.864+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c2c1a71cf0 0x55c2c1a741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.864+0000 7fb2dad15b80 4 client.26860 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:33:50.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.864+0000 7fb2dad15b80 1 -- 172.21.15.79:0/451074309 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c19b7000 con 0x55c2c1a71cf0 2024-06-03T02:33:50.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.864+0000 7fb2cda00700 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c2c1a71cf0 0x55c2c1a741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.865+0000 7fb2af400700 10 client.26860 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.865+0000 7fb2cda00700 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c2c1a71cf0 0x55c2c1a741b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=672d7d849a62c7e1 server_cookie=b1a50342ca60fdc7 in_seq=0 out_seq=0 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.866+0000 7fb2af400700 1 -- 172.21.15.79:0/451074309 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c2c19b7000 con 0x55c2c1a71cf0 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.866+0000 7fb2af400700 10 client.26860 handle_command_reply: tid=0 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 client.26860 shutdown 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 -- 172.21.15.79:0/451074309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 msgr2=0x55c2c1a619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a619e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000bfd0 tx=0x7fb2c009b660 comp rx=0 tx=0).stop 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 -- 172.21.15.79:0/451074309 shutdown_connections 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a619e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a58240 0x55c2c1a641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c2c1a71cf0 0x55c2c1a741b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/451074309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a5db90 0x55c2c1a614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 -- 172.21.15.79:0/451074309 >> 172.21.15.79:0/451074309 conn(0x55c2c1984170 msgr2=0x55c2c1a4c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:50.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2af400700 0 client.26860 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:33:50.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 -- 172.21.15.79:0/451074309 shutdown_connections 2024-06-03T02:33:50.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.867+0000 7fb2dad15b80 1 -- 172.21.15.79:0/451074309 wait complete. 2024-06-03T02:33:50.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.868+0000 7fb2dad15b80 20 client.26860 trim_cache size 0 max 16384 2024-06-03T02:33:50.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.904+0000 7fb2dad15b80 1 Processor -- start 2024-06-03T02:33:50.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.904+0000 7fb2dad15b80 1 -- start start 2024-06-03T02:33:50.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.904+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 0x55c2c1a57d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.904+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a531a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.904+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a4d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.904+0000 7fb2dad15b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a800eaa0 con 0x55c2c1a58240 2024-06-03T02:33:50.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.904+0000 7fb2dad15b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a80105e0 con 0x55c2c1a5db90 2024-06-03T02:33:50.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.904+0000 7fb2dad15b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a800f170 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.905+0000 7fb2cd000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.905+0000 7fb2ce400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a4d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.906+0000 7fb2cd000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48206/0 (socket says 172.21.15.79:48206) 2024-06-03T02:33:50.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.907+0000 7fb2cd000700 1 -- 172.21.15.79:0/2460717352 learned_addr learned my addr 172.21.15.79:0/2460717352 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:50.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.906+0000 7fb2cda00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 0x55c2c1a57d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.910+0000 7fb2ce400700 1 -- 172.21.15.79:0/2460717352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 msgr2=0x55c2c1a531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:50.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.910+0000 7fb2ce400700 1 --2- 172.21.15.79:0/2460717352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.910+0000 7fb2ce400700 1 -- 172.21.15.79:0/2460717352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 msgr2=0x55c2c1a57d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:50.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.910+0000 7fb2ce400700 1 --2- 172.21.15.79:0/2460717352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 0x55c2c1a57d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.910+0000 7fb2ce400700 1 -- 172.21.15.79:0/2460717352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c1a5e2b0 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.911+0000 7fb2ce400700 1 --2- 172.21.15.79:0/2460717352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a4d130 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0095ec0 tx=0x7fb2c00bb050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb350c24c2904582 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:50.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.913+0000 7fb2ba000700 1 -- 172.21.15.79:0/2460717352 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c009ad30 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.913+0000 7fb2ba000700 1 -- 172.21.15.79:0/2460717352 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c009a190 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.913+0000 7fb2ba000700 1 -- 172.21.15.79:0/2460717352 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00b9ba0 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.914+0000 7fb2dad15b80 1 -- 172.21.15.79:0/2460717352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 msgr2=0x55c2c1a4d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.914+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/2460717352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a4d130 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0095ec0 tx=0x7fb2c00bb050 comp rx=0 tx=0).stop 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.914+0000 7fb2dad15b80 1 -- 172.21.15.79:0/2460717352 shutdown_connections 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.914+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/2460717352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a4d130 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.914+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/2460717352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 0x55c2c1a57d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.914+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/2460717352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.914+0000 7fb2dad15b80 1 -- 172.21.15.79:0/2460717352 >> 172.21.15.79:0/2460717352 conn(0x55c2c1984170 msgr2=0x55c2c1a52a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.915+0000 7fb2dad15b80 1 -- 172.21.15.79:0/2460717352 shutdown_connections 2024-06-03T02:33:50.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.915+0000 7fb2dad15b80 1 -- 172.21.15.79:0/2460717352 wait complete. 2024-06-03T02:33:50.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.916+0000 7fb2dad15b80 1 -- start start 2024-06-03T02:33:50.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.916+0000 7fb2dad15b80 1 Processor -- start 2024-06-03T02:33:50.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.917+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 0x55c2c1a6a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.917+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a6a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.917+0000 7fb2dad15b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a78b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.917+0000 7fb2dad15b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a800ff10 con 0x55c2c1a58240 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.917+0000 7fb2dad15b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a800f840 con 0x55c2c1a5db90 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.917+0000 7fb2dad15b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a800ca40 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.917+0000 7fb2dad15b80 10 client.0 _ll_register_callbacks cb 0x55c2c193f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.918+0000 7fb2cda00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 0x55c2c1a6a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.918+0000 7fb2cd000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a6a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.918+0000 7fb2cd000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a6a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:48218/0 (socket says 172.21.15.79:48218) 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.918+0000 7fb2cd000700 1 -- 172.21.15.79:0/1754040531 learned_addr learned my addr 172.21.15.79:0/1754040531 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.918+0000 7fb2ce400700 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a78b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.918+0000 7fb2ce400700 1 -- 172.21.15.79:0/1754040531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 msgr2=0x55c2c1a6a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.918+0000 7fb2ce400700 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a6a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.918+0000 7fb2ce400700 1 -- 172.21.15.79:0/1754040531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 msgr2=0x55c2c1a6a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.919+0000 7fb2ce400700 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 0x55c2c1a6a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.919+0000 7fb2ce400700 1 -- 172.21.15.79:0/1754040531 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c1a79070 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.919+0000 7fb2cda00700 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 0x55c2c1a6a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:33:50.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.920+0000 7fb2ce400700 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a78b30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000bdf0 tx=0x7fb2c009b0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef9d3555b5bff322 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:50.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.920+0000 7fb2af400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:50.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.920+0000 7fb2af400700 1 -- 172.21.15.79:0/1754040531 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00840d0 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.920+0000 7fb2af400700 1 -- 172.21.15.79:0/1754040531 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fb2c0084270 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.920+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1754040531 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c009a190 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.920+0000 7fb2af400700 1 -- 172.21.15.79:0/1754040531 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2c00af120 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.920+0000 7fb2dad15b80 10 client.26866 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:50.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.920+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1754040531 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2c1adc250 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.921+0000 7fb2af400700 1 -- 172.21.15.79:0/1754040531 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c009a190 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.922+0000 7fb2af400700 1 -- 172.21.15.79:0/1754040531 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb2c009a3b0 con 0x55c2c1a5e4b0 2024-06-03T02:33:50.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.922+0000 7fb2dad15b80 10 client.26866 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:50.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.922+0000 7fb2dad15b80 10 client.26866 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:33:50.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.922+0000 7fb2dad15b80 20 client.26866 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:50.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.922+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c2c1adc520 0x55c2c1ade910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:50.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.922+0000 7fb2dad15b80 4 client.26866 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:33:50.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.922+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1754040531 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2c19b1580 con 0x55c2c1adc520 2024-06-03T02:33:50.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.923+0000 7fb2cda00700 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c2c1adc520 0x55c2c1ade910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.924+0000 7fb2cda00700 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c2c1adc520 0x55c2c1ade910 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24308169483ecbc5 server_cookie=b10f276d6f9130f6 in_seq=0 out_seq=0 2024-06-03T02:33:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.924+0000 7fb2af400700 10 client.26866 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:33:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.924+0000 7fb2af400700 1 -- 172.21.15.79:0/1754040531 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2c19b1580 con 0x55c2c1adc520 2024-06-03T02:33:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.924+0000 7fb2af400700 10 client.26866 handle_command_reply: tid=0 2024-06-03T02:33:50.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.924+0000 7fb2dad15b80 1 client.26866 shutdown 2024-06-03T02:33:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.925+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1754040531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 msgr2=0x55c2c1a78b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.925+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a78b30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000bdf0 tx=0x7fb2c009b0e0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.925+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1754040531 shutdown_connections 2024-06-03T02:33:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.925+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c2c1a5e4b0 0x55c2c1a78b30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.925+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c2c1adc520 0x55c2c1ade910 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.925+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c2c1a58240 0x55c2c1a6a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.925+0000 7fb2dad15b80 1 --2- 172.21.15.79:0/1754040531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2c1a5db90 0x55c2c1a6a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.925+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1754040531 >> 172.21.15.79:0/1754040531 conn(0x55c2c1984170 msgr2=0x55c2c1a5e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:50.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.925+0000 7fb2af400700 0 client.26866 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:33:50.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.927+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1754040531 shutdown_connections 2024-06-03T02:33:50.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.929+0000 7fb2dad15b80 1 -- 172.21.15.79:0/1754040531 wait complete. 2024-06-03T02:33:50.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.930+0000 7fb2dad15b80 20 client.26866 trim_cache size 0 max 16384 2024-06-03T02:33:50.931 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:33:50.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2b00a2ca0 msgr2=0x7fb2b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:50.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2b00a2ca0 0x7fb2b00a5160 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009680 tx=0x7fb2bc005b80 comp rx=0 tx=0).stop 2024-06-03T02:33:50.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2c81155e0 msgr2=0x7fb2c810c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:50.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2c81155e0 0x7fb2c810c160 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000be20 tx=0x7fb2c000bf00 comp rx=0 tx=0).stop 2024-06-03T02:33:50.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 shutdown_connections 2024-06-03T02:33:50.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2c81155e0 0x7fb2c810c160 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c8109490 0x7fb2c8114980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fb2b00a2ca0 0x7fb2b00a5160 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 --2- 172.21.15.79:0/3357164962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2c8109db0 0x7fb2c8114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:50.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.931+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 >> 172.21.15.79:0/3357164962 conn(0x7fb2c806d600 msgr2=0x7fb2c8073040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:50.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.932+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 shutdown_connections 2024-06-03T02:33:50.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:50.932+0000 7fb2d0400700 1 -- 172.21.15.79:0/3357164962 wait complete. 2024-06-03T02:33:50.956 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:50 smithi079 ceph-mon[31819]: pgmap v929: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-06-03T02:33:50.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:50 smithi123 ceph-mon[36640]: pgmap v929: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-06-03T02:33:51.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:50 smithi135 ceph-mon[36653]: pgmap v929: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-06-03T02:33:51.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:51 smithi123 ceph-mon[36640]: pgmap v930: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-06-03T02:33:52.022 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:51 smithi079 ceph-mon[31819]: pgmap v930: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-06-03T02:33:52.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:51 smithi135 ceph-mon[36653]: pgmap v930: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-06-03T02:33:53.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 825.0 s, 419.2 tps, lat 236.276 ms stddev 290.872 2024-06-03T02:33:53.460 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:33:54.165 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:33:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:53 smithi123 ceph-mon[36640]: pgmap v931: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 4.3 MiB/s wr, 454 op/s 2024-06-03T02:33:54.302 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:53 smithi079 ceph-mon[31819]: pgmap v931: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 4.3 MiB/s wr, 454 op/s 2024-06-03T02:33:54.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:53 smithi135 ceph-mon[36653]: pgmap v931: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 4.3 MiB/s wr, 454 op/s 2024-06-03T02:33:56.173 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:55 smithi079 ceph-mon[31819]: pgmap v932: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-06-03T02:33:56.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:55 smithi123 ceph-mon[36640]: pgmap v932: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-06-03T02:33:56.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:55 smithi135 ceph-mon[36653]: pgmap v932: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-06-03T02:33:58.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:57 smithi079 ceph-mon[31819]: pgmap v933: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-06-03T02:33:58.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:57 smithi123 ceph-mon[36640]: pgmap v933: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-06-03T02:33:58.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 830.0 s, 402.0 tps, lat 245.016 ms stddev 282.576 2024-06-03T02:33:58.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:57 smithi135 ceph-mon[36653]: pgmap v933: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-06-03T02:33:58.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.353+0000 7ff3cb800700 1 -- 172.21.15.79:0/150085671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3c4107840 msgr2=0x7ff3c410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:58.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.353+0000 7ff3cb800700 1 --2- 172.21.15.79:0/150085671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3c4107840 0x7ff3c410f3c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00b0d0 tx=0x7ff3bc00b3e0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.354+0000 7ff3cb800700 1 -- 172.21.15.79:0/150085671 shutdown_connections 2024-06-03T02:33:58.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.354+0000 7ff3cb800700 1 --2- 172.21.15.79:0/150085671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3c4107840 0x7ff3c410f3c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.354+0000 7ff3cb800700 1 --2- 172.21.15.79:0/150085671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c4102740 0x7ff3c4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.354+0000 7ff3cb800700 1 --2- 172.21.15.79:0/150085671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c41030f0 0x7ff3c4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.354+0000 7ff3cb800700 1 -- 172.21.15.79:0/150085671 >> 172.21.15.79:0/150085671 conn(0x7ff3c40795f0 msgr2=0x7ff3c4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:58.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.354+0000 7ff3cb800700 1 -- 172.21.15.79:0/150085671 shutdown_connections 2024-06-03T02:33:58.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.354+0000 7ff3cb800700 1 -- 172.21.15.79:0/150085671 wait complete. 2024-06-03T02:33:58.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.358+0000 7ff3cb800700 1 Processor -- start 2024-06-03T02:33:58.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.358+0000 7ff3cb800700 1 -- start start 2024-06-03T02:33:58.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.359+0000 7ff3cb800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4102740 0x7ff3c419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.359+0000 7ff3cb800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3c41030f0 0x7ff3c419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.359+0000 7ff3c8e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4102740 0x7ff3c419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3cb800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c4107840 0x7ff3c41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3c41030f0 0x7ff3c419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3c41030f0 0x7ff3c419ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:56758/0 (socket says 172.21.15.79:56758) 2024-06-03T02:33:58.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3c8e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4102740 0x7ff3c419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54988/0 (socket says 172.21.15.79:54988) 2024-06-03T02:33:58.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3cb800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41127b0 con 0x7ff3c4102740 2024-06-03T02:33:58.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3cb800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4112630 con 0x7ff3c41030f0 2024-06-03T02:33:58.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3cb800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4112930 con 0x7ff3c4107840 2024-06-03T02:33:58.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3c9800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c4107840 0x7ff3c41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3c9800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c4107840 0x7ff3c41989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59892/0 (socket says 172.21.15.79:59892) 2024-06-03T02:33:58.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.360+0000 7ff3c9800700 1 -- 172.21.15.79:0/3241027276 learned_addr learned my addr 172.21.15.79:0/3241027276 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:58.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.361+0000 7ff3c9800700 1 -- 172.21.15.79:0/3241027276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3c41030f0 msgr2=0x7ff3c419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:58.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.361+0000 7ff3c9800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3c41030f0 0x7ff3c419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.361+0000 7ff3c9800700 1 -- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4102740 msgr2=0x7ff3c419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:58.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.362+0000 7ff3c9800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4102740 0x7ff3c419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.362+0000 7ff3c9800700 1 -- 172.21.15.79:0/3241027276 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c4198f20 con 0x7ff3c4107840 2024-06-03T02:33:58.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.362+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4102740 0x7ff3c419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:33:58.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.364+0000 7ff3c9800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c4107840 0x7ff3c41989b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009850 tx=0x7ff3bc004dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:58.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.364+0000 7ff3c1600700 1 -- 172.21.15.79:0/3241027276 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc021070 con 0x7ff3c4107840 2024-06-03T02:33:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.365+0000 7ff3cb800700 1 -- 172.21.15.79:0/3241027276 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c41991b0 con 0x7ff3c4107840 2024-06-03T02:33:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.365+0000 7ff3c1600700 1 -- 172.21.15.79:0/3241027276 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3bc007b80 con 0x7ff3c4107840 2024-06-03T02:33:58.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.365+0000 7ff3c1600700 1 -- 172.21.15.79:0/3241027276 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc005370 con 0x7ff3c4107840 2024-06-03T02:33:58.370 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.366+0000 7ff3cb800700 1 -- 172.21.15.79:0/3241027276 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c4199690 con 0x7ff3c4107840 2024-06-03T02:33:58.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.372+0000 7ff3d611fb80 1 Processor -- start 2024-06-03T02:33:58.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.373+0000 7ff3c1600700 1 -- 172.21.15.79:0/3241027276 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff3bc024070 con 0x7ff3c4107840 2024-06-03T02:33:58.374 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.373+0000 7ff3d611fb80 1 -- start start 2024-06-03T02:33:58.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.374+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761ad3de40 0x55761adc1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.378 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.374+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761adc1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.375+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761adcc300 0x55761adce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.375+0000 7ff3d611fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55761acd0a10 con 0x55761ad3de40 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.375+0000 7ff3d611fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55761ad13b00 con 0x55761adcc300 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.375+0000 7ff3c8e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761adc1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.375+0000 7ff3d611fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55761acce8f0 con 0x55761adc15b0 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.375+0000 7ff3c8e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761adc1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59904/0 (socket says 172.21.15.79:59904) 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.375+0000 7ff3c8e00700 1 -- 172.21.15.79:0/1353619450 learned_addr learned my addr 172.21.15.79:0/1353619450 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.375+0000 7ff3c1600700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff3ac0a2c70 0x7ff3ac0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.376+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761adcc300 0x55761adce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.376+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761ad3de40 0x55761adc1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.376+0000 7ff3c8e00700 1 -- 172.21.15.79:0/1353619450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761adcc300 msgr2=0x55761adce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:58.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.376+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761adcc300 0x55761adce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.376+0000 7ff3c8e00700 1 -- 172.21.15.79:0/1353619450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761ad3de40 msgr2=0x55761adc1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.376+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761ad3de40 0x55761adc1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.376+0000 7ff3c9800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff3ac0a2c70 0x7ff3ac0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.377+0000 7ff3c8e00700 1 -- 172.21.15.79:0/1353619450 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761acbbf90 con 0x55761adc15b0 2024-06-03T02:33:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.377+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761ad3de40 0x55761adc1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:33:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.377+0000 7ff3c1600700 1 -- 172.21.15.79:0/3241027276 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff3bc00f020 con 0x7ff3c4107840 2024-06-03T02:33:58.380 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.377+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761adc1a30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800bd70 tx=0x7ff3b800f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f34552fccb89189 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.378+0000 7ff3c9800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff3ac0a2c70 0x7ff3ac0a5130 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0997e0 tx=0x7ff3bc099730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.380+0000 7ff3b6000700 1 -- 172.21.15.79:0/1353619450 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b800fe50 con 0x55761adc15b0 2024-06-03T02:33:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.380+0000 7ff3b6000700 1 -- 172.21.15.79:0/1353619450 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b8004510 con 0x55761adc15b0 2024-06-03T02:33:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.380+0000 7ff3b6000700 1 -- 172.21.15.79:0/1353619450 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80116a0 con 0x55761adc15b0 2024-06-03T02:33:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.381+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1353619450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 msgr2=0x55761adc1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.381+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761adc1a30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800bd70 tx=0x7ff3b800f640 comp rx=0 tx=0).stop 2024-06-03T02:33:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.381+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1353619450 shutdown_connections 2024-06-03T02:33:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.381+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761adcc300 0x55761adce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.381+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761ad3de40 0x55761adc1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.381+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/1353619450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761adc1a30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.381+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1353619450 >> 172.21.15.79:0/1353619450 conn(0x55761acee170 msgr2=0x55761adbd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:58.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.381+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1353619450 shutdown_connections 2024-06-03T02:33:58.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.381+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1353619450 wait complete. 2024-06-03T02:33:58.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.382+0000 7ff3d611fb80 1 -- start start 2024-06-03T02:33:58.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.384+0000 7ff3d611fb80 1 Processor -- start 2024-06-03T02:33:58.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.386+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761ad3de40 0x55761adcbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.387+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761adc15b0 0x55761adc8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.387+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 0x55761adc9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.387+0000 7ff3c3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761ad3de40 0x55761adcbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.387+0000 7ff3c3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761ad3de40 0x55761adcbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59912/0 (socket says 172.21.15.79:59912) 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.387+0000 7ff3c3e00700 1 -- 172.21.15.79:0/3074390337 learned_addr learned my addr 172.21.15.79:0/3074390337 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.387+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55761acaff60 con 0x55761adcc300 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.387+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55761ad0fac0 con 0x55761adc15b0 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.387+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55761adcefc0 con 0x55761ad3de40 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.388+0000 7ff3d611fb80 10 client.0 _ll_register_callbacks cb 0x55761acad720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.388+0000 7ff3c3e00700 1 -- 172.21.15.79:0/3074390337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761adc15b0 msgr2=0x55761adc8dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.388+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761adc15b0 0x55761adc8dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.388+0000 7ff3c3e00700 1 -- 172.21.15.79:0/3074390337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 msgr2=0x55761adc9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.388+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 0x55761adc9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.388+0000 7ff3c3e00700 1 -- 172.21.15.79:0/3074390337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761ac628d0 con 0x55761ad3de40 2024-06-03T02:33:58.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.389+0000 7ff3aaa00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:58.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.389+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761ad3de40 0x55761adcbac0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000aed0 tx=0x7ff3b0010d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a5881ace829057 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:58.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.390+0000 7ff3aaa00700 1 -- 172.21.15.79:0/3074390337 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b000fd20 con 0x55761ad3de40 2024-06-03T02:33:58.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.390+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b8004510 con 0x55761ad3de40 2024-06-03T02:33:58.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.390+0000 7ff3aaa00700 1 -- 172.21.15.79:0/3074390337 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff3b0004020 con 0x55761ad3de40 2024-06-03T02:33:58.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.390+0000 7ff3aaa00700 1 -- 172.21.15.79:0/3074390337 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff3b0024050 con 0x55761ad3de40 2024-06-03T02:33:58.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.391+0000 7ff3d611fb80 10 client.26878 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:58.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.391+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55761ade1f70 con 0x55761ad3de40 2024-06-03T02:33:58.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.391+0000 7ff3aaa00700 1 -- 172.21.15.79:0/3074390337 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b8004510 con 0x55761ad3de40 2024-06-03T02:33:58.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.392+0000 7ff3aaa00700 1 -- 172.21.15.79:0/3074390337 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b000e3b0 con 0x55761ad3de40 2024-06-03T02:33:58.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.393+0000 7ff3d611fb80 10 client.26878 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:58.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.393+0000 7ff3d611fb80 10 client.26878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:33:58.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.393+0000 7ff3d611fb80 20 client.26878 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:58.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.393+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55761ade4930 0x55761ade6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.393+0000 7ff3d611fb80 4 client.26878 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:33:58.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.393+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55761ad21000 con 0x55761ade4930 2024-06-03T02:33:58.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.394+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55761ade4930 0x55761ade6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.395+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55761ade4930 0x55761ade6d90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42615a2581468a07 server_cookie=c61afcf31340d2f0 in_seq=0 out_seq=0 2024-06-03T02:33:58.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.395+0000 7ff3aaa00700 10 client.26878 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:33:58.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.398+0000 7ff3aaa00700 1 -- 172.21.15.79:0/3074390337 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55761ad21000 con 0x55761ade4930 2024-06-03T02:33:58.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.398+0000 7ff3aaa00700 10 client.26878 handle_command_reply: tid=0 2024-06-03T02:33:58.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.398+0000 7ff3d611fb80 1 client.26878 shutdown 2024-06-03T02:33:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.399+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761ad3de40 msgr2=0x55761adcbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.399+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761ad3de40 0x55761adcbac0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000aed0 tx=0x7ff3b0010d00 comp rx=0 tx=0).stop 2024-06-03T02:33:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.399+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 shutdown_connections 2024-06-03T02:33:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.399+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 0x55761adc9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.399+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761ad3de40 0x55761adcbac0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.399+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55761ade4930 0x55761ade6d90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.399+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/3074390337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761adc15b0 0x55761adc8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.399+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 >> 172.21.15.79:0/3074390337 conn(0x55761acee170 msgr2=0x55761adbc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.400+0000 7ff3aaa00700 0 client.26878 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:33:58.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.400+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 shutdown_connections 2024-06-03T02:33:58.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.400+0000 7ff3d611fb80 1 -- 172.21.15.79:0/3074390337 wait complete. 2024-06-03T02:33:58.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.400+0000 7ff3d611fb80 20 client.26878 trim_cache size 0 max 16384 2024-06-03T02:33:58.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.435+0000 7ff3d611fb80 1 Processor -- start 2024-06-03T02:33:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.435+0000 7ff3d611fb80 1 -- start start 2024-06-03T02:33:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.436+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adc45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.436+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adc15b0 0x55761adcd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.436+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adcc300 0x55761adcdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.436+0000 7ff3d611fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ac110 con 0x55761adc15b0 2024-06-03T02:33:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.436+0000 7ff3d611fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41adc50 con 0x55761ad3de40 2024-06-03T02:33:58.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.436+0000 7ff3d611fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ac7e0 con 0x55761adcc300 2024-06-03T02:33:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.436+0000 7ff3c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adc45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.436+0000 7ff3c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adc45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:56778/0 (socket says 172.21.15.79:56778) 2024-06-03T02:33:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.436+0000 7ff3c3e00700 1 -- 172.21.15.79:0/1805576928 learned_addr learned my addr 172.21.15.79:0/1805576928 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.437+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/1805576928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adcc300 0x55761adcdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.437+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/1805576928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adc15b0 0x55761adcd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.437+0000 7ff3c3e00700 1 -- 172.21.15.79:0/1805576928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adcc300 msgr2=0x55761adcdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.437+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/1805576928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adcc300 0x55761adcdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.437+0000 7ff3c3e00700 1 -- 172.21.15.79:0/1805576928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adc15b0 msgr2=0x55761adcd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:58.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.437+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/1805576928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adc15b0 0x55761adcd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.437+0000 7ff3c3e00700 1 -- 172.21.15.79:0/1805576928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761adc2bd0 con 0x55761ad3de40 2024-06-03T02:33:58.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.438+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/1805576928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adc45f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0010770 tx=0x7ff3b00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71a05795e55dcb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:58.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.439+0000 7ff3b5600700 1 -- 172.21.15.79:0/1805576928 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0021040 con 0x55761ad3de40 2024-06-03T02:33:58.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.439+0000 7ff3b5600700 1 -- 172.21.15.79:0/1805576928 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b000e3b0 con 0x55761ad3de40 2024-06-03T02:33:58.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.439+0000 7ff3b5600700 1 -- 172.21.15.79:0/1805576928 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b000ebd0 con 0x55761ad3de40 2024-06-03T02:33:58.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.441+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1805576928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 msgr2=0x55761adc45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:58.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.441+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/1805576928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adc45f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0010770 tx=0x7ff3b00047c0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.441+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1805576928 shutdown_connections 2024-06-03T02:33:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.442+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/1805576928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adcc300 0x55761adcdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.442+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/1805576928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adc45f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.442+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/1805576928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adc15b0 0x55761adcd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.442+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1805576928 >> 172.21.15.79:0/1805576928 conn(0x55761acee170 msgr2=0x55761adc76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.442+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1805576928 shutdown_connections 2024-06-03T02:33:58.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.443+0000 7ff3d611fb80 1 -- 172.21.15.79:0/1805576928 wait complete. 2024-06-03T02:33:58.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.444+0000 7ff3d611fb80 1 -- start start 2024-06-03T02:33:58.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.444+0000 7ff3d611fb80 1 Processor -- start 2024-06-03T02:33:58.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.445+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.445+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761ae4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.445+0000 7ff3d611fb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 0x55761ae4cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.445+0000 7ff3d611fb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ad580 con 0x55761adcc300 2024-06-03T02:33:58.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.445+0000 7ff3d611fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41aceb0 con 0x55761ad3de40 2024-06-03T02:33:58.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.445+0000 7ff3d611fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41aa0b0 con 0x55761adc15b0 2024-06-03T02:33:58.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.445+0000 7ff3d611fb80 10 client.0 _ll_register_callbacks cb 0x55761acad720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:33:58.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.446+0000 7ff3c8e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761ae4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.447+0000 7ff3c8e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761ae4c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59924/0 (socket says 172.21.15.79:59924) 2024-06-03T02:33:58.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.447+0000 7ff3c8e00700 1 -- 172.21.15.79:0/4119190750 learned_addr learned my addr 172.21.15.79:0/4119190750 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:33:58.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.446+0000 7ff3c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.447+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 0x55761ae4cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.447+0000 7ff3c8e00700 1 -- 172.21.15.79:0/4119190750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 msgr2=0x55761adecbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:58.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.448+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adecbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.448+0000 7ff3c8e00700 1 -- 172.21.15.79:0/4119190750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 msgr2=0x55761ae4cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:33:58.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.448+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 0x55761ae4cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.448+0000 7ff3c8e00700 1 -- 172.21.15.79:0/4119190750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761ae4d170 con 0x55761adc15b0 2024-06-03T02:33:58.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.448+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 0x55761ae4cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:33:58.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.449+0000 7ff3c8e00700 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761ae4c670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8003cc0 tx=0x7ff3b8003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c049eb9468ee2a82 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:33:58.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.449+0000 7ff3aaa00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:33:58.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.450+0000 7ff3aaa00700 1 -- 172.21.15.79:0/4119190750 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b802c070 con 0x55761adc15b0 2024-06-03T02:33:58.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.450+0000 7ff3aaa00700 1 -- 172.21.15.79:0/4119190750 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7ff3b8003880 con 0x55761adc15b0 2024-06-03T02:33:58.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.451+0000 7ff3d611fb80 1 -- 172.21.15.79:0/4119190750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b000e3b0 con 0x55761adc15b0 2024-06-03T02:33:58.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.451+0000 7ff3aaa00700 1 -- 172.21.15.79:0/4119190750 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff3b800b420 con 0x55761adc15b0 2024-06-03T02:33:58.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.451+0000 7ff3d611fb80 10 client.26884 fetch_fsmap learned FSMap version 30 2024-06-03T02:33:58.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.451+0000 7ff3d611fb80 1 -- 172.21.15.79:0/4119190750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55761adec480 con 0x55761adc15b0 2024-06-03T02:33:58.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.453+0000 7ff3aaa00700 1 -- 172.21.15.79:0/4119190750 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b000e3b0 con 0x55761adc15b0 2024-06-03T02:33:58.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.453+0000 7ff3aaa00700 1 -- 172.21.15.79:0/4119190750 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b802f020 con 0x55761adc15b0 2024-06-03T02:33:58.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.453+0000 7ff3d611fb80 10 client.26884 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:33:58.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.453+0000 7ff3d611fb80 10 client.26884 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:33:58.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.453+0000 7ff3d611fb80 20 client.26884 populate_metadata read hostname 'smithi079' 2024-06-03T02:33:58.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.453+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55761add6630 0x55761add8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:33:58.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.454+0000 7ff3d611fb80 4 client.26884 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:33:58.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.454+0000 7ff3d611fb80 1 -- 172.21.15.79:0/4119190750 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55761ad1b580 con 0x55761add6630 2024-06-03T02:33:58.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.454+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55761add6630 0x55761add8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:33:58.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.455+0000 7ff3aaa00700 10 client.26884 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:33:58.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.455+0000 7ff3c3e00700 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55761add6630 0x55761add8a50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=898df126e57102ea server_cookie=f9cf92056dbcd878 in_seq=0 out_seq=0 2024-06-03T02:33:58.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.455+0000 7ff3aaa00700 1 -- 172.21.15.79:0/4119190750 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55761ad1b580 con 0x55761add6630 2024-06-03T02:33:58.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.455+0000 7ff3aaa00700 10 client.26884 handle_command_reply: tid=0 2024-06-03T02:33:58.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.455+0000 7ff3d611fb80 1 client.26884 shutdown 2024-06-03T02:33:58.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 -- 172.21.15.79:0/4119190750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 msgr2=0x55761ae4c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:58.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761ae4c670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8003cc0 tx=0x7ff3b8003cf0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 -- 172.21.15.79:0/4119190750 shutdown_connections 2024-06-03T02:33:58.458 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:33:58.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55761adcc300 0x55761ae4cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55761add6630 0x55761add8a50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55761ad3de40 0x55761adecbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 --2- 172.21.15.79:0/4119190750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55761adc15b0 0x55761ae4c670 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3aaa00700 0 client.26884 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 -- 172.21.15.79:0/4119190750 >> 172.21.15.79:0/4119190750 conn(0x55761acee170 msgr2=0x55761adbbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 -- 172.21.15.79:0/4119190750 shutdown_connections 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 1 -- 172.21.15.79:0/4119190750 wait complete. 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.456+0000 7ff3d611fb80 20 client.26884 trim_cache size 0 max 16384 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 -- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff3ac0a2c70 msgr2=0x7ff3ac0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff3ac0a2c70 0x7ff3ac0a5130 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0997e0 tx=0x7ff3bc099730 comp rx=0 tx=0).stop 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 -- 172.21.15.79:0/3241027276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c4107840 msgr2=0x7ff3c41989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c4107840 0x7ff3c41989b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009850 tx=0x7ff3bc004dc0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 -- 172.21.15.79:0/3241027276 shutdown_connections 2024-06-03T02:33:58.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c4107840 0x7ff3c41989b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7ff3ac0a2c70 0x7ff3ac0a5130 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4102740 0x7ff3c419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 --2- 172.21.15.79:0/3241027276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3c41030f0 0x7ff3c419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:33:58.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.458+0000 7ff3cb800700 1 -- 172.21.15.79:0/3241027276 >> 172.21.15.79:0/3241027276 conn(0x7ff3c40795f0 msgr2=0x7ff3c4100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:33:58.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.460+0000 7ff3cb800700 1 -- 172.21.15.79:0/3241027276 shutdown_connections 2024-06-03T02:33:58.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:33:58.463+0000 7ff3cb800700 1 -- 172.21.15.79:0/3241027276 wait complete. 2024-06-03T02:33:59.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:34:00.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:33:59 smithi123 ceph-mon[36640]: pgmap v934: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 3.9 MiB/s wr, 434 op/s 2024-06-03T02:34:00.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:33:59 smithi079 ceph-mon[31819]: pgmap v934: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 3.9 MiB/s wr, 434 op/s 2024-06-03T02:34:00.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:33:59 smithi135 ceph-mon[36653]: pgmap v934: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 3.9 MiB/s wr, 434 op/s 2024-06-03T02:34:00.728 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:34:01.911 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:01 smithi079 ceph-mon[31819]: pgmap v935: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 5.4 MiB/s wr, 560 op/s 2024-06-03T02:34:01.911 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:01 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:34:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:01 smithi123 ceph-mon[36640]: pgmap v935: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 5.4 MiB/s wr, 560 op/s 2024-06-03T02:34:02.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:01 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:34:02.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:01 smithi135 ceph-mon[36653]: pgmap v935: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 5.4 MiB/s wr, 560 op/s 2024-06-03T02:34:02.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:01 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:34:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:02 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.272 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.272 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.272 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.272 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.272 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.272 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.272 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.273 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:02 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 835.0 s, 374.0 tps, lat 267.743 ms stddev 363.113 2024-06-03T02:34:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:02 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T02:34:03.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.657+0000 7fa4c4a00700 1 -- 172.21.15.79:0/4226105583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4c0109490 msgr2=0x7fa4c0109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:03.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.657+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/4226105583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4c0109490 0x7fa4c0109870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800b3e0 tx=0x7fa4b800b6f0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.657+0000 7fa4c4a00700 1 -- 172.21.15.79:0/4226105583 shutdown_connections 2024-06-03T02:34:03.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.657+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/4226105583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4c01155e0 0x7fa4c01179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.657+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/4226105583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4c0109490 0x7fa4c0109870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.657+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/4226105583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4c0109db0 0x7fa4c01150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.658+0000 7fa4c4a00700 1 -- 172.21.15.79:0/4226105583 >> 172.21.15.79:0/4226105583 conn(0x7fa4c006d600 msgr2=0x7fa4c006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:03.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.659+0000 7fa4c4a00700 1 -- 172.21.15.79:0/4226105583 shutdown_connections 2024-06-03T02:34:03.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.659+0000 7fa4c4a00700 1 -- 172.21.15.79:0/4226105583 wait complete. 2024-06-03T02:34:03.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.665+0000 7fa4c4a00700 1 Processor -- start 2024-06-03T02:34:03.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.665+0000 7fa4c4a00700 1 -- start start 2024-06-03T02:34:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.666+0000 7fa4c4a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4c0109490 0x7fa4c0111cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.666+0000 7fa4c4a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4c0109db0 0x7fa4c0112200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.666+0000 7fa4c4a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4c01155e0 0x7fa4c0114570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.667+0000 7fa4c4a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c010c3e0 con 0x7fa4c0109db0 2024-06-03T02:34:03.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.667+0000 7fa4bde00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4c0109490 0x7fa4c0111cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.667+0000 7fa4be800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4c01155e0 0x7fa4c0114570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.667+0000 7fa4be800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4c01155e0 0x7fa4c0114570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59942/0 (socket says 172.21.15.79:59942) 2024-06-03T02:34:03.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.667+0000 7fa4be800700 1 -- 172.21.15.79:0/775854049 learned_addr learned my addr 172.21.15.79:0/775854049 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:03.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.667+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c010c260 con 0x7fa4c0109490 2024-06-03T02:34:03.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.667+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c010c560 con 0x7fa4c01155e0 2024-06-03T02:34:03.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.668+0000 7fa4bd400700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4c0109db0 0x7fa4c0112200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.668+0000 7fa4be800700 1 -- 172.21.15.79:0/775854049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4c0109490 msgr2=0x7fa4c0111cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:03.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.668+0000 7fa4be800700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4c0109490 0x7fa4c0111cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.668+0000 7fa4be800700 1 -- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4c0109db0 msgr2=0x7fa4c0112200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:03.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.668+0000 7fa4be800700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4c0109db0 0x7fa4c0112200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.668+0000 7fa4be800700 1 -- 172.21.15.79:0/775854049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c0114c70 con 0x7fa4c01155e0 2024-06-03T02:34:03.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.670+0000 7fa4be800700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4c01155e0 0x7fa4c0114570 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00c970 tx=0x7fa4ac00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:03.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.670+0000 7fa4b6a00700 1 -- 172.21.15.79:0/775854049 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac00ce60 con 0x7fa4c01155e0 2024-06-03T02:34:03.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.670+0000 7fa4b6a00700 1 -- 172.21.15.79:0/775854049 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac004830 con 0x7fa4c01155e0 2024-06-03T02:34:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.670+0000 7fa4b6a00700 1 -- 172.21.15.79:0/775854049 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0055a0 con 0x7fa4c01155e0 2024-06-03T02:34:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.670+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c0114f60 con 0x7fa4c01155e0 2024-06-03T02:34:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.670+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c01b46c0 con 0x7fa4c01155e0 2024-06-03T02:34:03.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.685+0000 7fa4cf3bdb80 1 Processor -- start 2024-06-03T02:34:03.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.685+0000 7fa4b6a00700 1 -- 172.21.15.79:0/775854049 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa4ac0049e0 con 0x7fa4c01155e0 2024-06-03T02:34:03.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.687+0000 7fa4cf3bdb80 1 -- start start 2024-06-03T02:34:03.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.690+0000 7fa4b6a00700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa4a40a2df0 0x7fa4a40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.690+0000 7fa4b6a00700 1 -- 172.21.15.79:0/775854049 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa4ac0cf150 con 0x7fa4c01155e0 2024-06-03T02:34:03.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.691+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d537ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.691+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d5379d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.691+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d537a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.692+0000 7fa4be800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d537a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.692+0000 7fa4be800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d537a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:56820/0 (socket says 172.21.15.79:56820) 2024-06-03T02:34:03.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.692+0000 7fa4be800700 1 -- 172.21.15.79:0/1670264355 learned_addr learned my addr 172.21.15.79:0/1670264355 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:03.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.693+0000 7fa4bd400700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa4a40a2df0 0x7fa4a40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.693+0000 7fa4bde00700 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d537ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.695+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1670264355 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d8d5288a10 con 0x55d8d53804b0 2024-06-03T02:34:03.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.695+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1670264355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8d52cbb00 con 0x55d8d537a240 2024-06-03T02:34:03.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.696+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1670264355 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8d52868f0 con 0x55d8d537fb90 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.697+0000 7fa4bd400700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa4a40a2df0 0x7fa4a40a52b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0005e50 tx=0x7fa4b0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.698+0000 7fa4be800700 1 -- 172.21.15.79:0/1670264355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 msgr2=0x55d8d537ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.698+0000 7fa4be800700 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d537ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.698+0000 7fa4be800700 1 -- 172.21.15.79:0/1670264355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 msgr2=0x55d8d5379d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.698+0000 7fa4bd400700 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d5379d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.700+0000 7fa4be800700 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d5379d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.700+0000 7fa4be800700 1 -- 172.21.15.79:0/1670264355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8d5273f90 con 0x55d8d537a240 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.700+0000 7fa4bde00700 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d537ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.700+0000 7fa4bd400700 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d5379d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.700+0000 7fa4be800700 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d537a680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac09b530 tx=0x7fa4ac004f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e03b40a926ec62 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:03.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.701+0000 7fa4ab400700 1 -- 172.21.15.79:0/1670264355 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0b4070 con 0x55d8d537a240 2024-06-03T02:34:03.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.701+0000 7fa4ab400700 1 -- 172.21.15.79:0/1670264355 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac09bca0 con 0x55d8d537a240 2024-06-03T02:34:03.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.703+0000 7fa4ab400700 1 -- 172.21.15.79:0/1670264355 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac01d040 con 0x55d8d537a240 2024-06-03T02:34:03.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.704+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1670264355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 msgr2=0x55d8d537a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:03.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.704+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d537a680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac09b530 tx=0x7fa4ac004f40 comp rx=0 tx=0).stop 2024-06-03T02:34:03.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.705+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1670264355 shutdown_connections 2024-06-03T02:34:03.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.705+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d537a680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.705+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d537ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.706+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1670264355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d5379d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.706+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1670264355 >> 172.21.15.79:0/1670264355 conn(0x55d8d52a6170 msgr2=0x55d8d5374ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:03.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.706+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1670264355 shutdown_connections 2024-06-03T02:34:03.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.706+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1670264355 wait complete. 2024-06-03T02:34:03.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.708+0000 7fa4cf3bdb80 1 -- start start 2024-06-03T02:34:03.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.708+0000 7fa4cf3bdb80 1 Processor -- start 2024-06-03T02:34:03.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.712+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537a240 0x55d8d53862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.715+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d537fb90 0x55d8d53835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.715+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5383af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.716+0000 7fa4cf3bdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d8d5267f60 con 0x55d8d537fb90 2024-06-03T02:34:03.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.716+0000 7fa4cf3bdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8d52c7ac0 con 0x55d8d53804b0 2024-06-03T02:34:03.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.716+0000 7fa4cf3bdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8d5386e90 con 0x55d8d537a240 2024-06-03T02:34:03.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.717+0000 7fa4be800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5383af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.717+0000 7fa4be800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5383af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:56824/0 (socket says 172.21.15.79:56824) 2024-06-03T02:34:03.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.717+0000 7fa4be800700 1 -- 172.21.15.79:0/1763979803 learned_addr learned my addr 172.21.15.79:0/1763979803 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:03.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.717+0000 7fa4cf3bdb80 10 client.0 _ll_register_callbacks cb 0x55d8d5261380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.717+0000 7fa4be800700 1 -- 172.21.15.79:0/1763979803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537a240 msgr2=0x55d8d53862b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.717+0000 7fa4be800700 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537a240 0x55d8d53862b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.717+0000 7fa4be800700 1 -- 172.21.15.79:0/1763979803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d537fb90 msgr2=0x55d8d53835b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.717+0000 7fa4be800700 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d537fb90 0x55d8d53835b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.717+0000 7fa4be800700 1 -- 172.21.15.79:0/1763979803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8d521a8d0 con 0x55d8d53804b0 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.720+0000 7fa4be800700 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5383af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0a3ce0 tx=0x7fa4ac0a4330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a41eb92f29c06bb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.720+0000 7fa49fe00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.720+0000 7fa49fe00700 1 -- 172.21.15.79:0/1763979803 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0a9060 con 0x55d8d53804b0 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.720+0000 7fa49fe00700 1 -- 172.21.15.79:0/1763979803 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa4ac09b800 con 0x55d8d53804b0 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.720+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1763979803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ac09bca0 con 0x55d8d53804b0 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.721+0000 7fa49fe00700 1 -- 172.21.15.79:0/1763979803 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac0b4050 con 0x55d8d53804b0 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.721+0000 7fa4cf3bdb80 10 client.26897 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.721+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1763979803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8d53915e0 con 0x55d8d53804b0 2024-06-03T02:34:03.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.721+0000 7fa49fe00700 1 -- 172.21.15.79:0/1763979803 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac09bca0 con 0x55d8d53804b0 2024-06-03T02:34:03.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.723+0000 7fa49fe00700 1 -- 172.21.15.79:0/1763979803 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa4ac007b00 con 0x55d8d53804b0 2024-06-03T02:34:03.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.723+0000 7fa4cf3bdb80 10 client.26897 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:03.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.723+0000 7fa4cf3bdb80 10 client.26897 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:34:03.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.723+0000 7fa4cf3bdb80 20 client.26897 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:03.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.723+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55d8d5393ab0 0x55d8d5395ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.723+0000 7fa4cf3bdb80 4 client.26897 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:34:03.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.723+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1763979803 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8d52d9000 con 0x55d8d5393ab0 2024-06-03T02:34:03.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.725+0000 7fa4bde00700 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55d8d5393ab0 0x55d8d5395ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.729+0000 7fa4bde00700 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55d8d5393ab0 0x55d8d5395ea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78acd2d351d8efaa server_cookie=366b37b61880f7ba in_seq=0 out_seq=0 2024-06-03T02:34:03.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.730+0000 7fa49fe00700 10 client.26897 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:34:03.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.731+0000 7fa49fe00700 1 -- 172.21.15.79:0/1763979803 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d8d52d9000 con 0x55d8d5393ab0 2024-06-03T02:34:03.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.731+0000 7fa49fe00700 10 client.26897 handle_command_reply: tid=0 2024-06-03T02:34:03.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.731+0000 7fa4cf3bdb80 1 client.26897 shutdown 2024-06-03T02:34:03.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.732+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1763979803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 msgr2=0x55d8d5383af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:03.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.732+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5383af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0a3ce0 tx=0x7fa4ac0a4330 comp rx=0 tx=0).stop 2024-06-03T02:34:03.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.732+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1763979803 shutdown_connections 2024-06-03T02:34:03.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.732+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5383af0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.732+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55d8d5393ab0 0x55d8d5395ea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.733+0000 7fa49fe00700 0 client.26897 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:34:03.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.733+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537a240 0x55d8d53862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.733+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/1763979803 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d537fb90 0x55d8d53835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.733+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1763979803 >> 172.21.15.79:0/1763979803 conn(0x55d8d52a6170 msgr2=0x55d8d5378cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:03.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.733+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1763979803 shutdown_connections 2024-06-03T02:34:03.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.733+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/1763979803 wait complete. 2024-06-03T02:34:03.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.734+0000 7fa4cf3bdb80 20 client.26897 trim_cache size 0 max 16384 2024-06-03T02:34:03.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.804+0000 7fa4cf3bdb80 1 Processor -- start 2024-06-03T02:34:03.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.804+0000 7fa4cf3bdb80 1 -- start start 2024-06-03T02:34:03.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.805+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537a240 0x55d8d537dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.805+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d537fb90 0x55d8d536f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.805+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5385df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.805+0000 7fa4cf3bdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01c5b10 con 0x55d8d537fb90 2024-06-03T02:34:03.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.805+0000 7fa4cf3bdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01c7650 con 0x55d8d53804b0 2024-06-03T02:34:03.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.805+0000 7fa4cf3bdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01c61e0 con 0x55d8d537a240 2024-06-03T02:34:03.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.805+0000 7fa4be800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5385df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.805+0000 7fa4be800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5385df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:56832/0 (socket says 172.21.15.79:56832) 2024-06-03T02:34:03.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.805+0000 7fa4be800700 1 -- 172.21.15.79:0/3416428180 learned_addr learned my addr 172.21.15.79:0/3416428180 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:03.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.806+0000 7fa4bde00700 1 --2- 172.21.15.79:0/3416428180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537a240 0x55d8d537dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.806+0000 7fa4bd400700 1 --2- 172.21.15.79:0/3416428180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d537fb90 0x55d8d536f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.806+0000 7fa4be800700 1 -- 172.21.15.79:0/3416428180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537a240 msgr2=0x55d8d537dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:03.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.806+0000 7fa4be800700 1 --2- 172.21.15.79:0/3416428180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537a240 0x55d8d537dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.806+0000 7fa4be800700 1 -- 172.21.15.79:0/3416428180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d537fb90 msgr2=0x55d8d536f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:03.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.806+0000 7fa4be800700 1 --2- 172.21.15.79:0/3416428180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d537fb90 0x55d8d536f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.806+0000 7fa4be800700 1 -- 172.21.15.79:0/3416428180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8d5375100 con 0x55d8d53804b0 2024-06-03T02:34:03.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.808+0000 7fa4be800700 1 --2- 172.21.15.79:0/3416428180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5385df0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac097f20 tx=0x7fa4ac0a40f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce278053ce3ee8f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:03.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.815+0000 7fa4aaa00700 1 -- 172.21.15.79:0/3416428180 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac01d030 con 0x55d8d53804b0 2024-06-03T02:34:03.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.815+0000 7fa4aaa00700 1 -- 172.21.15.79:0/3416428180 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac09bca0 con 0x55d8d53804b0 2024-06-03T02:34:03.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.815+0000 7fa4aaa00700 1 -- 172.21.15.79:0/3416428180 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0b4050 con 0x55d8d53804b0 2024-06-03T02:34:03.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.816+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/3416428180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 msgr2=0x55d8d5385df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:03.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.816+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/3416428180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5385df0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac097f20 tx=0x7fa4ac0a40f0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.817+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/3416428180 shutdown_connections 2024-06-03T02:34:03.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.817+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/3416428180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d53804b0 0x55d8d5385df0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.817+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/3416428180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537a240 0x55d8d537dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.817+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/3416428180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d537fb90 0x55d8d536f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.818+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/3416428180 >> 172.21.15.79:0/3416428180 conn(0x55d8d52a6170 msgr2=0x55d8d537d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:03.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.818+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/3416428180 shutdown_connections 2024-06-03T02:34:03.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.819+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/3416428180 wait complete. 2024-06-03T02:34:03.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.820+0000 7fa4cf3bdb80 1 -- start start 2024-06-03T02:34:03.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.820+0000 7fa4cf3bdb80 1 Processor -- start 2024-06-03T02:34:03.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.824+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d5396680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.824+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d5396bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.825+0000 7fa4cf3bdb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d53f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.825+0000 7fa4bde00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d5396680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.825+0000 7fa4cf3bdb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01c6f80 con 0x55d8d53804b0 2024-06-03T02:34:03.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.825+0000 7fa4cf3bdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c01c68b0 con 0x55d8d537a240 2024-06-03T02:34:03.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.825+0000 7fa4cf3bdb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0124870 con 0x55d8d537fb90 2024-06-03T02:34:03.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.825+0000 7fa4cf3bdb80 10 client.0 _ll_register_callbacks cb 0x55d8d5261380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:03.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.825+0000 7fa4bd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d5396bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.825+0000 7fa4bd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d5396bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:59970/0 (socket says 172.21.15.79:59970) 2024-06-03T02:34:03.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.825+0000 7fa4bd400700 1 -- 172.21.15.79:0/844889721 learned_addr learned my addr 172.21.15.79:0/844889721 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:03.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.827+0000 7fa4be800700 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d53f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.829+0000 7fa4bde00700 1 -- 172.21.15.79:0/844889721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 msgr2=0x55d8d5396bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:03.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.829+0000 7fa4bde00700 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d5396bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.829+0000 7fa4bde00700 1 -- 172.21.15.79:0/844889721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 msgr2=0x55d8d53f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:03.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.829+0000 7fa4bde00700 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d53f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.829+0000 7fa4bde00700 1 -- 172.21.15.79:0/844889721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8d53f6a90 con 0x55d8d537a240 2024-06-03T02:34:03.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.829+0000 7fa4be800700 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d53f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:03.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.830+0000 7fa4bde00700 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d5396680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80060b0 tx=0x7fa4b8009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ff37aa9543010f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:03.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.830+0000 7fa49fe00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:03.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.832+0000 7fa49fe00700 1 -- 172.21.15.79:0/844889721 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8003cd0 con 0x55d8d537a240 2024-06-03T02:34:03.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.832+0000 7fa49fe00700 1 -- 172.21.15.79:0/844889721 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fa4b8007420 con 0x55d8d537a240 2024-06-03T02:34:03.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.833+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/844889721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ac09bca0 con 0x55d8d537a240 2024-06-03T02:34:03.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.833+0000 7fa49fe00700 1 -- 172.21.15.79:0/844889721 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4b8033050 con 0x55d8d537a240 2024-06-03T02:34:03.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.834+0000 7fa4cf3bdb80 10 client.26903 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:03.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.834+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/844889721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8d53fde80 con 0x55d8d537a240 2024-06-03T02:34:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.834+0000 7fa49fe00700 1 -- 172.21.15.79:0/844889721 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac09bca0 con 0x55d8d537a240 2024-06-03T02:34:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.835+0000 7fa49fe00700 1 -- 172.21.15.79:0/844889721 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa4b8011070 con 0x55d8d537a240 2024-06-03T02:34:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.835+0000 7fa4cf3bdb80 10 client.26903 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.835+0000 7fa4cf3bdb80 10 client.26903 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:34:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.835+0000 7fa4cf3bdb80 20 client.26903 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.835+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55d8d53fe5e0 0x55d8d5400a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.835+0000 7fa4cf3bdb80 4 client.26903 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:34:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.835+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/844889721 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8d52d3580 con 0x55d8d53fe5e0 2024-06-03T02:34:03.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.836+0000 7fa4bd400700 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55d8d53fe5e0 0x55d8d5400a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:03.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.837+0000 7fa4bd400700 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55d8d53fe5e0 0x55d8d5400a00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=956c94874166aa7c server_cookie=4740aeac3215d167 in_seq=0 out_seq=0 2024-06-03T02:34:03.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.837+0000 7fa49fe00700 10 client.26903 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:34:03.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.838+0000 7fa49fe00700 1 -- 172.21.15.79:0/844889721 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8d52d3580 con 0x55d8d53fe5e0 2024-06-03T02:34:03.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.838+0000 7fa49fe00700 10 client.26903 handle_command_reply: tid=0 2024-06-03T02:34:03.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.838+0000 7fa4cf3bdb80 1 client.26903 shutdown 2024-06-03T02:34:03.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.838+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/844889721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 msgr2=0x55d8d5396680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:03.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.838+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d5396680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80060b0 tx=0x7fa4b8009360 comp rx=0 tx=0).stop 2024-06-03T02:34:03.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.839+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/844889721 shutdown_connections 2024-06-03T02:34:03.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.839+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d8d53804b0 0x55d8d53f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.839+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8d537a240 0x55d8d5396680 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.839+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55d8d53fe5e0 0x55d8d5400a00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.839+0000 7fa4cf3bdb80 1 --2- 172.21.15.79:0/844889721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d537fb90 0x55d8d5396bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.839+0000 7fa49fe00700 0 client.26903 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:34:03.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.839+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/844889721 >> 172.21.15.79:0/844889721 conn(0x55d8d52a6170 msgr2=0x55d8d5380950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:03.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.841+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/844889721 shutdown_connections 2024-06-03T02:34:03.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.841+0000 7fa4cf3bdb80 1 -- 172.21.15.79:0/844889721 wait complete. 2024-06-03T02:34:03.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.841+0000 7fa4cf3bdb80 20 client.26903 trim_cache size 0 max 16384 2024-06-03T02:34:03.842 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:34:03.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.845+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa4a40a2df0 msgr2=0x7fa4a40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:03.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.845+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa4a40a2df0 0x7fa4a40a52b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0005e50 tx=0x7fa4b0005dc0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.845+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4c01155e0 msgr2=0x7fa4c0114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:03.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.845+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4c01155e0 0x7fa4c0114570 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00c970 tx=0x7fa4ac00cc80 comp rx=0 tx=0).stop 2024-06-03T02:34:03.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.845+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 shutdown_connections 2024-06-03T02:34:03.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.845+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4c01155e0 0x7fa4c0114570 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.845+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4c0109490 0x7fa4c0111cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.845+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fa4a40a2df0 0x7fa4a40a52b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.846+0000 7fa4c4a00700 1 --2- 172.21.15.79:0/775854049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa4c0109db0 0x7fa4c0112200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:03.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.846+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 >> 172.21.15.79:0/775854049 conn(0x7fa4c006d600 msgr2=0x7fa4c0071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:03.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.846+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 shutdown_connections 2024-06-03T02:34:03.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:03.846+0000 7fa4c4a00700 1 -- 172.21.15.79:0/775854049 wait complete. 2024-06-03T02:34:04.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:03 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-06-03T02:34:04.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:03 smithi123 ceph-mon[36640]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-03T02:34:04.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:03 smithi123 ceph-mon[36640]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-03T02:34:04.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:03 smithi123 ceph-mon[36640]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-03T02:34:04.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:03 smithi123 ceph-mon[36640]: pgmap v936: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.6 MiB/s wr, 553 op/s 2024-06-03T02:34:04.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:03 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:34:04.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:03 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-06-03T02:34:04.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:03 smithi079 ceph-mon[31819]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-03T02:34:04.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:03 smithi079 ceph-mon[31819]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-03T02:34:04.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:03 smithi079 ceph-mon[31819]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-03T02:34:04.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:03 smithi079 ceph-mon[31819]: pgmap v936: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.6 MiB/s wr, 553 op/s 2024-06-03T02:34:04.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:03 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:34:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:03 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-06-03T02:34:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:03 smithi135 ceph-mon[36653]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-03T02:34:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:03 smithi135 ceph-mon[36653]: Unable to set osd_memory_target on smithi123 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-06-03T02:34:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:03 smithi135 ceph-mon[36653]: Unable to set osd_memory_target on smithi135 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-03T02:34:04.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:03 smithi135 ceph-mon[36653]: pgmap v936: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.6 MiB/s wr, 553 op/s 2024-06-03T02:34:04.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:03 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:34:05.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:34:05.295 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:04 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:34:05.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:04 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:34:05.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:04 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:34:05.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:04 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:34:05.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:04 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:34:06.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:05 smithi123 ceph-mon[36640]: pgmap v937: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 8.7 MiB/s wr, 801 op/s 2024-06-03T02:34:06.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:05 smithi079 ceph-mon[31819]: pgmap v937: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 8.7 MiB/s wr, 801 op/s 2024-06-03T02:34:06.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:05 smithi135 ceph-mon[36653]: pgmap v937: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 8.7 MiB/s wr, 801 op/s 2024-06-03T02:34:06.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:34:07.084 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:34:08.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:07 smithi079 ceph-mon[31819]: pgmap v938: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 8.4 MiB/s wr, 763 op/s 2024-06-03T02:34:08.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:07 smithi123 ceph-mon[36640]: pgmap v938: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 8.4 MiB/s wr, 763 op/s 2024-06-03T02:34:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 840.0 s, 404.6 tps, lat 243.186 ms stddev 330.590 2024-06-03T02:34:08.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:07 smithi135 ceph-mon[36653]: pgmap v938: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 8.4 MiB/s wr, 763 op/s 2024-06-03T02:34:10.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:09 smithi079 ceph-mon[31819]: pgmap v939: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 9.4 MiB/s wr, 840 op/s 2024-06-03T02:34:10.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:09 smithi135 ceph-mon[36653]: pgmap v939: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 9.4 MiB/s wr, 840 op/s 2024-06-03T02:34:10.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:09 smithi123 ceph-mon[36640]: pgmap v939: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 9.4 MiB/s wr, 840 op/s 2024-06-03T02:34:12.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:11 smithi079 ceph-mon[31819]: pgmap v940: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-06-03T02:34:12.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:11 smithi135 ceph-mon[36653]: pgmap v940: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-06-03T02:34:12.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:11 smithi123 ceph-mon[36640]: pgmap v940: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-06-03T02:34:13.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 845.0 s, 340.3 tps, lat 299.320 ms stddev 532.183 2024-06-03T02:34:13.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.681+0000 7f8cdaa00700 1 -- 172.21.15.79:0/2004983198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 msgr2=0x7f8cd4109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:13.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.681+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/2004983198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 0x7f8cd4109e30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0098f0 tx=0x7f8cbc009c00 comp rx=0 tx=0).stop 2024-06-03T02:34:13.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.681+0000 7f8cdaa00700 1 -- 172.21.15.79:0/2004983198 shutdown_connections 2024-06-03T02:34:13.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.681+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/2004983198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd4117520 0x7f8cd4119910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.681+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/2004983198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd410cbb0 0x7f8cd4109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.681+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/2004983198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 0x7f8cd4109e30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.681+0000 7f8cdaa00700 1 -- 172.21.15.79:0/2004983198 >> 172.21.15.79:0/2004983198 conn(0x7f8cd406d600 msgr2=0x7f8cd406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:13.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.681+0000 7f8cdaa00700 1 -- 172.21.15.79:0/2004983198 shutdown_connections 2024-06-03T02:34:13.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.681+0000 7f8cdaa00700 1 -- 172.21.15.79:0/2004983198 wait complete. 2024-06-03T02:34:13.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.683+0000 7f8cdaa00700 1 Processor -- start 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.683+0000 7f8cdaa00700 1 -- start start 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.683+0000 7f8cdaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 0x7f8cd410d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.683+0000 7f8cdaa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd410cbb0 0x7f8cd410d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cdaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4117520 0x7f8cd410db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cdaa00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd411d1d0 con 0x7f8cd410cbb0 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cdaa00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd411d050 con 0x7f8cd4117520 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cdaa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd411d350 con 0x7f8cd41099d0 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cd3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 0x7f8cd410d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cd3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 0x7f8cd410d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44064/0 (socket says 172.21.15.79:44064) 2024-06-03T02:34:13.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cd3e00700 1 -- 172.21.15.79:0/3215961812 learned_addr learned my addr 172.21.15.79:0/3215961812 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:13.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cd3e00700 1 -- 172.21.15.79:0/3215961812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4117520 msgr2=0x7f8cd410db90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T02:34:13.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cd3e00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4117520 0x7f8cd410db90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cd3e00700 1 -- 172.21.15.79:0/3215961812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd410cbb0 msgr2=0x7f8cd410d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:13.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cd3e00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd410cbb0 0x7f8cd410d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.684+0000 7f8cd3e00700 1 -- 172.21.15.79:0/3215961812 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd41a3480 con 0x7f8cd41099d0 2024-06-03T02:34:13.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.686+0000 7f8cd3e00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 0x7f8cd410d110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400e9f0 tx=0x7f8cc400edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:13.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.686+0000 7f8cd0c00700 1 -- 172.21.15.79:0/3215961812 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400cbd0 con 0x7f8cd41099d0 2024-06-03T02:34:13.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.686+0000 7f8cd0c00700 1 -- 172.21.15.79:0/3215961812 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc400cd70 con 0x7f8cd41099d0 2024-06-03T02:34:13.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.686+0000 7f8cd0c00700 1 -- 172.21.15.79:0/3215961812 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4010430 con 0x7f8cd41099d0 2024-06-03T02:34:13.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.686+0000 7f8cdaa00700 1 -- 172.21.15.79:0/3215961812 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd41a3770 con 0x7f8cd41099d0 2024-06-03T02:34:13.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.686+0000 7f8cdaa00700 1 -- 172.21.15.79:0/3215961812 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd41a3c60 con 0x7f8cd41099d0 2024-06-03T02:34:13.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.696+0000 7f8cd0c00700 1 -- 172.21.15.79:0/3215961812 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8cc4004750 con 0x7f8cd41099d0 2024-06-03T02:34:13.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.698+0000 7f8cd0c00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8cb80a2d70 0x7f8cb80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.698+0000 7f8cd0c00700 1 -- 172.21.15.79:0/3215961812 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8cc40cf5a0 con 0x7f8cd41099d0 2024-06-03T02:34:13.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.700+0000 7f8cd3400700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8cb80a2d70 0x7f8cb80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.701+0000 7f8ce52c1b80 1 Processor -- start 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.702+0000 7f8ce52c1b80 1 -- start start 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.702+0000 7f8cd3400700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8cb80a2d70 0x7f8cb80a5230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009680 tx=0x7f8cbc009f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.702+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f82064e40 0x559f820e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.702+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820e85b0 0x559f820e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.702+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820f3300 0x559f820f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.702+0000 7f8ce52c1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f81ff7a10 con 0x559f820e85b0 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.702+0000 7f8ce52c1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559f8203ab00 con 0x559f82064e40 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.702+0000 7f8ce52c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559f81ff58f0 con 0x559f820f3300 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.704+0000 7f8cd3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820e85b0 0x559f820e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.704+0000 7f8cd3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820e85b0 0x559f820e8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52618/0 (socket says 172.21.15.79:52618) 2024-06-03T02:34:13.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.704+0000 7f8cd3400700 1 -- 172.21.15.79:0/2584525667 learned_addr learned my addr 172.21.15.79:0/2584525667 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:13.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.705+0000 7f8cd3400700 1 -- 172.21.15.79:0/2584525667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820f3300 msgr2=0x559f820f56f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:34:13.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.705+0000 7f8cd3400700 1 --2- 172.21.15.79:0/2584525667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820f3300 0x559f820f56f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.705+0000 7f8cd3400700 1 -- 172.21.15.79:0/2584525667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f82064e40 msgr2=0x559f820e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:13.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.705+0000 7f8cd3400700 1 --2- 172.21.15.79:0/2584525667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f82064e40 0x559f820e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.705+0000 7f8cd3400700 1 -- 172.21.15.79:0/2584525667 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f81fe2f90 con 0x559f820e85b0 2024-06-03T02:34:13.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.706+0000 7f8cd3400700 1 --2- 172.21.15.79:0/2584525667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820e85b0 0x559f820e8a30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc016000 tx=0x7f8cbc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651c938ad9c5bf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:13.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.708+0000 7f8cc9600700 1 -- 172.21.15.79:0/2584525667 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc005040 con 0x559f820e85b0 2024-06-03T02:34:13.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.709+0000 7f8cc9600700 1 -- 172.21.15.79:0/2584525667 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc0056c0 con 0x559f820e85b0 2024-06-03T02:34:13.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.709+0000 7f8cc9600700 1 -- 172.21.15.79:0/2584525667 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc003e40 con 0x559f820e85b0 2024-06-03T02:34:13.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.710+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2584525667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820e85b0 msgr2=0x559f820e8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:13.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.710+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2584525667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820e85b0 0x559f820e8a30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc016000 tx=0x7f8cbc01c000 comp rx=0 tx=0).stop 2024-06-03T02:34:13.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.710+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2584525667 shutdown_connections 2024-06-03T02:34:13.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.710+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2584525667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820f3300 0x559f820f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.710+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2584525667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f82064e40 0x559f820e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.710+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2584525667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820e85b0 0x559f820e8a30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.710+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2584525667 >> 172.21.15.79:0/2584525667 conn(0x559f82015170 msgr2=0x559f820e44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:13.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.710+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2584525667 shutdown_connections 2024-06-03T02:34:13.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.710+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2584525667 wait complete. 2024-06-03T02:34:13.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.711+0000 7f8ce52c1b80 1 -- start start 2024-06-03T02:34:13.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.711+0000 7f8ce52c1b80 1 Processor -- start 2024-06-03T02:34:13.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f82064e40 0x559f820f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820e85b0 0x559f820efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 0x559f820f0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8ce52c1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559f81fd6f60 con 0x559f820f3300 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8ce52c1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559f82036ac0 con 0x559f820e85b0 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8ce52c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559f820f5f60 con 0x559f82064e40 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8ce52c1b80 10 client.0 _ll_register_callbacks cb 0x559f81fd4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8cd3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820e85b0 0x559f820efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8cd3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820e85b0 0x559f820efdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:56240/0 (socket says 172.21.15.79:56240) 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.713+0000 7f8cd3400700 1 -- 172.21.15.79:0/3916274687 learned_addr learned my addr 172.21.15.79:0/3916274687 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.714+0000 7f8cd3400700 1 -- 172.21.15.79:0/3916274687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f82064e40 msgr2=0x559f820f2ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.714+0000 7f8cd3400700 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f82064e40 0x559f820f2ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.714+0000 7f8cd3400700 1 -- 172.21.15.79:0/3916274687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 msgr2=0x559f820f0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:34:13.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.714+0000 7f8cd8a00700 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 0x559f820f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.714+0000 7f8cd3400700 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 0x559f820f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.714+0000 7f8cd3400700 1 -- 172.21.15.79:0/3916274687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f81f898d0 con 0x559f820e85b0 2024-06-03T02:34:13.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.714+0000 7f8cd8a00700 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 0x559f820f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:13.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.714+0000 7f8cd3400700 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820e85b0 0x559f820efdc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc003910 tx=0x7f8cbc0039f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f7f92cc6311ed5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:13.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.714+0000 7f8cc2000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:13.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.715+0000 7f8cc2000700 1 -- 172.21.15.79:0/3916274687 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc0054c0 con 0x559f820e85b0 2024-06-03T02:34:13.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.715+0000 7f8cc2000700 1 -- 172.21.15.79:0/3916274687 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8cbc01d040 con 0x559f820e85b0 2024-06-03T02:34:13.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.715+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3916274687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f82048000 con 0x559f820e85b0 2024-06-03T02:34:13.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.716+0000 7f8cc2000700 1 -- 172.21.15.79:0/3916274687 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8cbc03c050 con 0x559f820e85b0 2024-06-03T02:34:13.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.716+0000 7f8ce52c1b80 10 client.26909 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:13.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.716+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3916274687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f821006e0 con 0x559f820e85b0 2024-06-03T02:34:13.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.716+0000 7f8cc2000700 1 -- 172.21.15.79:0/3916274687 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559f82048000 con 0x559f820e85b0 2024-06-03T02:34:13.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.716+0000 7f8cc2000700 1 -- 172.21.15.79:0/3916274687 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8cbc02d940 con 0x559f820e85b0 2024-06-03T02:34:13.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.716+0000 7f8ce52c1b80 10 client.26909 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:13.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.716+0000 7f8ce52c1b80 10 client.26909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:34:13.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.717+0000 7f8ce52c1b80 20 client.26909 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:13.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.717+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559f82102d40 0x559f82105180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.717+0000 7f8ce52c1b80 4 client.26909 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:34:13.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.717+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3916274687 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f82042580 con 0x559f82102d40 2024-06-03T02:34:13.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.717+0000 7f8cd3e00700 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559f82102d40 0x559f82105180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.718+0000 7f8cc2000700 10 client.26909 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:13.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.718+0000 7f8cd3e00700 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559f82102d40 0x559f82105180 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60dc9884e2034f4 server_cookie=6a5b82eb1273ce45 in_seq=0 out_seq=0 2024-06-03T02:34:13.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.719+0000 7f8cc2000700 1 -- 172.21.15.79:0/3916274687 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f82042580 con 0x559f82102d40 2024-06-03T02:34:13.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.719+0000 7f8cc2000700 10 client.26909 handle_command_reply: tid=0 2024-06-03T02:34:13.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.719+0000 7f8ce52c1b80 1 client.26909 shutdown 2024-06-03T02:34:13.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.720+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3916274687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820e85b0 msgr2=0x559f820efdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:13.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.720+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820e85b0 0x559f820efdc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc003910 tx=0x7f8cbc0039f0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.720+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3916274687 shutdown_connections 2024-06-03T02:34:13.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.720+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559f82102d40 0x559f82105180 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.720+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 0x559f820f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.720+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f82064e40 0x559f820f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.720+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3916274687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820e85b0 0x559f820efdc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.720+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3916274687 >> 172.21.15.79:0/3916274687 conn(0x559f82015170 msgr2=0x559f820e3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:13.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.720+0000 7f8cc2000700 0 client.26909 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:13.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.721+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3916274687 shutdown_connections 2024-06-03T02:34:13.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.722+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3916274687 wait complete. 2024-06-03T02:34:13.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.723+0000 7f8ce52c1b80 20 client.26909 trim_cache size 0 max 16384 2024-06-03T02:34:13.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.756+0000 7f8ce52c1b80 1 Processor -- start 2024-06-03T02:34:13.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.756+0000 7f8ce52c1b80 1 -- start start 2024-06-03T02:34:13.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.757+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f82064e40 0x559f820f0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.757+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f820f2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.757+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 0x559f820f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.757+0000 7f8ce52c1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41c5b60 con 0x559f820f3300 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.757+0000 7f8ce52c1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41c76a0 con 0x559f82064e40 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.757+0000 7f8ce52c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41c6230 con 0x559f820e85b0 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.758+0000 7f8cd3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f820f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.758+0000 7f8cd3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f82064e40 0x559f820f0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.758+0000 7f8cd3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f820f2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44084/0 (socket says 172.21.15.79:44084) 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.758+0000 7f8cd3400700 1 -- 172.21.15.79:0/3548645042 learned_addr learned my addr 172.21.15.79:0/3548645042 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.758+0000 7f8cd3400700 1 -- 172.21.15.79:0/3548645042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f82064e40 msgr2=0x559f820f0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.758+0000 7f8cd3400700 1 --2- 172.21.15.79:0/3548645042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f82064e40 0x559f820f0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.758+0000 7f8cd3400700 1 -- 172.21.15.79:0/3548645042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 msgr2=0x559f820f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:13.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.758+0000 7f8cd3400700 1 --2- 172.21.15.79:0/3548645042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 0x559f820f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.758+0000 7f8cd3400700 1 -- 172.21.15.79:0/3548645042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f820f0cf0 con 0x559f820e85b0 2024-06-03T02:34:13.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.760+0000 7f8cd3400700 1 --2- 172.21.15.79:0/3548645042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f820f2600 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc003910 tx=0x7f8cbc003b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=313579e296d4d8bf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:13.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.764+0000 7f8cc8c00700 1 -- 172.21.15.79:0/3548645042 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc03c050 con 0x559f820e85b0 2024-06-03T02:34:13.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.764+0000 7f8cc8c00700 1 -- 172.21.15.79:0/3548645042 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc0353b0 con 0x559f820e85b0 2024-06-03T02:34:13.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.764+0000 7f8cc8c00700 1 -- 172.21.15.79:0/3548645042 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc03c050 con 0x559f820e85b0 2024-06-03T02:34:13.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.765+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3548645042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 msgr2=0x559f820f2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:13.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.765+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3548645042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f820f2600 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc003910 tx=0x7f8cbc003b10 comp rx=0 tx=0).stop 2024-06-03T02:34:13.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.767+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3548645042 shutdown_connections 2024-06-03T02:34:13.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.767+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3548645042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f820f3300 0x559f820f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.767+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3548645042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f82064e40 0x559f820f0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.767+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/3548645042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f820f2600 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.767+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3548645042 >> 172.21.15.79:0/3548645042 conn(0x559f82015170 msgr2=0x559f820ee6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:13.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.768+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3548645042 shutdown_connections 2024-06-03T02:34:13.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.768+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/3548645042 wait complete. 2024-06-03T02:34:13.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.768+0000 7f8ce52c1b80 1 -- start start 2024-06-03T02:34:13.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.769+0000 7f8ce52c1b80 1 Processor -- start 2024-06-03T02:34:13.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.770+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f82064e40 0x559f8210ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.771+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f8210b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.771+0000 7f8ce52c1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820f3300 0x559f82109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.771+0000 7f8cd3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f8210b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.771+0000 7f8cd3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f8210b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:44096/0 (socket says 172.21.15.79:44096) 2024-06-03T02:34:13.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.771+0000 7f8cd3400700 1 -- 172.21.15.79:0/2205422680 learned_addr learned my addr 172.21.15.79:0/2205422680 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:13.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.771+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41c6fd0 con 0x559f82064e40 2024-06-03T02:34:13.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.771+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41c6900 con 0x559f820f3300 2024-06-03T02:34:13.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.771+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41aa9c0 con 0x559f820e85b0 2024-06-03T02:34:13.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.771+0000 7f8cd8a00700 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820f3300 0x559f82109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.772+0000 7f8ce52c1b80 10 client.0 _ll_register_callbacks cb 0x559f81fd4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:13.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.772+0000 7f8cd3400700 1 -- 172.21.15.79:0/2205422680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820f3300 msgr2=0x559f82109c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:13.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.772+0000 7f8cd3400700 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820f3300 0x559f82109c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.772+0000 7f8cd3400700 1 -- 172.21.15.79:0/2205422680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f82064e40 msgr2=0x559f8210ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:13.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.772+0000 7f8cd3400700 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f82064e40 0x559f8210ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.772+0000 7f8cd3400700 1 -- 172.21.15.79:0/2205422680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8210a4c0 con 0x559f820e85b0 2024-06-03T02:34:13.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.772+0000 7f8cd8a00700 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820f3300 0x559f82109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:34:13.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.772+0000 7f8cd3400700 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f8210b320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc02bf10 tx=0x7f8cbc005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ae4a014041334f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:13.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.772+0000 7f8cc2000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:34:13.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.775+0000 7f8cc2000700 1 -- 172.21.15.79:0/2205422680 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc045020 con 0x559f820e85b0 2024-06-03T02:34:13.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.775+0000 7f8cc2000700 1 -- 172.21.15.79:0/2205422680 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8cbc029040 con 0x559f820e85b0 2024-06-03T02:34:13.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.775+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cbc0353b0 con 0x559f820e85b0 2024-06-03T02:34:13.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.778+0000 7f8cc2000700 1 -- 172.21.15.79:0/2205422680 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8cbc03c040 con 0x559f820e85b0 2024-06-03T02:34:13.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.778+0000 7f8ce52c1b80 10 client.26908 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:13.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.778+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f820fae50 con 0x559f820e85b0 2024-06-03T02:34:13.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.779+0000 7f8cc2000700 1 -- 172.21.15.79:0/2205422680 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc0353b0 con 0x559f820e85b0 2024-06-03T02:34:13.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.780+0000 7f8cc2000700 1 -- 172.21.15.79:0/2205422680 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8cbc02d940 con 0x559f820e85b0 2024-06-03T02:34:13.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.780+0000 7f8ce52c1b80 10 client.26908 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:13.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.780+0000 7f8ce52c1b80 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:34:13.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.780+0000 7f8ce52c1b80 20 client.26908 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:13.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.780+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559f820fb2b0 0x559f820fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:13.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.780+0000 7f8ce52c1b80 4 client.26908 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:34:13.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.780+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f81ebc250 con 0x559f820fb2b0 2024-06-03T02:34:13.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.781+0000 7f8cd3e00700 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559f820fb2b0 0x559f820fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:13.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.783+0000 7f8cc2000700 10 client.26908 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:13.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.783+0000 7f8cd3e00700 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559f820fb2b0 0x559f820fb6c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d589670f098017a server_cookie=4b2fb02a22efbddc in_seq=0 out_seq=0 2024-06-03T02:34:13.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.783+0000 7f8cc2000700 1 -- 172.21.15.79:0/2205422680 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f81ebc250 con 0x559f820fb2b0 2024-06-03T02:34:13.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.783+0000 7f8cc2000700 10 client.26908 handle_command_reply: tid=0 2024-06-03T02:34:13.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.783+0000 7f8ce52c1b80 1 client.26908 shutdown 2024-06-03T02:34:13.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.784+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 msgr2=0x559f8210b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:13.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.784+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f8210b320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc02bf10 tx=0x7f8cbc005c00 comp rx=0 tx=0).stop 2024-06-03T02:34:13.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.784+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 shutdown_connections 2024-06-03T02:34:13.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.784+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f820f3300 0x559f82109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.784+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559f820fb2b0 0x559f820fb6c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.784+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559f82064e40 0x559f8210ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.784+0000 7f8cc2000700 0 client.26908 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:13.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.785+0000 7f8ce52c1b80 1 --2- 172.21.15.79:0/2205422680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f820e85b0 0x559f8210b320 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.785+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 >> 172.21.15.79:0/2205422680 conn(0x559f82015170 msgr2=0x559f820e2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:13.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.785+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 shutdown_connections 2024-06-03T02:34:13.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.785+0000 7f8ce52c1b80 1 -- 172.21.15.79:0/2205422680 wait complete. 2024-06-03T02:34:13.787 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:34:13.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.786+0000 7f8ce52c1b80 20 client.26908 trim_cache size 0 max 16384 2024-06-03T02:34:13.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 -- 172.21.15.79:0/3215961812 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8cb80a2d70 msgr2=0x7f8cb80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:13.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8cb80a2d70 0x7f8cb80a5230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009680 tx=0x7f8cbc009f50 comp rx=0 tx=0).stop 2024-06-03T02:34:13.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 -- 172.21.15.79:0/3215961812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 msgr2=0x7f8cd410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:13.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 0x7f8cd410d110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400e9f0 tx=0x7f8cc400edb0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 -- 172.21.15.79:0/3215961812 shutdown_connections 2024-06-03T02:34:13.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4117520 0x7f8cd410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cd41099d0 0x7f8cd410d110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8cb80a2d70 0x7f8cb80a5230 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 --2- 172.21.15.79:0/3215961812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cd410cbb0 0x7f8cd410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:13.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.788+0000 7f8cdaa00700 1 -- 172.21.15.79:0/3215961812 >> 172.21.15.79:0/3215961812 conn(0x7f8cd406d600 msgr2=0x7f8cd4117900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:13.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.790+0000 7f8cdaa00700 1 -- 172.21.15.79:0/3215961812 shutdown_connections 2024-06-03T02:34:13.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:13.790+0000 7f8cdaa00700 1 -- 172.21.15.79:0/3215961812 wait complete. 2024-06-03T02:34:13.987 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:13 smithi079 ceph-mon[31819]: pgmap v941: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-06-03T02:34:14.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:13 smithi135 ceph-mon[36653]: pgmap v941: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-06-03T02:34:14.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:13 smithi123 ceph-mon[36640]: pgmap v941: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-06-03T02:34:16.341 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:15 smithi079 ceph-mon[31819]: pgmap v942: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2024-06-03T02:34:16.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:15 smithi135 ceph-mon[36653]: pgmap v942: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2024-06-03T02:34:16.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:15 smithi123 ceph-mon[36640]: pgmap v942: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2024-06-03T02:34:17.468 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T02:34:17.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T02:34:18.198 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:34:18.293 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:18 smithi079 ceph-mon[31819]: pgmap v943: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-06-03T02:34:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 850.0 s, 451.9 tps, lat 221.617 ms stddev 278.282 2024-06-03T02:34:18.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:18 smithi135 ceph-mon[36653]: pgmap v943: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-06-03T02:34:18.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:18 smithi123 ceph-mon[36640]: pgmap v943: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-06-03T02:34:20.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:20 smithi079 ceph-mon[31819]: pgmap v944: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-06-03T02:34:20.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:20 smithi135 ceph-mon[36653]: pgmap v944: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-06-03T02:34:20.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:20 smithi123 ceph-mon[36640]: pgmap v944: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-06-03T02:34:22.341 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:22 smithi079 ceph-mon[31819]: pgmap v945: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-06-03T02:34:22.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:22 smithi135 ceph-mon[36653]: pgmap v945: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-06-03T02:34:22.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:22 smithi123 ceph-mon[36640]: pgmap v945: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-06-03T02:34:23.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 855.0 s, 671.2 tps, lat 154.776 ms stddev 188.382 2024-06-03T02:34:24.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.026+0000 7f5d76000700 1 -- 172.21.15.79:0/2085332579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d7006a170 msgr2=0x7f5d7006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:24.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.026+0000 7f5d76000700 1 --2- 172.21.15.79:0/2085332579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d7006a170 0x7f5d7006a5f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009990 tx=0x7f5d5c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.026+0000 7f5d76000700 1 -- 172.21.15.79:0/2085332579 shutdown_connections 2024-06-03T02:34:24.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.026+0000 7f5d76000700 1 --2- 172.21.15.79:0/2085332579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d7006ab30 0x7f5d7010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.026+0000 7f5d76000700 1 --2- 172.21.15.79:0/2085332579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d70108500 0x7f5d701088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.026+0000 7f5d76000700 1 --2- 172.21.15.79:0/2085332579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d7006a170 0x7f5d7006a5f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.026+0000 7f5d76000700 1 -- 172.21.15.79:0/2085332579 >> 172.21.15.79:0/2085332579 conn(0x7f5d70078340 msgr2=0x7f5d70078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:24.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.029+0000 7f5d76000700 1 -- 172.21.15.79:0/2085332579 shutdown_connections 2024-06-03T02:34:24.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.029+0000 7f5d76000700 1 -- 172.21.15.79:0/2085332579 wait complete. 2024-06-03T02:34:24.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.030+0000 7f5d76000700 1 Processor -- start 2024-06-03T02:34:24.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.031+0000 7f5d76000700 1 -- start start 2024-06-03T02:34:24.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.033+0000 7f5d76000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d7006ab30 0x7f5d701a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.033+0000 7f5d76000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d70108500 0x7f5d701a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.033+0000 7f5d6f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d7006ab30 0x7f5d701a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.033+0000 7f5d6f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d7006ab30 0x7f5d701a4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:46452/0 (socket says 172.21.15.79:46452) 2024-06-03T02:34:24.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.034+0000 7f5d6f400700 1 -- 172.21.15.79:0/3566154455 learned_addr learned my addr 172.21.15.79:0/3566154455 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:24.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.034+0000 7f5d76000700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d7010d440 0x7f5d7010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.035+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701127b0 con 0x7f5d70108500 2024-06-03T02:34:24.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.035+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70112630 con 0x7f5d7006ab30 2024-06-03T02:34:24.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.035+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70112930 con 0x7f5d7010d440 2024-06-03T02:34:24.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.035+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d7010d440 0x7f5d7010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.036+0000 7f5d6fe00700 1 -- 172.21.15.79:0/3566154455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d7006ab30 msgr2=0x7f5d701a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:24.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.036+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d7006ab30 0x7f5d701a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.036+0000 7f5d6fe00700 1 -- 172.21.15.79:0/3566154455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d70108500 msgr2=0x7f5d701a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:24.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.036+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d70108500 0x7f5d701a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.036+0000 7f5d6fe00700 1 -- 172.21.15.79:0/3566154455 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7010dff0 con 0x7f5d7010d440 2024-06-03T02:34:24.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.039+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d7010d440 0x7f5d7010d8f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800cc50 tx=0x7f5d6800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:24.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.039+0000 7f5d67e00700 1 -- 172.21.15.79:0/3566154455 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68016020 con 0x7f5d7010d440 2024-06-03T02:34:24.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.039+0000 7f5d67e00700 1 -- 172.21.15.79:0/3566154455 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d6800ab40 con 0x7f5d7010d440 2024-06-03T02:34:24.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.039+0000 7f5d67e00700 1 -- 172.21.15.79:0/3566154455 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d680115e0 con 0x7f5d7010d440 2024-06-03T02:34:24.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.040+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d701abf80 con 0x7f5d7010d440 2024-06-03T02:34:24.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.040+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d701ac320 con 0x7f5d7010d440 2024-06-03T02:34:24.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.048+0000 7f5d67e00700 1 -- 172.21.15.79:0/3566154455 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5d68019020 con 0x7f5d7010d440 2024-06-03T02:34:24.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.050+0000 7f5d80a1bb80 1 Processor -- start 2024-06-03T02:34:24.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.051+0000 7f5d67e00700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d540a2d20 0x7f5d540a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.051+0000 7f5d67e00700 1 -- 172.21.15.79:0/3566154455 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5d680cf1e0 con 0x7f5d7010d440 2024-06-03T02:34:24.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.053+0000 7f5d80a1bb80 1 -- start start 2024-06-03T02:34:24.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.054+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957d99ec0 0x562957e1d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.054+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d540a2d20 0x7f5d540a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.054+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 0x562957e1dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.054+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e2a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.054+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562957d2ca90 con 0x562957e1d630 2024-06-03T02:34:24.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.054+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562957d6fb80 con 0x562957e28380 2024-06-03T02:34:24.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.054+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562957d2a970 con 0x562957d99ec0 2024-06-03T02:34:24.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.055+0000 7f5d6f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957d99ec0 0x562957e1d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.055+0000 7f5d6f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957d99ec0 0x562957e1d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52706/0 (socket says 172.21.15.79:52706) 2024-06-03T02:34:24.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.055+0000 7f5d6f400700 1 -- 172.21.15.79:0/971203715 learned_addr learned my addr 172.21.15.79:0/971203715 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:24.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.055+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e2a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.055+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 0x562957e1dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.057+0000 7f5d6f400700 1 -- 172.21.15.79:0/971203715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 msgr2=0x562957e2a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:24.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.057+0000 7f5d6f400700 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e2a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.057+0000 7f5d6f400700 1 -- 172.21.15.79:0/971203715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 msgr2=0x562957e1dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:24.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.057+0000 7f5d6f400700 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 0x562957e1dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.057+0000 7f5d6f400700 1 -- 172.21.15.79:0/971203715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562957d18010 con 0x562957d99ec0 2024-06-03T02:34:24.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.057+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e2a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:34:24.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.057+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 0x562957e1dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:34:24.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.057+0000 7f5d6f400700 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957d99ec0 0x562957e1d0f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5d60007cd0 tx=0x7f5d600103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3851164b45d9d164 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:24.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.058+0000 7f5d64c00700 1 -- 172.21.15.79:0/971203715 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010d00 con 0x562957d99ec0 2024-06-03T02:34:24.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.058+0000 7f5d64c00700 1 -- 172.21.15.79:0/971203715 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d60011920 con 0x562957d99ec0 2024-06-03T02:34:24.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.058+0000 7f5d64c00700 1 -- 172.21.15.79:0/971203715 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60019870 con 0x562957d99ec0 2024-06-03T02:34:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.059+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d540a2d20 0x7f5d540a51e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00b5c0 tx=0x7f5d5c009f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.059+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/971203715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957d99ec0 msgr2=0x562957e1d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.059+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957d99ec0 0x562957e1d0f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5d60007cd0 tx=0x7f5d600103d0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.059+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/971203715 shutdown_connections 2024-06-03T02:34:24.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.059+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e2a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.059+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957d99ec0 0x562957e1d0f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.059+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/971203715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 0x562957e1dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.059+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/971203715 >> 172.21.15.79:0/971203715 conn(0x562957d4a1f0 msgr2=0x562957e19530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:24.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.061+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/971203715 shutdown_connections 2024-06-03T02:34:24.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.061+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/971203715 wait complete. 2024-06-03T02:34:24.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.061+0000 7f5d80a1bb80 1 -- start start 2024-06-03T02:34:24.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.062+0000 7f5d80a1bb80 1 Processor -- start 2024-06-03T02:34:24.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.063+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e27ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.063+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e24dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e25310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562957d0bfe0 con 0x562957d99ec0 2024-06-03T02:34:24.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562957d6bb40 con 0x562957e28380 2024-06-03T02:34:24.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562957e2b040 con 0x562957e1d630 2024-06-03T02:34:24.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d6fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d6fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e25310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:46472/0 (socket says 172.21.15.79:46472) 2024-06-03T02:34:24.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d6fe00700 1 -- 172.21.15.79:0/3699625267 learned_addr learned my addr 172.21.15.79:0/3699625267 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:24.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e24dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d80a1bb80 10 client.0 _ll_register_callbacks cb 0x562957d097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:24.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.064+0000 7f5d6f400700 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e27ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.065+0000 7f5d6fe00700 1 -- 172.21.15.79:0/3699625267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 msgr2=0x562957e24dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:24.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.065+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e24dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.065+0000 7f5d6fe00700 1 -- 172.21.15.79:0/3699625267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 msgr2=0x562957e27ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:24.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.065+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e27ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.065+0000 7f5d6fe00700 1 -- 172.21.15.79:0/3699625267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562957cbe950 con 0x562957e28380 2024-06-03T02:34:24.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.065+0000 7f5d6f400700 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e27ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:34:24.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.066+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e25310 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5d680cc8d0 tx=0x7f5d6809bdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77da76b86fe87e04 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:24.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.066+0000 7f5d59600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:24.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.066+0000 7f5d59600700 1 -- 172.21.15.79:0/3699625267 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68086070 con 0x562957e28380 2024-06-03T02:34:24.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.066+0000 7f5d59600700 1 -- 172.21.15.79:0/3699625267 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5d680a3e10 con 0x562957e28380 2024-06-03T02:34:24.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.068+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/3699625267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d60011920 con 0x562957e28380 2024-06-03T02:34:24.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.068+0000 7f5d59600700 1 -- 172.21.15.79:0/3699625267 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d6809f9e0 con 0x562957e28380 2024-06-03T02:34:24.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.068+0000 7f5d80a1bb80 10 client.26927 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:24.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.068+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/3699625267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562957e357b0 con 0x562957e28380 2024-06-03T02:34:24.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.069+0000 7f5d59600700 1 -- 172.21.15.79:0/3699625267 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d60011920 con 0x562957e28380 2024-06-03T02:34:24.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.069+0000 7f5d59600700 1 -- 172.21.15.79:0/3699625267 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5d680afb50 con 0x562957e28380 2024-06-03T02:34:24.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.069+0000 7f5d80a1bb80 10 client.26927 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:24.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.069+0000 7f5d80a1bb80 10 client.26927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:34:24.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.069+0000 7f5d80a1bb80 20 client.26927 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:24.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.069+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562957e37df0 0x562957e3a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.069+0000 7f5d80a1bb80 4 client.26927 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:34:24.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.069+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/3699625267 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562957d7d080 con 0x562957e37df0 2024-06-03T02:34:24.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.070+0000 7f5d6f400700 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562957e37df0 0x562957e3a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.071+0000 7f5d6f400700 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562957e37df0 0x562957e3a1e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f46d8a6eb6932a server_cookie=ae0d8d1dbf23a2c0 in_seq=0 out_seq=0 2024-06-03T02:34:24.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.071+0000 7f5d59600700 10 client.26927 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:24.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.073+0000 7f5d59600700 1 -- 172.21.15.79:0/3699625267 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562957d7d080 con 0x562957e37df0 2024-06-03T02:34:24.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.073+0000 7f5d59600700 10 client.26927 handle_command_reply: tid=0 2024-06-03T02:34:24.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.073+0000 7f5d80a1bb80 1 client.26927 shutdown 2024-06-03T02:34:24.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.073+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/3699625267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 msgr2=0x562957e25310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:24.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.073+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e25310 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5d680cc8d0 tx=0x7f5d6809bdb0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.074+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/3699625267 shutdown_connections 2024-06-03T02:34:24.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.074+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e25310 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.074+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562957e37df0 0x562957e3a1e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.074+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e27ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.074+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/3699625267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e24dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.074+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/3699625267 >> 172.21.15.79:0/3699625267 conn(0x562957d4a1f0 msgr2=0x562957e187c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:24.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.074+0000 7f5d59600700 0 client.26927 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:24.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.075+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/3699625267 shutdown_connections 2024-06-03T02:34:24.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.075+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/3699625267 wait complete. 2024-06-03T02:34:24.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.075+0000 7f5d80a1bb80 20 client.26927 trim_cache size 0 max 16384 2024-06-03T02:34:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.110+0000 7f5d80a1bb80 1 Processor -- start 2024-06-03T02:34:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.110+0000 7f5d80a1bb80 1 -- start start 2024-06-03T02:34:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e29770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e27610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e236c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b5020 con 0x562957d99ec0 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b6b60 con 0x562957e28380 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b56f0 con 0x562957e1d630 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d6ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e27610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d6ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e27610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:52728/0 (socket says 172.21.15.79:52728) 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d6ea00700 1 -- 172.21.15.79:0/4226978791 learned_addr learned my addr 172.21.15.79:0/4226978791 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d6f400700 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e29770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d6fe00700 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d6ea00700 1 -- 172.21.15.79:0/4226978791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 msgr2=0x562957e236c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e236c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.111+0000 7f5d6ea00700 1 -- 172.21.15.79:0/4226978791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 msgr2=0x562957e29770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.112+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e29770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.112+0000 7f5d6ea00700 1 -- 172.21.15.79:0/4226978791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562957e1da10 con 0x562957e1d630 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.112+0000 7f5d6f400700 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e29770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:34:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.112+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e27610 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009990 tx=0x7f5d5c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2ccebc28be37251 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.112+0000 7f5d5be00700 1 -- 172.21.15.79:0/4226978791 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00bad0 con 0x562957e1d630 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.112+0000 7f5d5be00700 1 -- 172.21.15.79:0/4226978791 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5c022040 con 0x562957e1d630 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.112+0000 7f5d5be00700 1 -- 172.21.15.79:0/4226978791 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00f8e0 con 0x562957e1d630 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.113+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/4226978791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 msgr2=0x562957e27610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.113+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e27610 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009990 tx=0x7f5d5c005e80 comp rx=0 tx=0).stop 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.113+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/4226978791 shutdown_connections 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.113+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957e28380 0x562957e236c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.113+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957d99ec0 0x562957e29770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.113+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/4226978791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e1d630 0x562957e27610 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.113+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/4226978791 >> 172.21.15.79:0/4226978791 conn(0x562957d4a1f0 msgr2=0x562957e19030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.113+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/4226978791 shutdown_connections 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.113+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/4226978791 wait complete. 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.114+0000 7f5d80a1bb80 1 -- start start 2024-06-03T02:34:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.114+0000 7f5d80a1bb80 1 Processor -- start 2024-06-03T02:34:24.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.115+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957d99ec0 0x562957e30ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.115+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 0x562957e29900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.115+0000 7f5d80a1bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e28380 0x562957e29e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.116+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b6490 con 0x562957e1d630 2024-06-03T02:34:24.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.116+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b5dc0 con 0x562957d99ec0 2024-06-03T02:34:24.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.116+0000 7f5d80a1bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701b2fc0 con 0x562957e28380 2024-06-03T02:34:24.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.116+0000 7f5d80a1bb80 10 client.0 _ll_register_callbacks cb 0x562957d097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:24.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.117+0000 7f5d6f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957d99ec0 0x562957e30ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.117+0000 7f5d6f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957d99ec0 0x562957e30ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:46502/0 (socket says 172.21.15.79:46502) 2024-06-03T02:34:24.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.117+0000 7f5d6f400700 1 -- 172.21.15.79:0/869370464 learned_addr learned my addr 172.21.15.79:0/869370464 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:24.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.117+0000 7f5d6f400700 1 -- 172.21.15.79:0/869370464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e28380 msgr2=0x562957e29e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:24.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.118+0000 7f5d6f400700 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e28380 0x562957e29e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.118+0000 7f5d6f400700 1 -- 172.21.15.79:0/869370464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 msgr2=0x562957e29900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:24.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.118+0000 7f5d6f400700 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 0x562957e29900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.118+0000 7f5d6f400700 1 -- 172.21.15.79:0/869370464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562957e2a6a0 con 0x562957d99ec0 2024-06-03T02:34:24.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.122+0000 7f5d6f400700 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957d99ec0 0x562957e30ad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x562957e1a430 tx=0x7f5d60011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=242fdb3b73057a1e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:24.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.122+0000 7f5d59600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:24.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.122+0000 7f5d59600700 1 -- 172.21.15.79:0/869370464 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010cb0 con 0x562957d99ec0 2024-06-03T02:34:24.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.122+0000 7f5d59600700 1 -- 172.21.15.79:0/869370464 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f5d60011440 con 0x562957d99ec0 2024-06-03T02:34:24.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.122+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/869370464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5c022040 con 0x562957d99ec0 2024-06-03T02:34:24.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.123+0000 7f5d59600700 1 -- 172.21.15.79:0/869370464 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d60010e50 con 0x562957d99ec0 2024-06-03T02:34:24.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.123+0000 7f5d80a1bb80 10 client.26933 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:24.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.123+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/869370464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562957ea2b50 con 0x562957d99ec0 2024-06-03T02:34:24.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.123+0000 7f5d59600700 1 -- 172.21.15.79:0/869370464 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5c022040 con 0x562957d99ec0 2024-06-03T02:34:24.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.124+0000 7f5d59600700 1 -- 172.21.15.79:0/869370464 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5d6001a450 con 0x562957d99ec0 2024-06-03T02:34:24.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.124+0000 7f5d80a1bb80 10 client.26933 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:24.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.124+0000 7f5d80a1bb80 10 client.26933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:34:24.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.124+0000 7f5d80a1bb80 20 client.26933 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:24.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.124+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562957ea3150 0x562957ea5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:24.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.124+0000 7f5d80a1bb80 4 client.26933 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T02:34:24.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.124+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/869370464 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562957d77600 con 0x562957ea3150 2024-06-03T02:34:24.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.129+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562957ea3150 0x562957ea5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.134+0000 7f5d6ea00700 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562957ea3150 0x562957ea5590 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=401cfc81dd988a4b server_cookie=5ac58fa5d59edbb1 in_seq=0 out_seq=0 2024-06-03T02:34:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.134+0000 7f5d59600700 10 client.26933 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d59600700 1 -- 172.21.15.79:0/869370464 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562957d77600 con 0x562957ea3150 2024-06-03T02:34:24.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d59600700 10 client.26933 handle_command_reply: tid=0 2024-06-03T02:34:24.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d80a1bb80 1 client.26933 shutdown 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/869370464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957d99ec0 msgr2=0x562957e30ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957d99ec0 0x562957e30ad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x562957e1a430 tx=0x7f5d60011d90 comp rx=0 tx=0).stop 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/869370464 shutdown_connections 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562957e28380 0x562957e29e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562957d99ec0 0x562957e30ad0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x562957ea3150 0x562957ea5590 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d80a1bb80 1 --2- 172.21.15.79:0/869370464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562957e1d630 0x562957e29900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/869370464 >> 172.21.15.79:0/869370464 conn(0x562957d4a1f0 msgr2=0x562957e1f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.136+0000 7f5d59600700 0 client.26933 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.137+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/869370464 shutdown_connections 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.137+0000 7f5d80a1bb80 1 -- 172.21.15.79:0/869370464 wait complete. 2024-06-03T02:34:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.137+0000 7f5d80a1bb80 20 client.26933 trim_cache size 0 max 16384 2024-06-03T02:34:24.140 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:34:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "return_code": 0, 2024-06-03T02:34:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_tag": "1eb33220-70ee-4cff-98e2-38b3f44bb817", 2024-06-03T02:34:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": "asynchronous" 2024-06-03T02:34:24.140 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d540a2d20 msgr2=0x7f5d540a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d540a2d20 0x7f5d540a51e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00b5c0 tx=0x7f5d5c009f40 comp rx=0 tx=0).stop 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d7010d440 msgr2=0x7f5d7010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d7010d440 0x7f5d7010d8f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800cc50 tx=0x7f5d6800cf60 comp rx=0 tx=0).stop 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 shutdown_connections 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d7010d440 0x7f5d7010d8f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d7006ab30 0x7f5d701a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5d540a2d20 0x7f5d540a51e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 --2- 172.21.15.79:0/3566154455 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d70108500 0x7f5d701a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 >> 172.21.15.79:0/3566154455 conn(0x7f5d70078340 msgr2=0x7f5d70101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 shutdown_connections 2024-06-03T02:34:24.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:24.139+0000 7f5d76000700 1 -- 172.21.15.79:0/3566154455 wait complete. 2024-06-03T02:34:24.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:24 smithi079 ceph-mon[31819]: pgmap v946: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.9 MiB/s wr, 753 op/s 2024-06-03T02:34:24.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:24 smithi135 ceph-mon[36653]: pgmap v946: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.9 MiB/s wr, 753 op/s 2024-06-03T02:34:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:24 smithi123 ceph-mon[36640]: pgmap v946: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.9 MiB/s wr, 753 op/s 2024-06-03T02:34:25.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:25 smithi079 ceph-mon[31819]: scrub queued for path: / 2024-06-03T02:34:25.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:25 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:34:25.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:25 smithi079 ceph-mon[31819]: scrub summary: active paths [/] 2024-06-03T02:34:25.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:25 smithi135 ceph-mon[36653]: scrub queued for path: / 2024-06-03T02:34:25.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:25 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:34:25.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:25 smithi135 ceph-mon[36653]: scrub summary: active paths [/] 2024-06-03T02:34:25.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:25 smithi123 ceph-mon[36640]: scrub queued for path: / 2024-06-03T02:34:25.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:25 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:34:25.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:25 smithi123 ceph-mon[36640]: scrub summary: active paths [/] 2024-06-03T02:34:25.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T02:34:26.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:26 smithi079 ceph-mon[31819]: pgmap v947: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 602 op/s 2024-06-03T02:34:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:26 smithi135 ceph-mon[36653]: pgmap v947: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 602 op/s 2024-06-03T02:34:26.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:26 smithi123 ceph-mon[36640]: pgmap v947: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 602 op/s 2024-06-03T02:34:26.534 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:34:27.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:27 smithi079 ceph-mon[31819]: scrub summary: idle+waiting paths [/] 2024-06-03T02:34:27.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:27 smithi135 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-06-03T02:34:27.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:27 smithi123 ceph-mon[36640]: scrub summary: idle+waiting paths [/] 2024-06-03T02:34:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 860.0 s, 504.4 tps, lat 195.923 ms stddev 263.607 2024-06-03T02:34:28.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:28 smithi079 ceph-mon[31819]: pgmap v948: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.4 MiB/s wr, 549 op/s 2024-06-03T02:34:28.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:28 smithi135 ceph-mon[36653]: pgmap v948: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.4 MiB/s wr, 549 op/s 2024-06-03T02:34:28.374 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:28 smithi123 ceph-mon[36640]: pgmap v948: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.4 MiB/s wr, 549 op/s 2024-06-03T02:34:30.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:30 smithi135 ceph-mon[36653]: pgmap v949: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 542 op/s 2024-06-03T02:34:30.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:30 smithi135 ceph-mon[36653]: scrub summary: idle 2024-06-03T02:34:30.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:30 smithi079 ceph-mon[31819]: pgmap v949: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 542 op/s 2024-06-03T02:34:30.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:30 smithi079 ceph-mon[31819]: scrub summary: idle 2024-06-03T02:34:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:30 smithi123 ceph-mon[36640]: pgmap v949: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 542 op/s 2024-06-03T02:34:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:30 smithi123 ceph-mon[36640]: scrub summary: idle 2024-06-03T02:34:31.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.919+0000 7f5426c00700 1 -- 172.21.15.79:0/861131165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5420108500 msgr2=0x7f54201088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:31.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.919+0000 7f5426c00700 1 --2- 172.21.15.79:0/861131165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5420108500 0x7f54201088e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5410009990 tx=0x7f5410009ca0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.920+0000 7f5426c00700 1 -- 172.21.15.79:0/861131165 shutdown_connections 2024-06-03T02:34:31.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.920+0000 7f5426c00700 1 --2- 172.21.15.79:0/861131165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542006ab30 0x7f542010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.920+0000 7f5426c00700 1 --2- 172.21.15.79:0/861131165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5420108500 0x7f54201088e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.920+0000 7f5426c00700 1 --2- 172.21.15.79:0/861131165 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f542006a170 0x7f542006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.920+0000 7f5426c00700 1 -- 172.21.15.79:0/861131165 >> 172.21.15.79:0/861131165 conn(0x7f5420078340 msgr2=0x7f5420078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:31.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.921+0000 7f5426c00700 1 -- 172.21.15.79:0/861131165 shutdown_connections 2024-06-03T02:34:31.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.921+0000 7f5426c00700 1 -- 172.21.15.79:0/861131165 wait complete. 2024-06-03T02:34:31.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.925+0000 7f5426c00700 1 Processor -- start 2024-06-03T02:34:31.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.926+0000 7f5426c00700 1 -- start start 2024-06-03T02:34:31.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.927+0000 7f5426c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542006a170 0x7f542019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.927+0000 7f5426c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f542006ab30 0x7f542019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.927+0000 7f5426c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5420108500 0x7f5420198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.927+0000 7f5426c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f54201127b0 con 0x7f542006ab30 2024-06-03T02:34:31.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.927+0000 7f541fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542006a170 0x7f542019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:31.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.927+0000 7f541fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542006a170 0x7f542019ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47158/0 (socket says 172.21.15.79:47158) 2024-06-03T02:34:31.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.927+0000 7f541fe00700 1 -- 172.21.15.79:0/3444526646 learned_addr learned my addr 172.21.15.79:0/3444526646 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:31.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.928+0000 7f5424c00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5420108500 0x7f5420198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:31.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.928+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5420112630 con 0x7f542006a170 2024-06-03T02:34:31.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.928+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5420112930 con 0x7f5420108500 2024-06-03T02:34:31.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.928+0000 7f541fe00700 1 -- 172.21.15.79:0/3444526646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5420108500 msgr2=0x7f5420198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:31.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.929+0000 7f541fe00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5420108500 0x7f5420198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.929+0000 7f541fe00700 1 -- 172.21.15.79:0/3444526646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f542006ab30 msgr2=0x7f542019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:31.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.929+0000 7f541fe00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f542006ab30 0x7f542019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.929+0000 7f541fe00700 1 -- 172.21.15.79:0/3444526646 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54201993f0 con 0x7f542006a170 2024-06-03T02:34:31.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.929+0000 7f5424c00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5420108500 0x7f5420198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:34:31.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.930+0000 7f541fe00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542006a170 0x7f542019ea40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5410000c00 tx=0x7f5410003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:31.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.931+0000 7f541cc00700 1 -- 172.21.15.79:0/3444526646 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410027070 con 0x7f542006a170 2024-06-03T02:34:31.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.931+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5420199680 con 0x7f542006a170 2024-06-03T02:34:31.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.932+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54201a59c0 con 0x7f542006a170 2024-06-03T02:34:31.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.931+0000 7f541cc00700 1 -- 172.21.15.79:0/3444526646 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5410005690 con 0x7f542006a170 2024-06-03T02:34:31.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.932+0000 7f541cc00700 1 -- 172.21.15.79:0/3444526646 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410027070 con 0x7f542006a170 2024-06-03T02:34:31.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.939+0000 7f541cc00700 1 -- 172.21.15.79:0/3444526646 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f541000f460 con 0x7f542006a170 2024-06-03T02:34:31.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.943+0000 7f541cc00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54040a2f20 0x7f54040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.944+0000 7f541cc00700 1 -- 172.21.15.79:0/3444526646 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f54100cfed0 con 0x7f542006a170 2024-06-03T02:34:31.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.945+0000 7f541f400700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54040a2f20 0x7f54040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:31.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.946+0000 7f541f400700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54040a2f20 0x7f54040a53e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f540c005950 tx=0x7f540c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:31.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.950+0000 7f54315f0b80 1 Processor -- start 2024-06-03T02:34:31.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.950+0000 7f54315f0b80 1 -- start start 2024-06-03T02:34:31.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560146421e40 0x5601464a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464a55b0 0x5601464a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601464b0300 0x5601464b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f54315f0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5601463b4a10 con 0x5601464a55b0 2024-06-03T02:34:31.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f54315f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5601463f7b00 con 0x5601464b0300 2024-06-03T02:34:31.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f54315f0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601463b28f0 con 0x560146421e40 2024-06-03T02:34:31.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f541f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464a55b0 0x5601464a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:31.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f5424c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601464b0300 0x5601464b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:31.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f5424c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601464b0300 0x5601464b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47172/0 (socket says 172.21.15.79:47172) 2024-06-03T02:34:31.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.951+0000 7f5424c00700 1 -- 172.21.15.79:0/3819349606 learned_addr learned my addr 172.21.15.79:0/3819349606 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:31.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.952+0000 7f5424c00700 1 -- 172.21.15.79:0/3819349606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560146421e40 msgr2=0x5601464a5070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:34:31.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.952+0000 7f5424c00700 1 --2- 172.21.15.79:0/3819349606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560146421e40 0x5601464a5070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.952+0000 7f5424c00700 1 -- 172.21.15.79:0/3819349606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464a55b0 msgr2=0x5601464a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:31.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.952+0000 7f5424c00700 1 --2- 172.21.15.79:0/3819349606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464a55b0 0x5601464a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.952+0000 7f5424c00700 1 -- 172.21.15.79:0/3819349606 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56014639ff90 con 0x5601464b0300 2024-06-03T02:34:31.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.952+0000 7f5424c00700 1 --2- 172.21.15.79:0/3819349606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601464b0300 0x5601464b26f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5418007740 tx=0x7f5418009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5a415209841181d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:31.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.953+0000 7f5415600700 1 -- 172.21.15.79:0/3819349606 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418019020 con 0x5601464b0300 2024-06-03T02:34:31.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.953+0000 7f5415600700 1 -- 172.21.15.79:0/3819349606 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5418015d30 con 0x5601464b0300 2024-06-03T02:34:31.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.954+0000 7f54315f0b80 1 -- 172.21.15.79:0/3819349606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601464b0300 msgr2=0x5601464b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:31.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.954+0000 7f5415600700 1 -- 172.21.15.79:0/3819349606 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418019020 con 0x5601464b0300 2024-06-03T02:34:31.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.954+0000 7f54315f0b80 1 --2- 172.21.15.79:0/3819349606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601464b0300 0x5601464b26f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5418007740 tx=0x7f5418009f90 comp rx=0 tx=0).stop 2024-06-03T02:34:31.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.954+0000 7f54315f0b80 1 -- 172.21.15.79:0/3819349606 shutdown_connections 2024-06-03T02:34:31.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.954+0000 7f54315f0b80 1 --2- 172.21.15.79:0/3819349606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601464b0300 0x5601464b26f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.954+0000 7f54315f0b80 1 --2- 172.21.15.79:0/3819349606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560146421e40 0x5601464a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.954+0000 7f54315f0b80 1 --2- 172.21.15.79:0/3819349606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464a55b0 0x5601464a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.954+0000 7f54315f0b80 1 -- 172.21.15.79:0/3819349606 >> 172.21.15.79:0/3819349606 conn(0x5601463d2170 msgr2=0x5601464a14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:31.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.954+0000 7f54315f0b80 1 -- 172.21.15.79:0/3819349606 shutdown_connections 2024-06-03T02:34:31.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.955+0000 7f54315f0b80 1 -- 172.21.15.79:0/3819349606 wait complete. 2024-06-03T02:34:31.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.955+0000 7f54315f0b80 1 -- start start 2024-06-03T02:34:31.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.956+0000 7f54315f0b80 1 Processor -- start 2024-06-03T02:34:31.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.956+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464af980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.956+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464a55b0 0x5601464acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.956+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b0300 0x5601464ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.957+0000 7f54315f0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560146393f60 con 0x5601464b0300 2024-06-03T02:34:31.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.957+0000 7f54315f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5601463f3ac0 con 0x560146421e40 2024-06-03T02:34:31.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.957+0000 7f54315f0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601464b3010 con 0x5601464a55b0 2024-06-03T02:34:31.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.957+0000 7f54315f0b80 10 client.0 _ll_register_callbacks cb 0x560146391720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:31.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.957+0000 7f541f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464a55b0 0x5601464acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:31.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.958+0000 7f541f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464a55b0 0x5601464acc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37610/0 (socket says 172.21.15.79:37610) 2024-06-03T02:34:31.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.958+0000 7f541f400700 1 -- 172.21.15.79:0/1132621665 learned_addr learned my addr 172.21.15.79:0/1132621665 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:31.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.958+0000 7f541fe00700 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:31.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.959+0000 7f541f400700 1 -- 172.21.15.79:0/1132621665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 msgr2=0x5601464af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:31.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.959+0000 7f541f400700 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.959+0000 7f541f400700 1 -- 172.21.15.79:0/1132621665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b0300 msgr2=0x5601464ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:31.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.959+0000 7f541f400700 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b0300 0x5601464ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.962+0000 7f541f400700 1 -- 172.21.15.79:0/1132621665 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601463468d0 con 0x5601464a55b0 2024-06-03T02:34:31.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.962+0000 7f541f400700 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464a55b0 0x5601464acc80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f540c00de80 tx=0x7f540c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b83dd1f2e46d08c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:31.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.962+0000 7f540a000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:34:31.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.962+0000 7f540a000700 1 -- 172.21.15.79:0/1132621665 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c026400 con 0x5601464a55b0 2024-06-03T02:34:31.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.963+0000 7f540a000700 1 -- 172.21.15.79:0/1132621665 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f540c01e3f0 con 0x5601464a55b0 2024-06-03T02:34:31.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.963+0000 7f54315f0b80 1 -- 172.21.15.79:0/1132621665 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5418015d30 con 0x5601464a55b0 2024-06-03T02:34:31.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.963+0000 7f540a000700 1 -- 172.21.15.79:0/1132621665 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f540c01e5f0 con 0x5601464a55b0 2024-06-03T02:34:31.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.963+0000 7f54315f0b80 10 client.26929 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:31.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.963+0000 7f54315f0b80 1 -- 172.21.15.79:0/1132621665 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601464c19c0 con 0x5601464a55b0 2024-06-03T02:34:31.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.963+0000 7f540a000700 1 -- 172.21.15.79:0/1132621665 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5418015d30 con 0x5601464a55b0 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.964+0000 7f540a000700 1 -- 172.21.15.79:0/1132621665 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f540c0267f0 con 0x5601464a55b0 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.964+0000 7f54315f0b80 10 client.26929 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.964+0000 7f54315f0b80 10 client.26929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.964+0000 7f54315f0b80 20 client.26929 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.964+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601464c4050 0x5601464c6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.964+0000 7f54315f0b80 4 client.26929 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.964+0000 7f54315f0b80 1 -- 172.21.15.79:0/1132621665 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560146405000 con 0x5601464c4050 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.964+0000 7f541fe00700 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601464c4050 0x5601464c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.965+0000 7f540a000700 10 client.26929 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:31.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.965+0000 7f541fe00700 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601464c4050 0x5601464c6440 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3f014025db7462 server_cookie=63aa122a56766c9d in_seq=0 out_seq=0 2024-06-03T02:34:31.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.966+0000 7f540a000700 1 -- 172.21.15.79:0/1132621665 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560146405000 con 0x5601464c4050 2024-06-03T02:34:31.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.966+0000 7f540a000700 10 client.26929 handle_command_reply: tid=0 2024-06-03T02:34:31.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.966+0000 7f54315f0b80 1 client.26929 shutdown 2024-06-03T02:34:31.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.967+0000 7f54315f0b80 1 -- 172.21.15.79:0/1132621665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464a55b0 msgr2=0x5601464acc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:31.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.967+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464a55b0 0x5601464acc80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f540c00de80 tx=0x7f540c004750 comp rx=0 tx=0).stop 2024-06-03T02:34:31.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.967+0000 7f54315f0b80 1 -- 172.21.15.79:0/1132621665 shutdown_connections 2024-06-03T02:34:31.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.967+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601464c4050 0x5601464c6440 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.967+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b0300 0x5601464ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.967+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:31.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.967+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1132621665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464a55b0 0x5601464acc80 secure :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f540c00de80 tx=0x7f540c004750 comp rx=0 tx=0).stop 2024-06-03T02:34:31.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.967+0000 7f54315f0b80 1 -- 172.21.15.79:0/1132621665 >> 172.21.15.79:0/1132621665 conn(0x5601463d2170 msgr2=0x5601464a0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:31.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.967+0000 7f540a000700 0 client.26929 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:31.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.969+0000 7f54315f0b80 1 -- 172.21.15.79:0/1132621665 shutdown_connections 2024-06-03T02:34:31.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.969+0000 7f54315f0b80 1 -- 172.21.15.79:0/1132621665 wait complete. 2024-06-03T02:34:31.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:31.969+0000 7f54315f0b80 20 client.26929 trim_cache size 0 max 16384 2024-06-03T02:34:32.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.009+0000 7f54315f0b80 1 Processor -- start 2024-06-03T02:34:32.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.011+0000 7f54315f0b80 1 -- start start 2024-06-03T02:34:32.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.011+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:32.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.011+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b0300 0x5601464a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:32.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.011+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b1c30 0x5601464af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.011+0000 7f54315f0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f540000eb90 con 0x5601464b1c30 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.011+0000 7f54315f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54000106d0 con 0x560146421e40 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.011+0000 7f54315f0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f540000f260 con 0x5601464b0300 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.012+0000 7f541fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.012+0000 7f541fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:47200/0 (socket says 172.21.15.79:47200) 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.012+0000 7f541fe00700 1 -- 172.21.15.79:0/1135858820 learned_addr learned my addr 172.21.15.79:0/1135858820 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.012+0000 7f5424c00700 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b1c30 0x5601464af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.012+0000 7f541f400700 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b0300 0x5601464a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.013+0000 7f541fe00700 1 -- 172.21.15.79:0/1135858820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b0300 msgr2=0x5601464a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.013+0000 7f541fe00700 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b0300 0x5601464a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.013+0000 7f541fe00700 1 -- 172.21.15.79:0/1135858820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b1c30 msgr2=0x5601464af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.013+0000 7f541fe00700 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b1c30 0x5601464af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.013+0000 7f541fe00700 1 -- 172.21.15.79:0/1135858820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601464a53e0 con 0x560146421e40 2024-06-03T02:34:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.013+0000 7f541f400700 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b0300 0x5601464a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:32.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.013+0000 7f5424c00700 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b1c30 0x5601464af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:32.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.013+0000 7f541fe00700 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464b16f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5410022840 tx=0x7f54100cd490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9964b337ded3df13 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:32.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.014+0000 7f5414c00700 1 -- 172.21.15.79:0/1135858820 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541009c3b0 con 0x560146421e40 2024-06-03T02:34:32.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.014+0000 7f5414c00700 1 -- 172.21.15.79:0/1135858820 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54100a0760 con 0x560146421e40 2024-06-03T02:34:32.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.014+0000 7f5414c00700 1 -- 172.21.15.79:0/1135858820 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54100a4d60 con 0x560146421e40 2024-06-03T02:34:32.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.015+0000 7f54315f0b80 1 -- 172.21.15.79:0/1135858820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 msgr2=0x5601464b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:32.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.015+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464b16f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5410022840 tx=0x7f54100cd490 comp rx=0 tx=0).stop 2024-06-03T02:34:32.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.017+0000 7f54315f0b80 1 -- 172.21.15.79:0/1135858820 shutdown_connections 2024-06-03T02:34:32.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.017+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b1c30 0x5601464af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.017+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464b16f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.017+0000 7f54315f0b80 1 --2- 172.21.15.79:0/1135858820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b0300 0x5601464a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.017+0000 7f54315f0b80 1 -- 172.21.15.79:0/1135858820 >> 172.21.15.79:0/1135858820 conn(0x5601463d2170 msgr2=0x56014649feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:32.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.017+0000 7f54315f0b80 1 -- 172.21.15.79:0/1135858820 shutdown_connections 2024-06-03T02:34:32.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.017+0000 7f54315f0b80 1 -- 172.21.15.79:0/1135858820 wait complete. 2024-06-03T02:34:32.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.018+0000 7f54315f0b80 1 -- start start 2024-06-03T02:34:32.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.018+0000 7f54315f0b80 1 Processor -- start 2024-06-03T02:34:32.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.020+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464aa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:32.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.020+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b0300 0x56014652bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:32.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.020+0000 7f54315f0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b1c30 0x56014652c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:32.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.020+0000 7f54315f0b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5400010000 con 0x5601464b0300 2024-06-03T02:34:32.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.020+0000 7f54315f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f540000f930 con 0x560146421e40 2024-06-03T02:34:32.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.020+0000 7f54315f0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f540000cb30 con 0x5601464b1c30 2024-06-03T02:34:32.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.020+0000 7f54315f0b80 10 client.0 _ll_register_callbacks cb 0x560146391720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:32.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.020+0000 7f541fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:32.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.020+0000 7f5424c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b1c30 0x56014652c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:32.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.021+0000 7f5424c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b1c30 0x56014652c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:37632/0 (socket says 172.21.15.79:37632) 2024-06-03T02:34:32.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.021+0000 7f5424c00700 1 -- 172.21.15.79:0/258854073 learned_addr learned my addr 172.21.15.79:0/258854073 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:32.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.021+0000 7f541fe00700 1 -- 172.21.15.79:0/258854073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b1c30 msgr2=0x56014652c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:32.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.021+0000 7f541fe00700 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b1c30 0x56014652c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.021+0000 7f541fe00700 1 -- 172.21.15.79:0/258854073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b0300 msgr2=0x56014652bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:32.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.021+0000 7f541fe00700 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b0300 0x56014652bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.021+0000 7f541fe00700 1 -- 172.21.15.79:0/258854073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601464b2a60 con 0x560146421e40 2024-06-03T02:34:32.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.022+0000 7f541fe00700 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464aa630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x5601464a62e0 tx=0x7f541009cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e47ef95b0667b14 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:32.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.022+0000 7f540a000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:32.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.022+0000 7f540a000700 1 -- 172.21.15.79:0/258854073 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410022470 con 0x560146421e40 2024-06-03T02:34:32.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.022+0000 7f540a000700 1 -- 172.21.15.79:0/258854073 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f54100a4b00 con 0x560146421e40 2024-06-03T02:34:32.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.022+0000 7f54315f0b80 1 -- 172.21.15.79:0/258854073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54100a0760 con 0x560146421e40 2024-06-03T02:34:32.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.022+0000 7f540a000700 1 -- 172.21.15.79:0/258854073 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54100a4d00 con 0x560146421e40 2024-06-03T02:34:32.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.022+0000 7f54315f0b80 10 client.26951 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:32.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.022+0000 7f54315f0b80 1 -- 172.21.15.79:0/258854073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601464cc2f0 con 0x560146421e40 2024-06-03T02:34:32.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.023+0000 7f540a000700 1 -- 172.21.15.79:0/258854073 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54100a0760 con 0x560146421e40 2024-06-03T02:34:32.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.023+0000 7f540a000700 1 -- 172.21.15.79:0/258854073 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f541009b990 con 0x560146421e40 2024-06-03T02:34:32.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.023+0000 7f54315f0b80 10 client.26951 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:32.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.023+0000 7f54315f0b80 10 client.26951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:34:32.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.023+0000 7f54315f0b80 20 client.26951 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:32.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.023+0000 7f54315f0b80 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601464bbb20 0x5601464bbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:32.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.023+0000 7f54315f0b80 4 client.26951 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T02:34:32.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.023+0000 7f54315f0b80 1 -- 172.21.15.79:0/258854073 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5601463ff580 con 0x5601464bbb20 2024-06-03T02:34:32.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.023+0000 7f541f400700 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601464bbb20 0x5601464bbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:32.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.024+0000 7f540a000700 10 client.26951 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:32.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.026+0000 7f541f400700 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601464bbb20 0x5601464bbf30 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbfb6cf57f72f412 server_cookie=b87d5f107be5b6b3 in_seq=0 out_seq=0 2024-06-03T02:34:32.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.026+0000 7f540a000700 1 -- 172.21.15.79:0/258854073 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5601463ff580 con 0x5601464bbb20 2024-06-03T02:34:32.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.026+0000 7f540a000700 10 client.26951 handle_command_reply: tid=0 2024-06-03T02:34:32.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.026+0000 7f54315f0b80 1 client.26951 shutdown 2024-06-03T02:34:32.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.027+0000 7f54315f0b80 1 -- 172.21.15.79:0/258854073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 msgr2=0x5601464aa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.027+0000 7f54315f0b80 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464aa630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x5601464a62e0 tx=0x7f541009cb70 comp rx=0 tx=0).stop 2024-06-03T02:34:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.027+0000 7f54315f0b80 1 -- 172.21.15.79:0/258854073 shutdown_connections 2024-06-03T02:34:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.027+0000 7f54315f0b80 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601464b1c30 0x56014652c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.027+0000 7f54315f0b80 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560146421e40 0x5601464aa630 secure :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0x5601464a62e0 tx=0x7f541009cb70 comp rx=0 tx=0).stop 2024-06-03T02:34:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.027+0000 7f54315f0b80 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x5601464bbb20 0x5601464bbf30 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.027+0000 7f54315f0b80 1 --2- 172.21.15.79:0/258854073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5601464b0300 0x56014652bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.027+0000 7f540a000700 0 client.26951 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:34:32.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.027+0000 7f54315f0b80 1 -- 172.21.15.79:0/258854073 >> 172.21.15.79:0/258854073 conn(0x5601463d2170 msgr2=0x5601464a0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:32.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.028+0000 7f54315f0b80 1 -- 172.21.15.79:0/258854073 shutdown_connections 2024-06-03T02:34:32.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.029+0000 7f54315f0b80 1 -- 172.21.15.79:0/258854073 wait complete. 2024-06-03T02:34:32.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.029+0000 7f54315f0b80 20 client.26951 trim_cache size 0 max 16384 2024-06-03T02:34:32.030 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:34:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "status": "no active scrubs running", 2024-06-03T02:34:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "scrubs": {} 2024-06-03T02:34:32.031 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:34:32.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.030+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54040a2f20 msgr2=0x7f54040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:32.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.030+0000 7f5426c00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54040a2f20 0x7f54040a53e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f540c005950 tx=0x7f540c0058e0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.031+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542006a170 msgr2=0x7f542019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:32.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.031+0000 7f5426c00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542006a170 0x7f542019ea40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5410000c00 tx=0x7f5410003890 comp rx=0 tx=0).stop 2024-06-03T02:34:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.032+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 shutdown_connections 2024-06-03T02:34:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.032+0000 7f5426c00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5420108500 0x7f5420198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.032+0000 7f5426c00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f542006a170 0x7f542019ea40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.032+0000 7f5426c00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f54040a2f20 0x7f54040a53e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.032+0000 7f5426c00700 1 --2- 172.21.15.79:0/3444526646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f542006ab30 0x7f542019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.032+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 >> 172.21.15.79:0/3444526646 conn(0x7f5420078340 msgr2=0x7f5420100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.032+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 shutdown_connections 2024-06-03T02:34:32.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:32.032+0000 7f5426c00700 1 -- 172.21.15.79:0/3444526646 wait complete. 2024-06-03T02:34:32.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:32 smithi079 ceph-mon[31819]: pgmap v950: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.4 MiB/s wr, 601 op/s 2024-06-03T02:34:32.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:32 smithi123 ceph-mon[36640]: pgmap v950: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.4 MiB/s wr, 601 op/s 2024-06-03T02:34:32.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:32 smithi135 ceph-mon[36653]: pgmap v950: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.4 MiB/s wr, 601 op/s 2024-06-03T02:34:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 865.0 s, 505.8 tps, lat 194.058 ms stddev 248.268 2024-06-03T02:34:34.093 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T02:34:34.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:34:34.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:34 smithi079 ceph-mon[31819]: pgmap v951: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-06-03T02:34:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:34 smithi123 ceph-mon[36640]: pgmap v951: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-06-03T02:34:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:34 smithi135 ceph-mon[36653]: pgmap v951: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-06-03T02:34:34.771 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:34:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:36 smithi123 ceph-mon[36640]: pgmap v952: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-06-03T02:34:36.948 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:36 smithi135 ceph-mon[36653]: pgmap v952: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-06-03T02:34:36.948 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:36 smithi079 ceph-mon[31819]: pgmap v952: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-06-03T02:34:38.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 870.0 s, 457.4 tps, lat 221.304 ms stddev 254.112 2024-06-03T02:34:38.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.355+0000 7f9d62c00700 1 -- 172.21.15.79:0/2243152813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d5c109db0 msgr2=0x7f9d5c1150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:38.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.355+0000 7f9d62c00700 1 --2- 172.21.15.79:0/2243152813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d5c109db0 0x7f9d5c1150a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400b0a0 tx=0x7f9d5400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.355+0000 7f9d62c00700 1 -- 172.21.15.79:0/2243152813 shutdown_connections 2024-06-03T02:34:38.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.355+0000 7f9d62c00700 1 --2- 172.21.15.79:0/2243152813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d5c1155e0 0x7f9d5c1179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.355+0000 7f9d62c00700 1 --2- 172.21.15.79:0/2243152813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c109490 0x7f9d5c109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.355+0000 7f9d62c00700 1 --2- 172.21.15.79:0/2243152813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d5c109db0 0x7f9d5c1150a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.355+0000 7f9d62c00700 1 -- 172.21.15.79:0/2243152813 >> 172.21.15.79:0/2243152813 conn(0x7f9d5c06d600 msgr2=0x7f9d5c06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:38.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.358+0000 7f9d62c00700 1 -- 172.21.15.79:0/2243152813 shutdown_connections 2024-06-03T02:34:38.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.358+0000 7f9d62c00700 1 -- 172.21.15.79:0/2243152813 wait complete. 2024-06-03T02:34:38.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.359+0000 7f9d62c00700 1 Processor -- start 2024-06-03T02:34:38.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.360+0000 7f9d62c00700 1 -- start start 2024-06-03T02:34:38.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.360+0000 7f9d62c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d5c109490 0x7f9d5c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:38.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.360+0000 7f9d62c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d5c109db0 0x7f9d5c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.360+0000 7f9d62c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c1155e0 0x7f9d5c1a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d62c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c11b050 con 0x7f9d5c1155e0 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d62c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c11aed0 con 0x7f9d5c109db0 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d62c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c11b1d0 con 0x7f9d5c109490 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d5be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d5c109490 0x7f9d5c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d5b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d5c109db0 0x7f9d5c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d5be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d5c109490 0x7f9d5c1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:58362/0 (socket says 172.21.15.79:58362) 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d5be00700 1 -- 172.21.15.79:0/3038488824 learned_addr learned my addr 172.21.15.79:0/3038488824 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d5b400700 1 -- 172.21.15.79:0/3038488824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d5c109490 msgr2=0x7f9d5c1a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d5b400700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d5c109490 0x7f9d5c1a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d5b400700 1 -- 172.21.15.79:0/3038488824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c1155e0 msgr2=0x7f9d5c1a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.361+0000 7f9d5b400700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c1155e0 0x7f9d5c1a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.362+0000 7f9d5b400700 1 -- 172.21.15.79:0/3038488824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c1a6530 con 0x7f9d5c109db0 2024-06-03T02:34:38.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:38 smithi079 ceph-mon[31819]: pgmap v953: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.7 MiB/s wr, 534 op/s 2024-06-03T02:34:38.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.362+0000 7f9d5b400700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d5c109db0 0x7f9d5c1a1a60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400b780 tx=0x7f9d54004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:38.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.363+0000 7f9d58c00700 1 -- 172.21.15.79:0/3038488824 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54021070 con 0x7f9d5c109db0 2024-06-03T02:34:38.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.363+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5c1a67c0 con 0x7f9d5c109db0 2024-06-03T02:34:38.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.363+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5c07a6f0 con 0x7f9d5c109db0 2024-06-03T02:34:38.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.363+0000 7f9d58c00700 1 -- 172.21.15.79:0/3038488824 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d54007c80 con 0x7f9d5c109db0 2024-06-03T02:34:38.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.363+0000 7f9d58c00700 1 -- 172.21.15.79:0/3038488824 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54005350 con 0x7f9d5c109db0 2024-06-03T02:34:38.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.365+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5c069180 con 0x7f9d5c109db0 2024-06-03T02:34:38.369 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.368+0000 7f9d58c00700 1 -- 172.21.15.79:0/3038488824 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9d54024070 con 0x7f9d5c109db0 2024-06-03T02:34:38.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.371+0000 7f9d58c00700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9d400a2ca0 0x7f9d400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:38.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.371+0000 7f9d58c00700 1 -- 172.21.15.79:0/3038488824 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f9d5400f020 con 0x7f9d5c109db0 2024-06-03T02:34:38.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.373+0000 7f9d5be00700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9d400a2ca0 0x7f9d400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:38.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.375+0000 7f9d5be00700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9d400a2ca0 0x7f9d400a5160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9d48005950 tx=0x7f9d48004080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:38.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.376+0000 7f9d58c00700 1 -- 172.21.15.79:0/3038488824 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9d5402b020 con 0x7f9d5c109db0 2024-06-03T02:34:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:38 smithi123 ceph-mon[36640]: pgmap v953: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.7 MiB/s wr, 534 op/s 2024-06-03T02:34:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:38 smithi135 ceph-mon[36653]: pgmap v953: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.7 MiB/s wr, 534 op/s 2024-06-03T02:34:38.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.864+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d5c1a2880 con 0x7f9d5c109db0 2024-06-03T02:34:38.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.865+0000 7f9d58c00700 1 -- 172.21.15.79:0/3038488824 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f9d5409d620 con 0x7f9d5c109db0 2024-06-03T02:34:38.866 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:34:38.867 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:34:38.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.871+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9d400a2ca0 msgr2=0x7f9d400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:38.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.871+0000 7f9d62c00700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9d400a2ca0 0x7f9d400a5160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9d48005950 tx=0x7f9d48004080 comp rx=0 tx=0).stop 2024-06-03T02:34:38.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.871+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d5c109db0 msgr2=0x7f9d5c1a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:38.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.871+0000 7f9d62c00700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d5c109db0 0x7f9d5c1a1a60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400b780 tx=0x7f9d54004db0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.874+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 shutdown_connections 2024-06-03T02:34:38.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.874+0000 7f9d62c00700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9d400a2ca0 0x7f9d400a5160 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.874+0000 7f9d62c00700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c1155e0 0x7f9d5c1a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.874+0000 7f9d62c00700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d5c109490 0x7f9d5c1a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.874+0000 7f9d62c00700 1 --2- 172.21.15.79:0/3038488824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d5c109db0 0x7f9d5c1a1a60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:38.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.874+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 >> 172.21.15.79:0/3038488824 conn(0x7f9d5c06d600 msgr2=0x7f9d5c0711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:38.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.876+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 shutdown_connections 2024-06-03T02:34:38.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:38.876+0000 7f9d62c00700 1 -- 172.21.15.79:0/3038488824 wait complete. 2024-06-03T02:34:38.880 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:34:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:39 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3038488824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:34:39.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:39 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3038488824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:34:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:39 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3038488824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:34:40.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:40 smithi123 ceph-mon[36640]: pgmap v954: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-06-03T02:34:40.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:40 smithi079 ceph-mon[31819]: pgmap v954: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-06-03T02:34:40.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:40 smithi135 ceph-mon[36653]: pgmap v954: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-06-03T02:34:41.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T02:34:42.065 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:34:42.412 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:42 smithi079 ceph-mon[31819]: pgmap v955: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 487 op/s 2024-06-03T02:34:42.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:42 smithi123 ceph-mon[36640]: pgmap v955: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 487 op/s 2024-06-03T02:34:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:42 smithi135 ceph-mon[36653]: pgmap v955: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 487 op/s 2024-06-03T02:34:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 875.0 s, 489.8 tps, lat 202.950 ms stddev 246.513 2024-06-03T02:34:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:44 smithi123 ceph-mon[36640]: pgmap v956: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.8 MiB/s wr, 435 op/s 2024-06-03T02:34:44.501 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:44 smithi079 ceph-mon[31819]: pgmap v956: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.8 MiB/s wr, 435 op/s 2024-06-03T02:34:44.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:44 smithi135 ceph-mon[36653]: pgmap v956: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.8 MiB/s wr, 435 op/s 2024-06-03T02:34:46.298 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:46 smithi079 ceph-mon[31819]: pgmap v957: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-06-03T02:34:46.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:46 smithi123 ceph-mon[36640]: pgmap v957: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-06-03T02:34:46.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:46 smithi135 ceph-mon[36653]: pgmap v957: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-06-03T02:34:48.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.042+0000 7f7d57600700 1 -- 172.21.15.79:0/2162783210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d501027a0 msgr2=0x7f7d50102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:48.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.042+0000 7f7d57600700 1 --2- 172.21.15.79:0/2162783210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d501027a0 0x7f7d50102b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009990 tx=0x7f7d40009ca0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.043+0000 7f7d57600700 1 -- 172.21.15.79:0/2162783210 shutdown_connections 2024-06-03T02:34:48.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.043+0000 7f7d57600700 1 --2- 172.21.15.79:0/2162783210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d50107870 0x7f7d5010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.043+0000 7f7d57600700 1 --2- 172.21.15.79:0/2162783210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d501027a0 0x7f7d50102b80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.043+0000 7f7d57600700 1 --2- 172.21.15.79:0/2162783210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d50103150 0x7f7d50107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.043+0000 7f7d57600700 1 -- 172.21.15.79:0/2162783210 >> 172.21.15.79:0/2162783210 conn(0x7f7d50079570 msgr2=0x7f7d50079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:48.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.043+0000 7f7d57600700 1 -- 172.21.15.79:0/2162783210 shutdown_connections 2024-06-03T02:34:48.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.043+0000 7f7d57600700 1 -- 172.21.15.79:0/2162783210 wait complete. 2024-06-03T02:34:48.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.044+0000 7f7d57600700 1 Processor -- start 2024-06-03T02:34:48.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.044+0000 7f7d57600700 1 -- start start 2024-06-03T02:34:48.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.045+0000 7f7d57600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d501027a0 0x7f7d50074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.045+0000 7f7d57600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d50103150 0x7f7d50074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.045+0000 7f7d57600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d50107870 0x7f7d50075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.045+0000 7f7d57600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501127e0 con 0x7f7d50103150 2024-06-03T02:34:48.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.045+0000 7f7d57600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50112660 con 0x7f7d501027a0 2024-06-03T02:34:48.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.045+0000 7f7d57600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50112960 con 0x7f7d50107870 2024-06-03T02:34:48.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.045+0000 7f7d54c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d501027a0 0x7f7d50074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d55600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d50107870 0x7f7d50075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d55600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d50107870 0x7f7d50075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33252/0 (socket says 172.21.15.79:33252) 2024-06-03T02:34:48.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d55600700 1 -- 172.21.15.79:0/1972248269 learned_addr learned my addr 172.21.15.79:0/1972248269 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:48.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d47e00700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d50103150 0x7f7d50074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d55600700 1 -- 172.21.15.79:0/1972248269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d501027a0 msgr2=0x7f7d50074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:48.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d55600700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d501027a0 0x7f7d50074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d55600700 1 -- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d50103150 msgr2=0x7f7d50074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d55600700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d50103150 0x7f7d50074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d55600700 1 -- 172.21.15.79:0/1972248269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501a70c0 con 0x7f7d50107870 2024-06-03T02:34:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.046+0000 7f7d47e00700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d50103150 0x7f7d50074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:34:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.047+0000 7f7d55600700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d50107870 0x7f7d50075210 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800ef30 tx=0x7f7d4800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.047+0000 7f7d4e000700 1 -- 172.21.15.79:0/1972248269 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48004bb0 con 0x7f7d50107870 2024-06-03T02:34:48.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.047+0000 7f7d4e000700 1 -- 172.21.15.79:0/1972248269 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48004d50 con 0x7f7d50107870 2024-06-03T02:34:48.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.047+0000 7f7d57600700 1 -- 172.21.15.79:0/1972248269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d50198930 con 0x7f7d50107870 2024-06-03T02:34:48.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.048+0000 7f7d4e000700 1 -- 172.21.15.79:0/1972248269 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800cd30 con 0x7f7d50107870 2024-06-03T02:34:48.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.048+0000 7f7d57600700 1 -- 172.21.15.79:0/1972248269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d50198e70 con 0x7f7d50107870 2024-06-03T02:34:48.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.054+0000 7f7d4e000700 1 -- 172.21.15.79:0/1972248269 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7d48017a80 con 0x7f7d50107870 2024-06-03T02:34:48.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.057+0000 7f7d4e000700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d340a2ca0 0x7f7d340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.057+0000 7f7d4e000700 1 -- 172.21.15.79:0/1972248269 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7d480d7150 con 0x7f7d50107870 2024-06-03T02:34:48.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.057+0000 7f7d54c00700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d340a2ca0 0x7f7d340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.059+0000 7f7d61fabb80 1 Processor -- start 2024-06-03T02:34:48.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.059+0000 7f7d54c00700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d340a2ca0 0x7f7d340a5160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d40006010 tx=0x7f7d4000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:48.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.059+0000 7f7d61fabb80 1 -- start start 2024-06-03T02:34:48.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449e6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 0x5558449f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5558448f5a10 con 0x5558449f1300 2024-06-03T02:34:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555844938b00 con 0x5558449e65b0 2024-06-03T02:34:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5558448f38f0 con 0x555844962e40 2024-06-03T02:34:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d54c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d54c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449e6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33266/0 (socket says 172.21.15.79:33266) 2024-06-03T02:34:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d54c00700 1 -- 172.21.15.79:0/1161573619 learned_addr learned my addr 172.21.15.79:0/1161573619 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.060+0000 7f7d55600700 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 0x5558449f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.061+0000 7f7d54c00700 1 -- 172.21.15.79:0/1161573619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 msgr2=0x5558449e6a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:34:48.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.062+0000 7f7d47e00700 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.062+0000 7f7d54c00700 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449e6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.062+0000 7f7d54c00700 1 -- 172.21.15.79:0/1161573619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 msgr2=0x5558449f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:48.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.062+0000 7f7d54c00700 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 0x5558449f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.062+0000 7f7d54c00700 1 -- 172.21.15.79:0/1161573619 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558448e0f90 con 0x555844962e40 2024-06-03T02:34:48.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.062+0000 7f7d55600700 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 0x5558449f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:48.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.062+0000 7f7d47e00700 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:48.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.063+0000 7f7d54c00700 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449e6070 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d40015000 tx=0x7f7d4001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d161e6c41f6667eb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:48.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d46000700 1 -- 172.21.15.79:0/1161573619 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40003680 con 0x555844962e40 2024-06-03T02:34:48.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d46000700 1 -- 172.21.15.79:0/1161573619 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d40003d00 con 0x555844962e40 2024-06-03T02:34:48.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d46000700 1 -- 172.21.15.79:0/1161573619 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000fd90 con 0x555844962e40 2024-06-03T02:34:48.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d61fabb80 1 -- 172.21.15.79:0/1161573619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 msgr2=0x5558449e6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:48.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449e6070 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d40015000 tx=0x7f7d4001f000 comp rx=0 tx=0).stop 2024-06-03T02:34:48.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d61fabb80 1 -- 172.21.15.79:0/1161573619 shutdown_connections 2024-06-03T02:34:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 0x5558449f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449e6070 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/1161573619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.064+0000 7f7d61fabb80 1 -- 172.21.15.79:0/1161573619 >> 172.21.15.79:0/1161573619 conn(0x555844913170 msgr2=0x5558449e24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.065+0000 7f7d61fabb80 1 -- 172.21.15.79:0/1161573619 shutdown_connections 2024-06-03T02:34:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.065+0000 7f7d61fabb80 1 -- 172.21.15.79:0/1161573619 wait complete. 2024-06-03T02:34:48.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.066+0000 7f7d61fabb80 1 -- start start 2024-06-03T02:34:48.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.066+0000 7f7d61fabb80 1 Processor -- start 2024-06-03T02:34:48.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.066+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 0x5558449f0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.066+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449edc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.066+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449ee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.066+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5558448d4f60 con 0x555844962e40 2024-06-03T02:34:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.066+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555844934ac0 con 0x5558449e65b0 2024-06-03T02:34:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.066+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5558449f3f00 con 0x5558449f1300 2024-06-03T02:34:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.066+0000 7f7d61fabb80 10 client.0 _ll_register_callbacks cb 0x5558448d2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d47e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449edc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d47e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449edc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:57450/0 (socket says 172.21.15.79:57450) 2024-06-03T02:34:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d47e00700 1 -- 172.21.15.79:0/652117638 learned_addr learned my addr 172.21.15.79:0/652117638 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d55600700 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d54c00700 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 0x5558449f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d47e00700 1 -- 172.21.15.79:0/652117638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 msgr2=0x5558449ee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d47e00700 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449ee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d47e00700 1 -- 172.21.15.79:0/652117638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 msgr2=0x5558449f0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d47e00700 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 0x5558449f0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.067+0000 7f7d47e00700 1 -- 172.21.15.79:0/652117638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558448878d0 con 0x5558449e65b0 2024-06-03T02:34:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.068+0000 7f7d47e00700 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449edc20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800f660 tx=0x7f7d3800f970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14c2901f0ffddce8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.068+0000 7f7d3ea00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.068+0000 7f7d3ea00700 1 -- 172.21.15.79:0/652117638 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38004a30 con 0x5558449e65b0 2024-06-03T02:34:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.068+0000 7f7d3ea00700 1 -- 172.21.15.79:0/652117638 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7d38004bd0 con 0x5558449e65b0 2024-06-03T02:34:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.068+0000 7f7d61fabb80 1 -- 172.21.15.79:0/652117638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d40003d00 con 0x5558449e65b0 2024-06-03T02:34:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.068+0000 7f7d3ea00700 1 -- 172.21.15.79:0/652117638 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d38004dd0 con 0x5558449e65b0 2024-06-03T02:34:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d61fabb80 10 client.26966 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:48.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d61fabb80 1 -- 172.21.15.79:0/652117638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558449fe570 con 0x5558449e65b0 2024-06-03T02:34:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d3ea00700 1 -- 172.21.15.79:0/652117638 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d40003d00 con 0x5558449e65b0 2024-06-03T02:34:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d3ea00700 1 -- 172.21.15.79:0/652117638 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7d38011cd0 con 0x5558449e65b0 2024-06-03T02:34:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d61fabb80 10 client.26966 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d61fabb80 10 client.26966 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:34:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d61fabb80 20 client.26966 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x555844a00bb0 0x555844a02fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d61fabb80 4 client.26966 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:34:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.069+0000 7f7d61fabb80 1 -- 172.21.15.79:0/652117638 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555844946000 con 0x555844a00bb0 2024-06-03T02:34:48.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.070+0000 7f7d54c00700 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x555844a00bb0 0x555844a02fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.070+0000 7f7d54c00700 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x555844a00bb0 0x555844a02fa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=baf08e29a24b6316 server_cookie=9c9b45c8b99c3ee in_seq=0 out_seq=0 2024-06-03T02:34:48.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.070+0000 7f7d3ea00700 10 client.26966 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.071+0000 7f7d3ea00700 1 -- 172.21.15.79:0/652117638 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555844946000 con 0x555844a00bb0 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.071+0000 7f7d3ea00700 10 client.26966 handle_command_reply: tid=0 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.071+0000 7f7d61fabb80 1 client.26966 shutdown 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 -- 172.21.15.79:0/652117638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 msgr2=0x5558449edc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449edc20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800f660 tx=0x7f7d3800f970 comp rx=0 tx=0).stop 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 -- 172.21.15.79:0/652117638 shutdown_connections 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x555844a00bb0 0x555844a02fa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 0x5558449f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/652117638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449edc20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 -- 172.21.15.79:0/652117638 >> 172.21.15.79:0/652117638 conn(0x555844913170 msgr2=0x5558449e1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d3ea00700 0 client.26966 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:34:48.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 -- 172.21.15.79:0/652117638 shutdown_connections 2024-06-03T02:34:48.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 1 -- 172.21.15.79:0/652117638 wait complete. 2024-06-03T02:34:48.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.072+0000 7f7d61fabb80 20 client.26966 trim_cache size 0 max 16384 2024-06-03T02:34:48.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.109+0000 7f7d61fabb80 1 Processor -- start 2024-06-03T02:34:48.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.109+0000 7f7d61fabb80 1 -- start start 2024-06-03T02:34:48.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.109+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 0x5558449f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.109+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.109+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.109+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c00eaf0 con 0x555844962e40 2024-06-03T02:34:48.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.109+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c010630 con 0x5558449e65b0 2024-06-03T02:34:48.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.109+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c00f1c0 con 0x5558449f1300 2024-06-03T02:34:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.110+0000 7f7d55600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.110+0000 7f7d55600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33290/0 (socket says 172.21.15.79:33290) 2024-06-03T02:34:48.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.110+0000 7f7d55600700 1 -- 172.21.15.79:0/2559859433 learned_addr learned my addr 172.21.15.79:0/2559859433 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:48.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.110+0000 7f7d54c00700 1 --2- 172.21.15.79:0/2559859433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 0x5558449f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.111+0000 7f7d55600700 1 -- 172.21.15.79:0/2559859433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 msgr2=0x5558449f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:48.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.111+0000 7f7d55600700 1 --2- 172.21.15.79:0/2559859433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.111+0000 7f7d55600700 1 -- 172.21.15.79:0/2559859433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 msgr2=0x5558449f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:48.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.111+0000 7f7d55600700 1 --2- 172.21.15.79:0/2559859433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 0x5558449f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.111+0000 7f7d55600700 1 -- 172.21.15.79:0/2559859433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558449ec190 con 0x5558449f1300 2024-06-03T02:34:48.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.112+0000 7f7d55600700 1 --2- 172.21.15.79:0/2559859433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449f3070 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c7a0 tx=0x7f7d4800cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdaa98649f4e8bf0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:48.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.115+0000 7f7d45600700 1 -- 172.21.15.79:0/2559859433 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480bb070 con 0x5558449f1300 2024-06-03T02:34:48.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.115+0000 7f7d45600700 1 -- 172.21.15.79:0/2559859433 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48025300 con 0x5558449f1300 2024-06-03T02:34:48.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.115+0000 7f7d45600700 1 -- 172.21.15.79:0/2559859433 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48023040 con 0x5558449f1300 2024-06-03T02:34:48.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.116+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2559859433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 msgr2=0x5558449f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:48.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.116+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2559859433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449f3070 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c7a0 tx=0x7f7d4800cf80 comp rx=0 tx=0).stop 2024-06-03T02:34:48.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.117+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2559859433 shutdown_connections 2024-06-03T02:34:48.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.117+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2559859433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5558449f1300 0x5558449f3070 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.117+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2559859433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555844962e40 0x5558449f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.117+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2559859433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.118+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2559859433 >> 172.21.15.79:0/2559859433 conn(0x555844913170 msgr2=0x5558449ec510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:48.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.120+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2559859433 shutdown_connections 2024-06-03T02:34:48.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.120+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2559859433 wait complete. 2024-06-03T02:34:48.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.120+0000 7f7d61fabb80 1 -- start start 2024-06-03T02:34:48.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.121+0000 7f7d61fabb80 1 Processor -- start 2024-06-03T02:34:48.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.122+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.122+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449eab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.122+0000 7f7d61fabb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 0x5558449f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.122+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c00ff60 con 0x5558449f1300 2024-06-03T02:34:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.122+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c00f890 con 0x5558449e65b0 2024-06-03T02:34:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.122+0000 7f7d61fabb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c00ca90 con 0x555844962e40 2024-06-03T02:34:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.122+0000 7f7d61fabb80 10 client.0 _ll_register_callbacks cb 0x5558448d2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:48.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.123+0000 7f7d54c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.123+0000 7f7d54c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449ea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:33298/0 (socket says 172.21.15.79:33298) 2024-06-03T02:34:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.123+0000 7f7d54c00700 1 -- 172.21.15.79:0/2773224583 learned_addr learned my addr 172.21.15.79:0/2773224583 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.124+0000 7f7d55600700 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 0x5558449f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.124+0000 7f7d47e00700 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.124+0000 7f7d47e00700 1 -- 172.21.15.79:0/2773224583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 msgr2=0x5558449ea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.124+0000 7f7d47e00700 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449ea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.124+0000 7f7d47e00700 1 -- 172.21.15.79:0/2773224583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 msgr2=0x5558449f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.124+0000 7f7d47e00700 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 0x5558449f8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.124+0000 7f7d47e00700 1 -- 172.21.15.79:0/2773224583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558449f8810 con 0x5558449e65b0 2024-06-03T02:34:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.125+0000 7f7d47e00700 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449eab90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x5558449e9080 tx=0x7f7d3801ae00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be0417b80554d349 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.125+0000 7f7d3ea00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.125+0000 7f7d3ea00700 1 -- 172.21.15.79:0/2773224583 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3802c020 con 0x5558449e65b0 2024-06-03T02:34:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.125+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2773224583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d48025300 con 0x5558449e65b0 2024-06-03T02:34:48.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.125+0000 7f7d3ea00700 1 -- 172.21.15.79:0/2773224583 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f7d38005180 con 0x5558449e65b0 2024-06-03T02:34:48.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.126+0000 7f7d3ea00700 1 -- 172.21.15.79:0/2773224583 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d38037050 con 0x5558449e65b0 2024-06-03T02:34:48.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.126+0000 7f7d61fabb80 10 client.26969 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:48.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.126+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2773224583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558449f9af0 con 0x5558449e65b0 2024-06-03T02:34:48.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.128+0000 7f7d3ea00700 1 -- 172.21.15.79:0/2773224583 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48025300 con 0x5558449e65b0 2024-06-03T02:34:48.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.129+0000 7f7d3ea00700 1 -- 172.21.15.79:0/2773224583 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7d38005380 con 0x5558449e65b0 2024-06-03T02:34:48.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.129+0000 7f7d61fabb80 10 client.26969 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:48.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.129+0000 7f7d61fabb80 10 client.26969 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-03T02:34:48.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.129+0000 7f7d61fabb80 20 client.26969 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:48.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.129+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x555844a6b970 0x555844a6dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:48.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.129+0000 7f7d61fabb80 4 client.26969 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:34:48.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.129+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2773224583 --> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555844940580 con 0x555844a6b970 2024-06-03T02:34:48.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.131+0000 7f7d54c00700 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x555844a6b970 0x555844a6dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:48.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.132+0000 7f7d54c00700 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x555844a6b970 0x555844a6dd60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=459217fade816bde server_cookie=2708b1b13b719b80 in_seq=0 out_seq=0 2024-06-03T02:34:48.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.132+0000 7f7d3ea00700 10 client.26969 ms_handle_connect on v2:172.21.15.123:6834/4173566972 2024-06-03T02:34:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.135+0000 7f7d3ea00700 1 -- 172.21.15.79:0/2773224583 <== mds.4 v2:172.21.15.123:6834/4173566972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555844940580 con 0x555844a6b970 2024-06-03T02:34:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.135+0000 7f7d3ea00700 10 client.26969 handle_command_reply: tid=0 2024-06-03T02:34:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.135+0000 7f7d61fabb80 1 client.26969 shutdown 2024-06-03T02:34:48.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.137+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2773224583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 msgr2=0x5558449eab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:48.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.137+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449eab90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x5558449e9080 tx=0x7f7d3801ae00 comp rx=0 tx=0).stop 2024-06-03T02:34:48.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.137+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2773224583 shutdown_connections 2024-06-03T02:34:48.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.137+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.123:6834/4173566972,v1:172.21.15.123:6835/4173566972] conn(0x555844a6b970 0x555844a6dd60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.137+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5558449f1300 0x5558449f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.137+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555844962e40 0x5558449ea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.137+0000 7f7d61fabb80 1 --2- 172.21.15.79:0/2773224583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558449e65b0 0x5558449eab90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.137+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2773224583 >> 172.21.15.79:0/2773224583 conn(0x555844913170 msgr2=0x5558449e7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:48.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.139+0000 7f7d3ea00700 0 client.26969 ms_handle_reset on v2:172.21.15.123:6834/4173566972 2024-06-03T02:34:48.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.140+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2773224583 shutdown_connections 2024-06-03T02:34:48.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.140+0000 7f7d61fabb80 1 -- 172.21.15.79:0/2773224583 wait complete. 2024-06-03T02:34:48.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.141+0000 7f7d61fabb80 20 client.26969 trim_cache size 0 max 16384 2024-06-03T02:34:48.142 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:34:48.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 -- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d340a2ca0 msgr2=0x7f7d340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d340a2ca0 0x7f7d340a5160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d40006010 tx=0x7f7d4000b260 comp rx=0 tx=0).stop 2024-06-03T02:34:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 -- 172.21.15.79:0/1972248269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d50107870 msgr2=0x7f7d50075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d50107870 0x7f7d50075210 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800ef30 tx=0x7f7d4800c3a0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 -- 172.21.15.79:0/1972248269 shutdown_connections 2024-06-03T02:34:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d50107870 0x7f7d50075210 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d501027a0 0x7f7d50074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f7d340a2ca0 0x7f7d340a5160 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 --2- 172.21.15.79:0/1972248269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d50103150 0x7f7d50074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:48.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 -- 172.21.15.79:0/1972248269 >> 172.21.15.79:0/1972248269 conn(0x7f7d50079570 msgr2=0x7f7d501004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:48.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.143+0000 7f7d57600700 1 -- 172.21.15.79:0/1972248269 shutdown_connections 2024-06-03T02:34:48.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:48.144+0000 7f7d57600700 1 -- 172.21.15.79:0/1972248269 wait complete. 2024-06-03T02:34:48.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 880.0 s, 435.6 tps, lat 225.268 ms stddev 252.269 2024-06-03T02:34:48.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:48 smithi079 ceph-mon[31819]: pgmap v958: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-06-03T02:34:48.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:48 smithi123 ceph-mon[36640]: pgmap v958: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-06-03T02:34:48.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:48 smithi135 ceph-mon[36653]: pgmap v958: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-06-03T02:34:50.452 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:50 smithi079 ceph-mon[31819]: pgmap v959: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.1 MiB/s wr, 450 op/s 2024-06-03T02:34:50.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:50 smithi123 ceph-mon[36640]: pgmap v959: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.1 MiB/s wr, 450 op/s 2024-06-03T02:34:50.496 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T02:34:50.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:50 smithi135 ceph-mon[36653]: pgmap v959: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.1 MiB/s wr, 450 op/s 2024-06-03T02:34:51.244 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:34:52.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:52 smithi123 ceph-mon[36640]: pgmap v960: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-06-03T02:34:52.520 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:52 smithi079 ceph-mon[31819]: pgmap v960: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-06-03T02:34:52.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:52 smithi135 ceph-mon[36653]: pgmap v960: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-06-03T02:34:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 885.0 s, 473.6 tps, lat 211.515 ms stddev 234.563 2024-06-03T02:34:54.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:54 smithi123 ceph-mon[36640]: pgmap v961: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.1 MiB/s wr, 464 op/s 2024-06-03T02:34:54.477 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:54 smithi079 ceph-mon[31819]: pgmap v961: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.1 MiB/s wr, 464 op/s 2024-06-03T02:34:54.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:54 smithi135 ceph-mon[36653]: pgmap v961: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.1 MiB/s wr, 464 op/s 2024-06-03T02:34:56.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:56 smithi079 ceph-mon[31819]: pgmap v962: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.0 MiB/s wr, 451 op/s 2024-06-03T02:34:56.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:56 smithi123 ceph-mon[36640]: pgmap v962: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.0 MiB/s wr, 451 op/s 2024-06-03T02:34:56.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:56 smithi135 ceph-mon[36653]: pgmap v962: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.0 MiB/s wr, 451 op/s 2024-06-03T02:34:56.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.848+0000 7f28af200700 1 -- 172.21.15.79:0/1474058595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102620 msgr2=0x7f28b0102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:56.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.848+0000 7f28af200700 1 --2- 172.21.15.79:0/1474058595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102620 0x7f28b0102a00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f289c009990 tx=0x7f289c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.848+0000 7f28af200700 1 -- 172.21.15.79:0/1474058595 shutdown_connections 2024-06-03T02:34:56.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.848+0000 7f28af200700 1 --2- 172.21.15.79:0/1474058595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b0107700 0x7f28b010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.848+0000 7f28af200700 1 --2- 172.21.15.79:0/1474058595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102620 0x7f28b0102a00 secure :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f289c009990 tx=0x7f289c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.848+0000 7f28af200700 1 --2- 172.21.15.79:0/1474058595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28b0102fd0 0x7f28b0107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.848+0000 7f28af200700 1 -- 172.21.15.79:0/1474058595 >> 172.21.15.79:0/1474058595 conn(0x7f28b00fdf10 msgr2=0x7f28b0100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:56.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.849+0000 7f28af200700 1 -- 172.21.15.79:0/1474058595 shutdown_connections 2024-06-03T02:34:56.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.851+0000 7f28af200700 1 -- 172.21.15.79:0/1474058595 wait complete. 2024-06-03T02:34:56.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.853+0000 7f28af200700 1 Processor -- start 2024-06-03T02:34:56.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.855+0000 7f28af200700 1 -- start start 2024-06-03T02:34:56.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.855+0000 7f28af200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102fd0 0x7f28b019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.855+0000 7f28af200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b0107700 0x7f28b010e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.855+0000 7f28af200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28b010eb60 0x7f28b010f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.856+0000 7f28ad400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b0107700 0x7f28b010e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.856+0000 7f28ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102fd0 0x7f28b019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.856+0000 7f28ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102fd0 0x7f28b019fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49820/0 (socket says 172.21.15.79:49820) 2024-06-03T02:34:56.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.856+0000 7f28ad400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b0107700 0x7f28b010e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:57614/0 (socket says 172.21.15.79:57614) 2024-06-03T02:34:56.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.856+0000 7f28ade00700 1 -- 172.21.15.79:0/3932700523 learned_addr learned my addr 172.21.15.79:0/3932700523 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:56.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.856+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f28b01126e0 con 0x7f28b010eb60 2024-06-03T02:34:56.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.857+0000 7f28ae800700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28b010eb60 0x7f28b010f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.856+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28b0112560 con 0x7f28b0102fd0 2024-06-03T02:34:56.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.857+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28b0112860 con 0x7f28b0107700 2024-06-03T02:34:56.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.858+0000 7f28ade00700 1 -- 172.21.15.79:0/3932700523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b0107700 msgr2=0x7f28b010e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:56.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.858+0000 7f28ade00700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b0107700 0x7f28b010e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.858+0000 7f28ade00700 1 -- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28b010eb60 msgr2=0x7f28b010f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:56.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.859+0000 7f28ade00700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28b010eb60 0x7f28b010f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.859+0000 7f28ade00700 1 -- 172.21.15.79:0/3932700523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b019b450 con 0x7f28b0102fd0 2024-06-03T02:34:56.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.859+0000 7f28ad400700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b0107700 0x7f28b010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:34:56.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.859+0000 7f28ae800700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28b010eb60 0x7f28b010f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:34:56.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.860+0000 7f28ade00700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102fd0 0x7f28b019fa00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f289c0059d0 tx=0x7f289c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:56.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.861+0000 7f28a6a00700 1 -- 172.21.15.79:0/3932700523 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c026070 con 0x7f28b0102fd0 2024-06-03T02:34:56.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.861+0000 7f28a6a00700 1 -- 172.21.15.79:0/3932700523 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c00f460 con 0x7f28b0102fd0 2024-06-03T02:34:56.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.861+0000 7f28a6a00700 1 -- 172.21.15.79:0/3932700523 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c004230 con 0x7f28b0102fd0 2024-06-03T02:34:56.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.862+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b019b6e0 con 0x7f28b0102fd0 2024-06-03T02:34:56.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.864+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b019bb70 con 0x7f28b0102fd0 2024-06-03T02:34:56.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.877+0000 7f28a6a00700 1 -- 172.21.15.79:0/3932700523 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f289c029070 con 0x7f28b0102fd0 2024-06-03T02:34:56.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.880+0000 7f28a6a00700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f28940a2ca0 0x7f28940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.880+0000 7f28a6a00700 1 -- 172.21.15.79:0/3932700523 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f289c0d3170 con 0x7f28b0102fd0 2024-06-03T02:34:56.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.881+0000 7f28bf660b80 1 Processor -- start 2024-06-03T02:34:56.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.882+0000 7f28ad400700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f28940a2ca0 0x7f28940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.882+0000 7f28bf660b80 1 -- start start 2024-06-03T02:34:56.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.882+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.882+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a6680 0x55c48d5c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.882+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5c2540 0x55c48d5c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.883+0000 7f28ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.883+0000 7f28ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5a6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49824/0 (socket says 172.21.15.79:49824) 2024-06-03T02:34:56.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.883+0000 7f28ad400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a6680 0x55c48d5c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.883+0000 7f28ad400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a6680 0x55c48d5c2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:57628/0 (socket says 172.21.15.79:57628) 2024-06-03T02:34:56.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.883+0000 7f28ade00700 1 -- 172.21.15.79:0/1687372595 learned_addr learned my addr 172.21.15.79:0/1687372595 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:56.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.883+0000 7f28bf660b80 1 -- 172.21.15.79:0/1687372595 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c48d4b5a10 con 0x55c48d5c2540 2024-06-03T02:34:56.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.883+0000 7f28bf660b80 1 -- 172.21.15.79:0/1687372595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c48d4f8b00 con 0x55c48d5a5d60 2024-06-03T02:34:56.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.883+0000 7f28bf660b80 1 -- 172.21.15.79:0/1687372595 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c48d4b38f0 con 0x55c48d5a6680 2024-06-03T02:34:56.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.884+0000 7f28ae800700 1 --2- 172.21.15.79:0/1687372595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5c2540 0x55c48d5c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.884+0000 7f28ad400700 1 -- 172.21.15.79:0/1687372595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 msgr2=0x55c48d5a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:56.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.884+0000 7f28ad400700 1 --2- 172.21.15.79:0/1687372595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.884+0000 7f28ad400700 1 -- 172.21.15.79:0/1687372595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5c2540 msgr2=0x55c48d5c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:56.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.884+0000 7f28ad400700 1 --2- 172.21.15.79:0/1687372595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5c2540 0x55c48d5c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.884+0000 7f28ad400700 1 -- 172.21.15.79:0/1687372595 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c48d4a0f90 con 0x55c48d5a6680 2024-06-03T02:34:56.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.885+0000 7f28ad400700 1 --2- 172.21.15.79:0/1687372595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a6680 0x55c48d5c2000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f28b0198180 tx=0x7f28a0014e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d36b4323006b030 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:56.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.885+0000 7f289b400700 1 -- 172.21.15.79:0/1687372595 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0022070 con 0x55c48d5a6680 2024-06-03T02:34:56.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.886+0000 7f289b400700 1 -- 172.21.15.79:0/1687372595 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a001d410 con 0x55c48d5a6680 2024-06-03T02:34:56.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.886+0000 7f289b400700 1 -- 172.21.15.79:0/1687372595 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a000e620 con 0x55c48d5a6680 2024-06-03T02:34:56.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.886+0000 7f28bf660b80 1 -- 172.21.15.79:0/1687372595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a6680 msgr2=0x55c48d5c2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:56.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.886+0000 7f28bf660b80 1 --2- 172.21.15.79:0/1687372595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a6680 0x55c48d5c2000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f28b0198180 tx=0x7f28a0014e30 comp rx=0 tx=0).stop 2024-06-03T02:34:56.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.888+0000 7f28bf660b80 1 -- 172.21.15.79:0/1687372595 shutdown_connections 2024-06-03T02:34:56.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.889+0000 7f28bf660b80 1 --2- 172.21.15.79:0/1687372595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5c2540 0x55c48d5c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.889+0000 7f28bf660b80 1 --2- 172.21.15.79:0/1687372595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.889+0000 7f28bf660b80 1 --2- 172.21.15.79:0/1687372595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a6680 0x55c48d5c2000 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.889+0000 7f28bf660b80 1 -- 172.21.15.79:0/1687372595 >> 172.21.15.79:0/1687372595 conn(0x55c48d4d3170 msgr2=0x55c48d5a2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:56.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.890+0000 7f28bf660b80 1 -- 172.21.15.79:0/1687372595 shutdown_connections 2024-06-03T02:34:56.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.890+0000 7f28bf660b80 1 -- 172.21.15.79:0/1687372595 wait complete. 2024-06-03T02:34:56.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.890+0000 7f28ad400700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f28940a2ca0 0x7f28940a5160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f28a0011fd0 tx=0x7f28a0014e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:56.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.891+0000 7f28bf660b80 1 -- start start 2024-06-03T02:34:56.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.891+0000 7f28bf660b80 1 Processor -- start 2024-06-03T02:34:56.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.895+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.895+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5befc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.895+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5bf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.896+0000 7f28ade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.896+0000 7f28ade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5c1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:57640/0 (socket says 172.21.15.79:57640) 2024-06-03T02:34:56.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.898+0000 7f28ad400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5befc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.898+0000 7f28ae800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5bf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.898+0000 7f28ae800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5bf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49828/0 (socket says 172.21.15.79:49828) 2024-06-03T02:34:56.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.898+0000 7f28ad400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5befc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41756/0 (socket says 172.21.15.79:41756) 2024-06-03T02:34:56.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28ade00700 1 -- 172.21.15.79:0/2091572804 learned_addr learned my addr 172.21.15.79:0/2091572804 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:56.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c48d494f60 con 0x55c48d5a6680 2024-06-03T02:34:56.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c48d4f4ac0 con 0x55c48d5c2540 2024-06-03T02:34:56.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c48d5c4fd0 con 0x55c48d5a5d60 2024-06-03T02:34:56.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28bf660b80 10 client.0 _ll_register_callbacks cb 0x55c48d48e380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:56.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28ae800700 1 -- 172.21.15.79:0/2091572804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 msgr2=0x55c48d5c1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:56.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28ae800700 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5c1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28ae800700 1 -- 172.21.15.79:0/2091572804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 msgr2=0x55c48d5befc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:56.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28ae800700 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5befc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.899+0000 7f28ae800700 1 -- 172.21.15.79:0/2091572804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c48d4478d0 con 0x55c48d5c2540 2024-06-03T02:34:56.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.900+0000 7f28ad400700 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5befc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:56.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.901+0000 7f28ade00700 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:56.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.901+0000 7f288fe00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:56.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.901+0000 7f28ae800700 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5bf500 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f28a8008e60 tx=0x7f28a80114a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=317aa5c8104d910 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:56.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.901+0000 7f288fe00700 1 -- 172.21.15.79:0/2091572804 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a801a030 con 0x55c48d5c2540 2024-06-03T02:34:56.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.903+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28a001d410 con 0x55c48d5c2540 2024-06-03T02:34:56.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.903+0000 7f288fe00700 1 -- 172.21.15.79:0/2091572804 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f28a801b070 con 0x55c48d5c2540 2024-06-03T02:34:56.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.903+0000 7f288fe00700 1 -- 172.21.15.79:0/2091572804 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f28a801a030 con 0x55c48d5c2540 2024-06-03T02:34:56.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.904+0000 7f28bf660b80 10 client.26981 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:56.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.904+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c48d5cf7a0 con 0x55c48d5c2540 2024-06-03T02:34:56.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.904+0000 7f288fe00700 1 -- 172.21.15.79:0/2091572804 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a001d410 con 0x55c48d5c2540 2024-06-03T02:34:56.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.906+0000 7f288fe00700 1 -- 172.21.15.79:0/2091572804 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f28a8003a00 con 0x55c48d5c2540 2024-06-03T02:34:56.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.906+0000 7f28bf660b80 10 client.26981 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:56.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.906+0000 7f28bf660b80 10 client.26981 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:34:56.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.906+0000 7f28bf660b80 20 client.26981 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:56.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.906+0000 7f28bf660b80 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c48d5d1d30 0x55c48d5d4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.906+0000 7f28bf660b80 4 client.26981 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:34:56.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.906+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c48d506000 con 0x55c48d5d1d30 2024-06-03T02:34:56.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.906+0000 7f28ade00700 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c48d5d1d30 0x55c48d5d4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.907+0000 7f28ade00700 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c48d5d1d30 0x55c48d5d4120 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2575c75c99109ff6 server_cookie=c44a18e2a82ab48 in_seq=0 out_seq=0 2024-06-03T02:34:56.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.907+0000 7f288fe00700 10 client.26981 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:34:56.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.908+0000 7f288fe00700 1 -- 172.21.15.79:0/2091572804 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c48d506000 con 0x55c48d5d1d30 2024-06-03T02:34:56.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.908+0000 7f288fe00700 10 client.26981 handle_command_reply: tid=0 2024-06-03T02:34:56.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.909+0000 7f28bf660b80 1 client.26981 shutdown 2024-06-03T02:34:56.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.909+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 msgr2=0x55c48d5bf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:56.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.909+0000 7f28bf660b80 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5bf500 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f28a8008e60 tx=0x7f28a80114a0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.910+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 shutdown_connections 2024-06-03T02:34:56.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.910+0000 7f28bf660b80 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5bf500 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.910+0000 7f28bf660b80 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.910+0000 7f28bf660b80 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c48d5d1d30 0x55c48d5d4120 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.910+0000 7f28bf660b80 1 --2- 172.21.15.79:0/2091572804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5befc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.910+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 >> 172.21.15.79:0/2091572804 conn(0x55c48d4d3170 msgr2=0x55c48d5a1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:56.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.910+0000 7f288fe00700 0 client.26981 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:34:56.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.911+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 shutdown_connections 2024-06-03T02:34:56.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.911+0000 7f28bf660b80 1 -- 172.21.15.79:0/2091572804 wait complete. 2024-06-03T02:34:56.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.911+0000 7f28bf660b80 20 client.26981 trim_cache size 0 max 16384 2024-06-03T02:34:56.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.945+0000 7f28bf660b80 1 Processor -- start 2024-06-03T02:34:56.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.946+0000 7f28bf660b80 1 -- start start 2024-06-03T02:34:56.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.947+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5ba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.947+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5c1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.947+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.947+0000 7f28bf660b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f28b0074170 con 0x55c48d5a6680 2024-06-03T02:34:56.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.947+0000 7f28bf660b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28b01b69f0 con 0x55c48d5c2540 2024-06-03T02:34:56.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.947+0000 7f28bf660b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28b01b5580 con 0x55c48d5a5d60 2024-06-03T02:34:56.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.948+0000 7f28ade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5ba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.948+0000 7f28ade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5ba6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:57652/0 (socket says 172.21.15.79:57652) 2024-06-03T02:34:56.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.948+0000 7f28ade00700 1 -- 172.21.15.79:0/320255401 learned_addr learned my addr 172.21.15.79:0/320255401 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:56.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.948+0000 7f28ae800700 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.949+0000 7f28ade00700 1 -- 172.21.15.79:0/320255401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 msgr2=0x55c48d5d99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:56.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.949+0000 7f28ade00700 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5d99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.949+0000 7f28ade00700 1 -- 172.21.15.79:0/320255401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 msgr2=0x55c48d5c1800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:34:56.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.950+0000 7f28ad400700 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.950+0000 7f28ade00700 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5c1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.950+0000 7f28ade00700 1 -- 172.21.15.79:0/320255401 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c48d5a7bb0 con 0x55c48d5a5d60 2024-06-03T02:34:56.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.950+0000 7f28ad400700 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:34:56.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.951+0000 7f28ade00700 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5ba6b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f289c09bdf0 tx=0x7f289c0d2000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27efc1da3b814347 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:56.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.952+0000 7f289aa00700 1 -- 172.21.15.79:0/320255401 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c0d4060 con 0x55c48d5a5d60 2024-06-03T02:34:56.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.952+0000 7f289aa00700 1 -- 172.21.15.79:0/320255401 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c48d506000 con 0x55c48d5a5d60 2024-06-03T02:34:56.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.952+0000 7f289aa00700 1 -- 172.21.15.79:0/320255401 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c09a160 con 0x55c48d5a5d60 2024-06-03T02:34:56.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.953+0000 7f28bf660b80 1 -- 172.21.15.79:0/320255401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 msgr2=0x55c48d5ba6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:56.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.953+0000 7f28bf660b80 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5ba6b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f289c09bdf0 tx=0x7f289c0d2000 comp rx=0 tx=0).stop 2024-06-03T02:34:56.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.954+0000 7f28bf660b80 1 -- 172.21.15.79:0/320255401 shutdown_connections 2024-06-03T02:34:56.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.954+0000 7f28bf660b80 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5c2540 0x55c48d5d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.954+0000 7f28bf660b80 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5a5d60 0x55c48d5ba6b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.954+0000 7f28bf660b80 1 --2- 172.21.15.79:0/320255401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.954+0000 7f28bf660b80 1 -- 172.21.15.79:0/320255401 >> 172.21.15.79:0/320255401 conn(0x55c48d4d3170 msgr2=0x55c48d5b9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:56.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.954+0000 7f28bf660b80 1 -- 172.21.15.79:0/320255401 shutdown_connections 2024-06-03T02:34:56.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.954+0000 7f28bf660b80 1 -- 172.21.15.79:0/320255401 wait complete. 2024-06-03T02:34:56.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.955+0000 7f28bf660b80 1 -- start start 2024-06-03T02:34:56.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.955+0000 7f28bf660b80 1 Processor -- start 2024-06-03T02:34:56.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.957+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.957+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.957+0000 7f28bf660b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5c2540 0x55c48d5c6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.957+0000 7f28bf660b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f28b01b6320 con 0x55c48d5a6680 2024-06-03T02:34:56.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.957+0000 7f28bf660b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28b01b5c50 con 0x55c48d5a5d60 2024-06-03T02:34:56.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.957+0000 7f28bf660b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28b00720b0 con 0x55c48d5c2540 2024-06-03T02:34:56.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.957+0000 7f28bf660b80 10 client.0 _ll_register_callbacks cb 0x55c48d48e380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:34:56.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.957+0000 7f28ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.958+0000 7f28ade00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49834/0 (socket says 172.21.15.79:49834) 2024-06-03T02:34:56.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.958+0000 7f28ade00700 1 -- 172.21.15.79:0/4209530089 learned_addr learned my addr 172.21.15.79:0/4209530089 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:34:56.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.958+0000 7f28ae800700 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5c2540 0x55c48d5c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.958+0000 7f28ad400700 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.959+0000 7f28ade00700 1 -- 172.21.15.79:0/4209530089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5c2540 msgr2=0x55c48d5c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:56.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.959+0000 7f28ade00700 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5c2540 0x55c48d5c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.959+0000 7f28ade00700 1 -- 172.21.15.79:0/4209530089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 msgr2=0x55c48d5ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:34:56.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.959+0000 7f28ade00700 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.959+0000 7f28ade00700 1 -- 172.21.15.79:0/4209530089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c48d5c6f50 con 0x55c48d5a5d60 2024-06-03T02:34:56.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.959+0000 7f28ade00700 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5ca2f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f289c001eb0 tx=0x7f289c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=264486fea0e3eb9e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:34:56.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.959+0000 7f288fe00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:34:56.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.959+0000 7f288fe00700 1 -- 172.21.15.79:0/4209530089 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c0b8070 con 0x55c48d5a5d60 2024-06-03T02:34:56.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.960+0000 7f288fe00700 1 -- 172.21.15.79:0/4209530089 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f289c00f980 con 0x55c48d5a5d60 2024-06-03T02:34:56.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.960+0000 7f28bf660b80 1 -- 172.21.15.79:0/4209530089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c48d506000 con 0x55c48d5a5d60 2024-06-03T02:34:56.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.960+0000 7f288fe00700 1 -- 172.21.15.79:0/4209530089 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f289c0a0210 con 0x55c48d5a5d60 2024-06-03T02:34:56.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.960+0000 7f28bf660b80 10 client.26987 fetch_fsmap learned FSMap version 30 2024-06-03T02:34:56.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.960+0000 7f28bf660b80 1 -- 172.21.15.79:0/4209530089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c48d63d390 con 0x55c48d5a5d60 2024-06-03T02:34:56.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.960+0000 7f288fe00700 1 -- 172.21.15.79:0/4209530089 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c48d506000 con 0x55c48d5a5d60 2024-06-03T02:34:56.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.962+0000 7f288fe00700 1 -- 172.21.15.79:0/4209530089 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f289c0b1040 con 0x55c48d5a5d60 2024-06-03T02:34:56.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.962+0000 7f28bf660b80 10 client.26987 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:34:56.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.962+0000 7f28bf660b80 10 client.26987 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-06-03T02:34:56.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.962+0000 7f28bf660b80 20 client.26987 populate_metadata read hostname 'smithi079' 2024-06-03T02:34:56.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.962+0000 7f28bf660b80 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c48d63daa0 0x55c48d63fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:34:56.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.962+0000 7f28bf660b80 4 client.26987 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:34:56.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.962+0000 7f28bf660b80 1 -- 172.21.15.79:0/4209530089 --> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c48d500580 con 0x55c48d63daa0 2024-06-03T02:34:56.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.962+0000 7f28ad400700 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c48d63daa0 0x55c48d63fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:34:56.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.963+0000 7f288fe00700 10 client.26987 ms_handle_connect on v2:172.21.15.135:6838/414076797 2024-06-03T02:34:56.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.965+0000 7f28ad400700 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c48d63daa0 0x55c48d63fec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f382b7a5c7c6d3e7 server_cookie=78db43a1a2ff370b in_seq=0 out_seq=0 2024-06-03T02:34:56.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.965+0000 7f288fe00700 1 -- 172.21.15.79:0/4209530089 <== mds.3 v2:172.21.15.135:6838/414076797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c48d500580 con 0x55c48d63daa0 2024-06-03T02:34:56.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.965+0000 7f288fe00700 10 client.26987 handle_command_reply: tid=0 2024-06-03T02:34:56.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.966+0000 7f28bf660b80 1 client.26987 shutdown 2024-06-03T02:34:56.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.966+0000 7f28bf660b80 1 -- 172.21.15.79:0/4209530089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 msgr2=0x55c48d5ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:56.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.966+0000 7f28bf660b80 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5ca2f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f289c001eb0 tx=0x7f289c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.970+0000 7f28bf660b80 1 -- 172.21.15.79:0/4209530089 shutdown_connections 2024-06-03T02:34:56.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.970+0000 7f28bf660b80 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48d5c2540 0x55c48d5c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.970+0000 7f28bf660b80 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c48d5a5d60 0x55c48d5ca2f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.970+0000 7f28bf660b80 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.135:6838/414076797,v1:172.21.15.135:6839/414076797] conn(0x55c48d63daa0 0x55c48d63fec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.970+0000 7f28bf660b80 1 --2- 172.21.15.79:0/4209530089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c48d5a6680 0x55c48d5ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.970+0000 7f28bf660b80 1 -- 172.21.15.79:0/4209530089 >> 172.21.15.79:0/4209530089 conn(0x55c48d4d3170 msgr2=0x55c48d5a7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:56.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.970+0000 7f288fe00700 0 client.26987 ms_handle_reset on v2:172.21.15.135:6838/414076797 2024-06-03T02:34:56.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.970+0000 7f28bf660b80 1 -- 172.21.15.79:0/4209530089 shutdown_connections 2024-06-03T02:34:56.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.970+0000 7f28bf660b80 1 -- 172.21.15.79:0/4209530089 wait complete. 2024-06-03T02:34:56.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.971+0000 7f28bf660b80 20 client.26987 trim_cache size 0 max 16384 2024-06-03T02:34:56.972 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:34:56.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.973+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f28940a2ca0 msgr2=0x7f28940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:56.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.973+0000 7f28af200700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f28940a2ca0 0x7f28940a5160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f28a0011fd0 tx=0x7f28a0014e30 comp rx=0 tx=0).stop 2024-06-03T02:34:56.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.973+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102fd0 msgr2=0x7f28b019fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:34:56.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.973+0000 7f28af200700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102fd0 0x7f28b019fa00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f289c0059d0 tx=0x7f289c0057c0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.973+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 shutdown_connections 2024-06-03T02:34:56.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.973+0000 7f28af200700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f28940a2ca0 0x7f28940a5160 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.973+0000 7f28af200700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28b010eb60 0x7f28b010f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.973+0000 7f28af200700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b0102fd0 0x7f28b019fa00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.973+0000 7f28af200700 1 --2- 172.21.15.79:0/3932700523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b0107700 0x7f28b010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:34:56.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.974+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 >> 172.21.15.79:0/3932700523 conn(0x7f28b00fdf10 msgr2=0x7f28b0100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:34:56.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.974+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 shutdown_connections 2024-06-03T02:34:56.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:34:56.975+0000 7f28af200700 1 -- 172.21.15.79:0/3932700523 wait complete. 2024-06-03T02:34:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 890.0 s, 389.8 tps, lat 250.185 ms stddev 316.702 2024-06-03T02:34:58.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:34:58 smithi123 ceph-mon[36640]: pgmap v963: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.0 MiB/s wr, 447 op/s 2024-06-03T02:34:58.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:34:58 smithi079 ceph-mon[31819]: pgmap v963: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.0 MiB/s wr, 447 op/s 2024-06-03T02:34:58.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T02:34:58.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:34:58 smithi135 ceph-mon[36653]: pgmap v963: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.0 MiB/s wr, 447 op/s 2024-06-03T02:34:59.288 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:35:00.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:00 smithi123 ceph-mon[36640]: pgmap v964: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-06-03T02:35:00.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:00 smithi079 ceph-mon[31819]: pgmap v964: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-06-03T02:35:00.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:00 smithi135 ceph-mon[36653]: pgmap v964: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-06-03T02:35:02.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:02 smithi079 ceph-mon[31819]: pgmap v965: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 463 op/s 2024-06-03T02:35:02.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:02 smithi135 ceph-mon[36653]: pgmap v965: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 463 op/s 2024-06-03T02:35:02.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:02 smithi123 ceph-mon[36640]: pgmap v965: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 463 op/s 2024-06-03T02:35:03.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 895.0 s, 442.8 tps, lat 234.211 ms stddev 270.174 2024-06-03T02:35:03.946 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.945+0000 7f9e7ba00700 1 -- 172.21.15.79:0/4174399710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e741155e0 msgr2=0x7f9e741179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:03.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.945+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/4174399710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e741155e0 0x7f9e741179d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00b0a0 tx=0x7f9e6c00b3b0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.945+0000 7f9e7ba00700 1 -- 172.21.15.79:0/4174399710 shutdown_connections 2024-06-03T02:35:03.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.945+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/4174399710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e741155e0 0x7f9e741179d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.945+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/4174399710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e74109490 0x7f9e74109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.945+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/4174399710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e74109db0 0x7f9e741150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.945+0000 7f9e7ba00700 1 -- 172.21.15.79:0/4174399710 >> 172.21.15.79:0/4174399710 conn(0x7f9e7406d600 msgr2=0x7f9e7406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:03.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.946+0000 7f9e7ba00700 1 -- 172.21.15.79:0/4174399710 shutdown_connections 2024-06-03T02:35:03.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.946+0000 7f9e7ba00700 1 -- 172.21.15.79:0/4174399710 wait complete. 2024-06-03T02:35:03.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.947+0000 7f9e7ba00700 1 Processor -- start 2024-06-03T02:35:03.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.950+0000 7f9e7ba00700 1 -- start start 2024-06-03T02:35:03.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.950+0000 7f9e7ba00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e74109490 0x7f9e741a9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.951+0000 7f9e7ba00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e74109db0 0x7f9e741a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.951+0000 7f9e7ba00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e741155e0 0x7f9e741a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.951+0000 7f9e7ba00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7411b0d0 con 0x7f9e74109db0 2024-06-03T02:35:03.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.951+0000 7f9e7ba00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7411af50 con 0x7f9e74109490 2024-06-03T02:35:03.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.951+0000 7f9e7ba00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7411b250 con 0x7f9e741155e0 2024-06-03T02:35:03.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.951+0000 7f9e79a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e741155e0 0x7f9e741a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:03.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.951+0000 7f9e79a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e741155e0 0x7f9e741a35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:57670/0 (socket says 172.21.15.79:57670) 2024-06-03T02:35:03.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.951+0000 7f9e79a00700 1 -- 172.21.15.79:0/3770375938 learned_addr learned my addr 172.21.15.79:0/3770375938 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:03.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.952+0000 7f9e73e00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e74109db0 0x7f9e741a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:03.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.952+0000 7f9e79000700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e74109490 0x7f9e741a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:03.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.952+0000 7f9e73e00700 1 -- 172.21.15.79:0/3770375938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e741155e0 msgr2=0x7f9e741a35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:03.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.952+0000 7f9e73e00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e741155e0 0x7f9e741a35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.952+0000 7f9e73e00700 1 -- 172.21.15.79:0/3770375938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e74109490 msgr2=0x7f9e741a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:03.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.952+0000 7f9e73e00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e74109490 0x7f9e741a9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.952+0000 7f9e73e00700 1 -- 172.21.15.79:0/3770375938 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e741a3d30 con 0x7f9e74109db0 2024-06-03T02:35:03.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.953+0000 7f9e73e00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e74109db0 0x7f9e741a9950 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006fd0 tx=0x7f9e6000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:03.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.953+0000 7f9e71600700 1 -- 172.21.15.79:0/3770375938 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60015020 con 0x7f9e74109db0 2024-06-03T02:35:03.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.953+0000 7f9e71600700 1 -- 172.21.15.79:0/3770375938 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e60011d10 con 0x7f9e74109db0 2024-06-03T02:35:03.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.953+0000 7f9e71600700 1 -- 172.21.15.79:0/3770375938 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60010e10 con 0x7f9e74109db0 2024-06-03T02:35:03.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.954+0000 7f9e7ba00700 1 -- 172.21.15.79:0/3770375938 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e741a4020 con 0x7f9e74109db0 2024-06-03T02:35:03.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.954+0000 7f9e7ba00700 1 -- 172.21.15.79:0/3770375938 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e741b02e0 con 0x7f9e74109db0 2024-06-03T02:35:03.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.962+0000 7f9e71600700 1 -- 172.21.15.79:0/3770375938 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9e60018020 con 0x7f9e74109db0 2024-06-03T02:35:03.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.964+0000 7f9e71600700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9e5c0a2ca0 0x7f9e5c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.965+0000 7f9e71600700 1 -- 172.21.15.79:0/3770375938 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f9e600ceb00 con 0x7f9e74109db0 2024-06-03T02:35:03.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.965+0000 7f9e86308b80 1 Processor -- start 2024-06-03T02:35:03.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.965+0000 7f9e79000700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9e5c0a2ca0 0x7f9e5c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:03.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.966+0000 7f9e86308b80 1 -- start start 2024-06-03T02:35:03.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.966+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c353b90 0x55630c353f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.966+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c34dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.966+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c34e240 0x55630c34e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.966+0000 7f9e86308b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55630c25ca10 con 0x55630c353b90 2024-06-03T02:35:03.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.966+0000 7f9e86308b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55630c29fb00 con 0x55630c34e240 2024-06-03T02:35:03.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.966+0000 7f9e86308b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55630c25a8f0 con 0x55630c3544b0 2024-06-03T02:35:03.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.967+0000 7f9e79a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c34e240 0x55630c34e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:03.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.967+0000 7f9e79a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c34e240 0x55630c34e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49858/0 (socket says 172.21.15.79:49858) 2024-06-03T02:35:03.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.967+0000 7f9e79a00700 1 -- 172.21.15.79:0/2554897616 learned_addr learned my addr 172.21.15.79:0/2554897616 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:03.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e73e00700 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c34dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:03.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e79000700 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c353b90 0x55630c353f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:03.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e79a00700 1 -- 172.21.15.79:0/2554897616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 msgr2=0x55630c34dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:03.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e79a00700 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c34dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e79a00700 1 -- 172.21.15.79:0/2554897616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c353b90 msgr2=0x55630c353f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:03.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e79a00700 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c353b90 0x55630c353f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e79a00700 1 -- 172.21.15.79:0/2554897616 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55630c247f90 con 0x55630c34e240 2024-06-03T02:35:03.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e73e00700 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c34dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:35:03.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e79000700 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c353b90 0x55630c353f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:35:03.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e79000700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9e5c0a2ca0 0x7f9e5c0a5160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009a60 tx=0x7f9e68009e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:03.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.968+0000 7f9e79a00700 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c34e240 0x55630c34e680 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00bf00 tx=0x7f9e6c0090d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ed1f3fc588c2459 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:03.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.970+0000 7f9e66000700 1 -- 172.21.15.79:0/2554897616 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c004a50 con 0x55630c34e240 2024-06-03T02:35:03.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.970+0000 7f9e66000700 1 -- 172.21.15.79:0/2554897616 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c004bf0 con 0x55630c34e240 2024-06-03T02:35:03.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.970+0000 7f9e66000700 1 -- 172.21.15.79:0/2554897616 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0053b0 con 0x55630c34e240 2024-06-03T02:35:03.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.972+0000 7f9e86308b80 1 -- 172.21.15.79:0/2554897616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c34e240 msgr2=0x55630c34e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:03.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.972+0000 7f9e86308b80 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c34e240 0x55630c34e680 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00bf00 tx=0x7f9e6c0090d0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.972+0000 7f9e86308b80 1 -- 172.21.15.79:0/2554897616 shutdown_connections 2024-06-03T02:35:03.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.972+0000 7f9e86308b80 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c34e240 0x55630c34e680 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.972+0000 7f9e86308b80 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c353b90 0x55630c353f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.972+0000 7f9e86308b80 1 --2- 172.21.15.79:0/2554897616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c34dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.972+0000 7f9e86308b80 1 -- 172.21.15.79:0/2554897616 >> 172.21.15.79:0/2554897616 conn(0x55630c27a170 msgr2=0x55630c348ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:03.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.972+0000 7f9e86308b80 1 -- 172.21.15.79:0/2554897616 shutdown_connections 2024-06-03T02:35:03.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.972+0000 7f9e86308b80 1 -- 172.21.15.79:0/2554897616 wait complete. 2024-06-03T02:35:03.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.973+0000 7f9e86308b80 1 -- start start 2024-06-03T02:35:03.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.973+0000 7f9e86308b80 1 Processor -- start 2024-06-03T02:35:03.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.974+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c34e240 0x55630c35a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.974+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c357540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.974+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c357a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.974+0000 7f9e86308b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55630c23bf60 con 0x55630c34e240 2024-06-03T02:35:03.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.974+0000 7f9e86308b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55630c29bac0 con 0x55630c353b90 2024-06-03T02:35:03.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.974+0000 7f9e86308b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55630c35ae30 con 0x55630c3544b0 2024-06-03T02:35:03.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.974+0000 7f9e86308b80 10 client.0 _ll_register_callbacks cb 0x55630c235380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:35:03.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.977+0000 7f9e73e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c357540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:03.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.978+0000 7f9e73e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c357540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49860/0 (socket says 172.21.15.79:49860) 2024-06-03T02:35:03.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.978+0000 7f9e73e00700 1 -- 172.21.15.79:0/3470737740 learned_addr learned my addr 172.21.15.79:0/3470737740 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:03.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.978+0000 7f9e73e00700 1 -- 172.21.15.79:0/3470737740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 msgr2=0x55630c357a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T02:35:03.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.978+0000 7f9e73e00700 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c357a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.978+0000 7f9e73e00700 1 -- 172.21.15.79:0/3470737740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c34e240 msgr2=0x55630c35a240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:35:03.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.978+0000 7f9e73e00700 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c34e240 0x55630c35a240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.978+0000 7f9e73e00700 1 -- 172.21.15.79:0/3470737740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55630c1ee8d0 con 0x55630c353b90 2024-06-03T02:35:03.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.979+0000 7f9e5aa00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:35:03.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.979+0000 7f9e73e00700 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c357540 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9e60097980 tx=0x7f9e6009aa40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbed545b44c802e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:03.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.979+0000 7f9e5aa00700 1 -- 172.21.15.79:0/3470737740 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6009f360 con 0x55630c353b90 2024-06-03T02:35:03.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.979+0000 7f9e5aa00700 1 -- 172.21.15.79:0/3470737740 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f9e600a3790 con 0x55630c353b90 2024-06-03T02:35:03.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.980+0000 7f9e86308b80 1 -- 172.21.15.79:0/3470737740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6c004bf0 con 0x55630c353b90 2024-06-03T02:35:03.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.980+0000 7f9e5aa00700 1 -- 172.21.15.79:0/3470737740 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e600a3c20 con 0x55630c353b90 2024-06-03T02:35:03.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.980+0000 7f9e86308b80 10 client.26996 fetch_fsmap learned FSMap version 30 2024-06-03T02:35:03.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.981+0000 7f9e86308b80 1 -- 172.21.15.79:0/3470737740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55630c365580 con 0x55630c353b90 2024-06-03T02:35:03.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.981+0000 7f9e5aa00700 1 -- 172.21.15.79:0/3470737740 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c004bf0 con 0x55630c353b90 2024-06-03T02:35:03.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.981+0000 7f9e5aa00700 1 -- 172.21.15.79:0/3470737740 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9e60011830 con 0x55630c353b90 2024-06-03T02:35:03.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.983+0000 7f9e86308b80 10 client.26996 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:35:03.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.983+0000 7f9e86308b80 10 client.26996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:35:03.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.983+0000 7f9e86308b80 20 client.26996 populate_metadata read hostname 'smithi079' 2024-06-03T02:35:03.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.983+0000 7f9e86308b80 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55630c367c60 0x55630c36a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:03.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.983+0000 7f9e86308b80 4 client.26996 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:35:03.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.983+0000 7f9e86308b80 1 -- 172.21.15.79:0/3470737740 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55630c2ad000 con 0x55630c367c60 2024-06-03T02:35:03.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.984+0000 7f9e79000700 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55630c367c60 0x55630c36a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:03.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.986+0000 7f9e79000700 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55630c367c60 0x55630c36a050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e2d99d80eeaafe1 server_cookie=79b154f26bbc6aab in_seq=0 out_seq=0 2024-06-03T02:35:03.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.988+0000 7f9e5aa00700 10 client.26996 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:35:03.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.988+0000 7f9e5aa00700 1 -- 172.21.15.79:0/3470737740 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55630c2ad000 con 0x55630c367c60 2024-06-03T02:35:03.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.988+0000 7f9e5aa00700 10 client.26996 handle_command_reply: tid=0 2024-06-03T02:35:03.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.988+0000 7f9e86308b80 1 client.26996 shutdown 2024-06-03T02:35:03.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.989+0000 7f9e86308b80 1 -- 172.21.15.79:0/3470737740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 msgr2=0x55630c357540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:03.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.989+0000 7f9e86308b80 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c357540 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9e60097980 tx=0x7f9e6009aa40 comp rx=0 tx=0).stop 2024-06-03T02:35:03.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.990+0000 7f9e86308b80 1 -- 172.21.15.79:0/3470737740 shutdown_connections 2024-06-03T02:35:03.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.990+0000 7f9e86308b80 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c357a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.990+0000 7f9e86308b80 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c34e240 0x55630c35a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.990+0000 7f9e86308b80 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55630c367c60 0x55630c36a050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.990+0000 7f9e86308b80 1 --2- 172.21.15.79:0/3470737740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c357540 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:03.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.990+0000 7f9e5aa00700 0 client.26996 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:35:03.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.990+0000 7f9e86308b80 1 -- 172.21.15.79:0/3470737740 >> 172.21.15.79:0/3470737740 conn(0x55630c27a170 msgr2=0x55630c342790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:03.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.992+0000 7f9e86308b80 1 -- 172.21.15.79:0/3470737740 shutdown_connections 2024-06-03T02:35:03.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.992+0000 7f9e86308b80 1 -- 172.21.15.79:0/3470737740 wait complete. 2024-06-03T02:35:03.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:03.992+0000 7f9e86308b80 20 client.26996 trim_cache size 0 max 16384 2024-06-03T02:35:04.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.032+0000 7f9e86308b80 1 Processor -- start 2024-06-03T02:35:04.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.032+0000 7f9e86308b80 1 -- start start 2024-06-03T02:35:04.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.033+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c34e240 0x55630c34dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:04.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.033+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c3491a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:04.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.033+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c3544b0 0x55630c343130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:04.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.033+0000 7f9e86308b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741b8de0 con 0x55630c3544b0 2024-06-03T02:35:04.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.033+0000 7f9e86308b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741ba920 con 0x55630c353b90 2024-06-03T02:35:04.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.033+0000 7f9e86308b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741b94b0 con 0x55630c34e240 2024-06-03T02:35:04.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e79000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c34e240 0x55630c34dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:04.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e79000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c34e240 0x55630c34dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:57688/0 (socket says 172.21.15.79:57688) 2024-06-03T02:35:04.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e79000700 1 -- 172.21.15.79:0/4290509578 learned_addr learned my addr 172.21.15.79:0/4290509578 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:04.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e73e00700 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c3491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:04.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e79a00700 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c3544b0 0x55630c343130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:04.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e79000700 1 -- 172.21.15.79:0/4290509578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 msgr2=0x55630c3491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:04.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e79000700 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c3491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e79000700 1 -- 172.21.15.79:0/4290509578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c3544b0 msgr2=0x55630c343130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:04.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e79000700 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c3544b0 0x55630c343130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e79000700 1 -- 172.21.15.79:0/4290509578 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55630c36a6d0 con 0x55630c34e240 2024-06-03T02:35:04.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.034+0000 7f9e73e00700 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c3491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:35:04.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.035+0000 7f9e79000700 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c34e240 0x55630c34dd00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55630c36a900 tx=0x7f9e68017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b66ffe030b5dbe31 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:04.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.035+0000 7f9e65600700 1 -- 172.21.15.79:0/4290509578 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6802f070 con 0x55630c34e240 2024-06-03T02:35:04.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.035+0000 7f9e65600700 1 -- 172.21.15.79:0/4290509578 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6801a040 con 0x55630c34e240 2024-06-03T02:35:04.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.035+0000 7f9e65600700 1 -- 172.21.15.79:0/4290509578 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6802a9b0 con 0x55630c34e240 2024-06-03T02:35:04.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.036+0000 7f9e86308b80 1 -- 172.21.15.79:0/4290509578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c34e240 msgr2=0x55630c34dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:04.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.036+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c34e240 0x55630c34dd00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55630c36a900 tx=0x7f9e68017000 comp rx=0 tx=0).stop 2024-06-03T02:35:04.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.036+0000 7f9e86308b80 1 -- 172.21.15.79:0/4290509578 shutdown_connections 2024-06-03T02:35:04.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.036+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c3544b0 0x55630c343130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.036+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c34e240 0x55630c34dd00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.036+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4290509578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c3491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.036+0000 7f9e86308b80 1 -- 172.21.15.79:0/4290509578 >> 172.21.15.79:0/4290509578 conn(0x55630c27a170 msgr2=0x55630c348a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:04.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.037+0000 7f9e86308b80 1 -- 172.21.15.79:0/4290509578 shutdown_connections 2024-06-03T02:35:04.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.037+0000 7f9e86308b80 1 -- 172.21.15.79:0/4290509578 wait complete. 2024-06-03T02:35:04.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.037+0000 7f9e86308b80 1 -- start start 2024-06-03T02:35:04.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.037+0000 7f9e86308b80 1 Processor -- start 2024-06-03T02:35:04.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.038+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c360300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:04.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.039+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c360840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:04.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.039+0000 7f9e86308b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c36eb50 0x55630c36efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:04.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.039+0000 7f9e86308b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741ba250 con 0x55630c36eb50 2024-06-03T02:35:04.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.039+0000 7f9e86308b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741b9b80 con 0x55630c353b90 2024-06-03T02:35:04.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.039+0000 7f9e86308b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741b6d40 con 0x55630c3544b0 2024-06-03T02:35:04.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.039+0000 7f9e86308b80 10 client.0 _ll_register_callbacks cb 0x55630c235380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:35:04.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.039+0000 7f9e79a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c36eb50 0x55630c36efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:04.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.039+0000 7f9e79a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c36eb50 0x55630c36efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41830/0 (socket says 172.21.15.79:41830) 2024-06-03T02:35:04.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.039+0000 7f9e79a00700 1 -- 172.21.15.79:0/4243171912 learned_addr learned my addr 172.21.15.79:0/4243171912 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:04.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.040+0000 7f9e79000700 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c360300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:04.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.040+0000 7f9e73e00700 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c360840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:04.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.041+0000 7f9e79a00700 1 -- 172.21.15.79:0/4243171912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 msgr2=0x55630c360840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:04.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.041+0000 7f9e79a00700 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c360840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.041+0000 7f9e79a00700 1 -- 172.21.15.79:0/4243171912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 msgr2=0x55630c360300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:04.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.041+0000 7f9e79a00700 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c360300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.041+0000 7f9e79a00700 1 -- 172.21.15.79:0/4243171912 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55630c36f590 con 0x55630c36eb50 2024-06-03T02:35:04.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.041+0000 7f9e73e00700 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c360840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:35:04.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.041+0000 7f9e79000700 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c360300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:35:04.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.041+0000 7f9e79a00700 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c36eb50 0x55630c36efd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00bf00 tx=0x7f9e6c009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6bf364a8c17b77 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:04.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.042+0000 7f9e5aa00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-06-03T02:35:04.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.043+0000 7f9e5aa00700 1 -- 172.21.15.79:0/4243171912 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0074a0 con 0x55630c36eb50 2024-06-03T02:35:04.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.043+0000 7f9e5aa00700 1 -- 172.21.15.79:0/4243171912 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f9e6c007640 con 0x55630c36eb50 2024-06-03T02:35:04.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.043+0000 7f9e86308b80 1 -- 172.21.15.79:0/4243171912 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6801a040 con 0x55630c36eb50 2024-06-03T02:35:04.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.043+0000 7f9e5aa00700 1 -- 172.21.15.79:0/4243171912 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e6c039050 con 0x55630c36eb50 2024-06-03T02:35:04.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.044+0000 7f9e86308b80 10 client.16362 fetch_fsmap learned FSMap version 30 2024-06-03T02:35:04.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.044+0000 7f9e86308b80 1 -- 172.21.15.79:0/4243171912 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55630c3dae70 con 0x55630c36eb50 2024-06-03T02:35:04.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.044+0000 7f9e5aa00700 1 -- 172.21.15.79:0/4243171912 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6801a040 con 0x55630c36eb50 2024-06-03T02:35:04.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.044+0000 7f9e5aa00700 1 -- 172.21.15.79:0/4243171912 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9e6c011070 con 0x55630c36eb50 2024-06-03T02:35:04.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.046+0000 7f9e86308b80 10 client.16362 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:35:04.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.046+0000 7f9e86308b80 10 client.16362 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-03T02:35:04.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.046+0000 7f9e86308b80 20 client.16362 populate_metadata read hostname 'smithi079' 2024-06-03T02:35:04.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.046+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55630c3db0a0 0x55630c3dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:04.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.046+0000 7f9e86308b80 4 client.16362 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:35:04.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.046+0000 7f9e86308b80 1 -- 172.21.15.79:0/4243171912 --> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55630c2a7580 con 0x55630c3db0a0 2024-06-03T02:35:04.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.047+0000 7f9e79000700 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55630c3db0a0 0x55630c3dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:04.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.047+0000 7f9e79000700 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55630c3db0a0 0x55630c3dd490 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2475d0ed53f85434 server_cookie=837c3f3d1648d10c in_seq=0 out_seq=0 2024-06-03T02:35:04.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.047+0000 7f9e5aa00700 10 client.16362 ms_handle_connect on v2:172.21.15.123:6836/963974438 2024-06-03T02:35:04.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.048+0000 7f9e5aa00700 1 -- 172.21.15.79:0/4243171912 <== mds.2 v2:172.21.15.123:6836/963974438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55630c2a7580 con 0x55630c3db0a0 2024-06-03T02:35:04.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.048+0000 7f9e5aa00700 10 client.16362 handle_command_reply: tid=0 2024-06-03T02:35:04.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.048+0000 7f9e86308b80 1 client.16362 shutdown 2024-06-03T02:35:04.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.048+0000 7f9e86308b80 1 -- 172.21.15.79:0/4243171912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c36eb50 msgr2=0x55630c36efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:04.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.048+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c36eb50 0x55630c36efd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00bf00 tx=0x7f9e6c009450 comp rx=0 tx=0).stop 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e86308b80 1 -- 172.21.15.79:0/4243171912 shutdown_connections 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55630c36eb50 0x55630c36efd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.123:6836/963974438,v1:172.21.15.123:6837/963974438] conn(0x55630c3db0a0 0x55630c3dd490 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e5aa00700 0 client.16362 ms_handle_reset on v2:172.21.15.123:6836/963974438 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630c353b90 0x55630c360300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e86308b80 1 --2- 172.21.15.79:0/4243171912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55630c3544b0 0x55630c360840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e86308b80 1 -- 172.21.15.79:0/4243171912 >> 172.21.15.79:0/4243171912 conn(0x55630c27a170 msgr2=0x55630c354930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e86308b80 1 -- 172.21.15.79:0/4243171912 shutdown_connections 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e86308b80 1 -- 172.21.15.79:0/4243171912 wait complete. 2024-06-03T02:35:04.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.050+0000 7f9e86308b80 20 client.16362 trim_cache size 0 max 16384 2024-06-03T02:35:04.053 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:35:04.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 -- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9e5c0a2ca0 msgr2=0x7f9e5c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:04.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9e5c0a2ca0 0x7f9e5c0a5160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009a60 tx=0x7f9e68009e20 comp rx=0 tx=0).stop 2024-06-03T02:35:04.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 -- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e74109db0 msgr2=0x7f9e741a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:04.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e74109db0 0x7f9e741a9950 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006fd0 tx=0x7f9e6000b730 comp rx=0 tx=0).stop 2024-06-03T02:35:04.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 -- 172.21.15.79:0/3770375938 shutdown_connections 2024-06-03T02:35:04.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e741155e0 0x7f9e741a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e74109490 0x7f9e741a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f9e5c0a2ca0 0x7f9e5c0a5160 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009a60 tx=0x7f9e68009e20 comp rx=0 tx=0).stop 2024-06-03T02:35:04.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 --2- 172.21.15.79:0/3770375938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e74109db0 0x7f9e741a9950 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:04.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.053+0000 7f9e7ba00700 1 -- 172.21.15.79:0/3770375938 >> 172.21.15.79:0/3770375938 conn(0x7f9e7406d600 msgr2=0x7f9e74114640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:04.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.054+0000 7f9e7ba00700 1 -- 172.21.15.79:0/3770375938 shutdown_connections 2024-06-03T02:35:04.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:04.054+0000 7f9e7ba00700 1 -- 172.21.15.79:0/3770375938 wait complete. 2024-06-03T02:35:04.228 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:04 smithi079 ceph-mon[31819]: pgmap v966: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.9 MiB/s wr, 451 op/s 2024-06-03T02:35:04.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:04 smithi135 ceph-mon[36653]: pgmap v966: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.9 MiB/s wr, 451 op/s 2024-06-03T02:35:04.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:04 smithi123 ceph-mon[36640]: pgmap v966: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.9 MiB/s wr, 451 op/s 2024-06-03T02:35:05.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T02:35:05.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:05 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:35:05.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:05 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:35:05.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:05 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:35:05.904 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:35:06.475 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:06 smithi079 ceph-mon[31819]: pgmap v967: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.8 MiB/s wr, 462 op/s 2024-06-03T02:35:06.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:06 smithi135 ceph-mon[36653]: pgmap v967: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.8 MiB/s wr, 462 op/s 2024-06-03T02:35:06.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:06 smithi123 ceph-mon[36640]: pgmap v967: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.8 MiB/s wr, 462 op/s 2024-06-03T02:35:07.557 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:35:07.557 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:35:07.557 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:35:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:35:07.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:35:07.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:35:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:35:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:35:07.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:35:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:progress: 900.0 s, 489.4 tps, lat 202.178 ms stddev 252.281 2024-06-03T02:35:08.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:08 smithi079 ceph-mon[31819]: pgmap v968: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.3 MiB/s wr, 481 op/s 2024-06-03T02:35:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.714+0000 7f2e7d600700 1 -- 172.21.15.79:0/3826461171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e78108500 msgr2=0x7f2e781088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.714+0000 7f2e7d600700 1 --2- 172.21.15.79:0/3826461171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e78108500 0x7f2e781088e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009990 tx=0x7f2e60009ca0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.714+0000 7f2e7d600700 1 -- 172.21.15.79:0/3826461171 shutdown_connections 2024-06-03T02:35:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.714+0000 7f2e7d600700 1 --2- 172.21.15.79:0/3826461171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e7806ab30 0x7f2e7810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.714+0000 7f2e7d600700 1 --2- 172.21.15.79:0/3826461171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e78108500 0x7f2e781088e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.714+0000 7f2e7d600700 1 --2- 172.21.15.79:0/3826461171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e7806a170 0x7f2e7806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.714+0000 7f2e7d600700 1 -- 172.21.15.79:0/3826461171 >> 172.21.15.79:0/3826461171 conn(0x7f2e78078340 msgr2=0x7f2e78078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.715+0000 7f2e7d600700 1 -- 172.21.15.79:0/3826461171 shutdown_connections 2024-06-03T02:35:08.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.715+0000 7f2e7d600700 1 -- 172.21.15.79:0/3826461171 wait complete. 2024-06-03T02:35:08.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.716+0000 7f2e7d600700 1 Processor -- start 2024-06-03T02:35:08.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.717+0000 7f2e7d600700 1 -- start start 2024-06-03T02:35:08.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.717+0000 7f2e7d600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e7806a170 0x7f2e78101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.717+0000 7f2e7d600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e7806ab30 0x7f2e781018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.717+0000 7f2e7d600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e78108500 0x7f2e78101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.717+0000 7f2e7d600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78112820 con 0x7f2e7806a170 2024-06-03T02:35:08.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.717+0000 7f2e7d600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781126a0 con 0x7f2e7806ab30 2024-06-03T02:35:08.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.717+0000 7f2e7d600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781129a0 con 0x7f2e78108500 2024-06-03T02:35:08.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.718+0000 7f2e77400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e78108500 0x7f2e78101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.718+0000 7f2e77400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e78108500 0x7f2e78101e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60284/0 (socket says 172.21.15.79:60284) 2024-06-03T02:35:08.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.718+0000 7f2e77400700 1 -- 172.21.15.79:0/1912450107 learned_addr learned my addr 172.21.15.79:0/1912450107 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:08.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.719+0000 7f2e76000700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e7806ab30 0x7f2e781018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.719+0000 7f2e77400700 1 -- 172.21.15.79:0/1912450107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e7806ab30 msgr2=0x7f2e781018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:08.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.719+0000 7f2e77400700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e7806ab30 0x7f2e781018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.719+0000 7f2e77400700 1 -- 172.21.15.79:0/1912450107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e7806a170 msgr2=0x7f2e78101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:08.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.719+0000 7f2e77400700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e7806a170 0x7f2e78101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.719+0000 7f2e77400700 1 -- 172.21.15.79:0/1912450107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e78072e50 con 0x7f2e78108500 2024-06-03T02:35:08.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.720+0000 7f2e77400700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e78108500 0x7f2e78101e10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2e70007d70 tx=0x7f2e7000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:08.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.721+0000 7f2e6f400700 1 -- 172.21.15.79:0/1912450107 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70016020 con 0x7f2e78108500 2024-06-03T02:35:08.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.721+0000 7f2e6f400700 1 -- 172.21.15.79:0/1912450107 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7000ad40 con 0x7f2e78108500 2024-06-03T02:35:08.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.721+0000 7f2e6f400700 1 -- 172.21.15.79:0/1912450107 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e700116f0 con 0x7f2e78108500 2024-06-03T02:35:08.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:08 smithi123 ceph-mon[36640]: pgmap v968: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.3 MiB/s wr, 481 op/s 2024-06-03T02:35:08.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.724+0000 7f2e7d600700 1 -- 172.21.15.79:0/1912450107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e78073140 con 0x7f2e78108500 2024-06-03T02:35:08.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.724+0000 7f2e7d600700 1 -- 172.21.15.79:0/1912450107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e78073600 con 0x7f2e78108500 2024-06-03T02:35:08.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.729+0000 7f2e6f400700 1 -- 172.21.15.79:0/1912450107 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2e70019020 con 0x7f2e78108500 2024-06-03T02:35:08.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.730+0000 7f2e87edcb80 1 Processor -- start 2024-06-03T02:35:08.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.731+0000 7f2e87edcb80 1 -- start start 2024-06-03T02:35:08.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.731+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c476070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.731+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c4765b0 0x55e59c476a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.731+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c481300 0x55e59c4836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.731+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e59c385a10 con 0x55e59c481300 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.731+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e59c3c8b00 con 0x55e59c3f2e40 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.731+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e59c3838f0 con 0x55e59c4765b0 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.732+0000 7f2e6f400700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e5c0a2e30 0x7f2e5c0a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.732+0000 7f2e6f400700 1 -- 172.21.15.79:0/1912450107 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2e700cf050 con 0x7f2e78108500 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.732+0000 7f2e77400700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e5c0a2e30 0x7f2e5c0a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.733+0000 7f2e76000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c476070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.733+0000 7f2e76000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c476070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54524/0 (socket says 172.21.15.79:54524) 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.733+0000 7f2e76000700 1 -- 172.21.15.79:0/3239551657 learned_addr learned my addr 172.21.15.79:0/3239551657 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.734+0000 7f2e77400700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e5c0a2e30 0x7f2e5c0a52f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e700cf330 tx=0x7f2e700cf2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.734+0000 7f2e76000700 1 --2- 172.21.15.79:0/3239551657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c481300 0x55e59c4836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.734+0000 7f2e76a00700 1 --2- 172.21.15.79:0/3239551657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c4765b0 0x55e59c476a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.734+0000 7f2e76000700 1 -- 172.21.15.79:0/3239551657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c4765b0 msgr2=0x55e59c476a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:08.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.734+0000 7f2e76000700 1 --2- 172.21.15.79:0/3239551657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c4765b0 0x55e59c476a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.734+0000 7f2e76000700 1 -- 172.21.15.79:0/3239551657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 msgr2=0x55e59c476070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:08.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.734+0000 7f2e76000700 1 --2- 172.21.15.79:0/3239551657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c476070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.734+0000 7f2e76000700 1 -- 172.21.15.79:0/3239551657 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59c370f90 con 0x55e59c481300 2024-06-03T02:35:08.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.735+0000 7f2e76000700 1 --2- 172.21.15.79:0/3239551657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c481300 0x55e59c4836f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e68014700 tx=0x7f2e680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5bf48b9ebac37a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.735+0000 7f2e67e00700 1 -- 172.21.15.79:0/3239551657 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68004830 con 0x55e59c481300 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.736+0000 7f2e67e00700 1 -- 172.21.15.79:0/3239551657 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e68005020 con 0x55e59c481300 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.736+0000 7f2e87edcb80 1 -- 172.21.15.79:0/3239551657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c481300 msgr2=0x55e59c4836f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.737+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/3239551657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c481300 0x55e59c4836f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e68014700 tx=0x7f2e680047c0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.737+0000 7f2e87edcb80 1 -- 172.21.15.79:0/3239551657 shutdown_connections 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.737+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/3239551657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c481300 0x55e59c4836f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.737+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/3239551657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c476070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.737+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/3239551657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c4765b0 0x55e59c476a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.737+0000 7f2e87edcb80 1 -- 172.21.15.79:0/3239551657 >> 172.21.15.79:0/3239551657 conn(0x55e59c3a3170 msgr2=0x55e59c4724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:08.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.737+0000 7f2e87edcb80 1 -- 172.21.15.79:0/3239551657 shutdown_connections 2024-06-03T02:35:08.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.738+0000 7f2e87edcb80 1 -- 172.21.15.79:0/3239551657 wait complete. 2024-06-03T02:35:08.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.738+0000 7f2e87edcb80 1 -- start start 2024-06-03T02:35:08.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.739+0000 7f2e87edcb80 1 Processor -- start 2024-06-03T02:35:08.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.740+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c3f2e40 0x55e59c480970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.740+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c47dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.740+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c481300 0x55e59c47e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.740+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e59c364f60 con 0x55e59c4765b0 2024-06-03T02:35:08.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.740+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e59c3c4ac0 con 0x55e59c481300 2024-06-03T02:35:08.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.740+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e59c484060 con 0x55e59c3f2e40 2024-06-03T02:35:08.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.740+0000 7f2e87edcb80 10 client.0 _ll_register_callbacks cb 0x55e59c362720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:35:08.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.741+0000 7f2e76000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c3f2e40 0x55e59c480970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.741+0000 7f2e76000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c3f2e40 0x55e59c480970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:60310/0 (socket says 172.21.15.79:60310) 2024-06-03T02:35:08.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.741+0000 7f2e76000700 1 -- 172.21.15.79:0/474290965 learned_addr learned my addr 172.21.15.79:0/474290965 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:08.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.742+0000 7f2e76000700 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c481300 0x55e59c47e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.742+0000 7f2e76000700 1 -- 172.21.15.79:0/474290965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c481300 msgr2=0x55e59c47e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:08.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.742+0000 7f2e76000700 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c481300 0x55e59c47e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.742+0000 7f2e76000700 1 -- 172.21.15.79:0/474290965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 msgr2=0x55e59c47dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:08.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.742+0000 7f2e76a00700 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c47dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.742+0000 7f2e76000700 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c47dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.742+0000 7f2e76000700 1 -- 172.21.15.79:0/474290965 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e59c3178d0 con 0x55e59c3f2e40 2024-06-03T02:35:08.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.742+0000 7f2e76a00700 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c47dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T02:35:08.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.743+0000 7f2e76000700 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c3f2e40 0x55e59c480970 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2e68001eb0 tx=0x7f2e68004f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de5ae945cba9b4ce server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:08.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.743+0000 7f2e64c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-03T02:35:08.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.744+0000 7f2e64c00700 1 -- 172.21.15.79:0/474290965 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6802a060 con 0x55e59c3f2e40 2024-06-03T02:35:08.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.744+0000 7f2e64c00700 1 -- 172.21.15.79:0/474290965 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2e68012860 con 0x55e59c3f2e40 2024-06-03T02:35:08.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.744+0000 7f2e87edcb80 1 -- 172.21.15.79:0/474290965 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e68005020 con 0x55e59c3f2e40 2024-06-03T02:35:08.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.745+0000 7f2e64c00700 1 -- 172.21.15.79:0/474290965 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e68012a60 con 0x55e59c3f2e40 2024-06-03T02:35:08.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.746+0000 7f2e87edcb80 10 client.26989 fetch_fsmap learned FSMap version 30 2024-06-03T02:35:08.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.747+0000 7f2e87edcb80 1 -- 172.21.15.79:0/474290965 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e59c48e6d0 con 0x55e59c3f2e40 2024-06-03T02:35:08.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.747+0000 7f2e64c00700 1 -- 172.21.15.79:0/474290965 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e68005020 con 0x55e59c3f2e40 2024-06-03T02:35:08.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.747+0000 7f2e64c00700 1 -- 172.21.15.79:0/474290965 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2e6801fc20 con 0x55e59c3f2e40 2024-06-03T02:35:08.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.747+0000 7f2e87edcb80 10 client.26989 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:35:08.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.747+0000 7f2e87edcb80 10 client.26989 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:35:08.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.747+0000 7f2e87edcb80 20 client.26989 populate_metadata read hostname 'smithi079' 2024-06-03T02:35:08.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.747+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e59c490ba0 0x55e59c492f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.748+0000 7f2e76a00700 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e59c490ba0 0x55e59c492f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.749+0000 7f2e87edcb80 4 client.26989 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:35:08.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.749+0000 7f2e87edcb80 1 -- 172.21.15.79:0/474290965 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e59c3d6000 con 0x55e59c490ba0 2024-06-03T02:35:08.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.749+0000 7f2e64c00700 10 client.26989 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:35:08.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.751+0000 7f2e76a00700 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e59c490ba0 0x55e59c492f90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=541cd01d0ed45c42 server_cookie=ef69bb2acf5ae325 in_seq=0 out_seq=0 2024-06-03T02:35:08.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.753+0000 7f2e64c00700 1 -- 172.21.15.79:0/474290965 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e59c3d6000 con 0x55e59c490ba0 2024-06-03T02:35:08.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.753+0000 7f2e64c00700 10 client.26989 handle_command_reply: tid=0 2024-06-03T02:35:08.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.753+0000 7f2e87edcb80 1 client.26989 shutdown 2024-06-03T02:35:08.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.754+0000 7f2e87edcb80 1 -- 172.21.15.79:0/474290965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c3f2e40 msgr2=0x55e59c480970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:08.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.754+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c3f2e40 0x55e59c480970 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2e68001eb0 tx=0x7f2e68004f50 comp rx=0 tx=0).stop 2024-06-03T02:35:08.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.754+0000 7f2e87edcb80 1 -- 172.21.15.79:0/474290965 shutdown_connections 2024-06-03T02:35:08.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.754+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c481300 0x55e59c47e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.754+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e59c490ba0 0x55e59c492f90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.754+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c3f2e40 0x55e59c480970 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.754+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/474290965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c47dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.754+0000 7f2e87edcb80 1 -- 172.21.15.79:0/474290965 >> 172.21.15.79:0/474290965 conn(0x55e59c3a3170 msgr2=0x55e59c471740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:08.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.755+0000 7f2e64c00700 0 client.26989 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:35:08.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.755+0000 7f2e87edcb80 1 -- 172.21.15.79:0/474290965 shutdown_connections 2024-06-03T02:35:08.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.755+0000 7f2e87edcb80 1 -- 172.21.15.79:0/474290965 wait complete. 2024-06-03T02:35:08.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.756+0000 7f2e87edcb80 20 client.26989 trim_cache size 0 max 16384 2024-06-03T02:35:08.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.781+0000 7f2e87edcb80 1 Processor -- start 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.781+0000 7f2e87edcb80 1 -- start start 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.781+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c47cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.781+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c4826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.781+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c481300 0x55e59c4804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.781+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781b0a60 con 0x55e59c4765b0 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.781+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781c75e0 con 0x55e59c3f2e40 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.781+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781b1130 con 0x55e59c481300 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.782+0000 7f2e76000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c47cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.782+0000 7f2e76000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c47cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54550/0 (socket says 172.21.15.79:54550) 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.782+0000 7f2e76000700 1 -- 172.21.15.79:0/1288810258 learned_addr learned my addr 172.21.15.79:0/1288810258 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:08.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.782+0000 7f2e76000700 1 --2- 172.21.15.79:0/1288810258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c481300 0x55e59c4804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.782+0000 7f2e76a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c4826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.782+0000 7f2e76000700 1 -- 172.21.15.79:0/1288810258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c481300 msgr2=0x55e59c4804b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.782+0000 7f2e76000700 1 --2- 172.21.15.79:0/1288810258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c481300 0x55e59c4804b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.782+0000 7f2e76000700 1 -- 172.21.15.79:0/1288810258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 msgr2=0x55e59c4826f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.783+0000 7f2e76000700 1 --2- 172.21.15.79:0/1288810258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c4826f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.783+0000 7f2e76000700 1 -- 172.21.15.79:0/1288810258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59c47c1e0 con 0x55e59c3f2e40 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.783+0000 7f2e76a00700 1 --2- 172.21.15.79:0/1288810258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c4826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.783+0000 7f2e76000700 1 --2- 172.21.15.79:0/1288810258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c47cdf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2e68022870 tx=0x7f2e680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd371a83e4830c07 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.783+0000 7f2e67400700 1 -- 172.21.15.79:0/1288810258 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68034080 con 0x55e59c3f2e40 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.783+0000 7f2e67400700 1 -- 172.21.15.79:0/1288810258 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e6800cc30 con 0x55e59c3f2e40 2024-06-03T02:35:08.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.783+0000 7f2e67400700 1 -- 172.21.15.79:0/1288810258 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68012a60 con 0x55e59c3f2e40 2024-06-03T02:35:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.786+0000 7f2e87edcb80 1 -- 172.21.15.79:0/1288810258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 msgr2=0x55e59c47cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.786+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/1288810258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c47cdf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2e68022870 tx=0x7f2e680047c0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.786+0000 7f2e87edcb80 1 -- 172.21.15.79:0/1288810258 shutdown_connections 2024-06-03T02:35:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.786+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/1288810258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c481300 0x55e59c4804b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.786+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/1288810258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c3f2e40 0x55e59c47cdf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.786+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/1288810258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c4765b0 0x55e59c4826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.786+0000 7f2e87edcb80 1 -- 172.21.15.79:0/1288810258 >> 172.21.15.79:0/1288810258 conn(0x55e59c3a3170 msgr2=0x55e59c48c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.786+0000 7f2e87edcb80 1 -- 172.21.15.79:0/1288810258 shutdown_connections 2024-06-03T02:35:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.787+0000 7f2e87edcb80 1 -- 172.21.15.79:0/1288810258 wait complete. 2024-06-03T02:35:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.788+0000 7f2e87edcb80 1 -- start start 2024-06-03T02:35:08.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.788+0000 7f2e87edcb80 1 Processor -- start 2024-06-03T02:35:08.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c3f2e40 0x55e59c490ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c4765b0 0x55e59c4910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e87edcb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c481300 0x55e59c491620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781c6f10 con 0x55e59c3f2e40 2024-06-03T02:35:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781c6840 con 0x55e59c4765b0 2024-06-03T02:35:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e87edcb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781ae9c0 con 0x55e59c481300 2024-06-03T02:35:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e87edcb80 10 client.0 _ll_register_callbacks cb 0x55e59c362720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e76a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c4765b0 0x55e59c4910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e76000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c3f2e40 0x55e59c490ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e76a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c4765b0 0x55e59c4910e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:54564/0 (socket says 172.21.15.79:54564) 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.789+0000 7f2e76a00700 1 -- 172.21.15.79:0/4150431679 learned_addr learned my addr 172.21.15.79:0/4150431679 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e76a00700 1 -- 172.21.15.79:0/4150431679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c481300 msgr2=0x55e59c491620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e76a00700 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c481300 0x55e59c491620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e76a00700 1 -- 172.21.15.79:0/4150431679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c3f2e40 msgr2=0x55e59c490ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e76a00700 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c3f2e40 0x55e59c490ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e76a00700 1 -- 172.21.15.79:0/4150431679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e59c4f9510 con 0x55e59c4765b0 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e76a00700 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c4765b0 0x55e59c4910e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009990 tx=0x7f2e60005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e1ccda291a6661 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e64c00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:35:08.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e76000700 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c3f2e40 0x55e59c490ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:35:08.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e64c00700 1 -- 172.21.15.79:0/4150431679 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600050c0 con 0x55e59c4765b0 2024-06-03T02:35:08.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.790+0000 7f2e64c00700 1 -- 172.21.15.79:0/4150431679 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2e60005260 con 0x55e59c4765b0 2024-06-03T02:35:08.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.791+0000 7f2e87edcb80 1 -- 172.21.15.79:0/4150431679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e59c3d0580 con 0x55e59c4765b0 2024-06-03T02:35:08.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.791+0000 7f2e64c00700 1 -- 172.21.15.79:0/4150431679 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e60034050 con 0x55e59c4765b0 2024-06-03T02:35:08.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.791+0000 7f2e87edcb80 10 client.27014 fetch_fsmap learned FSMap version 30 2024-06-03T02:35:08.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.791+0000 7f2e87edcb80 1 -- 172.21.15.79:0/4150431679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e59c4fa860 con 0x55e59c4765b0 2024-06-03T02:35:08.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.792+0000 7f2e64c00700 1 -- 172.21.15.79:0/4150431679 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e59c3d0580 con 0x55e59c4765b0 2024-06-03T02:35:08.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.792+0000 7f2e64c00700 1 -- 172.21.15.79:0/4150431679 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2e6000f460 con 0x55e59c4765b0 2024-06-03T02:35:08.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.792+0000 7f2e87edcb80 10 client.27014 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:35:08.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.792+0000 7f2e87edcb80 10 client.27014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-06-03T02:35:08.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.792+0000 7f2e87edcb80 20 client.27014 populate_metadata read hostname 'smithi079' 2024-06-03T02:35:08.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.792+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e59c4fafa0 0x55e59c4fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:08.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.792+0000 7f2e87edcb80 4 client.27014 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:35:08.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.792+0000 7f2e87edcb80 1 -- 172.21.15.79:0/4150431679 --> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e59c24a250 con 0x55e59c4fafa0 2024-06-03T02:35:08.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.794+0000 7f2e76000700 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e59c4fafa0 0x55e59c4fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:08.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.794+0000 7f2e76000700 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e59c4fafa0 0x55e59c4fd3c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31290f2c38a29c71 server_cookie=1e566348f9c76edf in_seq=0 out_seq=0 2024-06-03T02:35:08.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.794+0000 7f2e64c00700 10 client.27014 ms_handle_connect on v2:172.21.15.79:6838/2483452059 2024-06-03T02:35:08.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.795+0000 7f2e64c00700 1 -- 172.21.15.79:0/4150431679 <== mds.1 v2:172.21.15.79:6838/2483452059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e59c24a250 con 0x55e59c4fafa0 2024-06-03T02:35:08.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.795+0000 7f2e64c00700 10 client.27014 handle_command_reply: tid=0 2024-06-03T02:35:08.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.795+0000 7f2e87edcb80 1 client.27014 shutdown 2024-06-03T02:35:08.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.795+0000 7f2e87edcb80 1 -- 172.21.15.79:0/4150431679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c4765b0 msgr2=0x55e59c4910e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:08.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.795+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c4765b0 0x55e59c4910e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009990 tx=0x7f2e60005f10 comp rx=0 tx=0).stop 2024-06-03T02:35:08.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.795+0000 7f2e87edcb80 1 -- 172.21.15.79:0/4150431679 shutdown_connections 2024-06-03T02:35:08.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.795+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.79:6838/2483452059,v1:172.21.15.79:6839/2483452059] conn(0x55e59c4fafa0 0x55e59c4fd3c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.796+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e59c481300 0x55e59c491620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.796+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e59c3f2e40 0x55e59c490ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.796+0000 7f2e64c00700 0 client.27014 ms_handle_reset on v2:172.21.15.79:6838/2483452059 2024-06-03T02:35:08.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.796+0000 7f2e87edcb80 1 --2- 172.21.15.79:0/4150431679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59c4765b0 0x55e59c4910e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.796+0000 7f2e87edcb80 1 -- 172.21.15.79:0/4150431679 >> 172.21.15.79:0/4150431679 conn(0x55e59c3a3170 msgr2=0x55e59c47cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:08.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.796+0000 7f2e87edcb80 1 -- 172.21.15.79:0/4150431679 shutdown_connections 2024-06-03T02:35:08.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.796+0000 7f2e87edcb80 1 -- 172.21.15.79:0/4150431679 wait complete. 2024-06-03T02:35:08.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.796+0000 7f2e87edcb80 20 client.27014 trim_cache size 0 max 16384 2024-06-03T02:35:08.798 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:35:08.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.797+0000 7f2e7d600700 1 -- 172.21.15.79:0/1912450107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e5c0a2e30 msgr2=0x7f2e5c0a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:08.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.797+0000 7f2e7d600700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e5c0a2e30 0x7f2e5c0a52f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e700cf330 tx=0x7f2e700cf2a0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.797+0000 7f2e7d600700 1 -- 172.21.15.79:0/1912450107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e78108500 msgr2=0x7f2e78101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:08.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.797+0000 7f2e7d600700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e78108500 0x7f2e78101e10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2e70007d70 tx=0x7f2e7000a340 comp rx=0 tx=0).stop 2024-06-03T02:35:08.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.797+0000 7f2e7d600700 1 -- 172.21.15.79:0/1912450107 shutdown_connections 2024-06-03T02:35:08.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.797+0000 7f2e7d600700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e78108500 0x7f2e78101e10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.797+0000 7f2e7d600700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2e5c0a2e30 0x7f2e5c0a52f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.797+0000 7f2e7d600700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e7806a170 0x7f2e78101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.797+0000 7f2e7d600700 1 --2- 172.21.15.79:0/1912450107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e7806ab30 0x7f2e781018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:08.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.798+0000 7f2e7d600700 1 -- 172.21.15.79:0/1912450107 >> 172.21.15.79:0/1912450107 conn(0x7f2e78078340 msgr2=0x7f2e78100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:08.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.798+0000 7f2e7d600700 1 -- 172.21.15.79:0/1912450107 shutdown_connections 2024-06-03T02:35:08.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:08.798+0000 7f2e7d600700 1 -- 172.21.15.79:0/1912450107 wait complete. 2024-06-03T02:35:08.819 INFO:teuthology.orchestra.run.smithi079.stdout:transaction type: 2024-06-03T02:35:08.819 INFO:teuthology.orchestra.run.smithi079.stdout:scaling factor: 32 2024-06-03T02:35:08.819 INFO:teuthology.orchestra.run.smithi079.stdout:query mode: simple 2024-06-03T02:35:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:number of clients: 100 2024-06-03T02:35:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:number of threads: 4 2024-06-03T02:35:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:duration: 900 s 2024-06-03T02:35:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:number of transactions actually processed: 719871 2024-06-03T02:35:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:latency average = 125.017 ms 2024-06-03T02:35:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:latency stddev = 196.681 ms 2024-06-03T02:35:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:tps = 799.405161 (including connections establishing) 2024-06-03T02:35:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:tps = 799.410850 (excluding connections establishing) 2024-06-03T02:35:08.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-06-03T02:35:08.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:08 smithi135 ceph-mon[36653]: pgmap v968: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.3 MiB/s wr, 481 op/s 2024-06-03T02:35:09.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T02:35:10.500 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:35:10.678 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:10 smithi079 ceph-mon[31819]: pgmap v969: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 464 op/s 2024-06-03T02:35:10.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:10 smithi123 ceph-mon[36640]: pgmap v969: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 464 op/s 2024-06-03T02:35:10.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:10 smithi135 ceph-mon[36653]: pgmap v969: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 464 op/s 2024-06-03T02:35:12.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:12 smithi123 ceph-mon[36640]: pgmap v970: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.5 MiB/s wr, 498 op/s 2024-06-03T02:35:12.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:12 smithi079 ceph-mon[31819]: pgmap v970: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.5 MiB/s wr, 498 op/s 2024-06-03T02:35:12.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:12 smithi135 ceph-mon[36653]: pgmap v970: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.5 MiB/s wr, 498 op/s 2024-06-03T02:35:14.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:14 smithi123 ceph-mon[36640]: pgmap v971: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.4 MiB/s wr, 553 op/s 2024-06-03T02:35:14.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:14 smithi079 ceph-mon[31819]: pgmap v971: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.4 MiB/s wr, 553 op/s 2024-06-03T02:35:14.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:14 smithi135 ceph-mon[36653]: pgmap v971: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.4 MiB/s wr, 553 op/s 2024-06-03T02:35:16.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:16 smithi123 ceph-mon[36640]: pgmap v972: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.6 MiB/s wr, 599 op/s 2024-06-03T02:35:16.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:16 smithi079 ceph-mon[31819]: pgmap v972: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.6 MiB/s wr, 599 op/s 2024-06-03T02:35:16.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:16 smithi135 ceph-mon[36653]: pgmap v972: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.6 MiB/s wr, 599 op/s 2024-06-03T02:35:18.705 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:18 smithi079 ceph-mon[31819]: pgmap v973: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.4 MiB/s wr, 550 op/s 2024-06-03T02:35:18.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:18 smithi123 ceph-mon[36640]: pgmap v973: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.4 MiB/s wr, 550 op/s 2024-06-03T02:35:18.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:18 smithi135 ceph-mon[36653]: pgmap v973: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.4 MiB/s wr, 550 op/s 2024-06-03T02:35:20.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.591+0000 7f652cc00700 1 -- 172.21.15.79:0/2790398616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6520099fe0 msgr2=0x7f652009a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:20.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.591+0000 7f652cc00700 1 --2- 172.21.15.79:0/2790398616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6520099fe0 0x7f652009a3c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f651c0099c0 tx=0x7f651c009cd0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.592+0000 7f652cc00700 1 -- 172.21.15.79:0/2790398616 shutdown_connections 2024-06-03T02:35:20.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.592+0000 7f652cc00700 1 --2- 172.21.15.79:0/2790398616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6520094be0 0x7f65200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.592+0000 7f652cc00700 1 --2- 172.21.15.79:0/2790398616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6520099fe0 0x7f652009a3c0 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f651c0099c0 tx=0x7f651c009cd0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.592+0000 7f652cc00700 1 --2- 172.21.15.79:0/2790398616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6520094220 0x7f65200946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.592+0000 7f652cc00700 1 -- 172.21.15.79:0/2790398616 >> 172.21.15.79:0/2790398616 conn(0x7f6520009ae0 msgr2=0x7f6520009ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:20.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.593+0000 7f652cc00700 1 -- 172.21.15.79:0/2790398616 shutdown_connections 2024-06-03T02:35:20.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.593+0000 7f652cc00700 1 -- 172.21.15.79:0/2790398616 wait complete. 2024-06-03T02:35:20.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.595+0000 7f652cc00700 1 Processor -- start 2024-06-03T02:35:20.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.598+0000 7f652cc00700 1 -- start start 2024-06-03T02:35:20.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.599+0000 7f652cc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6520094220 0x7f652013b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.600+0000 7f652cc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6520094be0 0x7f652013b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.600+0000 7f6527400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6520094220 0x7f652013b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.600+0000 7f6527400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6520094220 0x7f652013b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:53526/0 (socket says 172.21.15.79:53526) 2024-06-03T02:35:20.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.600+0000 7f652cc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f652013bf20 0x7f6520150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.601+0000 7f652cc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65200b5010 con 0x7f652013bf20 2024-06-03T02:35:20.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.601+0000 7f652cc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65200b4e90 con 0x7f6520094be0 2024-06-03T02:35:20.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.601+0000 7f652cc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65200b5190 con 0x7f6520094220 2024-06-03T02:35:20.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.601+0000 7f6527400700 1 -- 172.21.15.79:0/4211591176 learned_addr learned my addr 172.21.15.79:0/4211591176 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:20.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.601+0000 7f6526a00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6520094be0 0x7f652013b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.601+0000 7f6527400700 1 -- 172.21.15.79:0/4211591176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6520094be0 msgr2=0x7f652013b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:20.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.601+0000 7f6527400700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6520094be0 0x7f652013b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.601+0000 7f6527400700 1 -- 172.21.15.79:0/4211591176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f652013bf20 msgr2=0x7f6520150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:20.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.601+0000 7f6527400700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f652013bf20 0x7f6520150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.602+0000 7f6527400700 1 -- 172.21.15.79:0/4211591176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65201505d0 con 0x7f6520094220 2024-06-03T02:35:20.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.602+0000 7f6527400700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6520094220 0x7f652013b4a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f651c0098e0 tx=0x7f651c005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:20.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.602+0000 7f651be00700 1 -- 172.21.15.79:0/4211591176 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c026070 con 0x7f6520094220 2024-06-03T02:35:20.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.603+0000 7f652cc00700 1 -- 172.21.15.79:0/4211591176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65201507a0 con 0x7f6520094220 2024-06-03T02:35:20.604 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.603+0000 7f652cc00700 1 -- 172.21.15.79:0/4211591176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6520150ba0 con 0x7f6520094220 2024-06-03T02:35:20.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.603+0000 7f651be00700 1 -- 172.21.15.79:0/4211591176 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f651c004030 con 0x7f6520094220 2024-06-03T02:35:20.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.603+0000 7f651be00700 1 -- 172.21.15.79:0/4211591176 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c00fd80 con 0x7f6520094220 2024-06-03T02:35:20.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.616+0000 7f651be00700 1 -- 172.21.15.79:0/4211591176 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f651c029070 con 0x7f6520094220 2024-06-03T02:35:20.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.618+0000 7f651be00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65100a2ca0 0x7f65100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.619+0000 7f651be00700 1 -- 172.21.15.79:0/4211591176 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f651c0d0e60 con 0x7f6520094220 2024-06-03T02:35:20.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.619+0000 7f6526a00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65100a2ca0 0x7f65100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.620+0000 7f6526a00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65100a2ca0 0x7f65100a5160 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f652013cc10 tx=0x7f6514009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.623+0000 7f6538a90b80 1 Processor -- start 2024-06-03T02:35:20.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.625+0000 7f6538a90b80 1 -- start start 2024-06-03T02:35:20.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.625+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600c24e40 0x559600caa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.625+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600caa960 0x559600caade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.625+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.625+0000 7f6538a90b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559600bb7a10 con 0x559600caa960 2024-06-03T02:35:20.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.625+0000 7f6538a90b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559600bfab00 con 0x559600cb3570 2024-06-03T02:35:20.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.625+0000 7f6538a90b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559600bb58f0 con 0x559600c24e40 2024-06-03T02:35:20.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.626+0000 7f6526a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600caa960 0x559600caade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.626+0000 7f6527e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.626+0000 7f6527e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38390/0 (socket says 172.21.15.79:38390) 2024-06-03T02:35:20.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.626+0000 7f6527e00700 1 -- 172.21.15.79:0/2232251186 learned_addr learned my addr 172.21.15.79:0/2232251186 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:20.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.627+0000 7f6527e00700 1 -- 172.21.15.79:0/2232251186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600c24e40 msgr2=0x559600caa420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:35:20.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.627+0000 7f6527e00700 1 --2- 172.21.15.79:0/2232251186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600c24e40 0x559600caa420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.627+0000 7f6527e00700 1 -- 172.21.15.79:0/2232251186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600caa960 msgr2=0x559600caade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:20.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.627+0000 7f6527e00700 1 --2- 172.21.15.79:0/2232251186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600caa960 0x559600caade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.627+0000 7f6527e00700 1 -- 172.21.15.79:0/2232251186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559600ba2f90 con 0x559600cb3570 2024-06-03T02:35:20.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.627+0000 7f6527e00700 1 --2- 172.21.15.79:0/2232251186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb5960 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6528051b50 tx=0x7f6528068aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd042905c4703b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:20.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.628+0000 7f6518c00700 1 -- 172.21.15.79:0/2232251186 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528069890 con 0x559600cb3570 2024-06-03T02:35:20.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.628+0000 7f6518c00700 1 -- 172.21.15.79:0/2232251186 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652807c3a0 con 0x559600cb3570 2024-06-03T02:35:20.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.630+0000 7f6538a90b80 1 -- 172.21.15.79:0/2232251186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 msgr2=0x559600cb5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:20.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.630+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2232251186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb5960 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6528051b50 tx=0x7f6528068aa0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.630+0000 7f6538a90b80 1 -- 172.21.15.79:0/2232251186 shutdown_connections 2024-06-03T02:35:20.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.630+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2232251186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb5960 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.630+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2232251186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600c24e40 0x559600caa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.630+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2232251186 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600caa960 0x559600caade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.630+0000 7f6538a90b80 1 -- 172.21.15.79:0/2232251186 >> 172.21.15.79:0/2232251186 conn(0x559600bd5170 msgr2=0x559600ca4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:20.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.631+0000 7f6538a90b80 1 -- 172.21.15.79:0/2232251186 shutdown_connections 2024-06-03T02:35:20.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.631+0000 7f6538a90b80 1 -- 172.21.15.79:0/2232251186 wait complete. 2024-06-03T02:35:20.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.632+0000 7f6538a90b80 1 -- start start 2024-06-03T02:35:20.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.632+0000 7f6538a90b80 1 Processor -- start 2024-06-03T02:35:20.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.634+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600cb2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.634+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600caa960 0x559600caff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.634+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.634+0000 7f6538a90b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559600b96f60 con 0x559600c24e40 2024-06-03T02:35:20.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.634+0000 7f6538a90b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559600bf6ac0 con 0x559600cb3570 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.634+0000 7f6538a90b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559600cb6230 con 0x559600caa960 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.634+0000 7f6538a90b80 10 client.0 _ll_register_callbacks cb 0x559600b94720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.635+0000 7f6527e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.635+0000 7f6527e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38406/0 (socket says 172.21.15.79:38406) 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.635+0000 7f6527e00700 1 -- 172.21.15.79:0/2901162536 learned_addr learned my addr 172.21.15.79:0/2901162536 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.635+0000 7f6527e00700 1 -- 172.21.15.79:0/2901162536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600caa960 msgr2=0x559600caff40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.635+0000 7f6527e00700 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600caa960 0x559600caff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.635+0000 7f6527e00700 1 -- 172.21.15.79:0/2901162536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 msgr2=0x559600cb2c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.635+0000 7f6527e00700 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600cb2c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.635+0000 7f6527e00700 1 -- 172.21.15.79:0/2901162536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559600b498d0 con 0x559600cb3570 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.635+0000 7f6527e00700 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb0480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6528068340 tx=0x7f652807cd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=820ecbe82aa0fa03 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.636+0000 7f650d600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.636+0000 7f650d600700 1 -- 172.21.15.79:0/2901162536 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528069b60 con 0x559600cb3570 2024-06-03T02:35:20.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.636+0000 7f650d600700 1 -- 172.21.15.79:0/2901162536 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6528069d00 con 0x559600cb3570 2024-06-03T02:35:20.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.636+0000 7f6538a90b80 1 -- 172.21.15.79:0/2901162536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f652807c3a0 con 0x559600cb3570 2024-06-03T02:35:20.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.636+0000 7f650d600700 1 -- 172.21.15.79:0/2901162536 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6528078620 con 0x559600cb3570 2024-06-03T02:35:20.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.636+0000 7f6538a90b80 10 client.27026 fetch_fsmap learned FSMap version 30 2024-06-03T02:35:20.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.636+0000 7f6538a90b80 1 -- 172.21.15.79:0/2901162536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559600cc0960 con 0x559600cb3570 2024-06-03T02:35:20.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.637+0000 7f650d600700 1 -- 172.21.15.79:0/2901162536 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652807c3a0 con 0x559600cb3570 2024-06-03T02:35:20.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.637+0000 7f650d600700 1 -- 172.21.15.79:0/2901162536 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f65280897e0 con 0x559600cb3570 2024-06-03T02:35:20.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.637+0000 7f6538a90b80 10 client.27026 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:35:20.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.638+0000 7f6538a90b80 10 client.27026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:35:20.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.638+0000 7f6538a90b80 20 client.27026 populate_metadata read hostname 'smithi079' 2024-06-03T02:35:20.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.638+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559600cc2e30 0x559600cc5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.638+0000 7f6538a90b80 4 client.27026 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T02:35:20.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.638+0000 7f6538a90b80 1 -- 172.21.15.79:0/2901162536 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559600c08000 con 0x559600cc2e30 2024-06-03T02:35:20.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.639+0000 7f6527400700 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559600cc2e30 0x559600cc5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.640+0000 7f6527400700 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559600cc2e30 0x559600cc5220 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8165e01e8924b497 server_cookie=d135a2096b6f1a55 in_seq=0 out_seq=0 2024-06-03T02:35:20.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.640+0000 7f650d600700 10 client.27026 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:35:20.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.641+0000 7f650d600700 1 -- 172.21.15.79:0/2901162536 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559600c08000 con 0x559600cc2e30 2024-06-03T02:35:20.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.641+0000 7f650d600700 10 client.27026 handle_command_reply: tid=0 2024-06-03T02:35:20.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.641+0000 7f6538a90b80 1 client.27026 shutdown 2024-06-03T02:35:20.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 -- 172.21.15.79:0/2901162536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 msgr2=0x559600cb0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:20.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb0480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6528068340 tx=0x7f652807cd10 comp rx=0 tx=0).stop 2024-06-03T02:35:20.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 -- 172.21.15.79:0/2901162536 shutdown_connections 2024-06-03T02:35:20.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600cb3570 0x559600cb0480 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559600cc2e30 0x559600cc5220 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600cb2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 --2- 172.21.15.79:0/2901162536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600caa960 0x559600caff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f650d600700 0 client.27026 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:35:20.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 -- 172.21.15.79:0/2901162536 >> 172.21.15.79:0/2901162536 conn(0x559600bd5170 msgr2=0x559600ca4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 -- 172.21.15.79:0/2901162536 shutdown_connections 2024-06-03T02:35:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.643+0000 7f6538a90b80 1 -- 172.21.15.79:0/2901162536 wait complete. 2024-06-03T02:35:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.644+0000 7f6538a90b80 20 client.27026 trim_cache size 0 max 16384 2024-06-03T02:35:20.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.675+0000 7f6538a90b80 1 Processor -- start 2024-06-03T02:35:20.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.681+0000 7f6538a90b80 1 -- start start 2024-06-03T02:35:20.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600caf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600caff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 0x559600ccaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6538a90b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f650800eca0 con 0x559600c24e40 2024-06-03T02:35:20.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6538a90b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65080107e0 con 0x559600caa960 2024-06-03T02:35:20.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6538a90b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f650800f370 con 0x559600cb3570 2024-06-03T02:35:20.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6526a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600caff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6527e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 0x559600ccaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6526a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600caff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38420/0 (socket says 172.21.15.79:38420) 2024-06-03T02:35:20.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.682+0000 7f6526a00700 1 -- 172.21.15.79:0/700648831 learned_addr learned my addr 172.21.15.79:0/700648831 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:20.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.683+0000 7f6527e00700 1 -- 172.21.15.79:0/700648831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 msgr2=0x559600caff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:20.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.683+0000 7f6527e00700 1 --2- 172.21.15.79:0/700648831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600caff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.683+0000 7f6527e00700 1 -- 172.21.15.79:0/700648831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 msgr2=0x559600caf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:20.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.683+0000 7f6527e00700 1 --2- 172.21.15.79:0/700648831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600caf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.683+0000 7f6527e00700 1 -- 172.21.15.79:0/700648831 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559600cae4b0 con 0x559600cb3570 2024-06-03T02:35:20.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.683+0000 7f6526a00700 1 --2- 172.21.15.79:0/700648831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600caff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:35:20.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.684+0000 7f6527e00700 1 --2- 172.21.15.79:0/700648831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 0x559600ccaab0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f652807bb40 tx=0x7f6528068aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e916cbde0af5bfd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:20.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.685+0000 7f650fe00700 1 -- 172.21.15.79:0/700648831 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528078620 con 0x559600cb3570 2024-06-03T02:35:20.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.685+0000 7f650fe00700 1 -- 172.21.15.79:0/700648831 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6528081070 con 0x559600cb3570 2024-06-03T02:35:20.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.685+0000 7f650fe00700 1 -- 172.21.15.79:0/700648831 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528087cc0 con 0x559600cb3570 2024-06-03T02:35:20.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.686+0000 7f6538a90b80 1 -- 172.21.15.79:0/700648831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 msgr2=0x559600ccaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:20.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.686+0000 7f6538a90b80 1 --2- 172.21.15.79:0/700648831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 0x559600ccaab0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f652807bb40 tx=0x7f6528068aa0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.687 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:20 smithi079 ceph-mon[31819]: pgmap v974: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 607 op/s 2024-06-03T02:35:20.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.686+0000 7f6538a90b80 1 -- 172.21.15.79:0/700648831 shutdown_connections 2024-06-03T02:35:20.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.686+0000 7f6538a90b80 1 --2- 172.21.15.79:0/700648831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 0x559600ccaab0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.686+0000 7f6538a90b80 1 --2- 172.21.15.79:0/700648831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600caf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.686+0000 7f6538a90b80 1 --2- 172.21.15.79:0/700648831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600caff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.686+0000 7f6538a90b80 1 -- 172.21.15.79:0/700648831 >> 172.21.15.79:0/700648831 conn(0x559600bd5170 msgr2=0x559600cab510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:20.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.688+0000 7f6538a90b80 1 -- 172.21.15.79:0/700648831 shutdown_connections 2024-06-03T02:35:20.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.688+0000 7f6538a90b80 1 -- 172.21.15.79:0/700648831 wait complete. 2024-06-03T02:35:20.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.688+0000 7f6538a90b80 1 -- start start 2024-06-03T02:35:20.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.689+0000 7f6538a90b80 1 Processor -- start 2024-06-03T02:35:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.690+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600cb6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.691+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600cb6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.691+0000 7f6538a90b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 0x559600cb6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.691+0000 7f6538a90b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6508010110 con 0x559600c24e40 2024-06-03T02:35:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.691+0000 7f6538a90b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f650800fa40 con 0x559600caa960 2024-06-03T02:35:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.691+0000 7f6538a90b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f650800cc00 con 0x559600cb3570 2024-06-03T02:35:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.691+0000 7f6538a90b80 10 client.0 _ll_register_callbacks cb 0x559600b94720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T02:35:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.691+0000 7f6526a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600cb6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.692+0000 7f6526a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600cb6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:38432/0 (socket says 172.21.15.79:38432) 2024-06-03T02:35:20.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.692+0000 7f6527e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 0x559600cb6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.693+0000 7f6526a00700 1 -- 172.21.15.79:0/342945531 learned_addr learned my addr 172.21.15.79:0/342945531 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:20.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.692+0000 7f6527400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600cb6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.694+0000 7f6526a00700 1 -- 172.21.15.79:0/342945531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 msgr2=0x559600cb6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:20.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.694+0000 7f6526a00700 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 0x559600cb6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.694+0000 7f6526a00700 1 -- 172.21.15.79:0/342945531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 msgr2=0x559600cb6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T02:35:20.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.694+0000 7f6526a00700 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600cb6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.694+0000 7f6526a00700 1 -- 172.21.15.79:0/342945531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559600d2bc80 con 0x559600caa960 2024-06-03T02:35:20.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.695+0000 7f6526a00700 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600cb6a70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6514009750 tx=0x7f6514004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b741ebfcb4d3774d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:20.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.695+0000 7f650d600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-03T02:35:20.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.696+0000 7f650d600700 1 -- 172.21.15.79:0/342945531 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514026020 con 0x559600caa960 2024-06-03T02:35:20.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.697+0000 7f650d600700 1 -- 172.21.15.79:0/342945531 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f6514015be0 con 0x559600caa960 2024-06-03T02:35:20.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.697+0000 7f6538a90b80 1 -- 172.21.15.79:0/342945531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6528081070 con 0x559600caa960 2024-06-03T02:35:20.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.697+0000 7f650d600700 1 -- 172.21.15.79:0/342945531 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6514031050 con 0x559600caa960 2024-06-03T02:35:20.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.697+0000 7f6538a90b80 10 client.27032 fetch_fsmap learned FSMap version 30 2024-06-03T02:35:20.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.697+0000 7f6538a90b80 1 -- 172.21.15.79:0/342945531 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559600d2cfc0 con 0x559600caa960 2024-06-03T02:35:20.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.698+0000 7f650d600700 1 -- 172.21.15.79:0/342945531 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6528081070 con 0x559600caa960 2024-06-03T02:35:20.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.698+0000 7f650d600700 1 -- 172.21.15.79:0/342945531 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6514022440 con 0x559600caa960 2024-06-03T02:35:20.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.698+0000 7f6538a90b80 10 client.27032 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T02:35:20.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.698+0000 7f6538a90b80 10 client.27032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-03T02:35:20.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.698+0000 7f6538a90b80 20 client.27032 populate_metadata read hostname 'smithi079' 2024-06-03T02:35:20.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.698+0000 7f6538a90b80 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559600d2d700 0x559600d2fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:20.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.698+0000 7f6538a90b80 4 client.27032 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T02:35:20.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.698+0000 7f6538a90b80 1 -- 172.21.15.79:0/342945531 --> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559600c02580 con 0x559600d2d700 2024-06-03T02:35:20.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.699+0000 7f6527400700 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559600d2d700 0x559600d2fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:20.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6527400700 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559600d2d700 0x559600d2fb20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc05eb47611f6dd1 server_cookie=45ca0b04900bcafc in_seq=0 out_seq=0 2024-06-03T02:35:20.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f650d600700 10 client.27032 ms_handle_connect on v2:172.21.15.123:6838/1178082108 2024-06-03T02:35:20.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f650d600700 1 -- 172.21.15.79:0/342945531 <== mds.0 v2:172.21.15.123:6838/1178082108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559600c02580 con 0x559600d2d700 2024-06-03T02:35:20.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f650d600700 10 client.27032 handle_command_reply: tid=0 2024-06-03T02:35:20.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6538a90b80 1 client.27032 shutdown 2024-06-03T02:35:20.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6538a90b80 1 -- 172.21.15.79:0/342945531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 msgr2=0x559600cb6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:20.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6538a90b80 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600cb6a70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6514009750 tx=0x7f6514004770 comp rx=0 tx=0).stop 2024-06-03T02:35:20.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6538a90b80 1 -- 172.21.15.79:0/342945531 shutdown_connections 2024-06-03T02:35:20.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6538a90b80 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559600cb3570 0x559600cb6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6538a90b80 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.123:6838/1178082108,v1:172.21.15.123:6839/1178082108] conn(0x559600d2d700 0x559600d2fb20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6538a90b80 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559600c24e40 0x559600cb6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6538a90b80 1 --2- 172.21.15.79:0/342945531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559600caa960 0x559600cb6a70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f650d600700 0 client.27032 ms_handle_reset on v2:172.21.15.123:6838/1178082108 2024-06-03T02:35:20.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.700+0000 7f6538a90b80 1 -- 172.21.15.79:0/342945531 >> 172.21.15.79:0/342945531 conn(0x559600bd5170 msgr2=0x559600cafec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:20.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.701+0000 7f6538a90b80 1 -- 172.21.15.79:0/342945531 shutdown_connections 2024-06-03T02:35:20.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.701+0000 7f6538a90b80 1 -- 172.21.15.79:0/342945531 wait complete. 2024-06-03T02:35:20.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.701+0000 7f6538a90b80 20 client.27032 trim_cache size 0 max 16384 2024-06-03T02:35:20.703 INFO:teuthology.orchestra.run.smithi079.stdout:[] 2024-06-03T02:35:20.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.703+0000 7f652cc00700 1 -- 172.21.15.79:0/4211591176 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65100a2ca0 msgr2=0x7f65100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:20.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.703+0000 7f652cc00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65100a2ca0 0x7f65100a5160 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f652013cc10 tx=0x7f6514009190 comp rx=0 tx=0).stop 2024-06-03T02:35:20.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.703+0000 7f652cc00700 1 -- 172.21.15.79:0/4211591176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6520094220 msgr2=0x7f652013b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:20.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.703+0000 7f652cc00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6520094220 0x7f652013b4a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f651c0098e0 tx=0x7f651c005590 comp rx=0 tx=0).stop 2024-06-03T02:35:20.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.704+0000 7f652cc00700 1 -- 172.21.15.79:0/4211591176 shutdown_connections 2024-06-03T02:35:20.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.704+0000 7f652cc00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f65100a2ca0 0x7f65100a5160 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.704+0000 7f652cc00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f652013bf20 0x7f6520150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.704+0000 7f652cc00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6520094220 0x7f652013b4a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.704+0000 7f652cc00700 1 --2- 172.21.15.79:0/4211591176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6520094be0 0x7f652013b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:20.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.704+0000 7f652cc00700 1 -- 172.21.15.79:0/4211591176 >> 172.21.15.79:0/4211591176 conn(0x7f6520009ae0 msgr2=0x7f6520093210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:20.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.706+0000 7f652cc00700 1 -- 172.21.15.79:0/4211591176 shutdown_connections 2024-06-03T02:35:20.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:20.706+0000 7f652cc00700 1 -- 172.21.15.79:0/4211591176 wait complete. 2024-06-03T02:35:20.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:20 smithi123 ceph-mon[36640]: pgmap v974: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 607 op/s 2024-06-03T02:35:20.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:20 smithi135 ceph-mon[36653]: pgmap v974: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 607 op/s 2024-06-03T02:35:22.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:22 smithi123 ceph-mon[36640]: pgmap v975: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 666 op/s 2024-06-03T02:35:22.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:22 smithi079 ceph-mon[31819]: pgmap v975: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 666 op/s 2024-06-03T02:35:22.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:22 smithi135 ceph-mon[36653]: pgmap v975: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 666 op/s 2024-06-03T02:35:23.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-06-03T02:35:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/: 2024-06-03T02:35:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:35:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 .. 2024-06-03T02:35:23.623 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 data 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data: 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:total 36 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 . 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 .. 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 3 02:20 base 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Jun 3 02:19 current_logfiles 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jun 3 02:19 global 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 log 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_commit_ts 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_dynshmem 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Jun 3 02:19 pg_hba.conf 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jun 3 02:19 pg_ident.conf 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:35 pg_logical 2024-06-03T02:35:23.624 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 pg_multixact 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_notify 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_replslot 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_serial 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_snapshots 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:35 pg_stat 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:35 pg_stat_tmp 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:35 pg_subtrans 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_tblspc 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 pg_twophase 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 PG_VERSION 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 56 Jun 3 02:35 pg_wal 2024-06-03T02:35:23.625 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 pg_xact 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jun 3 02:19 postgresql.auto.conf 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Jun 3 02:19 postgresql.conf 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Jun 3 02:19 postmaster.opts 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/base: 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 3 02:20 . 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 1 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 13493 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Jun 3 02:26 13494 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:20 pgsql_tmp 2024-06-03T02:35:23.626 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/base/1: 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:total 7777 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 . 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 3 02:20 .. 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 112 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 113 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 1247 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1247_fsm 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1247_vm 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 3 02:19 1249 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1249_fsm 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1249_vm 2024-06-03T02:35:23.627 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 3 02:19 1255 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1255_fsm 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1255_vm 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 3 02:19 1259 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1259_fsm 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1259_vm 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 3 02:19 13344 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13344_fsm 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13344_vm 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13346 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13348 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13349_fsm 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349_vm 2024-06-03T02:35:23.628 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13351 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13353 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13354_fsm 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354_vm 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13356 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13358 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13359_fsm 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359_vm 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13361 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13363 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1417 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1418 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 174 2024-06-03T02:35:23.629 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 175 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2187 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2224 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2328 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2336 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2337 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2579 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2600 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2600_fsm 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2600_vm 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2601_fsm 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601_vm 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2602 2024-06-03T02:35:23.630 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2602_fsm 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2602_vm 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2603 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2603_fsm 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2603_vm 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2604 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2605 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2605_fsm 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2605_vm 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2606_fsm 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606_vm 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2607 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2607_fsm 2024-06-03T02:35:23.631 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2607_vm 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 3 02:19 2608 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2608_fsm 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2608_vm 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 3 02:19 2609 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2609_fsm 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2609_vm 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2610 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2610_fsm 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2610_vm 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2611 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2612_fsm 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612_vm 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2613 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615 2024-06-03T02:35:23.632 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2615_fsm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615_vm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2616 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2616_fsm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2616_vm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 3 02:19 2617 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2617_fsm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2617_vm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 3 02:19 2618 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2618_fsm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2618_vm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 3 02:19 2619 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2619_fsm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2619_vm 2024-06-03T02:35:23.633 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2620 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2650 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2651 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2652 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2653 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2654 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2655 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2656 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2657 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 3 02:19 2658 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 2659 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2660 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2661 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2662 2024-06-03T02:35:23.634 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2663 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2664 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2665 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2666 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2667 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2668 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2669 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2670 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 3 02:19 2673 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 3 02:19 2674 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 3 02:19 2675 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2678 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2679 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2680 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2681 2024-06-03T02:35:23.635 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2682 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2683 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2684 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2685 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2686 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2687 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2688 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2689 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 3 02:19 2690 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 3 02:19 2691 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2692 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2693 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2696 2024-06-03T02:35:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2699 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2701 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2702 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2703 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2704 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2753 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2753_fsm 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2753_vm 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2754 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2755 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2756 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2757 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2830 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2831 2024-06-03T02:35:23.637 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2832 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2833 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2834 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2835 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2836 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2837 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 3 02:19 2838 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2838_fsm 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2838_vm 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2839 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840_fsm 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2840_vm 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2841 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2995 2024-06-03T02:35:23.638 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2996 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3079_fsm 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079_vm 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3080 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3081 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3085 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3118 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3119 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 3 02:19 3164 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3256 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3257 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3258 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3350 2024-06-03T02:35:23.639 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3351 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3379 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3380 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3381 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3394 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3394_fsm 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3394_vm 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3395 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3429 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3430 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3431 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3433 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3439 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3440 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3455 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 3 02:19 3456 2024-06-03T02:35:23.640 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3456_fsm 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3456_vm 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3466 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3467 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3468 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3501 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3502 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3503 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3534 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3541_fsm 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541_vm 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3542 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3574 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3575 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3576 2024-06-03T02:35:23.641 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3596 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3597 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3598 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3599 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3600_fsm 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600_vm 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3601_fsm 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601_vm 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3602_fsm 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602_vm 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603_fsm 2024-06-03T02:35:23.642 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3603_vm 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3604 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3605 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3606 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3607 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3608 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 3609 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3712 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3764_fsm 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764_vm 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3766 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3767 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3997 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4143 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4144 2024-06-03T02:35:23.643 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4145 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4146 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4147 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4148 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4149 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4150 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4151 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4152 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4153 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4154 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4155 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4156 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4157 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4158 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4159 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4160 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4161 2024-06-03T02:35:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4162 2024-06-03T02:35:23.646 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4163 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4164 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4165 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4166 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4167 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4168 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4169 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4170 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4171 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4172 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4173 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4174 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 5002 2024-06-03T02:35:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 548 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 549 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6102 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6104 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6106 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6110 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6111 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6112 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6113 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6117 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 826 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 827 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 828 2024-06-03T02:35:23.648 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 3 02:19 pg_filenode.map 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 PG_VERSION 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:total 7777 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 3 02:19 . 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 3 02:20 .. 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 112 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 113 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 1247 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1247_fsm 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1247_vm 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 3 02:19 1249 2024-06-03T02:35:23.649 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1249_fsm 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1249_vm 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 3 02:19 1255 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1255_fsm 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1255_vm 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 3 02:19 1259 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1259_fsm 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1259_vm 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 3 02:19 13344 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13344_fsm 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13344_vm 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13346 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13348 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349 2024-06-03T02:35:23.650 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13349_fsm 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349_vm 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13351 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13353 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13354_fsm 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354_vm 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13356 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13358 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13359_fsm 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359_vm 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13361 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13363 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1417 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1418 2024-06-03T02:35:23.651 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 174 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 175 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2187 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2224 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2328 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2336 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2337 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2579 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2600 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2600_fsm 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2600_vm 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2601_fsm 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601_vm 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2602 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2602_fsm 2024-06-03T02:35:23.652 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2602_vm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2603 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2603_fsm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2603_vm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2604 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2605 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2605_fsm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2605_vm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2606_fsm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2606_vm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2607 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2607_fsm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2607_vm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 3 02:19 2608 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2608_fsm 2024-06-03T02:35:23.653 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2608_vm 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 3 02:19 2609 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2609_fsm 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2609_vm 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2610 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2610_fsm 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2610_vm 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2611 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2612_fsm 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612_vm 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2613 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2615_fsm 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615_vm 2024-06-03T02:35:23.654 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2616 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2616_fsm 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2616_vm 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 3 02:19 2617 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2617_fsm 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2617_vm 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 3 02:19 2618 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2618_fsm 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2618_vm 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 3 02:19 2619 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2619_fsm 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2619_vm 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2620 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2650 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2651 2024-06-03T02:35:23.655 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2652 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2653 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2654 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2655 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2656 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2657 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 3 02:19 2658 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:19 2659 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2660 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2661 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2662 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2663 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2664 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2665 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2666 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2667 2024-06-03T02:35:23.656 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2668 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2669 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2670 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 3 02:19 2673 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 3 02:19 2674 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 3 02:19 2675 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2678 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2679 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2680 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2681 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2682 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2683 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2684 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2685 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2686 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2687 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2688 2024-06-03T02:35:23.657 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2689 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 3 02:19 2690 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 3 02:19 2691 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2692 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2693 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2696 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2699 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2701 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2702 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2703 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2704 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2753 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2753_fsm 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2753_vm 2024-06-03T02:35:23.658 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2754 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2755 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2756 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2757 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2830 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2831 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2832 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2833 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2834 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2835 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2836 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2837 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 3 02:19 2838 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2838_fsm 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2838_vm 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2839 2024-06-03T02:35:23.659 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840_fsm 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2840_vm 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2841 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2995 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2996 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3079_fsm 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079_vm 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3080 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3081 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3085 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3118 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3119 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 3 02:19 3164 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3256 2024-06-03T02:35:23.660 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3257 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3258 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3350 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3351 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3379 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3380 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3381 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3394 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3394_fsm 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3394_vm 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3395 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3429 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3430 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3431 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3433 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3439 2024-06-03T02:35:23.661 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3440 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3455 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 3 02:19 3456 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3456_fsm 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3456_vm 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3466 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3467 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3468 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3501 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3502 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3503 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3534 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3541_fsm 2024-06-03T02:35:23.662 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541_vm 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3542 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3574 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3575 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3576 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3596 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3597 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3598 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3599 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3600_fsm 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600_vm 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3601_fsm 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601_vm 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602 2024-06-03T02:35:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3602_fsm 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602_vm 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603_fsm 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3603_vm 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3604 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3605 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3606 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3607 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3608 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 3609 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3712 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3764_fsm 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764_vm 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3766 2024-06-03T02:35:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3767 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3997 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4143 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4144 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4145 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4146 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4147 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4148 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4149 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4150 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4151 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4152 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4153 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4154 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4155 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4156 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4157 2024-06-03T02:35:23.665 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4158 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4159 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4160 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4161 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4162 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4163 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4164 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4165 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4166 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4167 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4168 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4169 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4170 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4171 2024-06-03T02:35:23.666 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4172 2024-06-03T02:35:23.670 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4173 2024-06-03T02:35:23.670 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4174 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 5002 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 548 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 549 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6102 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6104 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6106 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6110 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6111 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6112 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6113 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6117 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 826 2024-06-03T02:35:23.671 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 827 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 828 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 3 02:19 pg_filenode.map 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 PG_VERSION 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:total 545782 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Jun 3 02:26 . 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 3 02:20 .. 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 112 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 113 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:20 1247 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1247_fsm 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:20 1247_vm 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 3 02:20 1249 2024-06-03T02:35:23.672 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:20 1249_fsm 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:20 1249_vm 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 3 02:19 1255 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1255_fsm 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1255_vm 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 3 02:35 1259 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1259_fsm 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:20 1259_vm 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 3 02:19 13344 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13344_fsm 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13344_vm 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13346 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13348 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13349_fsm 2024-06-03T02:35:23.673 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13349_vm 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13351 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13353 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13354_fsm 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13354_vm 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13356 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13358 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 13359_fsm 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13359_vm 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 13361 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 13363 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1417 2024-06-03T02:35:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 1418 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 439902208 Jun 3 02:35 16396 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jun 3 02:35 16396_fsm 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:28 16396_vm 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Jun 3 02:35 16397 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:35 16397_fsm 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:35 16397_vm 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 507904 Jun 3 02:35 16399 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:35 16399_fsm 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:33 16399_vm 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:35 16400 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:35 16402 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Jun 3 02:35 16404 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 37650432 Jun 3 02:35 16406 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:35 16406_fsm 2024-06-03T02:35:23.675 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:31 16406_vm 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 174 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 175 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2187 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2224 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2328 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2336 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2337 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:20 2579 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2600 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2600_fsm 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2600_vm 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2601_fsm 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2601_vm 2024-06-03T02:35:23.676 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2602 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2602_fsm 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2602_vm 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2603 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2603_fsm 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2603_vm 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2604 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2605 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2605_fsm 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2605_vm 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:20 2606 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2606_fsm 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:20 2606_vm 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2607 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2607_fsm 2024-06-03T02:35:23.677 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2607_vm 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 3 02:20 2608 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2608_fsm 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:20 2608_vm 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 3 02:19 2609 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2609_fsm 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2609_vm 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:20 2610 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2610_fsm 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:20 2610_vm 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2611 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2612_fsm 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2612_vm 2024-06-03T02:35:23.678 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2613 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2615_fsm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2615_vm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2616 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2616_fsm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2616_vm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 3 02:19 2617 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2617_fsm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2617_vm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 3 02:19 2618 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2618_fsm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2618_vm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Jun 3 02:35 2619 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:23 2619_fsm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:20 2619_vm 2024-06-03T02:35:23.679 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2620 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2650 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2651 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2652 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2653 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2654 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2655 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2656 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2657 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 3 02:20 2658 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 3 02:20 2659 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2660 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2661 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:20 2662 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:20 2663 2024-06-03T02:35:23.680 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:20 2664 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:20 2665 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:20 2666 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:20 2667 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2668 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2669 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2670 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 3 02:20 2673 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 3 02:20 2674 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 3 02:19 2675 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:20 2678 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:20 2679 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2680 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2681 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2682 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2683 2024-06-03T02:35:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2684 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2685 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2686 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2687 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 2688 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 3 02:19 2689 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 3 02:19 2690 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 3 02:19 2691 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2692 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2693 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:24 2696 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2699 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2701 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2702 2024-06-03T02:35:23.682 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:20 2703 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:20 2704 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2753 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2753_fsm 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2753_vm 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2754 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2755 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2756 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 2757 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2830 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2831 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2832 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2833 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2834 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2835 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2836 2024-06-03T02:35:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2837 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 3 02:19 2838 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2838_fsm 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2838_vm 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2839 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2840_fsm 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2840_vm 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2841 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2995 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2996 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3079_fsm 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3079_vm 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3080 2024-06-03T02:35:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3081 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:19 3085 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3118 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3119 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 3 02:19 3164 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3256 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3257 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3258 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3350 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3351 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3379 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3380 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3381 2024-06-03T02:35:23.685 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3394 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3394_fsm 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3394_vm 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3395 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3429 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3430 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3431 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3433 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3439 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3440 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 3 02:20 3455 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 3 02:19 3456 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3456_fsm 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3456_vm 2024-06-03T02:35:23.686 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3466 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3467 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3468 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3501 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3502 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3503 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3534 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3541_fsm 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3541_vm 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3542 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3574 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3575 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3576 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3596 2024-06-03T02:35:23.687 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3597 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3598 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3599 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3600_fsm 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3600_vm 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3601_fsm 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3601_vm 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3602_fsm 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3602_vm 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3603_fsm 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3603_vm 2024-06-03T02:35:23.688 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3604 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3605 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3606 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3607 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3608 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 3 02:19 3609 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3712 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 3764_fsm 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3764_vm 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3766 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 3767 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3997 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4143 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4144 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4145 2024-06-03T02:35:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4146 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4147 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4148 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4149 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4150 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4151 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4152 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4153 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4154 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4155 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4156 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4157 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4158 2024-06-03T02:35:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4159 2024-06-03T02:35:23.691 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4160 2024-06-03T02:35:23.691 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4161 2024-06-03T02:35:23.691 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4162 2024-06-03T02:35:23.691 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4163 2024-06-03T02:35:23.691 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4164 2024-06-03T02:35:23.691 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4165 2024-06-03T02:35:23.691 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4166 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4167 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4168 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4169 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4170 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4171 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4172 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4173 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4174 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 5002 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 548 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 549 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6102 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6104 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6106 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6110 2024-06-03T02:35:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6111 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6112 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6113 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6117 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 826 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 827 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 828 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 3 02:19 pg_filenode.map 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Jun 3 02:19 pg_internal.init 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:19 PG_VERSION 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:20 . 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 3 02:20 .. 2024-06-03T02:35:23.693 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/global: 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:total 560 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jun 3 02:19 . 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1213 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1213_fsm 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1213_vm 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1214 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1214_fsm 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1214_vm 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 1232 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 1233 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1260 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1260_fsm 2024-06-03T02:35:23.694 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1260_vm 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1261 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1261_fsm 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1261_vm 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1262 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 1262_fsm 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 1262_vm 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2396 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 3 02:19 2396_fsm 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2396_vm 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2397 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2671 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2672 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2676 2024-06-03T02:35:23.695 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2677 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2694 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2695 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2697 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 3 02:19 2698 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2846 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2847 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2964 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2965 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 2966 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 2967 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 3592 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 3593 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4060 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4061 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4175 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4176 2024-06-03T02:35:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4177 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4178 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4181 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4182 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4183 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4184 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 4185 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 4186 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6000 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6001 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6002 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 6100 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6114 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 6115 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:35 pg_control 2024-06-03T02:35:23.697 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 3 02:19 pg_filenode.map 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Jun 3 02:19 pg_internal.init 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/log: 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:total 2 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 Jun 3 02:35 postgresql-Mon.log 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.698 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:total 1 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:35 . 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 mappings 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 3 02:35 replorigin_checkpoint 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 snapshots 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.699 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:35 .. 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:35 .. 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 . 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 members 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 offsets 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-06-03T02:35:23.700 INFO:teuthology.orchestra.run.smithi079.stdout:total 8 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 .. 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:19 0000 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:total 8 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 3 02:19 .. 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 3 02:24 0000 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.701 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-06-03T02:35:23.702 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:total 15 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 3 02:35 . 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Jun 3 02:35 db_0.stat 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Jun 3 02:35 db_13494.stat 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Jun 3 02:35 global.stat 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:35 . 2024-06-03T02:35:23.703 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:total 256 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:35 . 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Jun 3 02:35 000A 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.704 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:total 901120 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 56 Jun 3 02:35 . 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:35 000000010000000000000063 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 000000010000000000000064 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:28 000000010000000000000065 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 000000010000000000000066 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:27 000000010000000000000067 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:26 000000010000000000000068 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 000000010000000000000069 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:29 00000001000000000000006A 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:29 00000001000000000000006B 2024-06-03T02:35:23.705 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:26 00000001000000000000006C 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:26 00000001000000000000006D 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:28 00000001000000000000006E 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:27 00000001000000000000006F 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:28 000000010000000000000070 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 000000010000000000000071 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 000000010000000000000072 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 000000010000000000000073 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:26 000000010000000000000074 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 000000010000000000000075 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 000000010000000000000076 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:26 000000010000000000000077 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:26 000000010000000000000078 2024-06-03T02:35:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 000000010000000000000079 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:27 00000001000000000000007A 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:26 00000001000000000000007B 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 00000001000000000000007C 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:25 00000001000000000000007D 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 00000001000000000000007E 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:31 00000001000000000000007F 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 000000010000000000000080 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:31 000000010000000000000081 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:32 000000010000000000000082 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:32 000000010000000000000083 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 000000010000000000000084 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:31 000000010000000000000085 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:33 000000010000000000000086 2024-06-03T02:35:23.707 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:31 000000010000000000000087 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:34 000000010000000000000088 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:31 000000010000000000000089 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:33 00000001000000000000008A 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 00000001000000000000008B 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:32 00000001000000000000008C 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 00000001000000000000008D 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:32 00000001000000000000008E 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:33 00000001000000000000008F 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:29 000000010000000000000090 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 000000010000000000000091 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 000000010000000000000092 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:31 000000010000000000000093 2024-06-03T02:35:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 000000010000000000000094 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:31 000000010000000000000095 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 000000010000000000000096 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 000000010000000000000097 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:30 000000010000000000000098 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 3 02:34 000000010000000000000099 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 archive_status 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:total 0 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 3 02:19 . 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 56 Jun 3 02:35 .. 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:total 176 2024-06-03T02:35:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 3 02:19 . 2024-06-03T02:35:23.710 INFO:teuthology.orchestra.run.smithi079.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 3 02:35 .. 2024-06-03T02:35:23.710 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 3 02:35 0000 2024-06-03T02:35:23.711 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-03T02:35:23.721 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-03T02:35:23.721 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-03T02:35:23.722 INFO:tasks.fwd_scrub:done joining 2024-06-03T02:35:23.722 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-03T02:35:23.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-03T02:35:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.142+0000 7f3f1f200640 1 -- 172.21.15.79:0/2018595208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f201000f0 msgr2=0x7f3f20100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.142+0000 7f3f1f200640 1 --2- 172.21.15.79:0/2018595208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f201000f0 0x7f3f20100510 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009860 tx=0x7f3f1002a340 comp rx=0 tx=0).stop 2024-06-03T02:35:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.142+0000 7f3f1f200640 1 -- 172.21.15.79:0/2018595208 shutdown_connections 2024-06-03T02:35:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.142+0000 7f3f1f200640 1 --2- 172.21.15.79:0/2018595208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f201013f0 0x7f3f20101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.142+0000 7f3f1f200640 1 --2- 172.21.15.79:0/2018595208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f20100a50 0x7f3f20100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.142+0000 7f3f1f200640 1 --2- 172.21.15.79:0/2018595208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f201000f0 0x7f3f20100510 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.142+0000 7f3f1f200640 1 -- 172.21.15.79:0/2018595208 >> 172.21.15.79:0/2018595208 conn(0x7f3f200fbc70 msgr2=0x7f3f200fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:24.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.142+0000 7f3f1f200640 1 -- 172.21.15.79:0/2018595208 shutdown_connections 2024-06-03T02:35:24.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.142+0000 7f3f1f200640 1 -- 172.21.15.79:0/2018595208 wait complete. 2024-06-03T02:35:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.143+0000 7f3f1f200640 1 Processor -- start 2024-06-03T02:35:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.143+0000 7f3f1f200640 1 -- start start 2024-06-03T02:35:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f201000f0 0x7f3f20199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f20100a50 0x7f3f2019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f201013f0 0x7f3f201a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1f200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2010f680 con 0x7f3f20100a50 2024-06-03T02:35:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2010f500 con 0x7f3f201000f0 2024-06-03T02:35:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1f200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2010f800 con 0x7f3f201013f0 2024-06-03T02:35:24.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f20100a50 0x7f3f2019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f201000f0 0x7f3f20199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f20100a50 0x7f3f2019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38748/0 (socket says 172.21.15.79:38748) 2024-06-03T02:35:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1d400640 1 -- 172.21.15.79:0/1200794478 learned_addr learned my addr 172.21.15.79:0/1200794478 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1e800640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f201013f0 0x7f3f201a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1de00640 1 -- 172.21.15.79:0/1200794478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f201013f0 msgr2=0x7f3f201a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.144+0000 7f3f1de00640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f201013f0 0x7f3f201a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.145+0000 7f3f1de00640 1 -- 172.21.15.79:0/1200794478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f20100a50 msgr2=0x7f3f2019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.145+0000 7f3f1de00640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f20100a50 0x7f3f2019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.145+0000 7f3f1de00640 1 -- 172.21.15.79:0/1200794478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f201a1930 con 0x7f3f201000f0 2024-06-03T02:35:24.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.145+0000 7f3f1de00640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f201000f0 0x7f3f20199d20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3f10035040 tx=0x7f3f10033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:24.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.145+0000 7f3f16a00640 1 -- 172.21.15.79:0/1200794478 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037040 con 0x7f3f201000f0 2024-06-03T02:35:24.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.145+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f201a1bc0 con 0x7f3f201000f0 2024-06-03T02:35:24.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.145+0000 7f3f16a00640 1 -- 172.21.15.79:0/1200794478 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f10004040 con 0x7f3f201000f0 2024-06-03T02:35:24.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.145+0000 7f3f16a00640 1 -- 172.21.15.79:0/1200794478 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f100055f0 con 0x7f3f201000f0 2024-06-03T02:35:24.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.145+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f201a2100 con 0x7f3f201000f0 2024-06-03T02:35:24.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.146+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2010b020 con 0x7f3f201000f0 2024-06-03T02:35:24.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.148+0000 7f3f16a00640 1 -- 172.21.15.79:0/1200794478 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3f10042020 con 0x7f3f201000f0 2024-06-03T02:35:24.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.150+0000 7f3f16a00640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3f040a2c10 0x7f3f040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:24.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.150+0000 7f3f16a00640 1 -- 172.21.15.79:0/1200794478 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f3f100eef60 con 0x7f3f201000f0 2024-06-03T02:35:24.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.150+0000 7f3f1d400640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3f040a2c10 0x7f3f040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:24.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.151+0000 7f3f1d400640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3f040a2c10 0x7f3f040a50d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009cd0 tx=0x7f3f0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:24.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.154+0000 7f3f16a00640 1 -- 172.21.15.79:0/1200794478 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3f100bbab0 con 0x7f3f201000f0 2024-06-03T02:35:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.361+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f200625d0 con 0x7f3f201000f0 2024-06-03T02:35:24.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.364+0000 7f3f16a00640 1 -- 172.21.15.79:0/1200794478 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+24805 (secure 0 0 0) 0x7f3f100bb3b0 con 0x7f3f201000f0 2024-06-03T02:35:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:24.365 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":101,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:15:54.257434+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:35:24.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3f040a2c10 msgr2=0x7f3f040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3f040a2c10 0x7f3f040a50d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009cd0 tx=0x7f3f0c005f90 comp rx=0 tx=0).stop 2024-06-03T02:35:24.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f201000f0 msgr2=0x7f3f20199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f201000f0 0x7f3f20199d20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3f10035040 tx=0x7f3f10033d90 comp rx=0 tx=0).stop 2024-06-03T02:35:24.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 shutdown_connections 2024-06-03T02:35:24.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f3f040a2c10 0x7f3f040a50d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f201013f0 0x7f3f201a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3f20100a50 0x7f3f2019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 --2- 172.21.15.79:0/1200794478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f201000f0 0x7f3f20199d20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.366+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 >> 172.21.15.79:0/1200794478 conn(0x7f3f200fbc70 msgr2=0x7f3f200fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:24.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.367+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 shutdown_connections 2024-06-03T02:35:24.368 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.367+0000 7f3f1f200640 1 -- 172.21.15.79:0/1200794478 wait complete. 2024-06-03T02:35:24.396 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-03T02:08:38.699457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-03T02:14:03.801578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-03T02:14:08.648682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-03T02:14:21.759252+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-03T02:35:24.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-03T02:35:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:24 smithi079 ceph-mon[31819]: pgmap v976: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 MiB/s wr, 668 op/s 2024-06-03T02:35:24.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:24 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1200794478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:35:24.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.605+0000 7f08bc800640 1 -- 172.21.15.79:0/1707122596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08b41057b0 msgr2=0x7f08b410c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.605+0000 7f08bc800640 1 --2- 172.21.15.79:0/1707122596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08b41057b0 0x7f08b410c040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009ca0 tx=0x7f08b002a8c0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.605+0000 7f08bc800640 1 -- 172.21.15.79:0/1707122596 shutdown_connections 2024-06-03T02:35:24.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.605+0000 7f08bc800640 1 --2- 172.21.15.79:0/1707122596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08b41057b0 0x7f08b410c040 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.605+0000 7f08bc800640 1 --2- 172.21.15.79:0/1707122596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08b4104df0 0x7f08b4105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.605+0000 7f08bc800640 1 --2- 172.21.15.79:0/1707122596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08b4103c00 0x7f08b4104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.605+0000 7f08bc800640 1 -- 172.21.15.79:0/1707122596 >> 172.21.15.79:0/1707122596 conn(0x7f08b40ff190 msgr2=0x7f08b41015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:24.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.606+0000 7f08bc800640 1 -- 172.21.15.79:0/1707122596 shutdown_connections 2024-06-03T02:35:24.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.606+0000 7f08bc800640 1 -- 172.21.15.79:0/1707122596 wait complete. 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.606+0000 7f08bc800640 1 Processor -- start 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08bc800640 1 -- start start 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08bc800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08b4103c00 0x7f08b4199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08bc800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08b4104df0 0x7f08b419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08bc800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08b41057b0 0x7f08b41a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08bc800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08b410f880 con 0x7f08b41057b0 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08bc800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08b410f700 con 0x7f08b4103c00 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08bc800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08b410fa00 con 0x7f08b4104df0 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08ba800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08b41057b0 0x7f08b41a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08b9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08b4103c00 0x7f08b4199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:24.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08ba800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08b41057b0 0x7f08b41a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38758/0 (socket says 172.21.15.79:38758) 2024-06-03T02:35:24.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08ba800640 1 -- 172.21.15.79:0/1333476368 learned_addr learned my addr 172.21.15.79:0/1333476368 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:24.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.607+0000 7f08b9400640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08b4104df0 0x7f08b419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08b9e00640 1 -- 172.21.15.79:0/1333476368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08b4104df0 msgr2=0x7f08b419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08b9e00640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08b4104df0 0x7f08b419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08b9e00640 1 -- 172.21.15.79:0/1333476368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08b41057b0 msgr2=0x7f08b41a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08b9e00640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08b41057b0 0x7f08b41a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08b9e00640 1 -- 172.21.15.79:0/1333476368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b41a1af0 con 0x7f08b4103c00 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08b9e00640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08b4103c00 0x7f08b4199ed0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f08b4104c50 tx=0x7f08a800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08aea00640 1 -- 172.21.15.79:0/1333476368 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8017070 con 0x7f08b4103c00 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b41a1de0 con 0x7f08b4103c00 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08aea00640 1 -- 172.21.15.79:0/1333476368 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a8009c30 con 0x7f08b4103c00 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.608+0000 7f08aea00640 1 -- 172.21.15.79:0/1333476368 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a800f620 con 0x7f08b4103c00 2024-06-03T02:35:24.610 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.609+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b41a22a0 con 0x7f08b4103c00 2024-06-03T02:35:24.611 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.609+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b410af20 con 0x7f08b4103c00 2024-06-03T02:35:24.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.612+0000 7f08aea00640 1 -- 172.21.15.79:0/1333476368 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f08a801a070 con 0x7f08b4103c00 2024-06-03T02:35:24.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.613+0000 7f08aea00640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f089c0a2b90 0x7f089c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:24.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.613+0000 7f08aea00640 1 -- 172.21.15.79:0/1333476368 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f08a80cf210 con 0x7f08b4103c00 2024-06-03T02:35:24.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.613+0000 7f08b9400640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f089c0a2b90 0x7f089c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:24.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.614+0000 7f08b9400640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f089c0a2b90 0x7f089c0a5050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f08b419b3e0 tx=0x7f08a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:24.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.615+0000 7f08aea00640 1 -- 172.21.15.79:0/1333476368 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f08a809b710 con 0x7f08b4103c00 2024-06-03T02:35:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:24 smithi123 ceph-mon[36640]: pgmap v976: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 MiB/s wr, 668 op/s 2024-06-03T02:35:24.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:24 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1200794478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:35:24.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.815+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08b402d2e0 con 0x7f08b4103c00 2024-06-03T02:35:24.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.816+0000 7f08aea00640 1 -- 172.21.15.79:0/1333476368 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f08a809fa10 con 0x7f08b4103c00 2024-06-03T02:35:24.817 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f089c0a2b90 msgr2=0x7f089c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f089c0a2b90 0x7f089c0a5050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f08b419b3e0 tx=0x7f08a400b390 comp rx=0 tx=0).stop 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08b4103c00 msgr2=0x7f08b4199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08b4103c00 0x7f08b4199ed0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f08b4104c50 tx=0x7f08a800dde0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 shutdown_connections 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08b41057b0 0x7f08b41a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f089c0a2b90 0x7f089c0a5050 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08b4104df0 0x7f08b419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 --2- 172.21.15.79:0/1333476368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08b4103c00 0x7f08b4199ed0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 >> 172.21.15.79:0/1333476368 conn(0x7f08b40ff190 msgr2=0x7f08b41080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 shutdown_connections 2024-06-03T02:35:24.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:24.819+0000 7f08bc800640 1 -- 172.21.15.79:0/1333476368 wait complete. 2024-06-03T02:35:24.839 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-03T02:35:24.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:24 smithi135 ceph-mon[36653]: pgmap v976: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 MiB/s wr, 668 op/s 2024-06-03T02:35:24.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:24 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1200794478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:35:25.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.034+0000 7f2dfae00640 1 -- 172.21.15.79:0/1042708865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4109a30 msgr2=0x7f2df4110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.034+0000 7f2dfae00640 1 --2- 172.21.15.79:0/1042708865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4109a30 0x7f2df4110350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009ca0 tx=0x7f2dec02a8c0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.034+0000 7f2dfae00640 1 -- 172.21.15.79:0/1042708865 shutdown_connections 2024-06-03T02:35:25.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.034+0000 7f2dfae00640 1 --2- 172.21.15.79:0/1042708865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4109a30 0x7f2df4110350 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.034+0000 7f2dfae00640 1 --2- 172.21.15.79:0/1042708865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df4109070 0x7f2df41094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.034+0000 7f2dfae00640 1 --2- 172.21.15.79:0/1042708865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df4107e90 0x7f2df41082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.034+0000 7f2dfae00640 1 -- 172.21.15.79:0/1042708865 >> 172.21.15.79:0/1042708865 conn(0x7f2df4076e70 msgr2=0x7f2df40792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:25.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.034+0000 7f2dfae00640 1 -- 172.21.15.79:0/1042708865 shutdown_connections 2024-06-03T02:35:25.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.034+0000 7f2dfae00640 1 -- 172.21.15.79:0/1042708865 wait complete. 2024-06-03T02:35:25.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.035+0000 7f2dfae00640 1 Processor -- start 2024-06-03T02:35:25.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.035+0000 7f2dfae00640 1 -- start start 2024-06-03T02:35:25.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2dfae00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4107e90 0x7f2df419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2dfae00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df4109070 0x7f2df419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2dfae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df4109a30 0x7f2df41a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2dfae00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2df4113b60 con 0x7f2df4107e90 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2dfae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2df41139e0 con 0x7f2df4109a30 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2dfae00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2df4113ce0 con 0x7f2df4109070 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2df9a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4107e90 0x7f2df419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2df9a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4107e90 0x7f2df419e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52222/0 (socket says 172.21.15.79:52222) 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2df9a00640 1 -- 172.21.15.79:0/677462149 learned_addr learned my addr 172.21.15.79:0/677462149 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2dfa400640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df4109a30 0x7f2df41a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2df9000640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df4109070 0x7f2df419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2df9a00640 1 -- 172.21.15.79:0/677462149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df4109070 msgr2=0x7f2df419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2df9a00640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df4109070 0x7f2df419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2df9a00640 1 -- 172.21.15.79:0/677462149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df4109a30 msgr2=0x7f2df41a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2df9a00640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df4109a30 0x7f2df41a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2df9a00640 1 -- 172.21.15.79:0/677462149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df41a5d10 con 0x7f2df4107e90 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.036+0000 7f2dfa400640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df4109a30 0x7f2df41a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.037+0000 7f2df9a00640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4107e90 0x7f2df419e0f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9c0 tx=0x7f2de800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.037+0000 7f2df2000640 1 -- 172.21.15.79:0/677462149 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de800cd80 con 0x7f2df4107e90 2024-06-03T02:35:25.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.037+0000 7f2df2000640 1 -- 172.21.15.79:0/677462149 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de80035f0 con 0x7f2df4107e90 2024-06-03T02:35:25.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.037+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df41a5fa0 con 0x7f2df4107e90 2024-06-03T02:35:25.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.037+0000 7f2df2000640 1 -- 172.21.15.79:0/677462149 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8010690 con 0x7f2df4107e90 2024-06-03T02:35:25.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.037+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df41a6460 con 0x7f2df4107e90 2024-06-03T02:35:25.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.038+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df410f160 con 0x7f2df4107e90 2024-06-03T02:35:25.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.041+0000 7f2df2000640 1 -- 172.21.15.79:0/677462149 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2de80037a0 con 0x7f2df4107e90 2024-06-03T02:35:25.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.042+0000 7f2df2000640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ddc0a2b90 0x7f2ddc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.042+0000 7f2df2000640 1 -- 172.21.15.79:0/677462149 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f2de80cf990 con 0x7f2df4107e90 2024-06-03T02:35:25.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.042+0000 7f2df9000640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ddc0a2b90 0x7f2ddc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.043+0000 7f2df9000640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ddc0a2b90 0x7f2ddc0a5050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2df419f600 tx=0x7f2de0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:25.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.047+0000 7f2df2000640 1 -- 172.21.15.79:0/677462149 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2de809c4e0 con 0x7f2df4107e90 2024-06-03T02:35:25.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.248+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2df40625a0 con 0x7f2df4107e90 2024-06-03T02:35:25.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.249+0000 7f2df2000640 1 -- 172.21.15.79:0/677462149 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f2de809bde0 con 0x7f2df4107e90 2024-06-03T02:35:25.250 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-06-03T02:35:25.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ddc0a2b90 msgr2=0x7f2ddc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ddc0a2b90 0x7f2ddc0a5050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2df419f600 tx=0x7f2de0005f90 comp rx=0 tx=0).stop 2024-06-03T02:35:25.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4107e90 msgr2=0x7f2df419e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4107e90 0x7f2df419e0f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9c0 tx=0x7f2de800ee80 comp rx=0 tx=0).stop 2024-06-03T02:35:25.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 shutdown_connections 2024-06-03T02:35:25.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df4109a30 0x7f2df41a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f2ddc0a2b90 0x7f2ddc0a5050 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df4109070 0x7f2df419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 --2- 172.21.15.79:0/677462149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2df4107e90 0x7f2df419e0f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 >> 172.21.15.79:0/677462149 conn(0x7f2df4076e70 msgr2=0x7f2df410c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:25.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 shutdown_connections 2024-06-03T02:35:25.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.252+0000 7f2dfae00640 1 -- 172.21.15.79:0/677462149 wait complete. 2024-06-03T02:35:25.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-03T02:35:25.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.468+0000 7f280da00640 1 -- 172.21.15.79:0/25978337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2808104cd0 msgr2=0x7f2808105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.468+0000 7f280da00640 1 --2- 172.21.15.79:0/25978337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2808104cd0 0x7f2808105150 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009860 tx=0x7f27f402a340 comp rx=0 tx=0).stop 2024-06-03T02:35:25.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.468+0000 7f280da00640 1 -- 172.21.15.79:0/25978337 shutdown_connections 2024-06-03T02:35:25.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.468+0000 7f280da00640 1 --2- 172.21.15.79:0/25978337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2808105690 0x7f280810bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.468+0000 7f280da00640 1 --2- 172.21.15.79:0/25978337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2808104cd0 0x7f2808105150 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.468+0000 7f280da00640 1 --2- 172.21.15.79:0/25978337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2808103ae0 0x7f2808103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.468+0000 7f280da00640 1 -- 172.21.15.79:0/25978337 >> 172.21.15.79:0/25978337 conn(0x7f28080ff050 msgr2=0x7f2808101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:25.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.469+0000 7f280da00640 1 -- 172.21.15.79:0/25978337 shutdown_connections 2024-06-03T02:35:25.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.469+0000 7f280da00640 1 -- 172.21.15.79:0/25978337 wait complete. 2024-06-03T02:35:25.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.470+0000 7f280da00640 1 Processor -- start 2024-06-03T02:35:25.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.470+0000 7f280da00640 1 -- start start 2024-06-03T02:35:25.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.470+0000 7f280da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2808103ae0 0x7f2808199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f280da00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2808104cd0 0x7f280819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f280da00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2808105690 0x7f28081a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f280da00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f280810f740 con 0x7f2808104cd0 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f280da00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f280810f5c0 con 0x7f2808103ae0 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f280da00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f280810f8c0 con 0x7f2808105690 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f2807e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2808103ae0 0x7f2808199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f2807e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2808103ae0 0x7f2808199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:59046/0 (socket says 172.21.15.79:59046) 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f2807e00640 1 -- 172.21.15.79:0/4173357086 learned_addr learned my addr 172.21.15.79:0/4173357086 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f2807400640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2808104cd0 0x7f280819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f280d000640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2808105690 0x7f28081a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f2807e00640 1 -- 172.21.15.79:0/4173357086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2808105690 msgr2=0x7f28081a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f2807e00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2808105690 0x7f28081a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f2807e00640 1 -- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2808104cd0 msgr2=0x7f280819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f2807e00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2808104cd0 0x7f280819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f2807e00640 1 -- 172.21.15.79:0/4173357086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28081a1a00 con 0x7f2808103ae0 2024-06-03T02:35:25.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.471+0000 7f280d000640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2808105690 0x7f28081a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:35:25.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.472+0000 7f2807e00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2808103ae0 0x7f2808199de0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9c0 tx=0x7f27f800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:25.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.472+0000 7f2804c00640 1 -- 172.21.15.79:0/4173357086 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f800cd80 con 0x7f2808103ae0 2024-06-03T02:35:25.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.472+0000 7f2804c00640 1 -- 172.21.15.79:0/4173357086 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f80035f0 con 0x7f2808103ae0 2024-06-03T02:35:25.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.472+0000 7f2804c00640 1 -- 172.21.15.79:0/4173357086 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8010690 con 0x7f2808103ae0 2024-06-03T02:35:25.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.472+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28081a1cf0 con 0x7f2808103ae0 2024-06-03T02:35:25.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.472+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28081a21c0 con 0x7f2808103ae0 2024-06-03T02:35:25.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.473+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280810ae50 con 0x7f2808103ae0 2024-06-03T02:35:25.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.475+0000 7f2804c00640 1 -- 172.21.15.79:0/4173357086 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f27f80037a0 con 0x7f2808103ae0 2024-06-03T02:35:25.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.476+0000 7f2804c00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27ec0a2b90 0x7f27ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.476+0000 7f2804c00640 1 -- 172.21.15.79:0/4173357086 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f27f80cf8c0 con 0x7f2808103ae0 2024-06-03T02:35:25.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.476+0000 7f2807400640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27ec0a2b90 0x7f27ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.477+0000 7f2807400640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27ec0a2b90 0x7f27ec0a5050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009830 tx=0x7f27f4030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:25.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.481+0000 7f2804c00640 1 -- 172.21.15.79:0/4173357086 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f27f809c410 con 0x7f2808103ae0 2024-06-03T02:35:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:25 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1333476368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T02:35:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:25 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/677462149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T02:35:25.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.677+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f280802d2e0 con 0x7f2808103ae0 2024-06-03T02:35:25.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.678+0000 7f2804c00640 1 -- 172.21.15.79:0/4173357086 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f280802d2e0 con 0x7f2808103ae0 2024-06-03T02:35:25.679 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27ec0a2b90 msgr2=0x7f27ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27ec0a2b90 0x7f27ec0a5050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009830 tx=0x7f27f4030550 comp rx=0 tx=0).stop 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2808103ae0 msgr2=0x7f2808199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2808103ae0 0x7f2808199de0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9c0 tx=0x7f27f800ee80 comp rx=0 tx=0).stop 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 shutdown_connections 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f27ec0a2b90 0x7f27ec0a5050 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2808105690 0x7f28081a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2808104cd0 0x7f280819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 --2- 172.21.15.79:0/4173357086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2808103ae0 0x7f2808199de0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 >> 172.21.15.79:0/4173357086 conn(0x7f28080ff050 msgr2=0x7f2808108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:25.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 shutdown_connections 2024-06-03T02:35:25.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.681+0000 7f280da00640 1 -- 172.21.15.79:0/4173357086 wait complete. 2024-06-03T02:35:25.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-03T02:35:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:25 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1333476368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T02:35:25.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:25 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/677462149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T02:35:25.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:25 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1333476368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T02:35:25.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:25 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/677462149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T02:35:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.883+0000 7f96ef000640 1 -- 172.21.15.79:0/1673147849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 msgr2=0x7f96e8104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.883+0000 7f96ef000640 1 --2- 172.21.15.79:0/1673147849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 0x7f96e8104020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009900 tx=0x7f96d802a410 comp rx=0 tx=0).stop 2024-06-03T02:35:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.883+0000 7f96ef000640 1 -- 172.21.15.79:0/1673147849 shutdown_connections 2024-06-03T02:35:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.883+0000 7f96ef000640 1 --2- 172.21.15.79:0/1673147849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e81057b0 0x7f96e810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.883+0000 7f96ef000640 1 --2- 172.21.15.79:0/1673147849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96e8104df0 0x7f96e8105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.883+0000 7f96ef000640 1 --2- 172.21.15.79:0/1673147849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 0x7f96e8104020 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.883+0000 7f96ef000640 1 -- 172.21.15.79:0/1673147849 >> 172.21.15.79:0/1673147849 conn(0x7f96e80ff190 msgr2=0x7f96e81015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:25.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.883+0000 7f96ef000640 1 -- 172.21.15.79:0/1673147849 shutdown_connections 2024-06-03T02:35:25.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.883+0000 7f96ef000640 1 -- 172.21.15.79:0/1673147849 wait complete. 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.884+0000 7f96ef000640 1 Processor -- start 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.884+0000 7f96ef000640 1 -- start start 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.884+0000 7f96ef000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 0x7f96e8199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96ef000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e8104df0 0x7f96e819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96ef000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96e81057b0 0x7f96e81a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96ef000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f96e810f8c0 con 0x7f96e8103c00 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96ef000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96e810f740 con 0x7f96e8104df0 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96ef000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96e810fa40 con 0x7f96e81057b0 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96e7e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 0x7f96e8199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96e7e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 0x7f96e8199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52248/0 (socket says 172.21.15.79:52248) 2024-06-03T02:35:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96e7e00640 1 -- 172.21.15.79:0/4273363344 learned_addr learned my addr 172.21.15.79:0/4273363344 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96dfe00640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e8104df0 0x7f96e819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96ed000640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96e81057b0 0x7f96e81a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96e7e00640 1 -- 172.21.15.79:0/4273363344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96e81057b0 msgr2=0x7f96e81a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96e7e00640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96e81057b0 0x7f96e81a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96e7e00640 1 -- 172.21.15.79:0/4273363344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e8104df0 msgr2=0x7f96e819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96e7e00640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e8104df0 0x7f96e819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96e7e00640 1 -- 172.21.15.79:0/4273363344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e81a1b80 con 0x7f96e8103c00 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96dfe00640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e8104df0 0x7f96e819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.885+0000 7f96ed000640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96e81057b0 0x7f96e81a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.886+0000 7f96e7e00640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 0x7f96e8199f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f96d8031440 tx=0x7f96d8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.886+0000 7f96e5600640 1 -- 172.21.15.79:0/4273363344 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d803f030 con 0x7f96e8103c00 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.886+0000 7f96e5600640 1 -- 172.21.15.79:0/4273363344 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d8003ce0 con 0x7f96e8103c00 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.886+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e81a1e10 con 0x7f96e8103c00 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.886+0000 7f96e5600640 1 -- 172.21.15.79:0/4273363344 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8035070 con 0x7f96e8103c00 2024-06-03T02:35:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.886+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e81a2220 con 0x7f96e8103c00 2024-06-03T02:35:25.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.887+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e810b040 con 0x7f96e8103c00 2024-06-03T02:35:25.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.889+0000 7f96e5600640 1 -- 172.21.15.79:0/4273363344 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f96d8042030 con 0x7f96e8103c00 2024-06-03T02:35:25.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.890+0000 7f96e5600640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96cc0a2b90 0x7f96cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:25.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.890+0000 7f96e5600640 1 -- 172.21.15.79:0/4273363344 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f96d80efd30 con 0x7f96e8103c00 2024-06-03T02:35:25.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.891+0000 7f96dfe00640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96cc0a2b90 0x7f96cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:25.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.891+0000 7f96dfe00640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96cc0a2b90 0x7f96cc0a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f96e819b470 tx=0x7f96d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:25.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:25.895+0000 7f96e5600640 1 -- 172.21.15.79:0/4273363344 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f96d80bc880 con 0x7f96e8103c00 2024-06-03T02:35:26.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.095+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f96e80626e0 con 0x7f96e8103c00 2024-06-03T02:35:26.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.097+0000 7f96e5600640 1 -- 172.21.15.79:0/4273363344 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v101) ==== 103+0+11 (secure 0 0 0) 0x7f96d80bc180 con 0x7f96e8103c00 2024-06-03T02:35:26.098 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-06-03T02:35:26.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96cc0a2b90 msgr2=0x7f96cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96cc0a2b90 0x7f96cc0a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f96e819b470 tx=0x7f96d0005f90 comp rx=0 tx=0).stop 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 msgr2=0x7f96e8199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 0x7f96e8199f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f96d8031440 tx=0x7f96d8032ce0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 shutdown_connections 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f96cc0a2b90 0x7f96cc0a5050 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96e81057b0 0x7f96e81a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e8104df0 0x7f96e819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 --2- 172.21.15.79:0/4273363344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96e8103c00 0x7f96e8199f60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 >> 172.21.15.79:0/4273363344 conn(0x7f96e80ff190 msgr2=0x7f96e8108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 shutdown_connections 2024-06-03T02:35:26.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.100+0000 7f96ef000640 1 -- 172.21.15.79:0/4273363344 wait complete. 2024-06-03T02:35:26.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-03T02:35:26.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.314+0000 7f0355e00640 1 -- 172.21.15.79:0/1985301527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03500759e0 msgr2=0x7f0350073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:26.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.314+0000 7f0355e00640 1 --2- 172.21.15.79:0/1985301527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03500759e0 0x7f0350073e30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0344009930 tx=0x7f034402a440 comp rx=0 tx=0).stop 2024-06-03T02:35:26.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.314+0000 7f0355e00640 1 -- 172.21.15.79:0/1985301527 shutdown_connections 2024-06-03T02:35:26.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.314+0000 7f0355e00640 1 --2- 172.21.15.79:0/1985301527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f035010dde0 0x7f03501101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.314+0000 7f0355e00640 1 --2- 172.21.15.79:0/1985301527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0350074400 0x7f0350074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.314+0000 7f0355e00640 1 --2- 172.21.15.79:0/1985301527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03500759e0 0x7f0350073e30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.314+0000 7f0355e00640 1 -- 172.21.15.79:0/1985301527 >> 172.21.15.79:0/1985301527 conn(0x7f03500fcd80 msgr2=0x7f03500ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:26.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.314+0000 7f0355e00640 1 -- 172.21.15.79:0/1985301527 shutdown_connections 2024-06-03T02:35:26.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.314+0000 7f0355e00640 1 -- 172.21.15.79:0/1985301527 wait complete. 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.315+0000 7f0355e00640 1 Processor -- start 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.315+0000 7f0355e00640 1 -- start start 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.315+0000 7f0355e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350074400 0x7f035019e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.315+0000 7f0355e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500759e0 0x7f035019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.315+0000 7f0355e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010dde0 0x7f03501a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0355e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0350113ab0 con 0x7f0350074400 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0355e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0350113930 con 0x7f035010dde0 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0355e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0350113c30 con 0x7f03500759e0 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0354a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350074400 0x7f035019e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0355400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010dde0 0x7f03501a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0354a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350074400 0x7f035019e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52266/0 (socket says 172.21.15.79:52266) 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0354a00640 1 -- 172.21.15.79:0/1495315403 learned_addr learned my addr 172.21.15.79:0/1495315403 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:35:26.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f034fe00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500759e0 0x7f035019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:26.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0355400640 1 -- 172.21.15.79:0/1495315403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500759e0 msgr2=0x7f035019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:26.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0355400640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500759e0 0x7f035019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0355400640 1 -- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350074400 msgr2=0x7f035019e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:26.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0355400640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350074400 0x7f035019e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:26.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0355400640 1 -- 172.21.15.79:0/1495315403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03501a5db0 con 0x7f035010dde0 2024-06-03T02:35:26.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.316+0000 7f0354a00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350074400 0x7f035019e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:35:26.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.317+0000 7f0355400640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010dde0 0x7f03501a5740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0348009fd0 tx=0x7f034800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:26.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.317+0000 7f034d600640 1 -- 172.21.15.79:0/1495315403 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034800ce20 con 0x7f035010dde0 2024-06-03T02:35:26.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.317+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03501a60a0 con 0x7f035010dde0 2024-06-03T02:35:26.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.317+0000 7f034d600640 1 -- 172.21.15.79:0/1495315403 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0348005020 con 0x7f035010dde0 2024-06-03T02:35:26.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.317+0000 7f034d600640 1 -- 172.21.15.79:0/1495315403 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348010720 con 0x7f035010dde0 2024-06-03T02:35:26.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.317+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03501a65f0 con 0x7f035010dde0 2024-06-03T02:35:26.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.318+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0350103fb0 con 0x7f035010dde0 2024-06-03T02:35:26.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.320+0000 7f034d600640 1 -- 172.21.15.79:0/1495315403 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0348005540 con 0x7f035010dde0 2024-06-03T02:35:26.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.322+0000 7f034d600640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03380a2b90 0x7f03380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:35:26.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.322+0000 7f034d600640 1 -- 172.21.15.79:0/1495315403 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f03480cfac0 con 0x7f035010dde0 2024-06-03T02:35:26.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.322+0000 7f0354a00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03380a2b90 0x7f03380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:35:26.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.323+0000 7f0354a00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03380a2b90 0x7f03380a5050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0344000c00 tx=0x7f03440315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:35:26.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.324+0000 7f034d600640 1 -- 172.21.15.79:0/1495315403 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f034809c610 con 0x7f035010dde0 2024-06-03T02:35:26.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.571+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f03500625a0 con 0x7f035010dde0 2024-06-03T02:35:26.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.582+0000 7f034d600640 1 -- 172.21.15.79:0/1495315403 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740360 (secure 0 0 0) 0x7f034809bf10 con 0x7f035010dde0 2024-06-03T02:35:26.595 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:35:26.595 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:35:26.595 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 20, 2024-06-03T02:35:26.595 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.595 INFO:teuthology.orchestra.run.smithi079.stdout: "active_gid": 14152, 2024-06-03T02:35:26.595 INFO:teuthology.orchestra.run.smithi079.stdout: "active_name": "x", 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: "active_addrs": { 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2278164 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2278164 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:35:26.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "active_addr": "172.21.15.79:6801/2278164", 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "active_change": "2024-06-03T02:05:52.009056+0000", 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "active_mgr_features": 4540701547738038271, 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "available": true, 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "standbys": [ 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "gid": 24109, 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "z", 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_features": 4540701547738038271, 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "available_modules": [ 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "alerts", 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.597 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How frequently to reexamine health status", 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.599 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.600 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.601 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_destination": { 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_destination", 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Email address to send alerts to", 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.604 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_from_name": { 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_from_name", 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "Ceph", 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Email From: name", 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_host": { 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_host", 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SMTP server", 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.606 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_password": { 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_password", 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Password to authenticate with", 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.607 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_port": { 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_port", 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "465", 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SMTP port", 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_sender": { 2024-06-03T02:35:26.608 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_sender", 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SMTP envelope sender", 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_ssl": { 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_ssl", 2024-06-03T02:35:26.609 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_user": { 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_user", 2024-06-03T02:35:26.610 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "User to authenticate as", 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.611 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "balancer", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "active": { 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "active", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "automatically balance PGs across cluster", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.612 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "begin_time": { 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "begin_time", 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0000", 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "beginning time of day to automatically balance", 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.613 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "begin_weekday": { 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "begin_weekday", 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "0", 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "6", 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.614 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_compat_max_iterations": { 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crush_compat_max_iterations", 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "25", 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "250", 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_compat_metrics": { 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crush_compat_metrics", 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "pgs,objects,bytes", 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.616 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_compat_step": { 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crush_compat_step", 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0.5", 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "0.001", 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "0.999", 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "aggressiveness of optimization", 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "end_time": { 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "end_time", 2024-06-03T02:35:26.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "2359", 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "ending time of day to automatically balance", 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "end_weekday": { 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "end_weekday", 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:26.618 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "0", 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "6", 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.622 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.623 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "min_score": { 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "min_score", 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.624 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": { 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mode", 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "upmap", 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.625 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-compat", 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "none", 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "read", 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap", 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap-read" 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Balancer mode", 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_ids": { 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pool_ids", 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.626 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.627 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:26.628 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.628 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.628 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.628 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-03T02:35:26.628 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.629 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap_max_deviation": { 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "upmap_max_deviation", 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:26.630 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap_max_optimizations": { 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "upmap_max_optimizations", 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:26.631 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cephadm", 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.632 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "agent_down_multiplier": { 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "agent_down_multiplier", 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "3.0", 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.633 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "agent_refresh_rate": { 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "agent_refresh_rate", 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "20", 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.634 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "agent_starting_port": { 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "agent_starting_port", 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "4721", 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-03T02:35:26.635 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_ptrace": { 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "allow_ptrace", 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.636 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "autotune_interval": { 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "autotune_interval", 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.637 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "autotune_memory_target_ratio": { 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "autotune_memory_target_ratio", 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0.7", 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm_log_destination": { 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cephadm_log_destination", 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.639 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "file", 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "file,syslog", 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "syslog" 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "cgroups_split": { 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cgroups_split", 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.640 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "config_checks_enabled": { 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "config_checks_enabled", 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.641 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "config_dashboard": { 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "config_dashboard", 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.642 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_alertmanager": { 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_alertmanager", 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.643 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_base": { 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_base", 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.644 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Container image name, without the tag", 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_elasticsearch": { 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_elasticsearch", 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.645 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "elasticsearch container image", 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_grafana": { 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_grafana", 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.646 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_haproxy": { 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_haproxy", 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.647 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "HAproxy container image", 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_jaeger_agent": { 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_jaeger_agent", 2024-06-03T02:35:26.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Jaeger agent container image", 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_jaeger_collector": { 2024-06-03T02:35:26.649 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_jaeger_collector", 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Jaeger collector container image", 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_jaeger_query": { 2024-06-03T02:35:26.650 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_jaeger_query", 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Jaeger query container image", 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_keepalived": { 2024-06-03T02:35:26.651 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_keepalived", 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Keepalived container image", 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_loki": { 2024-06-03T02:35:26.652 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_loki", 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Loki container image", 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_node_exporter": { 2024-06-03T02:35:26.653 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_node_exporter", 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.654 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_nvmeof": { 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_nvmeof", 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Nvme-of container image", 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.655 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_prometheus": { 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_prometheus", 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.656 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_promtail": { 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_promtail", 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Promtail container image", 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_samba": { 2024-06-03T02:35:26.657 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_samba", 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Samba/SMB container image", 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.658 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_snmp_gateway": { 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_snmp_gateway", 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SNMP Gateway container image", 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "container_init": { 2024-06-03T02:35:26.659 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_init", 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Run podman/docker with `--init`", 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.660 INFO:teuthology.orchestra.run.smithi079.stdout: "daemon_cache_timeout": { 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "daemon_cache_timeout", 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.661 INFO:teuthology.orchestra.run.smithi079.stdout: "default_cephadm_command_timeout": { 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "default_cephadm_command_timeout", 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "900", 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.662 INFO:teuthology.orchestra.run.smithi079.stdout: "default_registry": { 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "default_registry", 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io", 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "device_cache_timeout": { 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "device_cache_timeout", 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "1800", 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "seconds to cache device inventory", 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.664 INFO:teuthology.orchestra.run.smithi079.stdout: "device_enhanced_scan": { 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "device_enhanced_scan", 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: "facts_cache_timeout": { 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "facts_cache_timeout", 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "seconds to cache host facts data", 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.666 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "grafana_dashboards_path": { 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "grafana_dashboards_path", 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "host_check_interval": { 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "host_check_interval", 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to perform a host check", 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.668 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "hw_monitoring": { 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "hw_monitoring", 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.669 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "inventory_list_all": { 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "inventory_list_all", 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.670 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.671 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "log_refresh_metadata": { 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_refresh_metadata", 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.672 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.673 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.675 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.676 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.677 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "*", 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.678 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "max_count_per_host": { 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_count_per_host", 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.679 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "max number of daemons per service per host", 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd_draining_count": { 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_osd_draining_count", 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.680 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "migration_current": { 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "migration_current", 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.681 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "internal - do not modify", 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": { 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mode", 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "root", 2024-06-03T02:35:26.682 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm-package", 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "root" 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "mode for remote execution of cephadm", 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "oob_default_addr": { 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "oob_default_addr", 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.683 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "169.254.1.1", 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "prometheus_alerts_path": { 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "prometheus_alerts_path", 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.684 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_insecure": { 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_insecure", 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.685 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_password": { 2024-06-03T02:35:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_password", 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_url": { 2024-06-03T02:35:26.687 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_url", 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.688 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_username": { 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_username", 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.689 INFO:teuthology.orchestra.run.smithi079.stdout: "secure_monitoring_stack": { 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "secure_monitoring_stack", 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.690 INFO:teuthology.orchestra.run.smithi079.stdout: "service_discovery_port": { 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "service_discovery_port", 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8765", 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "cephadm service discovery port", 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.691 INFO:teuthology.orchestra.run.smithi079.stdout: "ssh_config_file": { 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssh_config_file", 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "use_agent": { 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "use_agent", 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "use_repo_digest": { 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "use_repo_digest", 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.694 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_on_failed_host_check": { 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_on_failed_host_check", 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise a health warning if the host check fails", 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_on_stray_daemons": { 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_on_stray_daemons", 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_on_stray_hosts": { 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_on_stray_hosts", 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crash", 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.698 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.699 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.701 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.702 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "retain_interval": { 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "retain_interval", 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.703 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "31536000", 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_recent_interval": { 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_recent_interval", 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.704 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "1209600", 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "dashboard", 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.705 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.706 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "ALERTMANAGER_API_HOST": { 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.707 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "AUDIT_API_ENABLED": { 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "AUDIT_API_ENABLED", 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "ENABLE_BROWSABLE_API": { 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.711 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.712 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.714 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.717 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.718 INFO:teuthology.orchestra.run.smithi079.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_PASSWORD": { 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "admin", 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.720 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.721 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_URL": { 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_URL", 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.722 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_USERNAME": { 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "admin", 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.723 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.724 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "PROMETHEUS_API_HOST": { 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.729 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.730 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.731 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.732 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.733 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.735 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_ENABLED": { 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8", 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "45", 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_ACCESS_KEY": { 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "admin", 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_SECRET_KEY": { 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.744 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_SSL_VERIFY": { 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.745 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.746 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "cross_origin_url": { 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cross_origin_url", 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "crt_file": { 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crt_file", 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "debug": { 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "debug", 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Enable/disable debug options", 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.751 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "jwt_token_ttl": { 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "jwt_token_ttl", 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "28800", 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.752 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "key_file": { 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "key_file", 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.753 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.754 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.755 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.757 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.758 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "motd": { 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "motd", 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.759 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "The message of the day", 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "redirect_resolve_ip_addr": { 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "redirect_resolve_ip_addr", 2024-06-03T02:35:26.760 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "server_addr": { 2024-06-03T02:35:26.761 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_addr", 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "::", 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.762 INFO:teuthology.orchestra.run.smithi079.stdout: "server_port": { 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_port", 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8080", 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "ssl": { 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssl", 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "ssl_server_port": { 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssl_server_port", 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8443", 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.765 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_behaviour": { 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_behaviour", 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "redirect", 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: "redirect" 2024-06-03T02:35:26.766 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_error_status_code": { 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_error_status_code", 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "500", 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "400", 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "599", 2024-06-03T02:35:26.767 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "url_prefix": { 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "url_prefix", 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.768 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "devicehealth", 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_monitoring": { 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enable_monitoring", 2024-06-03T02:35:26.769 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "monitor device health metrics", 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.770 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.771 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.772 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.773 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.774 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "mark_out_threshold": { 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mark_out_threshold", 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "2419200", 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": { 2024-06-03T02:35:26.776 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pool_name", 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "device_health_metrics", 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "retention_period": { 2024-06-03T02:35:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "retention_period", 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "15552000", 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how long to retain device health metrics", 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "scrape_frequency": { 2024-06-03T02:35:26.778 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "scrape_frequency", 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "86400", 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.779 INFO:teuthology.orchestra.run.smithi079.stdout: "self_heal": { 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "self_heal", 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.780 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to wake up and check device health", 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.781 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_threshold": { 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_threshold", 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "7257600", 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "diskprediction_local", 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.783 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.786 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.787 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "predict_interval": { 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "predict_interval", 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "86400", 2024-06-03T02:35:26.788 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "predictor_model": { 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "predictor_model", 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "prophetstor", 2024-06-03T02:35:26.789 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:26.790 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "influx", 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": false, 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "influxdb python module not found", 2024-06-03T02:35:26.791 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "batch_size": { 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "batch_size", 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5000", 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "database": { 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "database", 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "ceph", 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.793 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": { 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "hostname", 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "InfluxDB server hostname", 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.794 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "30", 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "5", 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.796 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.797 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.798 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.799 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "password": { 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "password", 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "password of InfluxDB server user", 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "port": { 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "port", 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8086", 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "InfluxDB server port", 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.802 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "ssl": { 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssl", 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "false", 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T02:35:26.803 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "threads": { 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "threads", 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "32", 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-03T02:35:26.804 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "username": { 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "username", 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "username of InfluxDB server user", 2024-06-03T02:35:26.805 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "verify_ssl": { 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "verify_ssl", 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "true", 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.806 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "insights", 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.807 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.808 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.809 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.810 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.811 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.812 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "iostat", 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.813 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.814 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.815 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.816 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "k8sevents", 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_event_retention_days": { 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ceph_event_retention_days", 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.818 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "7", 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "config_check_secs": { 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "config_check_secs", 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.819 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "10", 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.820 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.821 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.823 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.824 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "localpool", 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.825 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "failure_domain": { 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "failure_domain", 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "host", 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "failure domain for any created local pool", 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.827 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.828 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.829 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.831 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": { 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "min_size", 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "default min_size for any created local pool", 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.832 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "num_rep": { 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "num_rep", 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "3", 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "default replica count for any created local pool", 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.833 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": { 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pg_num", 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "128", 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "default pg_num for any created local pool", 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "prefix": { 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "prefix", 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "name prefix for any created local pool", 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.835 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "subtree": { 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "subtree", 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "rack", 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-03T02:35:26.836 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mds_autoscaler", 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.837 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.839 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.840 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.841 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.842 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mirroring", 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.843 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.844 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.845 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.846 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.847 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "nfs", 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.849 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.850 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.851 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "orchestrator", 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "fail_fs": { 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "fail_fs", 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.854 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.855 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.856 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.857 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.858 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.859 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "orchestrator": { 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "orchestrator", 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.860 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm", 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: "rook", 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: "test_orchestrator" 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Orchestrator backend", 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "osd_perf_query", 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.862 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.863 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.864 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.865 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.866 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "osd_support", 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.868 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.869 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.870 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.871 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pg_autoscaler", 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.873 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.874 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.875 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.877 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:26.878 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "threshold": { 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "threshold", 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "3.0", 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1.0", 2024-06-03T02:35:26.879 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "scaling threshold", 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "progress", 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.880 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_pg_recovery_event": { 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "allow_pg_recovery_event", 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "enabled": { 2024-06-03T02:35:26.881 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enabled", 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.882 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.883 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.884 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.885 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.886 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "max_completed_events": { 2024-06-03T02:35:26.887 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_completed_events", 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "50", 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "number of past completed events to remember", 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:26.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how long the module is going to sleep", 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "prometheus", 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "cache": { 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cache", 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.890 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "exclude_perf_counters": { 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "exclude_perf_counters", 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.891 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.892 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.893 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.894 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.895 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.896 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_stats_pools": { 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_stats_pools", 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.897 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "300", 2024-06-03T02:35:26.898 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "scrape_interval": { 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "scrape_interval", 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "15.0", 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.899 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "server_addr": { 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_addr", 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "::", 2024-06-03T02:35:26.900 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "server_port": { 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_port", 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "9283", 2024-06-03T02:35:26.901 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "stale_cache_strategy": { 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "stale_cache_strategy", 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.902 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "log", 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_behaviour": { 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_behaviour", 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.903 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "default", 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "default", 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "error" 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_error_status_code": { 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_error_status_code", 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "500", 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "400", 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "599", 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.905 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_support", 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.906 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.907 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.908 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.909 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.910 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "max_concurrent_snap_create": { 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_concurrent_snap_create", 2024-06-03T02:35:26.911 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "mirror_snapshot_schedule": { 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mirror_snapshot_schedule", 2024-06-03T02:35:26.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "trash_purge_schedule": { 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "trash_purge_schedule", 2024-06-03T02:35:26.913 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "restful", 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_auth": { 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enable_auth", 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.915 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "key_file": { 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "key_file", 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.916 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.917 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.919 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.920 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "server_addr": { 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_addr", 2024-06-03T02:35:26.922 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "server_port": { 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_port", 2024-06-03T02:35:26.923 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.924 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rgw", 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.925 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.926 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.927 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.928 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.929 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rook", 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.930 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.931 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.932 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.933 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.934 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.935 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "storage_class": { 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "storage_class", 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "local", 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.936 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "selftest", 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.937 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.938 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.939 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.940 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.941 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "roption1": { 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "roption1", 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.942 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "roption2": { 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "roption2", 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "xyz", 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption1": { 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption1", 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.944 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption2": { 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption2", 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.945 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption3": { 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption3", 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:26.946 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption4": { 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption4", 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.947 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption5": { 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption5", 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.948 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption6": { 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption6", 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.949 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption7": { 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption7", 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "42", 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "testkey": { 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "testkey", 2024-06-03T02:35:26.951 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "testlkey": { 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "testlkey", 2024-06-03T02:35:26.952 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "testnewline": { 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "testnewline", 2024-06-03T02:35:26.953 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "snap_schedule", 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_m_granularity": { 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "allow_m_granularity", 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.955 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "allow minute scheduled snapshots", 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "dump_on_update": { 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "dump_on_update", 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.956 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "dump database to debug log on update", 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.957 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.958 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.959 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.960 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.961 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "stats", 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.962 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.963 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.964 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.965 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.966 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "status", 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.968 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.971 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.972 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "telegraf", 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.973 INFO:teuthology.orchestra.run.smithi079.stdout: "address": { 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "address", 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.974 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "15", 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.975 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.976 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.977 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.978 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.979 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.980 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "telemetry", 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_basic": { 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_basic", 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.981 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_crash": { 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_crash", 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.982 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_device": { 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_device", 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.983 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_ident": { 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_ident", 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.984 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_perf": { 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_perf", 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.985 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "contact": { 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "contact", 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.986 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "description": { 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "description", 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.987 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "device_url": { 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "device_url", 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.988 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "enabled": { 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enabled", 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.989 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.990 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "24", 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "8", 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "last_opt_revision": { 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "last_opt_revision", 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.991 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "1", 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "leaderboard": { 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "leaderboard", 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.992 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "leaderboard_description": { 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "leaderboard_description", 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.993 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.994 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.995 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.996 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:26.997 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "organization": { 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "organization", 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "proxy": { 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "proxy", 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.000 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "url": { 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "url", 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.001 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "test_orchestrator", 2024-06-03T02:35:27.002 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.003 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.004 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.005 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.006 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.007 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "volumes", 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.008 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.009 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.010 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.012 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "max_concurrent_clones": { 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_concurrent_clones", 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.013 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "4", 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "periodic_async_work": { 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "periodic_async_work", 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.014 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Periodically check for async work", 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "snapshot_clone_delay": { 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "snapshot_clone_delay", 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.015 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "snapshot_clone_no_wait": { 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "snapshot_clone_no_wait", 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.016 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix", 2024-06-03T02:35:27.017 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "discovery_interval": { 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "discovery_interval", 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "100", 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.018 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "identifier": { 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "identifier", 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.019 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.020 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.021 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.022 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.023 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.024 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.025 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "zabbix_host": { 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix_host", 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.026 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "zabbix_port": { 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix_port", 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10051", 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.027 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "zabbix_sender": { 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix_sender", 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: "gid": 24115, 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "y", 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_features": 4540701547738038271, 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: "available_modules": [ 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "alerts", 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How frequently to reexamine health status", 2024-06-03T02:35:27.030 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.031 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.032 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.033 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.034 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.037 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.038 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_destination": { 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_destination", 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.039 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Email address to send alerts to", 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_from_name": { 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_from_name", 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "Ceph", 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.040 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Email From: name", 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_host": { 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_host", 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.041 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SMTP server", 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_password": { 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_password", 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.042 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Password to authenticate with", 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_port": { 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_port", 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "465", 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.043 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SMTP port", 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_sender": { 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_sender", 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.044 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SMTP envelope sender", 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_ssl": { 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_ssl", 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.045 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_user": { 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_user", 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.046 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "User to authenticate as", 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "balancer", 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.047 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "active": { 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "active", 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "automatically balance PGs across cluster", 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "begin_time": { 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "begin_time", 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0000", 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "beginning time of day to automatically balance", 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "begin_weekday": { 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "begin_weekday", 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "0", 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "6", 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_compat_max_iterations": { 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crush_compat_max_iterations", 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "25", 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "250", 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_compat_metrics": { 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crush_compat_metrics", 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "pgs,objects,bytes", 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_compat_step": { 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crush_compat_step", 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0.5", 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "0.001", 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "0.999", 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "aggressiveness of optimization", 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "end_time": { 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "end_time", 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "2359", 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "ending time of day to automatically balance", 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "end_weekday": { 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "end_weekday", 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "0", 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "6", 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.056 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.057 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.058 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "min_score": { 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "min_score", 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": { 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mode", 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "upmap", 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-compat", 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "none", 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "read", 2024-06-03T02:35:27.062 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap", 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap-read" 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Balancer mode", 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_ids": { 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pool_ids", 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.063 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap_max_deviation": { 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "upmap_max_deviation", 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:27.065 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap_max_optimizations": { 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "upmap_max_optimizations", 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.066 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cephadm", 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.067 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "agent_down_multiplier": { 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "agent_down_multiplier", 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "3.0", 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.068 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "agent_refresh_rate": { 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "agent_refresh_rate", 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "20", 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "agent_starting_port": { 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "agent_starting_port", 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "4721", 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.070 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_ptrace": { 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "allow_ptrace", 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-03T02:35:27.071 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "autotune_interval": { 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "autotune_interval", 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.072 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "autotune_memory_target_ratio": { 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "autotune_memory_target_ratio", 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0.7", 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.073 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm_log_destination": { 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cephadm_log_destination", 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "file", 2024-06-03T02:35:27.074 INFO:teuthology.orchestra.run.smithi079.stdout: "file,syslog", 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "syslog" 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "cgroups_split": { 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cgroups_split", 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.075 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "config_checks_enabled": { 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "config_checks_enabled", 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.076 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "config_dashboard": { 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "config_dashboard", 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.077 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_alertmanager": { 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_alertmanager", 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.078 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_base": { 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_base", 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-03T02:35:27.079 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Container image name, without the tag", 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_elasticsearch": { 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_elasticsearch", 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.080 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "elasticsearch container image", 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_grafana": { 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_grafana", 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-03T02:35:27.081 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_haproxy": { 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_haproxy", 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-03T02:35:27.082 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "HAproxy container image", 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_jaeger_agent": { 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_jaeger_agent", 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-03T02:35:27.083 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Jaeger agent container image", 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_jaeger_collector": { 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_jaeger_collector", 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-03T02:35:27.084 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Jaeger collector container image", 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_jaeger_query": { 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_jaeger_query", 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.085 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Jaeger query container image", 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_keepalived": { 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_keepalived", 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.086 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Keepalived container image", 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_loki": { 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_loki", 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.087 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Loki container image", 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_node_exporter": { 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_node_exporter", 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.088 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_nvmeof": { 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_nvmeof", 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.089 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Nvme-of container image", 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_prometheus": { 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_prometheus", 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.090 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_promtail": { 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_promtail", 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.091 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Promtail container image", 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_samba": { 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_samba", 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-03T02:35:27.092 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Samba/SMB container image", 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_snmp_gateway": { 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_snmp_gateway", 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-03T02:35:27.093 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SNMP Gateway container image", 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "container_init": { 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_init", 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.094 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Run podman/docker with `--init`", 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "daemon_cache_timeout": { 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "daemon_cache_timeout", 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.095 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "default_cephadm_command_timeout": { 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "default_cephadm_command_timeout", 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "900", 2024-06-03T02:35:27.096 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "default_registry": { 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "default_registry", 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.097 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io", 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "device_cache_timeout": { 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "device_cache_timeout", 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.098 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "1800", 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "seconds to cache device inventory", 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "device_enhanced_scan": { 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "device_enhanced_scan", 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.099 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "facts_cache_timeout": { 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "facts_cache_timeout", 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.100 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "seconds to cache host facts data", 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "grafana_dashboards_path": { 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "grafana_dashboards_path", 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-03T02:35:27.101 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "host_check_interval": { 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "host_check_interval", 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.102 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to perform a host check", 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "hw_monitoring": { 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "hw_monitoring", 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.103 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "inventory_list_all": { 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "inventory_list_all", 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.104 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.105 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "log_refresh_metadata": { 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_refresh_metadata", 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.108 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.109 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.110 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.111 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.112 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "*", 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "max_count_per_host": { 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_count_per_host", 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.113 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "max number of daemons per service per host", 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd_draining_count": { 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_osd_draining_count", 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.114 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "migration_current": { 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "migration_current", 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.115 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "internal - do not modify", 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": { 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mode", 2024-06-03T02:35:27.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "root", 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm-package", 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "root" 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "mode for remote execution of cephadm", 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "oob_default_addr": { 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "oob_default_addr", 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "169.254.1.1", 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-03T02:35:27.118 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "prometheus_alerts_path": { 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "prometheus_alerts_path", 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-03T02:35:27.119 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_insecure": { 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_insecure", 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.120 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_password": { 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_password", 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_url": { 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_url", 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.122 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_username": { 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_username", 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.123 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "secure_monitoring_stack": { 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "secure_monitoring_stack", 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.124 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "service_discovery_port": { 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "service_discovery_port", 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8765", 2024-06-03T02:35:27.125 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "cephadm service discovery port", 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "ssh_config_file": { 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssh_config_file", 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.126 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "use_agent": { 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "use_agent", 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.127 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "use_repo_digest": { 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "use_repo_digest", 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.128 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_on_failed_host_check": { 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_on_failed_host_check", 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.129 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise a health warning if the host check fails", 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_on_stray_daemons": { 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_on_stray_daemons", 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.130 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_on_stray_hosts": { 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_on_stray_hosts", 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.131 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crash", 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.132 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.133 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.134 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.135 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.136 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "retain_interval": { 2024-06-03T02:35:27.137 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "retain_interval", 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "31536000", 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.138 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_recent_interval": { 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_recent_interval", 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "1209600", 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "dashboard", 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "ALERTMANAGER_API_HOST": { 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "AUDIT_API_ENABLED": { 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "AUDIT_API_ENABLED", 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "ENABLE_BROWSABLE_API": { 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-03T02:35:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_PASSWORD": { 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-03T02:35:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "admin", 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-03T02:35:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_URL": { 2024-06-03T02:35:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_URL", 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_USERNAME": { 2024-06-03T02:35:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "admin", 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-03T02:35:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-03T02:35:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-03T02:35:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "PROMETHEUS_API_HOST": { 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_ENABLED": { 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8", 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "45", 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_ACCESS_KEY": { 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "admin", 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_SECRET_KEY": { 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_SSL_VERIFY": { 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cross_origin_url": { 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cross_origin_url", 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "crt_file": { 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crt_file", 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "debug": { 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "debug", 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Enable/disable debug options", 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "jwt_token_ttl": { 2024-06-03T02:35:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "jwt_token_ttl", 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "28800", 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "key_file": { 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "key_file", 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "motd": { 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "motd", 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "The message of the day", 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "redirect_resolve_ip_addr": { 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "redirect_resolve_ip_addr", 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "server_addr": { 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_addr", 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "::", 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "server_port": { 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_port", 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8080", 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "ssl": { 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssl", 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "ssl_server_port": { 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssl_server_port", 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8443", 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_behaviour": { 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_behaviour", 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "redirect", 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "redirect" 2024-06-03T02:35:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_error_status_code": { 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_error_status_code", 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "500", 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "400", 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "599", 2024-06-03T02:35:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "url_prefix": { 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "url_prefix", 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "devicehealth", 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_monitoring": { 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enable_monitoring", 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "monitor device health metrics", 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "mark_out_threshold": { 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mark_out_threshold", 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "2419200", 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": { 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pool_name", 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "device_health_metrics", 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "retention_period": { 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "retention_period", 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "15552000", 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how long to retain device health metrics", 2024-06-03T02:35:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "scrape_frequency": { 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "scrape_frequency", 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "86400", 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "self_heal": { 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "self_heal", 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to wake up and check device health", 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_threshold": { 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_threshold", 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "7257600", 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "diskprediction_local", 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "predict_interval": { 2024-06-03T02:35:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "predict_interval", 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "86400", 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "predictor_model": { 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "predictor_model", 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "prophetstor", 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "influx", 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": false, 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "influxdb python module not found", 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "batch_size": { 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "batch_size", 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5000", 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "database": { 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "database", 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "ceph", 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": { 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "hostname", 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "InfluxDB server hostname", 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "30", 2024-06-03T02:35:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "5", 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "password": { 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "password", 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "password of InfluxDB server user", 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "port": { 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "port", 2024-06-03T02:35:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8086", 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "InfluxDB server port", 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "ssl": { 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssl", 2024-06-03T02:35:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "false", 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "threads": { 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "threads", 2024-06-03T02:35:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "32", 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "username": { 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "username", 2024-06-03T02:35:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "username of InfluxDB server user", 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "verify_ssl": { 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "verify_ssl", 2024-06-03T02:35:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "true", 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "insights", 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "iostat", 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "k8sevents", 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_event_retention_days": { 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ceph_event_retention_days", 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "7", 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "config_check_secs": { 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "config_check_secs", 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "10", 2024-06-03T02:35:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "localpool", 2024-06-03T02:35:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "failure_domain": { 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "failure_domain", 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "host", 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "failure domain for any created local pool", 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-03T02:35:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": { 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "min_size", 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "default min_size for any created local pool", 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "num_rep": { 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "num_rep", 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "3", 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "default replica count for any created local pool", 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": { 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pg_num", 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "128", 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "default pg_num for any created local pool", 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "prefix": { 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "prefix", 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "name prefix for any created local pool", 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "subtree": { 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "subtree", 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "rack", 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mds_autoscaler", 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mirroring", 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "nfs", 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "orchestrator", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "fail_fs": { 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "fail_fs", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "orchestrator": { 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "orchestrator", 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm", 2024-06-03T02:35:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "rook", 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "test_orchestrator" 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Orchestrator backend", 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "osd_perf_query", 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "osd_support", 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pg_autoscaler", 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "threshold": { 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "threshold", 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "3.0", 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1.0", 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "scaling threshold", 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "progress", 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_pg_recovery_event": { 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "allow_pg_recovery_event", 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "enabled": { 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enabled", 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "max_completed_events": { 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_completed_events", 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "50", 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "number of past completed events to remember", 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how long the module is going to sleep", 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "prometheus", 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "cache": { 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cache", 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "exclude_perf_counters": { 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "exclude_perf_counters", 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_stats_pools": { 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_stats_pools", 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "300", 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "scrape_interval": { 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "scrape_interval", 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "15.0", 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "server_addr": { 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_addr", 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "::", 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "server_port": { 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_port", 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "9283", 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "stale_cache_strategy": { 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "stale_cache_strategy", 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "log", 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_behaviour": { 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_behaviour", 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "default", 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "default", 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "error" 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_error_status_code": { 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_error_status_code", 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "500", 2024-06-03T02:35:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "400", 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "599", 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_support", 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "max_concurrent_snap_create": { 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_concurrent_snap_create", 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "mirror_snapshot_schedule": { 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mirror_snapshot_schedule", 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "trash_purge_schedule": { 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "trash_purge_schedule", 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "restful", 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_auth": { 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enable_auth", 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "key_file": { 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "key_file", 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "server_addr": { 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_addr", 2024-06-03T02:35:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "server_port": { 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_port", 2024-06-03T02:35:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rgw", 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rook", 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "storage_class": { 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "storage_class", 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "local", 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "selftest", 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "roption1": { 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "roption1", 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "roption2": { 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "roption2", 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "xyz", 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption1": { 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption1", 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption2": { 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption2", 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption3": { 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption3", 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption4": { 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption4", 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption5": { 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption5", 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption6": { 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption6", 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption7": { 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption7", 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "42", 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "testkey": { 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "testkey", 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "testlkey": { 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "testlkey", 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "testnewline": { 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "testnewline", 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "snap_schedule", 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_m_granularity": { 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "allow_m_granularity", 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "allow minute scheduled snapshots", 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "dump_on_update": { 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "dump_on_update", 2024-06-03T02:35:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "dump database to debug log on update", 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "stats", 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "status", 2024-06-03T02:35:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "telegraf", 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "address": { 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "address", 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "15", 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "telemetry", 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_basic": { 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_basic", 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_crash": { 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_crash", 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_device": { 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_device", 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_ident": { 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_ident", 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_perf": { 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_perf", 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-03T02:35:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "contact": { 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "contact", 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "description": { 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "description", 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "device_url": { 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "device_url", 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "enabled": { 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enabled", 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "24", 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "8", 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "last_opt_revision": { 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "last_opt_revision", 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "1", 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "leaderboard": { 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "leaderboard", 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "leaderboard_description": { 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "leaderboard_description", 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "organization": { 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "organization", 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "proxy": { 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "proxy", 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "url": { 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "url", 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "test_orchestrator", 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "volumes", 2024-06-03T02:35:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "max_concurrent_clones": { 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_concurrent_clones", 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "4", 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "periodic_async_work": { 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "periodic_async_work", 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Periodically check for async work", 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "snapshot_clone_delay": { 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "snapshot_clone_delay", 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "snapshot_clone_no_wait": { 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "snapshot_clone_no_wait", 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix", 2024-06-03T02:35:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "discovery_interval": { 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "discovery_interval", 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "100", 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "identifier": { 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "identifier", 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "zabbix_host": { 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix_host", 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "zabbix_port": { 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix_port", 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10051", 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "zabbix_sender": { 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix_sender", 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "modules": [ 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm", 2024-06-03T02:35:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "dashboard", 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "iostat", 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "nfs", 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "restful" 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "available_modules": [ 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "alerts", 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How frequently to reexamine health status", 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_destination": { 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_destination", 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Email address to send alerts to", 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_from_name": { 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_from_name", 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "Ceph", 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Email From: name", 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_host": { 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_host", 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SMTP server", 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_password": { 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_password", 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Password to authenticate with", 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_port": { 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_port", 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "465", 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SMTP port", 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_sender": { 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_sender", 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SMTP envelope sender", 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_ssl": { 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_ssl", 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "smtp_user": { 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "smtp_user", 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "User to authenticate as", 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "balancer", 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "active": { 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "active", 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "automatically balance PGs across cluster", 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "begin_time": { 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "begin_time", 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0000", 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "beginning time of day to automatically balance", 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "begin_weekday": { 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "begin_weekday", 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "0", 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "6", 2024-06-03T02:35:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_compat_max_iterations": { 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crush_compat_max_iterations", 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "25", 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "250", 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_compat_metrics": { 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crush_compat_metrics", 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "pgs,objects,bytes", 2024-06-03T02:35:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_compat_step": { 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crush_compat_step", 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0.5", 2024-06-03T02:35:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "0.001", 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "0.999", 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "aggressiveness of optimization", 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "end_time": { 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "end_time", 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "2359", 2024-06-03T02:35:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "ending time of day to automatically balance", 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "end_weekday": { 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "end_weekday", 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "0", 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "6", 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "min_score": { 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "min_score", 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": { 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mode", 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "upmap", 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-compat", 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "none", 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "read", 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap", 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap-read" 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Balancer mode", 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_ids": { 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pool_ids", 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap_max_deviation": { 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "upmap_max_deviation", 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-03T02:35:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "upmap_max_optimizations": { 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "upmap_max_optimizations", 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cephadm", 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "agent_down_multiplier": { 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "agent_down_multiplier", 2024-06-03T02:35:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "3.0", 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "agent_refresh_rate": { 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "agent_refresh_rate", 2024-06-03T02:35:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "20", 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "agent_starting_port": { 2024-06-03T02:35:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "agent_starting_port", 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "4721", 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_ptrace": { 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "allow_ptrace", 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "autotune_interval": { 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "autotune_interval", 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "autotune_memory_target_ratio": { 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "autotune_memory_target_ratio", 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0.7", 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm_log_destination": { 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cephadm_log_destination", 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "file", 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "file,syslog", 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "syslog" 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-03T02:35:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "cgroups_split": { 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cgroups_split", 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "config_checks_enabled": { 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "config_checks_enabled", 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-03T02:35:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "config_dashboard": { 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "config_dashboard", 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_alertmanager": { 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_alertmanager", 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_base": { 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_base", 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Container image name, without the tag", 2024-06-03T02:35:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_elasticsearch": { 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_elasticsearch", 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "elasticsearch container image", 2024-06-03T02:35:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_grafana": { 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_grafana", 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_haproxy": { 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_haproxy", 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "HAproxy container image", 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_jaeger_agent": { 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_jaeger_agent", 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Jaeger agent container image", 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_jaeger_collector": { 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_jaeger_collector", 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Jaeger collector container image", 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_jaeger_query": { 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_jaeger_query", 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Jaeger query container image", 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_keepalived": { 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_keepalived", 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Keepalived container image", 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_loki": { 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_loki", 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Loki container image", 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_node_exporter": { 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_node_exporter", 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_nvmeof": { 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_nvmeof", 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Nvme-of container image", 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_prometheus": { 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_prometheus", 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Prometheus container image", 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_promtail": { 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_promtail", 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Promtail container image", 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_samba": { 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_samba", 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Samba/SMB container image", 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image_snmp_gateway": { 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_image_snmp_gateway", 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "SNMP Gateway container image", 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "container_init": { 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "container_init", 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Run podman/docker with `--init`", 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "daemon_cache_timeout": { 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "daemon_cache_timeout", 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "default_cephadm_command_timeout": { 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "default_cephadm_command_timeout", 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "900", 2024-06-03T02:35:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "default_registry": { 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "default_registry", 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "docker.io", 2024-06-03T02:35:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "device_cache_timeout": { 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "device_cache_timeout", 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "1800", 2024-06-03T02:35:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "seconds to cache device inventory", 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "device_enhanced_scan": { 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "device_enhanced_scan", 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "facts_cache_timeout": { 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "facts_cache_timeout", 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "seconds to cache host facts data", 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "grafana_dashboards_path": { 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "grafana_dashboards_path", 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "host_check_interval": { 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "host_check_interval", 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to perform a host check", 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "hw_monitoring": { 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "hw_monitoring", 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "inventory_list_all": { 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "inventory_list_all", 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "log_refresh_metadata": { 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_refresh_metadata", 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "*", 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "max_count_per_host": { 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_count_per_host", 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "max number of daemons per service per host", 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd_draining_count": { 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_osd_draining_count", 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "migration_current": { 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "migration_current", 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "internal - do not modify", 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "mode": { 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mode", 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "root", 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm-package", 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "root" 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "mode for remote execution of cephadm", 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "oob_default_addr": { 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "oob_default_addr", 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "169.254.1.1", 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "prometheus_alerts_path": { 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "prometheus_alerts_path", 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_insecure": { 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_insecure", 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_password": { 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_password", 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_url": { 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_url", 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "registry_username": { 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "registry_username", 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "secure_monitoring_stack": { 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "secure_monitoring_stack", 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "service_discovery_port": { 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "service_discovery_port", 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8765", 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "cephadm service discovery port", 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "ssh_config_file": { 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssh_config_file", 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "use_agent": { 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "use_agent", 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "use_repo_digest": { 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "use_repo_digest", 2024-06-03T02:35:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_on_failed_host_check": { 2024-06-03T02:35:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_on_failed_host_check", 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise a health warning if the host check fails", 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_on_stray_daemons": { 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_on_stray_daemons", 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_on_stray_hosts": { 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_on_stray_hosts", 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crash", 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "retain_interval": { 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "retain_interval", 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "31536000", 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_recent_interval": { 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_recent_interval", 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "1209600", 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "dashboard", 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "ALERTMANAGER_API_HOST": { 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "AUDIT_API_ENABLED": { 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "AUDIT_API_ENABLED", 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "ENABLE_BROWSABLE_API": { 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_PASSWORD": { 2024-06-03T02:35:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "admin", 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-03T02:35:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_URL": { 2024-06-03T02:35:27.594 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_URL", 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_API_USERNAME": { 2024-06-03T02:35:27.595 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "admin", 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-03T02:35:27.596 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-03T02:35:27.597 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-03T02:35:27.598 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-03T02:35:27.599 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "PROMETHEUS_API_HOST": { 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-03T02:35:27.600 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-03T02:35:27.601 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-03T02:35:27.602 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.603 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-03T02:35:27.604 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-03T02:35:27.605 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-03T02:35:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-03T02:35:27.607 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-03T02:35:27.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_ENABLED": { 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-03T02:35:27.609 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-03T02:35:27.610 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-03T02:35:27.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-03T02:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8", 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.613 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "45", 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_ACCESS_KEY": { 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-03T02:35:27.614 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-03T02:35:27.615 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "admin", 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_SECRET_KEY": { 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-03T02:35:27.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "RGW_API_SSL_VERIFY": { 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.617 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-03T02:35:27.618 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-03T02:35:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-03T02:35:27.620 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "cross_origin_url": { 2024-06-03T02:35:27.621 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cross_origin_url", 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "crt_file": { 2024-06-03T02:35:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "crt_file", 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.623 INFO:teuthology.orchestra.run.smithi079.stdout: "debug": { 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "debug", 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Enable/disable debug options", 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "jwt_token_ttl": { 2024-06-03T02:35:27.624 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "jwt_token_ttl", 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "28800", 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.625 INFO:teuthology.orchestra.run.smithi079.stdout: "key_file": { 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "key_file", 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.626 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.627 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.628 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.629 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.630 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.631 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "motd": { 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "motd", 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "The message of the day", 2024-06-03T02:35:27.632 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "redirect_resolve_ip_addr": { 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "redirect_resolve_ip_addr", 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.633 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "server_addr": { 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_addr", 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "::", 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.634 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "server_port": { 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_port", 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8080", 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.635 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "ssl": { 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssl", 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "ssl_server_port": { 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssl_server_port", 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8443", 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.637 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_behaviour": { 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_behaviour", 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "redirect", 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "redirect" 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.638 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_error_status_code": { 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_error_status_code", 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "500", 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "400", 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "599", 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.639 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "url_prefix": { 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "url_prefix", 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.640 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "devicehealth", 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_monitoring": { 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enable_monitoring", 2024-06-03T02:35:27.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "monitor device health metrics", 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.642 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.643 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.644 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.645 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "mark_out_threshold": { 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mark_out_threshold", 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "2419200", 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.648 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": { 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pool_name", 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "device_health_metrics", 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.649 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "retention_period": { 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "retention_period", 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "15552000", 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how long to retain device health metrics", 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.650 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "scrape_frequency": { 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "scrape_frequency", 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "86400", 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.651 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "self_heal": { 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "self_heal", 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.652 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how frequently to wake up and check device health", 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.653 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "warn_threshold": { 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "warn_threshold", 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "7257600", 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.654 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "diskprediction_local", 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.655 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.656 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.657 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.658 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.659 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "predict_interval": { 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "predict_interval", 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.660 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "86400", 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "predictor_model": { 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "predictor_model", 2024-06-03T02:35:27.661 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "prophetstor", 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "600", 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "influx", 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": false, 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "influxdb python module not found", 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "batch_size": { 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "batch_size", 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5000", 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-03T02:35:27.664 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "database": { 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "database", 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "ceph", 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.665 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": { 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "hostname", 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.666 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "InfluxDB server hostname", 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "30", 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "5", 2024-06-03T02:35:27.667 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.668 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.669 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.671 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.672 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "password": { 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "password", 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.673 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "password of InfluxDB server user", 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "port": { 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "port", 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.674 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "8086", 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "InfluxDB server port", 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "ssl": { 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ssl", 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "false", 2024-06-03T02:35:27.675 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "threads": { 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "threads", 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:27.676 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "32", 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "username": { 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "username", 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.677 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "username of InfluxDB server user", 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "verify_ssl": { 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "verify_ssl", 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "true", 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "insights", 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.679 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.680 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.681 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.682 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.683 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.684 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "iostat", 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.685 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.686 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.687 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.688 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.689 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "k8sevents", 2024-06-03T02:35:27.690 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_event_retention_days": { 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "ceph_event_retention_days", 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "7", 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.691 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "config_check_secs": { 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "config_check_secs", 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "10", 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.692 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.693 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.694 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.696 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.697 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "localpool", 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "failure_domain": { 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "failure_domain", 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.698 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "host", 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "failure domain for any created local pool", 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.699 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.700 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.702 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.703 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": { 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "min_size", 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.704 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "default min_size for any created local pool", 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "num_rep": { 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "num_rep", 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.705 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "3", 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "default replica count for any created local pool", 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": { 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pg_num", 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.706 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "128", 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "default pg_num for any created local pool", 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "prefix": { 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "prefix", 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.707 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "name prefix for any created local pool", 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "subtree": { 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "subtree", 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.708 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "rack", 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.709 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mds_autoscaler", 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.710 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.711 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.712 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.713 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.714 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mirroring", 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.715 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.716 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.717 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.718 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "nfs", 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.721 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.722 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.723 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.724 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.725 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "orchestrator", 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "fail_fs": { 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "fail_fs", 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.726 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.728 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.729 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.730 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.731 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "orchestrator": { 2024-06-03T02:35:27.732 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "orchestrator", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "rook", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "test_orchestrator" 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Orchestrator backend", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.733 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "osd_perf_query", 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.734 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.736 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.737 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.738 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "osd_support", 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.741 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.742 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.743 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "pg_autoscaler", 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "threshold": { 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "threshold", 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "3.0", 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1.0", 2024-06-03T02:35:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "scaling threshold", 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "progress", 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_pg_recovery_event": { 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "allow_pg_recovery_event", 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "enabled": { 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enabled", 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "max_completed_events": { 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_completed_events", 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "50", 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "number of past completed events to remember", 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "sleep_interval": { 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "sleep_interval", 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "5", 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "how long the module is going to sleep", 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "prometheus", 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "cache": { 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cache", 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "exclude_perf_counters": { 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "exclude_perf_counters", 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_stats_pools": { 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_stats_pools", 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "300", 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "scrape_interval": { 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "scrape_interval", 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "15.0", 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "server_addr": { 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_addr", 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "::", 2024-06-03T02:35:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "server_port": { 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_port", 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "9283", 2024-06-03T02:35:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "stale_cache_strategy": { 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "stale_cache_strategy", 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.774 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "log", 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_behaviour": { 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_behaviour", 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "default", 2024-06-03T02:35:27.775 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "default", 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "error" 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "standby_error_status_code": { 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "standby_error_status_code", 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "500", 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "400", 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "599", 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_support", 2024-06-03T02:35:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "max_concurrent_snap_create": { 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_concurrent_snap_create", 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10", 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "mirror_snapshot_schedule": { 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mirror_snapshot_schedule", 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "trash_purge_schedule": { 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "trash_purge_schedule", 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "restful", 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_auth": { 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enable_auth", 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "key_file": { 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "key_file", 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "server_addr": { 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_addr", 2024-06-03T02:35:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "server_port": { 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "server_port", 2024-06-03T02:35:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rgw", 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rook", 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "storage_class": { 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "storage_class", 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "local", 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "selftest", 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "roption1": { 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "roption1", 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "roption2": { 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "roption2", 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "xyz", 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption1": { 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption1", 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption2": { 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption2", 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption3": { 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption3", 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "float", 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption4": { 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption4", 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption5": { 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption5", 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption6": { 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption6", 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "rwoption7": { 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rwoption7", 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "1", 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "42", 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "testkey": { 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "testkey", 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "testlkey": { 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "testlkey", 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "testnewline": { 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "testnewline", 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "snap_schedule", 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_m_granularity": { 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "allow_m_granularity", 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "allow minute scheduled snapshots", 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "dump_on_update": { 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "dump_on_update", 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "dump database to debug log on update", 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "stats", 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "status", 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "telegraf", 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "address": { 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "address", 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "15", 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.845 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "telemetry", 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_basic": { 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_basic", 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_crash": { 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_crash", 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_device": { 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_device", 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_ident": { 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_ident", 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "channel_perf": { 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "channel_perf", 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "contact": { 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "contact", 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "description": { 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "description", 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "device_url": { 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "device_url", 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "enabled": { 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "enabled", 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "24", 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "8", 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "last_opt_revision": { 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "last_opt_revision", 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "1", 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "leaderboard": { 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "leaderboard", 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "leaderboard_description": { 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "leaderboard_description", 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "organization": { 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "organization", 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "proxy": { 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "proxy", 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "url": { 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "url", 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "test_orchestrator", 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "volumes", 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "max_concurrent_clones": { 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "max_concurrent_clones", 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "4", 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "periodic_async_work": { 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "periodic_async_work", 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Periodically check for async work", 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "snapshot_clone_delay": { 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "snapshot_clone_delay", 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "0", 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "snapshot_clone_no_wait": { 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "snapshot_clone_no_wait", 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "True", 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix", 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "can_run": true, 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "error_string": "", 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "module_options": { 2024-06-03T02:35:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "discovery_interval": { 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "discovery_interval", 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "uint", 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "100", 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "identifier": { 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "identifier", 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "interval": { 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "interval", 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "secs", 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "60", 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "log_level": { 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_level", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster": { 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster", 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_cluster_level": { 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_cluster_level", 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "info", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [ 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "critical", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "debug", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "error", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "info", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "warning" 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": { 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "log_to_file", 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "bool", 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "False", 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "zabbix_host": { 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix_host", 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "", 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "zabbix_port": { 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix_port", 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "int", 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "10051", 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "zabbix_sender": { 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "zabbix_sender", 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "str", 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "level": "advanced", 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 0, 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "min": "", 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "max": "", 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "enum_allowed": [], 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "desc": "", 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "long_desc": "", 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tags": [], 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "see_also": [] 2024-06-03T02:35:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "services": { 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "dashboard": "https://172.21.15.79:8443/" 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "always_on_modules": { 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus": [ 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "balancer", 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "crash", 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "devicehealth", 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "orchestrator", 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscaler", 2024-06-03T02:35:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "progress", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_support", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "status", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "telemetry", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "volumes" 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific": [ 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "balancer", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "crash", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "devicehealth", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "orchestrator", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscaler", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "progress", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_support", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "status", 2024-06-03T02:35:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "telemetry", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "volumes" 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy": [ 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "balancer", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "crash", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "devicehealth", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "orchestrator", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscaler", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "progress", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_support", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "status", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "telemetry", 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "volumes" 2024-06-03T02:35:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "reef": [ 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "balancer", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "crash", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "devicehealth", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "orchestrator", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscaler", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "progress", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_support", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "status", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "telemetry", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "volumes" 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "squid": [ 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "balancer", 2024-06-03T02:35:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "crash", 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "devicehealth", 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "orchestrator", 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscaler", 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "progress", 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_support", 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "status", 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "telemetry", 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "volumes" 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "last_failure_osd_epoch": 3, 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "active_clients": [ 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "devicehealth", 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:0", 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3816023784 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "libcephsqlite", 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:0", 2024-06-03T02:35:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3483233144 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "rbd_support", 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:0", 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3617441405 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "volumes", 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:0", 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 21664158 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-06-03T02:35:27.907 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:35:27.909 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:26 smithi135 ceph-mon[36653]: pgmap v977: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 623 op/s 2024-06-03T02:35:27.909 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:26 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4173357086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T02:35:27.909 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:26 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/4273363344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T02:35:27.909 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:27 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1495315403' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-03T02:35:27.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:26 smithi123 ceph-mon[36640]: pgmap v977: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 623 op/s 2024-06-03T02:35:27.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:26 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4173357086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T02:35:27.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:26 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/4273363344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T02:35:27.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:27 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1495315403' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-03T02:35:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.593+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03380a2b90 msgr2=0x7f03380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.593+0000 7f0355e00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03380a2b90 0x7f03380a5050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0344000c00 tx=0x7f03440315b0 comp rx=0 tx=0).stop 2024-06-03T02:35:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.593+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010dde0 msgr2=0x7f03501a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:35:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.593+0000 7f0355e00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010dde0 0x7f03501a5740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0348009fd0 tx=0x7f034800ef10 comp rx=0 tx=0).stop 2024-06-03T02:35:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.594+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 shutdown_connections 2024-06-03T02:35:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.594+0000 7f0355e00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010dde0 0x7f03501a5740 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.594+0000 7f0355e00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f03380a2b90 0x7f03380a5050 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.594+0000 7f0355e00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500759e0 0x7f035019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.594+0000 7f0355e00640 1 --2- 172.21.15.79:0/1495315403 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350074400 0x7f035019e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:35:27.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.594+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 >> 172.21.15.79:0/1495315403 conn(0x7f03500fcd80 msgr2=0x7f0350101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:35:27.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.594+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 shutdown_connections 2024-06-03T02:35:27.911 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:35:26.594+0000 7f0355e00640 1 -- 172.21.15.79:0/1495315403 wait complete. 2024-06-03T02:35:27.917 DEBUG:tasks.check_counter:Getting stats from a 2024-06-03T02:35:27.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-03T02:35:27.917 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:26 smithi079 ceph-mon[31819]: pgmap v977: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 623 op/s 2024-06-03T02:35:27.918 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:26 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4173357086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T02:35:27.918 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:26 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/4273363344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T02:35:27.918 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:27 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1495315403' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-03T02:35:28.369 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:35:28.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:28 smithi123 ceph-mon[36640]: pgmap v978: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.3 MiB/s wr, 498 op/s 2024-06-03T02:35:28.741 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:28 smithi079 ceph-mon[31819]: pgmap v978: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.3 MiB/s wr, 498 op/s 2024-06-03T02:35:28.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:28 smithi135 ceph-mon[36653]: pgmap v978: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.3 MiB/s wr, 498 op/s 2024-06-03T02:35:30.522 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:30 smithi079 ceph-mon[31819]: pgmap v979: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.9 MiB/s wr, 468 op/s 2024-06-03T02:35:30.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:30 smithi135 ceph-mon[36653]: pgmap v979: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.9 MiB/s wr, 468 op/s 2024-06-03T02:35:30.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:30 smithi123 ceph-mon[36640]: pgmap v979: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.9 MiB/s wr, 468 op/s 2024-06-03T02:35:31.174 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:35:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:35:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 0, 2024-06-03T02:35:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 2, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 0, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 0, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 0, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.005313039, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.000000000, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.000737384, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 352, 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 347, 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 791968, 2024-06-03T02:35:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 214656, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 2, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 1, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.134129687, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.068181625, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.047322514, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.016818358, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 346, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.092894341, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000268480 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 791968, 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 214656 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 6, 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 267, 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 3392, 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 203184, 2024-06-03T02:35:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 2, 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.048498639, 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.040514100, 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.004090972, 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 266, 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.068645830, 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000258067 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 203184 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 0, 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-06-03T02:35:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-06-03T02:35:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:35:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 148704, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 29, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-06-03T02:35:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 928, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 2, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-06-03T02:35:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 354, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 742230, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 354, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 354, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 742230, 2024-06-03T02:35:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:31.186 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:35:32.025 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-06-03T02:35:32.025 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-06-03T02:35:32.025 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-06-03T02:35:32.026 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-06-03T02:35:32.026 DEBUG:tasks.check_counter:Getting stats from d 2024-06-03T02:35:32.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-03T02:35:32.473 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:35:32.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:32 smithi079 ceph-mon[31819]: pgmap v980: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.7 MiB/s wr, 329 op/s 2024-06-03T02:35:32.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:32 smithi135 ceph-mon[36653]: pgmap v980: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.7 MiB/s wr, 329 op/s 2024-06-03T02:35:32.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:32 smithi123 ceph-mon[36640]: pgmap v980: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.7 MiB/s wr, 329 op/s 2024-06-03T02:35:34.602 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:34 smithi079 ceph-mon[31819]: pgmap v981: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-06-03T02:35:34.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:34 smithi135 ceph-mon[36653]: pgmap v981: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-06-03T02:35:34.948 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:35:34.948 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:35:34.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 3446, 2024-06-03T02:35:34.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 3446, 2024-06-03T02:35:34.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 977463, 2024-06-03T02:35:34.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 1095011, 2024-06-03T02:35:34.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 6, 2024-06-03T02:35:34.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 1, 2024-06-03T02:35:34.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.503095533, 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.134280607, 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.115521540, 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.208741328, 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3445, 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.521347464, 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000151334 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 330, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 325, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 813740, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 196186, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 5, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 3, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.087519922, 2024-06-03T02:35:34.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.026315732, 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.042854962, 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.016240509, 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 322, 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.054891338, 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000170469 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 813184, 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 195488 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 46, 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 314, 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 152316, 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 637918, 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 7, 2024-06-03T02:35:34.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 3, 2024-06-03T02:35:34.953 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.063702373, 2024-06-03T02:35:34.953 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.045094038, 2024-06-03T02:35:34.953 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.012123076, 2024-06-03T02:35:34.953 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.002400480, 2024-06-03T02:35:34.953 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:34.953 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 312, 2024-06-03T02:35:34.954 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.077669076, 2024-06-03T02:35:34.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000248939 2024-06-03T02:35:34.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.954 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:34.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.954 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.955 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 152048, 2024-06-03T02:35:34.955 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 637600 2024-06-03T02:35:34.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.955 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-06-03T02:35:34.956 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-06-03T02:35:34.956 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-06-03T02:35:34.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.956 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-06-03T02:35:34.956 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-06-03T02:35:34.956 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-06-03T02:35:34.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 5382, 2024-06-03T02:35:34.957 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.248734385, 2024-06-03T02:35:34.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000046215 2024-06-03T02:35:34.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:34.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.957 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-06-03T02:35:34.957 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-06-03T02:35:34.958 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-06-03T02:35:34.958 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.958 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.958 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:34.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.958 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-06-03T02:35:34.959 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 0, 2024-06-03T02:35:34.959 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 0, 2024-06-03T02:35:34.959 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-06-03T02:35:34.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.959 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.960 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-06-03T02:35:34.960 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-06-03T02:35:34.960 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-06-03T02:35:34.960 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-06-03T02:35:34.960 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 0, 2024-06-03T02:35:34.961 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-06-03T02:35:34.961 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-06-03T02:35:34.961 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 0, 2024-06-03T02:35:34.961 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 0, 2024-06-03T02:35:34.961 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 0, 2024-06-03T02:35:34.961 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-06-03T02:35:34.962 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 0, 2024-06-03T02:35:34.962 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-06-03T02:35:34.962 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-06-03T02:35:34.962 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-06-03T02:35:34.962 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 1, 2024-06-03T02:35:34.962 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 0, 2024-06-03T02:35:34.963 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 0, 2024-06-03T02:35:34.963 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-06-03T02:35:34.963 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-06-03T02:35:34.963 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:35:34.963 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-06-03T02:35:34.963 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-06-03T02:35:34.964 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-06-03T02:35:34.964 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-06-03T02:35:34.964 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-06-03T02:35:34.964 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-06-03T02:35:34.964 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-06-03T02:35:34.964 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-06-03T02:35:34.965 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-06-03T02:35:34.965 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-06-03T02:35:34.965 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-06-03T02:35:34.965 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 0, 2024-06-03T02:35:34.965 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 0, 2024-06-03T02:35:34.965 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-06-03T02:35:34.966 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T02:35:34.966 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-06-03T02:35:34.966 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:35:34.966 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:35:34.966 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-06-03T02:35:34.966 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T02:35:34.967 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T02:35:34.967 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-06-03T02:35:34.967 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:35:34.967 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T02:35:34.967 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-06-03T02:35:34.967 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:35:34.968 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:35:34.968 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:35:34.968 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-06-03T02:35:34.968 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-06-03T02:35:34.968 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-06-03T02:35:34.968 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 0 2024-06-03T02:35:34.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.969 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-06-03T02:35:34.969 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-06-03T02:35:34.969 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-06-03T02:35:34.969 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-06-03T02:35:34.969 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 0, 2024-06-03T02:35:34.970 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-06-03T02:35:34.970 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-06-03T02:35:34.970 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-06-03T02:35:34.970 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:35:34.970 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-06-03T02:35:34.970 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-06-03T02:35:34.970 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-06-03T02:35:34.971 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-06-03T02:35:34.971 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-06-03T02:35:34.971 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:35:34.971 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:35:34.971 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-06-03T02:35:34.971 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-06-03T02:35:34.972 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:35:34.972 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:35:34.972 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T02:35:34.972 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-06-03T02:35:34.972 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-06-03T02:35:34.972 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-06-03T02:35:34.973 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-06-03T02:35:34.973 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-06-03T02:35:34.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.973 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-06-03T02:35:34.973 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-06-03T02:35:34.973 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 0, 2024-06-03T02:35:34.974 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-06-03T02:35:34.974 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-06-03T02:35:34.974 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 7, 2024-06-03T02:35:34.974 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-06-03T02:35:34.974 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-06-03T02:35:34.975 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 0, 2024-06-03T02:35:34.975 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-06-03T02:35:34.975 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-06-03T02:35:34.975 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 2, 2024-06-03T02:35:34.975 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 2, 2024-06-03T02:35:34.975 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-06-03T02:35:34.976 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-06-03T02:35:34.976 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 4194304, 2024-06-03T02:35:34.976 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 4196260, 2024-06-03T02:35:34.976 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 4196119, 2024-06-03T02:35:34.976 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-06-03T02:35:34.976 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.977 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.977 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 7 2024-06-03T02:35:34.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.977 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-06-03T02:35:34.978 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 3, 2024-06-03T02:35:34.978 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 3, 2024-06-03T02:35:34.978 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 0, 2024-06-03T02:35:34.978 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 1, 2024-06-03T02:35:34.978 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 1, 2024-06-03T02:35:34.979 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 0, 2024-06-03T02:35:34.979 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 0, 2024-06-03T02:35:34.979 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 0, 2024-06-03T02:35:34.979 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 0, 2024-06-03T02:35:34.979 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-06-03T02:35:34.979 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 0, 2024-06-03T02:35:34.980 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 0, 2024-06-03T02:35:34.980 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 49152, 2024-06-03T02:35:34.980 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 215420 2024-06-03T02:35:34.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.980 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-06-03T02:35:34.980 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 0, 2024-06-03T02:35:34.981 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-06-03T02:35:34.981 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 0, 2024-06-03T02:35:34.981 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 0, 2024-06-03T02:35:34.981 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-06-03T02:35:34.981 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-06-03T02:35:34.981 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.981 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.982 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.982 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-06-03T02:35:34.982 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.982 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.982 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.983 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-06-03T02:35:34.983 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.983 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.983 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.984 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-06-03T02:35:34.984 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.984 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.984 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.984 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-06-03T02:35:34.984 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.985 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.985 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.985 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-06-03T02:35:34.985 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.985 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.986 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.986 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-06-03T02:35:34.986 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.986 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.986 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.987 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-06-03T02:35:34.987 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.987 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.987 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.988 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-06-03T02:35:34.988 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.988 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.988 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.988 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-06-03T02:35:34.988 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.988 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.989 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.989 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-06-03T02:35:34.989 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.989 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.989 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.989 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-06-03T02:35:34.989 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.990 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:34 smithi123 ceph-mon[36640]: pgmap v981: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-06-03T02:35:34.991 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.991 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.991 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-06-03T02:35:34.991 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.991 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.991 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.991 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.992 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.993 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.994 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.995 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-06-03T02:35:34.996 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.996 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.996 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.996 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-06-03T02:35:34.996 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.996 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.996 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.997 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-06-03T02:35:34.997 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.997 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.997 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.997 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-06-03T02:35:34.997 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.997 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.997 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.998 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-06-03T02:35:34.998 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.998 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.998 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.998 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-06-03T02:35:34.998 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.999 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.999 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:34.999 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-06-03T02:35:34.999 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:34.999 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:34.999 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:34.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.000 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-06-03T02:35:35.000 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:35.000 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:35.000 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:35.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.000 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:35:35.001 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:35:35.001 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-06-03T02:35:35.001 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:35.001 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:35.001 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:35.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:35.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-06-03T02:35:35.002 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-06-03T02:35:35.002 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 3, 2024-06-03T02:35:35.002 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 2, 2024-06-03T02:35:35.002 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 2, 2024-06-03T02:35:35.003 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-06-03T02:35:35.003 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 0, 2024-06-03T02:35:35.003 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 0, 2024-06-03T02:35:35.003 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 4012, 2024-06-03T02:35:35.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-06-03T02:35:35.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-06-03T02:35:35.004 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:35:35.004 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-06-03T02:35:35.004 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:35:35.004 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:35:35.005 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:35:35.005 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:35:35.005 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:35:35.005 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:35:35.005 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:35:35.005 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:35:35.006 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:35:35.006 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:35:35.006 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:35:35.006 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:35:35.006 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:35:35.006 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-06-03T02:35:35.007 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:35:35.007 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-06-03T02:35:35.007 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:35:35.007 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:35:35.007 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:35:35.007 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:35:35.008 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:35:35.008 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:35:35.008 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:35:35.008 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-06-03T02:35:35.008 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:35:35.008 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:35:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:35:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-06-03T02:35:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-06-03T02:35:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-06-03T02:35:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:35:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:35:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:35:35.010 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:35:35.010 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 182007, 2024-06-03T02:35:35.010 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 45, 2024-06-03T02:35:35.010 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:35:35.010 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-06-03T02:35:35.010 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-06-03T02:35:35.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-06-03T02:35:35.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:35:35.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-06-03T02:35:35.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:35:35.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-06-03T02:35:35.012 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8984, 2024-06-03T02:35:35.012 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 172, 2024-06-03T02:35:35.012 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:35:35.012 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:35:35.012 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-06-03T02:35:35.012 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 8464, 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 13, 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-06-03T02:35:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 3445, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 21, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 3, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 3445, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3445, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 33.243275939, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.009649717 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3445, 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 3446 2024-06-03T02:35:35.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 3445, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 3445, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 0, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 2254, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 0, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 1189, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 0, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 0, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-06-03T02:35:35.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 1, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-06-03T02:35:35.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-06-03T02:35:35.017 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 101, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 4, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 5, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 1, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 0, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 2, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 0 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.018 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 0, 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-06-03T02:35:35.019 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 0, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 3818, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 1577732, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 3818, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3818, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1577732, 2024-06-03T02:35:35.020 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 3445, 2024-06-03T02:35:35.021 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 1961, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3445, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1961, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:35.022 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 3445, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 3445, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3445, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 3445, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x55fc9c066780": { 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:35.025 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:35.026 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:35.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:35.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:35.026 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:35:36.201 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-06-03T02:35:36.201 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-06-03T02:35:36.202 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-06-03T02:35:36.202 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-06-03T02:35:36.202 DEBUG:tasks.check_counter:Getting stats from g 2024-06-03T02:35:36.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-03T02:35:36.646 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:35:36.795 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:36 smithi079 ceph-mon[31819]: pgmap v982: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 914 KiB/s wr, 120 op/s 2024-06-03T02:35:36.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:36 smithi135 ceph-mon[36653]: pgmap v982: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 914 KiB/s wr, 120 op/s 2024-06-03T02:35:36.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:36 smithi123 ceph-mon[36640]: pgmap v982: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 914 KiB/s wr, 120 op/s 2024-06-03T02:35:38.603 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:38 smithi079 ceph-mon[31819]: pgmap v983: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-06-03T02:35:38.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:38 smithi135 ceph-mon[36653]: pgmap v983: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-06-03T02:35:38.946 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:35:38.946 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:35:38.946 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 75, 2024-06-03T02:35:38.946 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 75, 2024-06-03T02:35:38.946 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 22171, 2024-06-03T02:35:38.946 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 141022, 2024-06-03T02:35:38.946 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 19, 2024-06-03T02:35:38.946 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.074328560, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.034020692, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.032152425, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000813915, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 74, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.021141378, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000285694 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 94, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000030636, 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000325 2024-06-03T02:35:38.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 58, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 326, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 11778, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 943769, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 39, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 6, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.094305145, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.047627342, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.040203497, 2024-06-03T02:35:38.948 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000567774, 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 322, 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.087228264, 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000270895 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 48, 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000010224, 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000213 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 616960 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.949 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:35:38.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 2549, 2024-06-03T02:35:38.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 1723, 2024-06-03T02:35:38.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 1418412, 2024-06-03T02:35:38.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 655607, 2024-06-03T02:35:38.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 8, 2024-06-03T02:35:38.950 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.354370675, 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.135112157, 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.245323080, 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.025246705, 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1724, 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.361302686, 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000209572 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.952 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3304, 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.002488587, 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000753 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 851120, 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 221920 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-06-03T02:35:38.953 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 53, 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.002872674, 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000054201 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2, 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000008065, 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000004032 2024-06-03T02:35:38.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 33, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 3, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.002247126, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000749042 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 30, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-06-03T02:35:38.955 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 11, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 34, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 11, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 23, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 18, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 4, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 3, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 99, 2024-06-03T02:35:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 48, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 29, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 22, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-06-03T02:35:38.957 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 1020, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 1547899950, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T02:35:38.958 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 0 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.959 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 106, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 64, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 63, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-06-03T02:35:38.960 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.961 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 7, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 7, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 2, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 2, 2024-06-03T02:35:38.962 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 2, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 4194304, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 4196260, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 0, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 6, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.018429783, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.003071630 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 0 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-06-03T02:35:38.963 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 38, 2024-06-03T02:35:38.964 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 40, 2024-06-03T02:35:38.964 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 2, 2024-06-03T02:35:38.964 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 58, 2024-06-03T02:35:38.964 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 62, 2024-06-03T02:35:38.964 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 4, 2024-06-03T02:35:38.964 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 34, 2024-06-03T02:35:38.964 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 38, 2024-06-03T02:35:38.964 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 4, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 3, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 3, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 50944, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 223612 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 36, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 33, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 21, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 1, 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-06-03T02:35:38.965 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1, 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.001866504, 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001866504 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.966 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2, 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000380622, 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000190311 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.968 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.969 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-06-03T02:35:38.970 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.970 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.970 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.970 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-06-03T02:35:38.970 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-06-03T02:35:38.971 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-06-03T02:35:38.972 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.973 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-06-03T02:35:38.974 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.974 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.974 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.974 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-06-03T02:35:38.974 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.974 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.974 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 3, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 2, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 2, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 2, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 939, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.978 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:35:38.979 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-06-03T02:35:38.980 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 251517, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 118, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-06-03T02:35:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8984, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 172, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 188791, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 943, 2024-06-03T02:35:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 46, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 12663, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 29, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 46, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 46, 2024-06-03T02:35:38.983 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.291993440, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.006347683 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 46, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 70 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 46, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 0, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 46, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 0, 2024-06-03T02:35:38.984 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 10, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 0, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 6, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 9, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 6, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 0, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 20, 2024-06-03T02:35:38.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:35:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-06-03T02:35:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:35:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-06-03T02:35:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-06-03T02:35:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-06-03T02:35:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:35:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-06-03T02:35:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:35:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-06-03T02:35:38.987 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 101, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 10, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 15, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 5, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 17, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 0, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 2 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-06-03T02:35:38.988 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 1, 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T02:35:38.989 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 0, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 2678, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 1187420, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 2678, 2024-06-03T02:35:38.990 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 2678, 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1187420, 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:38.991 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 46, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 12663, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 46, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 12663, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 46, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 46, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 46, 2024-06-03T02:35:38.993 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 46, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:38.994 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x55b68e374500": { 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:38.995 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 7, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 1956, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 7, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 6, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1956, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:38.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:38.997 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:35:38.997 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:38 smithi123 ceph-mon[36640]: pgmap v983: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-06-03T02:35:40.124 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-06-03T02:35:40.124 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-06-03T02:35:40.124 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-06-03T02:35:40.124 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=106 2024-06-03T02:35:40.124 DEBUG:tasks.check_counter:Getting stats from j 2024-06-03T02:35:40.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-03T02:35:40.575 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:35:40.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:40 smithi079 ceph-mon[31819]: pgmap v984: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T02:35:40.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:40 smithi135 ceph-mon[36653]: pgmap v984: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T02:35:40.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:40 smithi123 ceph-mon[36640]: pgmap v984: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T02:35:42.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:42 smithi079 ceph-mon[31819]: pgmap v985: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-06-03T02:35:42.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:42 smithi135 ceph-mon[36653]: pgmap v985: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-06-03T02:35:42.932 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-06-03T02:35:42.932 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:35:42.932 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 1545, 2024-06-03T02:35:42.932 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 1538, 2024-06-03T02:35:42.932 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 1220208, 2024-06-03T02:35:42.932 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 595714, 2024-06-03T02:35:42.932 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 5, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 1, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.356153180, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.140102844, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.103365524, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.094247017, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1537, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.396966110, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000258273 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.933 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 825408, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 222064 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 2250, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 2521, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 573703, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 1239719, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 5, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-06-03T02:35:42.934 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.343340543, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.121212556, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.069599717, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.127688439, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2519, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.436837850, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000173417 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.935 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 524480 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 8, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 8, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 4701, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 1732, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 8, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.005428363, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.000231832, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.004800027, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000112423, 2024-06-03T02:35:42.936 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 7, 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.005710671, 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000815810 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-06-03T02:35:42.937 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 5440, 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.218776358, 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000040216 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-06-03T02:35:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 0, 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 0, 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-06-03T02:35:42.939 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-06-03T02:35:42.940 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 1, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-06-03T02:35:42.941 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T02:35:42.942 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 0 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-06-03T02:35:42.943 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:35:42.944 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 0, 2024-06-03T02:35:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 5, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 0, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 2, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 2, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 4194304, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 4195729, 2024-06-03T02:35:42.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 4195423, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 5 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 3, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 3, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 0, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 1, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 1, 2024-06-03T02:35:42.947 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 48760, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 215420 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 0, 2024-06-03T02:35:42.948 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.949 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.950 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.951 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-06-03T02:35:42.952 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.953 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.953 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.953 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-06-03T02:35:42.953 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.955 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-06-03T02:35:42.956 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-06-03T02:35:42.958 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-06-03T02:35:42.959 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-06-03T02:35:42.960 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 1, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 0, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 0, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 2500, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:35:42.961 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:35:42.962 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-06-03T02:35:42.963 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 200309, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 58, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:35:42.964 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8984, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 172, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 8464, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 13, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-06-03T02:35:42.965 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 3428, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 21, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 2, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 3428, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3428, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 32.222295260, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.009399736 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 3428, 2024-06-03T02:35:42.966 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 3429 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 3428, 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 3428, 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 0, 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 2244, 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 0, 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 1182, 2024-06-03T02:35:42.967 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 1, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-06-03T02:35:42.968 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-06-03T02:35:42.969 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 101, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 2, 2024-06-03T02:35:42.970 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 6, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 9, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 3, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 0, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 2, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 0 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 0, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.971 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 0, 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-06-03T02:35:42.972 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 3799, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 1434597, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 3799, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3799, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1434597, 2024-06-03T02:35:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:42.974 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 3428, 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 1430, 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3428, 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1430, 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.975 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 3428, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 3428, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3428, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 3428, 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x5572d3d0c780": { 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-06-03T02:35:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-06-03T02:35:42.979 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-06-03T02:35:42.980 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:42 smithi123 ceph-mon[36640]: pgmap v985: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-06-03T02:35:44.003 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-06-03T02:35:44.003 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-06-03T02:35:44.004 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update=0 2024-06-03T02:35:44.004 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update_receipt=0 2024-06-03T02:35:44.004 DEBUG:tasks.check_counter:Getting stats from b 2024-06-03T02:35:44.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-03T02:35:44.482 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:35:44.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:44 smithi079 ceph-mon[31819]: pgmap v986: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-03T02:35:44.840 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:44 smithi123 ceph-mon[36640]: pgmap v986: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-03T02:35:44.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:44 smithi135 ceph-mon[36653]: pgmap v986: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-03T02:35:46.525 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-03T02:35:46.525 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 16530, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 19716, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 5000716, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 17174936, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 24, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 1.076840339, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.362536266, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.548874565, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.208295856, 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 19711, 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.930513736, 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000097940 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 20872, 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003907544, 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000187 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 61872, 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 29856 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.527 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:35:46.528 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 7697, 2024-06-03T02:35:46.528 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 8768, 2024-06-03T02:35:46.528 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2075529, 2024-06-03T02:35:46.528 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 7579863, 2024-06-03T02:35:46.528 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 9, 2024-06-03T02:35:46.528 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-06-03T02:35:46.528 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.478913183, 2024-06-03T02:35:46.528 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.173348510, 2024-06-03T02:35:46.528 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.136378424, 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.140820076, 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 8769, 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.526801741, 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000060075 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3317, 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000792313, 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000238 2024-06-03T02:35:46.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 557664 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 10056, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 11560, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3510900, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 9274985, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 106, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.744372421, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.256532307, 2024-06-03T02:35:46.530 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.256106878, 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.193443997, 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 11560, 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.721549431, 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000062417 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6565, 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001573309, 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000239 2024-06-03T02:35:46.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 916624, 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 245392 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 15926, 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 5.358768271, 2024-06-03T02:35:46.532 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000336479 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 941, 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.044115993, 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000046882 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 3408, 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 3408, 2024-06-03T02:35:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3408, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 15.668771642, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004597644 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 39, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 3, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 1379, 2024-06-03T02:35:46.534 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 1379, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 1378, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 1042, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 1042, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 2, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 5616, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 4540, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-06-03T02:35:46.535 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 241, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 15765, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1016, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1498619404, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 15765, 2024-06-03T02:35:46.536 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 549, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 16, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 533, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 15216, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 315, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 2299, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:35:46.537 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 3683, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 3109, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 143, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 976 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 167, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 252, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-06-03T02:35:46.538 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 401, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 248, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 153, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 16, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-06-03T02:35:46.539 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 3, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 6267, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 6267, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 10, 2024-06-03T02:35:46.540 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 10, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 21046173, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2256, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 36.410836024, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.016139554 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-06-03T02:35:46.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 1064, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 1328, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 264, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 68, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 73, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 5, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 1379, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 1905, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 526, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 1042, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 1428, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 386, 2024-06-03T02:35:46.542 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 69196, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 231804 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 4098, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 1, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 3408, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 336, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 2, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1264, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 11.067318474, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.008755789 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 54, 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.055960784, 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001036310 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 153, 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.312575048, 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002042974 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 784, 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.927839004, 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001183468 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.545 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.547 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-06-03T02:35:46.547 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.547 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.547 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.547 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 37, 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.573012467, 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.015486823 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 473, 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.177736791, 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000375764 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 64, 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.142467401, 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002226053 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 381, 2024-06-03T02:35:46.549 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2.018271937, 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005297301 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.550 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.016243169, 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.008121584 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001279903, 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000639951 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.552 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 9, 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.025697122, 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002855235 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-06-03T02:35:46.553 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 185, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.350369542, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001893889 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.554 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 4, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 3, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 501, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 501, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 983, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.555 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:35:46.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:35:46.557 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 355036, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 1177, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 968, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 11, 2024-06-03T02:35:46.558 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8984, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 172, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 4091099, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 56846, 2024-06-03T02:35:46.559 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 19779, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 17445341, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 37, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 19779, 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-06-03T02:35:46.560 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 19779, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 133.898773599, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006769744 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 19779, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 20118 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 19779, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 15908, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 3871, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-06-03T02:35:46.561 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 11, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 127, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 2367, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 154, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 3, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 1038, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 15781, 2024-06-03T02:35:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 22, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-06-03T02:35:46.563 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-06-03T02:35:46.564 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 101, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 11, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 509, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 106 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:35:46.565 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 17, 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 5, 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 522, 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 248, 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-06-03T02:35:46.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 34279, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 7465160, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 34279, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 34279, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 7465160, 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:46.567 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:46.568 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 19779, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 17472357, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 19779, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 17472357, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:46.569 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 19779, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 19779, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 19779, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 19779, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:46.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 248, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 27016, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 248, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 111, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 27016, 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:46.571 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x5576a72aa500": { 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 6267, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 16851869, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 6267, 2024-06-03T02:35:46.572 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2256, 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 16851869, 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:46.573 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-03T02:35:46.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:46 smithi079 ceph-mon[31819]: pgmap v987: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-06-03T02:35:46.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:46 smithi135 ceph-mon[36653]: pgmap v987: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-06-03T02:35:46.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:46 smithi123 ceph-mon[36640]: pgmap v987: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-06-03T02:35:47.329 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-06-03T02:35:47.329 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-06-03T02:35:47.329 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=167 2024-06-03T02:35:47.329 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-06-03T02:35:47.330 DEBUG:tasks.check_counter:Getting stats from e 2024-06-03T02:35:47.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-03T02:35:47.594 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:35:48.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:48 smithi079 ceph-mon[31819]: pgmap v988: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-06-03T02:35:48.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:48 smithi135 ceph-mon[36653]: pgmap v988: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-06-03T02:35:48.908 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:48 smithi123 ceph-mon[36640]: pgmap v988: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-06-03T02:35:49.517 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-03T02:35:49.517 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:35:49.517 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 454, 2024-06-03T02:35:49.517 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 445, 2024-06-03T02:35:49.517 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 847715, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 626376, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 39, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.067536666, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.020477061, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.036234113, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.008881113, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 444, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.029776374, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000067063 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.518 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 32, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000004625, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000144 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 838032, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 227040 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 35, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 308, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 9477, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 677932, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 17, 2024-06-03T02:35:49.519 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.025022342, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.013117635, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.008998487, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000193736, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 304, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.023724806, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000078042 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 44, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000009268, 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000210 2024-06-03T02:35:49.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2256, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 616976 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2165, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1347, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 575755, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 433780, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 8, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.138631854, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.062990331, 2024-06-03T02:35:49.521 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.090075391, 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.005613585, 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1348, 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.069282981, 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000051396 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3329, 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000724555, 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000217 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-03T02:35:49.522 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 41, 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001610931, 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000039291 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.523 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000005389, 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000002694 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 4, 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 2, 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000385989, 2024-06-03T02:35:49.524 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000192994 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 2, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 11, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 34, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 11, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 23, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 18, 2024-06-03T02:35:49.525 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 4, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 1, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 1, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 3, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 66, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 43, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 21, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-06-03T02:35:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1020, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1547899950, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-06-03T02:35:49.527 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-06-03T02:35:49.528 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 2 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 102, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 61, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 63, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-06-03T02:35:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-06-03T02:35:49.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 5, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 5, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 2, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-06-03T02:35:49.531 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 4195729, 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4, 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.058187403, 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.014546850 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-06-03T02:35:49.532 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 38, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 40, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 2, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 58, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 61, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 3, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 34, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 38, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 4, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 1, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 2, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 1, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 53040, 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223612 2024-06-03T02:35:49.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 5, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 4, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 17, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-06-03T02:35:49.534 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000226034, 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000226034 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-06-03T02:35:49.535 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000159955, 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000159955 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-06-03T02:35:49.536 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.537 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.538 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-06-03T02:35:49.538 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.538 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.538 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.538 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.539 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-06-03T02:35:49.540 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.541 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.542 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.543 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.544 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.545 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 2, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 1, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 1, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 966, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:35:49.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:35:49.548 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 248401, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 102, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-06-03T02:35:49.549 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8984, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 172, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 189071, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 934, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-06-03T02:35:49.550 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 40, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 11727, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 33, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 40, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 40, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.297992755, 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.007449818 2024-06-03T02:35:49.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 40, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 62 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 40, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 0, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 40, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 0, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 4, 2024-06-03T02:35:49.552 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 7, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 6, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-06-03T02:35:49.553 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-06-03T02:35:49.554 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 101, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 10, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 14, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 4, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-06-03T02:35:49.555 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 14, 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 1 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-06-03T02:35:49.556 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 2650, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1170476, 2024-06-03T02:35:49.557 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 2650, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2650, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1170476, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-03T02:35:49.558 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 40, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 11727, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 40, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 11727, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.559 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 40, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 40, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 40, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 40, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.560 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-06-03T02:35:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55a4650ee500": { 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 5, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1425, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:49.562 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 5, 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 4, 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1425, 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:49.563 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-03T02:35:50.404 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-03T02:35:50.404 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-03T02:35:50.404 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-06-03T02:35:50.404 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=102 2024-06-03T02:35:50.404 DEBUG:tasks.check_counter:Getting stats from h 2024-06-03T02:35:50.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-03T02:35:50.676 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:35:50.723 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:50 smithi123 ceph-mon[36640]: pgmap v989: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-03T02:35:50.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:50 smithi079 ceph-mon[31819]: pgmap v989: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-03T02:35:50.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:50 smithi135 ceph-mon[36653]: pgmap v989: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-03T02:35:52.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:52 smithi123 ceph-mon[36640]: pgmap v990: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-06-03T02:35:52.587 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-03T02:35:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:35:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 452, 2024-06-03T02:35:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 437, 2024-06-03T02:35:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 860918, 2024-06-03T02:35:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 528462, 2024-06-03T02:35:52.587 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 35, 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.065696198, 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.020199863, 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.034650525, 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.008589626, 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 437, 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.027467407, 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000062854 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:52.588 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 38, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000005501, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000144 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 853264, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 226880 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 83, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 354, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 22259, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 748587, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 19, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-06-03T02:35:52.589 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.029591693, 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.014420539, 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.012203240, 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000361299, 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 352, 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.032918650, 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000093518 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 121, 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000023027, 2024-06-03T02:35:52.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000190 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 607728 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2163, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1357, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 571764, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 469617, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 9, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.146422875, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.066149867, 2024-06-03T02:35:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.090497510, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.005909446, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1355, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.085951419, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000063432 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3318, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000736128, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000221 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.592 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 63, 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001582755, 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000025123 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-06-03T02:35:52.593 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000023982, 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000011991 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 43, 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 1, 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000142120, 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000142120 2024-06-03T02:35:52.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 42, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 11, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 34, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 11, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 23, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 18, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 4, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 1, 2024-06-03T02:35:52.595 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 1, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 3, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 104, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 42, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 42, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 20, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-06-03T02:35:52.596 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1020, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1547899950, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T02:35:52.597 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 104, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 62, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 63, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-06-03T02:35:52.598 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-06-03T02:35:52.599 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 7, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 7, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 2, 2024-06-03T02:35:52.600 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 4196260, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.011554575, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001925762 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.601 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 38, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 40, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 2, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 58, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 62, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 4, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 34, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 38, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 4, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 1, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 1, 2024-06-03T02:35:52.602 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 52588, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223612 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 43, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 43, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 21, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-06-03T02:35:52.603 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-06-03T02:35:52.604 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000142120, 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000142120 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.605 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-06-03T02:35:52.606 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.606 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.606 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.606 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-06-03T02:35:52.606 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.606 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.607 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-06-03T02:35:52.608 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.608 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.609 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-06-03T02:35:52.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.613 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:35:52.615 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 3, 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 2, 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 1, 2024-06-03T02:35:52.616 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 970, 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:35:52.617 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-06-03T02:35:52.618 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-06-03T02:35:52.619 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 251457, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 116, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:35:52.620 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8984, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 172, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 189071, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 934, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-06-03T02:35:52.621 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 45, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 12649, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 28, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 45, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 45, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.290992940, 2024-06-03T02:35:52.622 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006466509 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 45, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 68 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 45, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 0, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 45, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-06-03T02:35:52.623 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 6, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 9, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 6, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:35:52.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-06-03T02:35:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 101, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 8, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-06-03T02:35:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 4, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 16, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 1 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.627 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 2694, 2024-06-03T02:35:52.628 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1188621, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 2694, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2694, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1188621, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-06-03T02:35:52.629 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 45, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 12649, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 45, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 12649, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.630 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 45, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 45, 2024-06-03T02:35:52.631 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 45, 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 45, 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-03T02:35:52.633 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:52.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x557366c1e500": { 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 7, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1956, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 7, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 6, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1956, 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:52.636 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:52.636 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:52.636 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:52.636 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:52.636 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-03T02:35:52.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:52 smithi079 ceph-mon[31819]: pgmap v990: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-06-03T02:35:52.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:52 smithi135 ceph-mon[36653]: pgmap v990: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-06-03T02:35:53.434 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-06-03T02:35:53.434 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-06-03T02:35:53.434 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-06-03T02:35:53.434 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=104 2024-06-03T02:35:53.434 DEBUG:tasks.check_counter:Getting stats from k 2024-06-03T02:35:53.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-03T02:35:53.672 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T02:35:54.825 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:54 smithi123 ceph-mon[36640]: pgmap v991: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-03T02:35:54.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:54 smithi079 ceph-mon[31819]: pgmap v991: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-03T02:35:54.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:54 smithi135 ceph-mon[36653]: pgmap v991: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-03T02:35:55.593 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-03T02:35:55.593 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:35:55.593 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2435, 2024-06-03T02:35:55.593 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2435, 2024-06-03T02:35:55.593 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 731136, 2024-06-03T02:35:55.593 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 773507, 2024-06-03T02:35:55.593 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2024-06-03T02:35:55.593 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-06-03T02:35:55.593 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.190144580, 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.063470712, 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.048695501, 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.061806842, 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2434, 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.125682949, 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000051636 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1364, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1357, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1070602, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 539370, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.125144037, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.037089600, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.040892826, 2024-06-03T02:35:55.595 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.041595729, 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1356, 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.052280749, 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000038555 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 826656, 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 224800 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 3, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 278, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 588, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 618030, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.014437705, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.010498771, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.001554068, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000025022, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 276, 2024-06-03T02:35:55.597 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.020338214, 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000073689 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 617712 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-06-03T02:35:55.598 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5767, 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.134079438, 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000023249 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.599 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-06-03T02:35:55.600 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 1, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2024-06-03T02:35:55.601 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-06-03T02:35:55.602 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-06-03T02:35:55.603 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-06-03T02:35:55.604 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-06-03T02:35:55.605 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-06-03T02:35:55.606 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 5, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 4195729, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 4195588, 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-06-03T02:35:55.607 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 5 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 3, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 3, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 0, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 1, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 1, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 0, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 0, 2024-06-03T02:35:55.608 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 50548, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 215420 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-06-03T02:35:55.609 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.611 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-06-03T02:35:55.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.613 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.614 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-06-03T02:35:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.617 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.619 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.620 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 2, 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2024-06-03T02:35:55.621 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 4487, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:35:55.622 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:35:55.623 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-06-03T02:35:55.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 190891, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 53, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:35:55.625 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8984, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 172, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 8464, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 13, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-06-03T02:35:55.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 3421, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 21, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 3, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 3421, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3421, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 7.086829859, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002071566 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3421, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3422 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 3421, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 3421, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 2199, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 1220, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2024-06-03T02:35:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:35:55.629 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-06-03T02:35:55.630 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 101, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 5, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 7, 2024-06-03T02:35:55.631 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 2, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.632 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3798, 2024-06-03T02:35:55.633 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1438104, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3798, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3798, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1438104, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-06-03T02:35:55.634 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3421, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 1430, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3421, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1430, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.635 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3421, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 3421, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3421, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 3421, 2024-06-03T02:35:55.636 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:55.637 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x556829cde6e0": { 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-03T02:35:55.638 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-03T02:35:55.639 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-03T02:35:56.309 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-06-03T02:35:56.310 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-06-03T02:35:56.310 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-06-03T02:35:56.310 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-06-03T02:35:56.310 DEBUG:tasks.check_counter:Getting stats from c 2024-06-03T02:35:56.310 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-03T02:35:56.627 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:35:56.758 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:56 smithi135 ceph-mon[36653]: pgmap v992: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-06-03T02:35:56.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:56 smithi079 ceph-mon[31819]: pgmap v992: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-06-03T02:35:56.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:56 smithi123 ceph-mon[36640]: pgmap v992: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-06-03T02:35:58.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:35:58 smithi135 ceph-mon[36653]: pgmap v993: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-06-03T02:35:58.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:35:58 smithi079 ceph-mon[31819]: pgmap v993: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-06-03T02:35:58.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:35:58 smithi123 ceph-mon[36640]: pgmap v993: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1767, 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1767, 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 4839760, 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 561092, 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 6, 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.168611017, 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.048497688, 2024-06-03T02:35:59.239 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.046125226, 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.059318086, 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1766, 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.098102368, 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000055550 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-03T02:35:59.240 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1697, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1690, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 9551030, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 643816, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.164823329, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.048708996, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.057717740, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.050797924, 2024-06-03T02:35:59.241 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1689, 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.103990050, 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000061569 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 825728, 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 222400 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.242 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 851, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1123, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 4410253, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 888783, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 6, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.086692643, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.034417483, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.020707416, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.023510800, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1121, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.053859335, 2024-06-03T02:35:59.243 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000048045 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 618800 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.244 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 6608, 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.170948865, 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000025869 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.245 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 0, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 0, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-06-03T02:35:59.246 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 0, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 1625, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 0, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 1625, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 1625, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 0, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 0, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 2, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 0, 2024-06-03T02:35:59.247 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-06-03T02:35:59.248 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 1016, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 1498633557, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:35:59.249 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 0, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 0 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-06-03T02:35:59.250 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:35:59.251 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 6272, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-06-03T02:35:59.252 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 0, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 10, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 2, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 21066821, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 21060364, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.253 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 6272 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 1063, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 1311, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 248, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 67, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 72, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 5, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 1625, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 1629, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 4, 2024-06-03T02:35:59.254 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 0, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 0, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 56516, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 215420 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 0, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 0, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 0, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-06-03T02:35:59.255 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.256 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.257 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.258 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.259 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.260 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-06-03T02:35:59.260 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.260 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.260 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.260 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.260 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.261 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.262 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.263 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-06-03T02:35:59.264 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.265 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.266 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-06-03T02:35:59.267 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 4, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 3, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 2, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 0, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 0, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 4488, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:35:59.268 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:35:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:35:59.270 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 236726, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 1096, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-03T02:35:59.271 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 8984, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 172, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 3756211, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 42643, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-03T02:35:59.272 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 3937, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 26, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 4, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 3937, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3937, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 25.062464317, 2024-06-03T02:35:59.273 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.006365878 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3937, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 3938 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 3937, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 3937, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 0, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 2396, 2024-06-03T02:35:59.274 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 1538, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 2, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:35:59.275 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-06-03T02:35:59.276 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 101, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 2, 2024-06-03T02:35:59.277 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 6, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 6, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 0, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 2, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 0 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 0, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-06-03T02:35:59.278 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:35:59.279 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 4311, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 18390813, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 4311, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4311, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 18390813, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.280 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 3937, 2024-06-03T02:35:59.281 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 16874375, 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3937, 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 16874375, 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:35:59.282 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 3937, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 3937, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3937, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 3937, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.283 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-06-03T02:35:59.284 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x555c1188a6e0": { 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:35:59.285 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:35:59.286 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:35:59.287 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:35:59.287 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:35:59.287 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-03T02:36:00.002 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-03T02:36:00.003 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-06-03T02:36:00.003 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-06-03T02:36:00.003 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-06-03T02:36:00.003 DEBUG:tasks.check_counter:Getting stats from f 2024-06-03T02:36:00.003 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-03T02:36:00.242 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:36:00.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:00 smithi135 ceph-mon[36653]: pgmap v994: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-03T02:36:00.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:00 smithi079 ceph-mon[31819]: pgmap v994: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-03T02:36:00.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:00 smithi123 ceph-mon[36640]: pgmap v994: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1217, 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1219, 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 321353, 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 387070, 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 7, 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.095934106, 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.028798931, 2024-06-03T02:36:02.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.025778967, 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.031886255, 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1217, 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.068107293, 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000055963 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.010 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1477, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1470, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 1097718, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 574597, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.150337253, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.041783554, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.048413602, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.053542966, 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:36:02.011 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1469, 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.078949088, 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000053743 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 826112, 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 223520 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:36:02.012 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1213, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1484, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 409120, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 915888, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.139918711, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.046719760, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.035253681, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.045229415, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1483, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.088586583, 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000059734 2024-06-03T02:36:02.013 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 531744 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-03T02:36:02.014 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 5913, 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.161952919, 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000027389 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-03T02:36:02.015 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 0, 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 0, 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.016 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 0, 2024-06-03T02:36:02.017 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 1, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-06-03T02:36:02.018 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T02:36:02.019 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-06-03T02:36:02.020 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 0 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-06-03T02:36:02.021 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-06-03T02:36:02.022 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-06-03T02:36:02.022 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-06-03T02:36:02.022 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:36:02.022 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:36:02.022 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-06-03T02:36:02.022 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-06-03T02:36:02.022 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:36:02.022 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:36:02.022 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T02:36:02.023 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-06-03T02:36:02.023 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-06-03T02:36:02.023 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-06-03T02:36:02.023 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-06-03T02:36:02.023 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 0, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 7, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 0, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-06-03T02:36:02.024 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 2, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 2, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 4196260, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 4196119, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 7 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.025 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 3, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 3, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 0, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 1, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 1, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 0, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 0, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 0, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 0, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 0, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 0, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 50480, 2024-06-03T02:36:02.026 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 215420 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 0, 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 0, 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 0, 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.027 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-06-03T02:36:02.028 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-06-03T02:36:02.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.030 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-06-03T02:36:02.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.032 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.032 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.032 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.032 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.033 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.034 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.035 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.036 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.037 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-06-03T02:36:02.038 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.038 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.038 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.038 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.038 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-06-03T02:36:02.038 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.038 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.038 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.038 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-06-03T02:36:02.039 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 3, 2024-06-03T02:36:02.040 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 2, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 2, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 0, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 0, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 4490, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:36:02.041 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:36:02.042 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-03T02:36:02.043 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 190929, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 54, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-03T02:36:02.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 8984, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 172, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 8464, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 13, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.045 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 3529, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 21, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 3, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 3529, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3529, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 18.932593333, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.005364860 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-06-03T02:36:02.046 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3529, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 3530 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 3529, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 3529, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 0, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 2310, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 0, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 1217, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 0, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 0, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-06-03T02:36:02.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 0, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 1, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 0, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:36:02.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-06-03T02:36:02.049 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 101, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 2, 2024-06-03T02:36:02.050 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 5, 2024-06-03T02:36:02.051 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 6, 2024-06-03T02:36:02.051 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 1, 2024-06-03T02:36:02.051 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-06-03T02:36:02.051 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 0, 2024-06-03T02:36:02.051 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 2, 2024-06-03T02:36:02.051 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 0 2024-06-03T02:36:02.051 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.051 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-06-03T02:36:02.052 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 0, 2024-06-03T02:36:02.052 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:36:02.052 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-06-03T02:36:02.052 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-06-03T02:36:02.052 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.052 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-06-03T02:36:02.053 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-06-03T02:36:02.053 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.053 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-06-03T02:36:02.053 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-06-03T02:36:02.053 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 3903, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 1454666, 2024-06-03T02:36:02.054 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 3903, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3903, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1454666, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-03T02:36:02.055 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 3529, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 1961, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3529, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1961, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:02.056 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 3529, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 3529, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3529, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 3529, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.057 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:36:02.058 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-06-03T02:36:02.059 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-06-03T02:36:02.059 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:02.059 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.059 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.059 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.059 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:02.059 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:02.059 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x562b78dd86e0": { 2024-06-03T02:36:02.059 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:02.060 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:02.061 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:02.061 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:02.061 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:02.061 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-03T02:36:02.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:02 smithi079 ceph-mon[31819]: pgmap v995: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-03T02:36:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:02 smithi135 ceph-mon[36653]: pgmap v995: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-03T02:36:02.924 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-03T02:36:02.924 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-06-03T02:36:02.924 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-06-03T02:36:02.924 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-06-03T02:36:02.924 DEBUG:tasks.check_counter:Getting stats from i 2024-06-03T02:36:02.925 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-03T02:36:02.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:02 smithi123 ceph-mon[36640]: pgmap v995: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-03T02:36:03.163 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:36:04.716 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:04 smithi135 ceph-mon[36653]: pgmap v996: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-03T02:36:04.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:04 smithi079 ceph-mon[31819]: pgmap v996: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-03T02:36:04.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:04 smithi123 ceph-mon[36640]: pgmap v996: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-03T02:36:05.007 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-03T02:36:05.007 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:36:05.007 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 444, 2024-06-03T02:36:05.007 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 435, 2024-06-03T02:36:05.007 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 851941, 2024-06-03T02:36:05.007 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 556471, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 37, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.068265507, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.020879114, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.035253653, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.009427207, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 432, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.042500424, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000098380 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 33, 2024-06-03T02:36:05.008 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000004740, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000143 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 845184, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 227936 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 2172, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1632, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 576422, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 1038420, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 10, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.136496220, 2024-06-03T02:36:05.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.056242310, 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.089488785, 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.007203438, 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1633, 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.109184450, 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000066861 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3332, 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000610501, 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000183 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.010 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 611760 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 75, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 77, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 20165, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 134105, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 17, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.017030086, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.003662861, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.012850406, 2024-06-03T02:36:05.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000325545, 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 75, 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.018173604, 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000242314 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 110, 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000022156, 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000201 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-03T02:36:05.012 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-06-03T02:36:05.013 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-03T02:36:05.013 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.013 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-06-03T02:36:05.013 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-03T02:36:05.013 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-03T02:36:05.013 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 56, 2024-06-03T02:36:05.013 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.001751263, 2024-06-03T02:36:05.013 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000031272 2024-06-03T02:36:05.013 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000007907, 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000003953 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 37, 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 0, 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-06-03T02:36:05.014 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 37, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 11, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 34, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 11, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 23, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-06-03T02:36:05.015 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 18, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 4, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 0, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 0, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 3, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 100, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 42, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 37, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 21, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-06-03T02:36:05.016 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 1020, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 1547899950, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-06-03T02:36:05.017 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-06-03T02:36:05.018 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 0 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 105, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 63, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 63, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-06-03T02:36:05.019 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-06-03T02:36:05.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 5, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 5, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 2, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 2, 2024-06-03T02:36:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 2, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 4195729, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.010414543, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002603635 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-06-03T02:36:05.022 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 38, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 40, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 2, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 58, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 62, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 4, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 34, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 38, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 4, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 0, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 0, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 51672, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 223612 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.023 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 37, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 37, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 17, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.024 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.025 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.026 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.027 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.028 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.028 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.028 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-06-03T02:36:05.028 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.028 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.028 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.029 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.030 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.030 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.030 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-06-03T02:36:05.030 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.030 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.030 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.030 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.030 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-06-03T02:36:05.030 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.031 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-06-03T02:36:05.035 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 2, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 1, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 0, 2024-06-03T02:36:05.036 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 0, 2024-06-03T02:36:05.037 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 964, 2024-06-03T02:36:05.037 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-06-03T02:36:05.037 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.037 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-03T02:36:05.037 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:36:05.037 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-03T02:36:05.037 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:36:05.037 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:36:05.037 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:36:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-03T02:36:05.039 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 235257, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 110, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:36:05.040 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 8984, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 172, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 188679, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 929, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-03T02:36:05.041 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 41, 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 11817, 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 30, 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 41, 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-06-03T02:36:05.042 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 41, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.307993276, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.007512031 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 41, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 63 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 41, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 0, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 41, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-06-03T02:36:05.043 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 10, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 4, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 8, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 6, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 0, 2024-06-03T02:36:05.044 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 20, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-06-03T02:36:05.045 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-06-03T02:36:05.046 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 101, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 2, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 9, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 14, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 5, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 14, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 0, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 1 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.047 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-06-03T02:36:05.048 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 2687, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 1182859, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 2687, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:36:05.049 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 2687, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1182859, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:36:05.050 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 41, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 11817, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 41, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 11817, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.051 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 41, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 41, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 41, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 41, 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:05.052 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:36:05.053 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x55c0c5c9e500": { 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 5, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 1425, 2024-06-03T02:36:05.054 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 5, 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4, 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1425, 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:05.055 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-03T02:36:05.782 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-06-03T02:36:05.782 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-06-03T02:36:05.782 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-06-03T02:36:05.782 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=105 2024-06-03T02:36:05.782 DEBUG:tasks.check_counter:Getting stats from l 2024-06-03T02:36:05.783 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-03T02:36:06.019 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T02:36:06.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:06 smithi079 ceph-mon[31819]: pgmap v997: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-03T02:36:06.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:06 smithi135 ceph-mon[36653]: pgmap v997: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-03T02:36:06.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:06 smithi123 ceph-mon[36640]: pgmap v997: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-03T02:36:07.781 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:07 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:36:07.830 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-03T02:36:07.830 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T02:36:07.830 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 4, 2024-06-03T02:36:07.830 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 4, 2024-06-03T02:36:07.830 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 3072, 2024-06-03T02:36:07.830 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 448, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 0, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.001237777, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000028729, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000473603, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000662556, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000220852 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.831 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 2, 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 281, 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 320, 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 283776, 2024-06-03T02:36:07.832 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 5, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.015172736, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.011431898, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.001124834, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 279, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.021475386, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000076972 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:07.833 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:07.834 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.834 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T02:36:07.834 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 283776 2024-06-03T02:36:07.834 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.834 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T02:36:07.834 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 370, 2024-06-03T02:36:07.834 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 365, 2024-06-03T02:36:07.834 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 796864, 2024-06-03T02:36:07.834 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 226176, 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 2, 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.052585084, 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.019838175, 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.023978173, 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.008854201, 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 364, 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.017520870, 2024-06-03T02:36:07.835 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000048134 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 796864, 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 226176 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.836 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 0, 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T02:36:07.837 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-03T02:36:07.838 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-03T02:36:07.839 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T02:36:07.839 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T02:36:07.839 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T02:36:07.839 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T02:36:07.839 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T02:36:07.839 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-03T02:36:07.839 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-03T02:36:07.839 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-03T02:36:07.839 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 148704, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 29, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-03T02:36:07.840 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-03T02:36:07.841 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 928, 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 2, 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-03T02:36:07.842 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 372, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 744570, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 372, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 372, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 744570, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-03T02:36:07.843 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-03T02:36:07.844 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:07.844 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-03T02:36:07.844 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-03T02:36:07.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:07 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:36:08.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:07 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T02:36:08.551 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-06-03T02:36:08.551 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-06-03T02:36:08.551 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-06-03T02:36:08.551 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-06-03T02:36:08.552 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-06-03T02:36:08.552 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-03T02:36:08.563 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-03T02:36:08.563 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:36:08.563 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T02:36:08.627 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:36:08.628 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T02:36:08.667 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T02:36:08.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:08 smithi135 ceph-mon[36653]: pgmap v998: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-03T02:36:08.864 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:08 smithi123 ceph-mon[36640]: pgmap v998: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-03T02:36:09.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:08 smithi079 ceph-mon[31819]: pgmap v998: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-03T02:36:09.144 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:36:09.918 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:36:09.918 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:36:09.919 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:36:09.919 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:36:09.919 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:09 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:36:09.919 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:09 smithi079 ceph-mon[31819]: pgmap v999: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-06-03T02:36:10.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:36:10.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:36:10.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:36:10.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:36:10.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:09 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:36:10.096 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:09 smithi135 ceph-mon[36653]: pgmap v999: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-06-03T02:36:10.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:36:10.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:36:10.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T02:36:10.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T02:36:10.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:09 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' 2024-06-03T02:36:10.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:09 smithi123 ceph-mon[36640]: pgmap v999: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-06-03T02:36:11.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.694+0000 7f0eca600700 1 -- 172.21.15.79:0/899330283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 msgr2=0x7f0ec4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:11.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.694+0000 7f0eca600700 1 --2- 172.21.15.79:0/899330283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 0x7f0ec4074aa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009990 tx=0x7f0eb4009ca0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.695+0000 7f0eca600700 1 -- 172.21.15.79:0/899330283 shutdown_connections 2024-06-03T02:36:11.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.695+0000 7f0eca600700 1 --2- 172.21.15.79:0/899330283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec4113660 0x7f0ec4115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.695+0000 7f0eca600700 1 --2- 172.21.15.79:0/899330283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 0x7f0ec4074aa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.695+0000 7f0eca600700 1 --2- 172.21.15.79:0/899330283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ec4074fe0 0x7f0ec4113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.695+0000 7f0eca600700 1 -- 172.21.15.79:0/899330283 >> 172.21.15.79:0/899330283 conn(0x7f0ec40795f0 msgr2=0x7f0ec4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:11.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.695+0000 7f0eca600700 1 -- 172.21.15.79:0/899330283 shutdown_connections 2024-06-03T02:36:11.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.695+0000 7f0eca600700 1 -- 172.21.15.79:0/899330283 wait complete. 2024-06-03T02:36:11.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.696+0000 7f0eca600700 1 Processor -- start 2024-06-03T02:36:11.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.696+0000 7f0eca600700 1 -- start start 2024-06-03T02:36:11.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.696+0000 7f0eca600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 0x7f0ec419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:11.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0eca600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ec4074fe0 0x7f0ec419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0eca600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec4113660 0x7f0ec41a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0eca600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4118e10 con 0x7f0ec40746c0 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0eca600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4118c90 con 0x7f0ec4113660 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0eca600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4118f90 con 0x7f0ec4074fe0 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0ec3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 0x7f0ec419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0ec3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 0x7f0ec419f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38402/0 (socket says 172.21.15.79:38402) 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0ec3400700 1 -- 172.21.15.79:0/1529724502 learned_addr learned my addr 172.21.15.79:0/1529724502 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0ec2a00700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ec4074fe0 0x7f0ec419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0ec3e00700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec4113660 0x7f0ec41a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0ec3400700 1 -- 172.21.15.79:0/1529724502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ec4074fe0 msgr2=0x7f0ec419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.697+0000 7f0ec3400700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ec4074fe0 0x7f0ec419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.698+0000 7f0ec3400700 1 -- 172.21.15.79:0/1529724502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec4113660 msgr2=0x7f0ec41a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:11.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.698+0000 7f0ec3400700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec4113660 0x7f0ec41a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.698+0000 7f0ec3400700 1 -- 172.21.15.79:0/1529724502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec41a42b0 con 0x7f0ec40746c0 2024-06-03T02:36:11.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.698+0000 7f0ec3400700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 0x7f0ec419f2a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000c00 tx=0x7f0eb400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:11.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.698+0000 7f0ebbe00700 1 -- 172.21.15.79:0/1529724502 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4027070 con 0x7f0ec40746c0 2024-06-03T02:36:11.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.698+0000 7f0ebbe00700 1 -- 172.21.15.79:0/1529724502 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb40056d0 con 0x7f0ec40746c0 2024-06-03T02:36:11.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.698+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec41a4540 con 0x7f0ec40746c0 2024-06-03T02:36:11.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.698+0000 7f0ebbe00700 1 -- 172.21.15.79:0/1529724502 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4022e10 con 0x7f0ec40746c0 2024-06-03T02:36:11.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.698+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec41abe20 con 0x7f0ec40746c0 2024-06-03T02:36:11.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.700+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec4075db0 con 0x7f0ec40746c0 2024-06-03T02:36:11.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.702+0000 7f0ebbe00700 1 -- 172.21.15.79:0/1529724502 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0eb402a070 con 0x7f0ec40746c0 2024-06-03T02:36:11.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.703+0000 7f0ebbe00700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ea80a2c20 0x7f0ea80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:11.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.703+0000 7f0ebbe00700 1 -- 172.21.15.79:0/1529724502 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f0eb40d0740 con 0x7f0ec40746c0 2024-06-03T02:36:11.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.703+0000 7f0ec2a00700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ea80a2c20 0x7f0ea80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:11.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.704+0000 7f0ec2a00700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ea80a2c20 0x7f0ea80a50e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ec410fee0 tx=0x7f0eb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:11.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.706+0000 7f0ebbe00700 1 -- 172.21.15.79:0/1529724502 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0eb4013070 con 0x7f0ec40746c0 2024-06-03T02:36:11.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.950+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0ec4069180 con 0x7f0ec40746c0 2024-06-03T02:36:11.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.951+0000 7f0ebbe00700 1 -- 172.21.15.79:0/1529724502 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v101) ==== 81+0+584 (secure 0 0 0) 0x7f0eb4087080 con 0x7f0ec40746c0 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6801/54480245 2024-06-04T02:05:52.008734+0000 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6800/54480245 2024-06-04T02:05:52.008734+0000 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3960540784 2024-06-04T02:05:52.008734+0000 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3085681251 2024-06-04T02:05:16.591415+0000 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6800/1845575623 2024-06-04T02:05:16.591415+0000 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/37254677 2024-06-04T02:05:16.591415+0000 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6801/1845575623 2024-06-04T02:05:16.591415+0000 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/603700731 2024-06-04T02:05:16.591415+0000 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/161718363 2024-06-04T02:05:52.008734+0000 2024-06-03T02:36:11.952 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3870624303 2024-06-04T02:05:52.008734+0000 2024-06-03T02:36:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.953+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ea80a2c20 msgr2=0x7f0ea80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ea80a2c20 0x7f0ea80a50e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ec410fee0 tx=0x7f0eb0009260 comp rx=0 tx=0).stop 2024-06-03T02:36:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 msgr2=0x7f0ec419f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:11.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 0x7f0ec419f2a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000c00 tx=0x7f0eb400bab0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 shutdown_connections 2024-06-03T02:36:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec4113660 0x7f0ec41a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f0ea80a2c20 0x7f0ea80a50e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ec40746c0 0x7f0ec419f2a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 --2- 172.21.15.79:0/1529724502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ec4074fe0 0x7f0ec419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:11.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 >> 172.21.15.79:0/1529724502 conn(0x7f0ec40795f0 msgr2=0x7f0ec4112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:11.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 shutdown_connections 2024-06-03T02:36:11.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:11.954+0000 7f0eca600700 1 -- 172.21.15.79:0/1529724502 wait complete. 2024-06-03T02:36:11.956 INFO:teuthology.orchestra.run.smithi079.stderr:listed 10 entries 2024-06-03T02:36:12.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:11 smithi079 ceph-mon[31819]: pgmap v1000: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-03T02:36:12.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:11 smithi123 ceph-mon[36640]: pgmap v1000: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-03T02:36:12.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:11 smithi135 ceph-mon[36653]: pgmap v1000: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-03T02:36:13.032 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-03T02:36:13.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:36:13.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo umount /tmp/cephfs 2024-06-03T02:36:13.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:12 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1529724502' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T02:36:13.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:12 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1529724502' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T02:36:13.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:12 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1529724502' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T02:36:13.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2024-06-03T02:36:13.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:36:13.680 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-06-03T02:36:13.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:36:13.747 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-06-03T02:36:13.797 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-06-03T02:36:13.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:36:13.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-06-03T02:36:13.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:36:13.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2024-06-03T02:36:13.969 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-03T02:36:13.969 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-06-03T02:36:13.969 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T02:36:14.010 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/1ea6ec915164f245ddb3df05634f9d6f9a9c79502f33aaa3f540bf6024aa9544/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/1ea6ec915164f245ddb3df05634f9d6f9a9c79502f33aaa3f540bf6024aa9544/diff,workdir=/var/lib/containers/storage/overlay/1ea6ec915164f245ddb3df05634f9d6f9a9c79502f33aaa3f540bf6024aa9544/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4e600d2783c181eee4075b878752b4b14217dce8c31f75708c0e282da9756ef8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/4e600d2783c181eee4075b878752b4b14217dce8c31f75708c0e282da9756ef8/diff,workdir=/var/lib/containers/storage/overlay/4e600d2783c181eee4075b878752b4b14217dce8c31f75708c0e282da9756ef8/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/1c9cea24a38e19eecda0a5413e089ebf41d4b65450bf7c592b16c64bfb9ee1ce/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/1c9cea24a38e19eecda0a5413e089ebf41d4b65450bf7c592b16c64bfb9ee1ce/diff,workdir=/var/lib/containers/storage/overlay/1c9cea24a38e19eecda0a5413e089ebf41d4b65450bf7c592b16c64bfb9ee1ce/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/21166499dd8746897f126033c21120aff54f7e6ae6f1443aaa07ac204778d007/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/21166499dd8746897f126033c21120aff54f7e6ae6f1443aaa07ac204778d007/diff,workdir=/var/lib/containers/storage/overlay/21166499dd8746897f126033c21120aff54f7e6ae6f1443aaa07ac204778d007/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1ec742493c09d25e4d7c52255c9b47a48fdcdc02d14621b8a236aa827ddc1728/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/1ec742493c09d25e4d7c52255c9b47a48fdcdc02d14621b8a236aa827ddc1728/diff,workdir=/var/lib/containers/storage/overlay/1ec742493c09d25e4d7c52255c9b47a48fdcdc02d14621b8a236aa827ddc1728/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e52bdb640a349c3aa400a620ed1a6dafb7e722f87ff840286ece4961b917f7a4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/e52bdb640a349c3aa400a620ed1a6dafb7e722f87ff840286ece4961b917f7a4/diff,workdir=/var/lib/containers/storage/overlay/e52bdb640a349c3aa400a620ed1a6dafb7e722f87ff840286ece4961b917f7a4/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b9dd7f362ec4fc3ef0ea19b63868e4a5d971462c4055f8e47bbcf66f315b2bb3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/b9dd7f362ec4fc3ef0ea19b63868e4a5d971462c4055f8e47bbcf66f315b2bb3/diff,workdir=/var/lib/containers/storage/overlay/b9dd7f362ec4fc3ef0ea19b63868e4a5d971462c4055f8e47bbcf66f315b2bb3/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ce98d459e0c4ccab3682c7089f1afae5a2aeb0bd3a18366bf35e56efde1f40b7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/ce98d459e0c4ccab3682c7089f1afae5a2aeb0bd3a18366bf35e56efde1f40b7/diff,workdir=/var/lib/containers/storage/overlay/ce98d459e0c4ccab3682c7089f1afae5a2aeb0bd3a18366bf35e56efde1f40b7/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/165525a93cfcc72ab8901c2d34ad7fba829a3fe3450b6ddb06c471fc530a18ca/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/165525a93cfcc72ab8901c2d34ad7fba829a3fe3450b6ddb06c471fc530a18ca/diff,workdir=/var/lib/containers/storage/overlay/165525a93cfcc72ab8901c2d34ad7fba829a3fe3450b6ddb06c471fc530a18ca/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/723bd6c6c92a1200415f5ebeffecf34b9ad64f1ff4a0b9887104a54eccf0931c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRWISXSN6XPQBEDAHATKXH5JLI:/var/lib/containers/storage/overlay/l/TDCAZ6XVTHKXRURLDSPUK3OJYD,upperdir=/var/lib/containers/storage/overlay/723bd6c6c92a1200415f5ebeffecf34b9ad64f1ff4a0b9887104a54eccf0931c/diff,workdir=/var/lib/containers/storage/overlay/723bd6c6c92a1200415f5ebeffecf34b9ad64f1ff4a0b9887104a54eccf0931c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 2024-06-03T02:36:14.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:36:14.010 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-06-03T02:36:14.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T02:36:14.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2024-06-03T02:36:14.179 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:13 smithi079 ceph-mon[31819]: pgmap v1001: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T02:36:14.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:13 smithi123 ceph-mon[36640]: pgmap v1001: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T02:36:14.247 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:13 smithi135 ceph-mon[36653]: pgmap v1001: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T02:36:14.330 INFO:teuthology.orchestra.run.smithi079.stderr:Cannot find device "ceph-brx" 2024-06-03T02:36:14.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T02:36:14.336 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-03T02:36:14.507 INFO:tasks.cephadm:Teardown begin 2024-06-03T02:36:14.507 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-03T02:36:14.508 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:36:14.961 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:36:16.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:15 smithi079 ceph-mon[31819]: pgmap v1002: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 211 KiB/s wr, 13 op/s 2024-06-03T02:36:16.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:15 smithi123 ceph-mon[36640]: pgmap v1002: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 211 KiB/s wr, 13 op/s 2024-06-03T02:36:16.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:15 smithi135 ceph-mon[36653]: pgmap v1002: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 211 KiB/s wr, 13 op/s 2024-06-03T02:36:17.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.561+0000 7f9768c00700 1 -- 172.21.15.79:0/2347919847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 msgr2=0x7f9764111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:17.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.562+0000 7f9768c00700 1 --2- 172.21.15.79:0/2347919847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 0x7f9764111070 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f975c009990 tx=0x7f975c009ca0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.562+0000 7f9768c00700 1 -- 172.21.15.79:0/2347919847 shutdown_connections 2024-06-03T02:36:17.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.562+0000 7f9768c00700 1 --2- 172.21.15.79:0/2347919847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 0x7f9764111070 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.562+0000 7f9768c00700 1 --2- 172.21.15.79:0/2347919847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97640748c0 0x7f9764074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.562+0000 7f9768c00700 1 --2- 172.21.15.79:0/2347919847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97640751e0 0x7f976410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.562+0000 7f9768c00700 1 -- 172.21.15.79:0/2347919847 >> 172.21.15.79:0/2347919847 conn(0x7f97640fdf10 msgr2=0x7f9764100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:17.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.562+0000 7f9768c00700 1 -- 172.21.15.79:0/2347919847 shutdown_connections 2024-06-03T02:36:17.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.562+0000 7f9768c00700 1 -- 172.21.15.79:0/2347919847 wait complete. 2024-06-03T02:36:17.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.563+0000 7f9768c00700 1 Processor -- start 2024-06-03T02:36:17.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.563+0000 7f9768c00700 1 -- start start 2024-06-03T02:36:17.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.563+0000 7f9768c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97640748c0 0x7f97641a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:17.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.563+0000 7f9768c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97640751e0 0x7f97641a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:17.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.563+0000 7f9768c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 0x7f976419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:17.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.563+0000 7f9768c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f97641148f0 con 0x7f976410ec00 2024-06-03T02:36:17.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.563+0000 7f9768c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9764114770 con 0x7f97640748c0 2024-06-03T02:36:17.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.563+0000 7f9768c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9764114a70 con 0x7f97640751e0 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 0x7f976419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 0x7f976419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:47212/0 (socket says 172.21.15.79:47212) 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763e00700 1 -- 172.21.15.79:0/314538256 learned_addr learned my addr 172.21.15.79:0/314538256 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9762a00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97640751e0 0x7f97641a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97640748c0 0x7f97641a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763e00700 1 -- 172.21.15.79:0/314538256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97640751e0 msgr2=0x7f97641a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763e00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97640751e0 0x7f97641a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763e00700 1 -- 172.21.15.79:0/314538256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97640748c0 msgr2=0x7f97641a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763e00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97640748c0 0x7f97641a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763e00700 1 -- 172.21.15.79:0/314538256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976419b3f0 con 0x7f976410ec00 2024-06-03T02:36:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.564+0000 7f9763e00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 0x7f976419ab10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f975c0098e0 tx=0x7f975c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.565+0000 7f975be00700 1 -- 172.21.15.79:0/314538256 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c027070 con 0x7f976410ec00 2024-06-03T02:36:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.565+0000 7f975be00700 1 -- 172.21.15.79:0/314538256 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975c003d70 con 0x7f976410ec00 2024-06-03T02:36:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.565+0000 7f975be00700 1 -- 172.21.15.79:0/314538256 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c00f460 con 0x7f976410ec00 2024-06-03T02:36:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.565+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976419b680 con 0x7f976410ec00 2024-06-03T02:36:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.565+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97641a77f0 con 0x7f976410ec00 2024-06-03T02:36:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.566+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9764075f70 con 0x7f976410ec00 2024-06-03T02:36:17.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.568+0000 7f975be00700 1 -- 172.21.15.79:0/314538256 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f975c02a070 con 0x7f976410ec00 2024-06-03T02:36:17.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.570+0000 7f975be00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f97480a2ca0 0x7f97480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:17.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.570+0000 7f975be00700 1 -- 172.21.15.79:0/314538256 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f975c0d0b80 con 0x7f976410ec00 2024-06-03T02:36:17.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.570+0000 7f9763400700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f97480a2ca0 0x7f97480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:17.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.571+0000 7f9763400700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f97480a2ca0 0x7f97480a5160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f975400f540 tx=0x7f975400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:17.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.573+0000 7f975be00700 1 -- 172.21.15.79:0/314538256 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f975c031020 con 0x7f976410ec00 2024-06-03T02:36:17.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.849+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97641a79c0 con 0x7f976410ec00 2024-06-03T02:36:17.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.851+0000 7f975be00700 1 -- 172.21.15.79:0/314538256 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f975c013070 con 0x7f976410ec00 2024-06-03T02:36:17.852 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:36:17.853 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:36:17.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.854+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f97480a2ca0 msgr2=0x7f97480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:17.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.854+0000 7f9768c00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f97480a2ca0 0x7f97480a5160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f975400f540 tx=0x7f975400b040 comp rx=0 tx=0).stop 2024-06-03T02:36:17.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.854+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 msgr2=0x7f976419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:17.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.854+0000 7f9768c00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 0x7f976419ab10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f975c0098e0 tx=0x7f975c00bab0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.854+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 shutdown_connections 2024-06-03T02:36:17.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.854+0000 7f9768c00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f97480a2ca0 0x7f97480a5160 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.854+0000 7f9768c00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f976410ec00 0x7f976419ab10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.854+0000 7f9768c00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97640748c0 0x7f97641a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.854+0000 7f9768c00700 1 --2- 172.21.15.79:0/314538256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97640751e0 0x7f97641a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:17.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.855+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 >> 172.21.15.79:0/314538256 conn(0x7f97640fdf10 msgr2=0x7f9764110e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:17.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.855+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 shutdown_connections 2024-06-03T02:36:17.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:17.855+0000 7f9768c00700 1 -- 172.21.15.79:0/314538256 wait complete. 2024-06-03T02:36:17.859 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:36:18.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:17 smithi123 ceph-mon[36640]: pgmap v1003: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 211 KiB/s wr, 9 op/s 2024-06-03T02:36:18.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:17 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/314538256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:36:18.273 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:17 smithi079 ceph-mon[31819]: pgmap v1003: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 211 KiB/s wr, 9 op/s 2024-06-03T02:36:18.273 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:17 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/314538256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:36:18.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:17 smithi135 ceph-mon[36653]: pgmap v1003: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 211 KiB/s wr, 9 op/s 2024-06-03T02:36:18.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:17 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/314538256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:36:18.752 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T02:36:19.224 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:36:20.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:19 smithi079 ceph-mon[31819]: pgmap v1004: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 212 KiB/s wr, 11 op/s 2024-06-03T02:36:20.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:19 smithi123 ceph-mon[36640]: pgmap v1004: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 212 KiB/s wr, 11 op/s 2024-06-03T02:36:20.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:19 smithi135 ceph-mon[36653]: pgmap v1004: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 212 KiB/s wr, 11 op/s 2024-06-03T02:36:21.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.839+0000 7f8ff0200700 1 -- 172.21.15.79:0/2124008970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 msgr2=0x7f8fe8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:21.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.839+0000 7f8ff0200700 1 --2- 172.21.15.79:0/2124008970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 0x7f8fe8102b20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0099c0 tx=0x7f8fdc009cd0 comp rx=0 tx=0).stop 2024-06-03T02:36:21.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.839+0000 7f8ff0200700 1 -- 172.21.15.79:0/2124008970 shutdown_connections 2024-06-03T02:36:21.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.839+0000 7f8ff0200700 1 --2- 172.21.15.79:0/2124008970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fe8107840 0x7f8fe810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:21.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.839+0000 7f8ff0200700 1 --2- 172.21.15.79:0/2124008970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 0x7f8fe8102b20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:21.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.839+0000 7f8ff0200700 1 --2- 172.21.15.79:0/2124008970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe81030f0 0x7f8fe8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:21.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.839+0000 7f8ff0200700 1 -- 172.21.15.79:0/2124008970 >> 172.21.15.79:0/2124008970 conn(0x7f8fe80795f0 msgr2=0x7f8fe8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:21.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.839+0000 7f8ff0200700 1 -- 172.21.15.79:0/2124008970 shutdown_connections 2024-06-03T02:36:21.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.839+0000 7f8ff0200700 1 -- 172.21.15.79:0/2124008970 wait complete. 2024-06-03T02:36:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.840+0000 7f8ff0200700 1 Processor -- start 2024-06-03T02:36:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.840+0000 7f8ff0200700 1 -- start start 2024-06-03T02:36:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8ff0200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 0x7f8fe8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8ff0200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe81030f0 0x7f8fe8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8ff0200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fe8107840 0x7f8fe8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8ff0200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8112760 con 0x7f8fe81030f0 2024-06-03T02:36:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8ff0200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81125e0 con 0x7f8fe8102740 2024-06-03T02:36:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8ff0200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81128e0 con 0x7f8fe8107840 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8fed800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 0x7f8fe8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8fece00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe81030f0 0x7f8fe8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8fed800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 0x7f8fe8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49932/0 (socket says 172.21.15.79:49932) 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8fed800700 1 -- 172.21.15.79:0/1311640107 learned_addr learned my addr 172.21.15.79:0/1311640107 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.841+0000 7f8fee200700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fe8107840 0x7f8fe8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fed800700 1 -- 172.21.15.79:0/1311640107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fe8107840 msgr2=0x7f8fe8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fed800700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fe8107840 0x7f8fe8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fed800700 1 -- 172.21.15.79:0/1311640107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe81030f0 msgr2=0x7f8fe8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fed800700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe81030f0 0x7f8fe8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fed800700 1 -- 172.21.15.79:0/1311640107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe81a70d0 con 0x7f8fe8102740 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fee200700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fe8107840 0x7f8fe8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fed800700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 0x7f8fe8074790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006010 tx=0x7f8fdc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fe6000700 1 -- 172.21.15.79:0/1311640107 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc003680 con 0x7f8fe8102740 2024-06-03T02:36:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fe6000700 1 -- 172.21.15.79:0/1311640107 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fdc0055b0 con 0x7f8fe8102740 2024-06-03T02:36:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8fe6000700 1 -- 172.21.15.79:0/1311640107 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0173f0 con 0x7f8fe8102740 2024-06-03T02:36:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.842+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe81a7360 con 0x7f8fe8102740 2024-06-03T02:36:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.843+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81a7760 con 0x7f8fe8102740 2024-06-03T02:36:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.844+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe810aa80 con 0x7f8fe8102740 2024-06-03T02:36:21.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.846+0000 7f8fe6000700 1 -- 172.21.15.79:0/1311640107 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8fdc00fca0 con 0x7f8fe8102740 2024-06-03T02:36:21.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.847+0000 7f8fe6000700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd00a2e40 0x7f8fd00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:21.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.847+0000 7f8fe6000700 1 -- 172.21.15.79:0/1311640107 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f8fdc0d0660 con 0x7f8fe8102740 2024-06-03T02:36:21.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.847+0000 7f8fece00700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd00a2e40 0x7f8fd00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:21.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.848+0000 7f8fece00700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd00a2e40 0x7f8fd00a5300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009680 tx=0x7f8fd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:21.849+0000 7f8fe6000700 1 -- 172.21.15.79:0/1311640107 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fdc013070 con 0x7f8fe8102740 2024-06-03T02:36:22.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:21 smithi079 ceph-mon[31819]: pgmap v1005: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 12 op/s 2024-06-03T02:36:22.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.126+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fe8069180 con 0x7f8fe8102740 2024-06-03T02:36:22.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.127+0000 7f8fe6000700 1 -- 172.21.15.79:0/1311640107 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f8fdc09d240 con 0x7f8fe8102740 2024-06-03T02:36:22.128 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:36:22.129 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6841/1411940491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6840","nonce":1411940491},{"type":"v1","addr":"172.21.15.79:6841","nonce":1411940491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1231185643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1231185643},{"type":"v1","addr":"172.21.15.135:6833","nonce":1231185643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T02:14:12.550646+0000","modified":"2024-06-03T02:19:22.186781+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24515,"mds_2":24500,"mds_3":24490,"mds_4":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/859282294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":859282294},{"type":"v1","addr":"172.21.15.79:6835","nonce":859282294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/296204924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":296204924},{"type":"v1","addr":"172.21.15.123:6833","nonce":296204924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3846200406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3846200406},{"type":"v1","addr":"172.21.15.135:6835","nonce":3846200406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/476881413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":476881413},{"type":"v1","addr":"172.21.15.79:6837","nonce":476881413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/755294241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":755294241},{"type":"v1","addr":"172.21.15.135:6837","nonce":755294241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.123:6835/4173566972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4173566972},{"type":"v1","addr":"172.21.15.123:6835","nonce":4173566972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6839/414076797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":414076797},{"type":"v1","addr":"172.21.15.135:6839","nonce":414076797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.123:6837/963974438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":963974438},{"type":"v1","addr":"172.21.15.123:6837","nonce":963974438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.79:6839/2483452059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2483452059},{"type":"v1","addr":"172.21.15.79:6839","nonce":2483452059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6839/1178082108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1178082108},{"type":"v1","addr":"172.21.15.123:6839","nonce":1178082108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24515,24500,24482,24539,24490]},"id":1}]} 2024-06-03T02:36:22.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.130+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd00a2e40 msgr2=0x7f8fd00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.130+0000 7f8ff0200700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd00a2e40 0x7f8fd00a5300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009680 tx=0x7f8fd4009260 comp rx=0 tx=0).stop 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.130+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 msgr2=0x7f8fe8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.130+0000 7f8ff0200700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 0x7f8fe8074790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006010 tx=0x7f8fdc00f690 comp rx=0 tx=0).stop 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.131+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 shutdown_connections 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.131+0000 7f8ff0200700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fe8107840 0x7f8fe8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.131+0000 7f8ff0200700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8102740 0x7f8fe8074790 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.131+0000 7f8ff0200700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f8fd00a2e40 0x7f8fd00a5300 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.131+0000 7f8ff0200700 1 --2- 172.21.15.79:0/1311640107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8fe81030f0 0x7f8fe8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.131+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 >> 172.21.15.79:0/1311640107 conn(0x7f8fe80795f0 msgr2=0x7f8fe8101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.131+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 shutdown_connections 2024-06-03T02:36:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:22.131+0000 7f8ff0200700 1 -- 172.21.15.79:0/1311640107 wait complete. 2024-06-03T02:36:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 30 2024-06-03T02:36:22.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:21 smithi123 ceph-mon[36640]: pgmap v1005: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 12 op/s 2024-06-03T02:36:22.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:21 smithi135 ceph-mon[36653]: pgmap v1005: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 12 op/s 2024-06-03T02:36:23.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T02:36:23.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:22 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1311640107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:36:23.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:22 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1311640107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:36:23.458 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:36:23.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:22 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1311640107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T02:36:24.296 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:23 smithi079 ceph-mon[31819]: pgmap v1006: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 213 KiB/s wr, 10 op/s 2024-06-03T02:36:24.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:23 smithi135 ceph-mon[36653]: pgmap v1006: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 213 KiB/s wr, 10 op/s 2024-06-03T02:36:24.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:23 smithi123 ceph-mon[36640]: pgmap v1006: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 213 KiB/s wr, 10 op/s 2024-06-03T02:36:25.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.917+0000 7fca62800700 1 -- 172.21.15.79:0/2127922947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca5c10a670 msgr2=0x7fca5c10aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:25.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.917+0000 7fca62800700 1 --2- 172.21.15.79:0/2127922947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca5c10a670 0x7fca5c10aa50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fca500099c0 tx=0x7fca50009cd0 comp rx=0 tx=0).stop 2024-06-03T02:36:25.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.917+0000 7fca62800700 1 -- 172.21.15.79:0/2127922947 shutdown_connections 2024-06-03T02:36:25.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.917+0000 7fca62800700 1 --2- 172.21.15.79:0/2127922947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca5c104f80 0x7fca5c111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:25.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.917+0000 7fca62800700 1 --2- 172.21.15.79:0/2127922947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca5c10a670 0x7fca5c10aa50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:25.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.917+0000 7fca62800700 1 --2- 172.21.15.79:0/2127922947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca5c1045c0 0x7fca5c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:25.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.917+0000 7fca62800700 1 -- 172.21.15.79:0/2127922947 >> 172.21.15.79:0/2127922947 conn(0x7fca5c100070 msgr2=0x7fca5c102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:25.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.917+0000 7fca62800700 1 -- 172.21.15.79:0/2127922947 shutdown_connections 2024-06-03T02:36:25.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.917+0000 7fca62800700 1 -- 172.21.15.79:0/2127922947 wait complete. 2024-06-03T02:36:25.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.918+0000 7fca62800700 1 Processor -- start 2024-06-03T02:36:25.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.919+0000 7fca62800700 1 -- start start 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.919+0000 7fca62800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca5c1045c0 0x7fca5c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.919+0000 7fca62800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca5c104f80 0x7fca5c1a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.919+0000 7fca62800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca5c10a670 0x7fca5c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.919+0000 7fca62800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c114910 con 0x7fca5c10a670 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.919+0000 7fca62800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c114790 con 0x7fca5c104f80 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.919+0000 7fca62800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c114a90 con 0x7fca5c1045c0 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca60a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca5c104f80 0x7fca5c1a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.919+0000 7fca61400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca5c1045c0 0x7fca5c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca60a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca5c104f80 0x7fca5c1a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:51430/0 (socket says 172.21.15.79:51430) 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca60a00700 1 -- 172.21.15.79:0/1518965968 learned_addr learned my addr 172.21.15.79:0/1518965968 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca61e00700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca5c10a670 0x7fca5c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:25.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca61400700 1 -- 172.21.15.79:0/1518965968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca5c104f80 msgr2=0x7fca5c1a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:25.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca61400700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca5c104f80 0x7fca5c1a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:25.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca61400700 1 -- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca5c10a670 msgr2=0x7fca5c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:25.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca61400700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca5c10a670 0x7fca5c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:25.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca61400700 1 -- 172.21.15.79:0/1518965968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c19b250 con 0x7fca5c1045c0 2024-06-03T02:36:25.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca61e00700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca5c10a670 0x7fca5c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T02:36:25.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca60a00700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca5c104f80 0x7fca5c1a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:36:25.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.920+0000 7fca61400700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca5c1045c0 0x7fca5c1a0930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca50005b40 tx=0x7fca50003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:25.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.921+0000 7fca5a000700 1 -- 172.21.15.79:0/1518965968 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50003680 con 0x7fca5c1045c0 2024-06-03T02:36:25.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.921+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c19b4e0 con 0x7fca5c1045c0 2024-06-03T02:36:25.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.921+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c1a7800 con 0x7fca5c1045c0 2024-06-03T02:36:25.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.921+0000 7fca5a000700 1 -- 172.21.15.79:0/1518965968 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca500055a0 con 0x7fca5c1045c0 2024-06-03T02:36:25.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.921+0000 7fca5a000700 1 -- 172.21.15.79:0/1518965968 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50026060 con 0x7fca5c1045c0 2024-06-03T02:36:25.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.922+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5c10cb00 con 0x7fca5c1045c0 2024-06-03T02:36:25.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.924+0000 7fca5a000700 1 -- 172.21.15.79:0/1518965968 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fca5000f580 con 0x7fca5c1045c0 2024-06-03T02:36:25.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.925+0000 7fca5a000700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca440a2e40 0x7fca440a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:25.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.925+0000 7fca5a000700 1 -- 172.21.15.79:0/1518965968 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7fca500cf8e0 con 0x7fca5c1045c0 2024-06-03T02:36:25.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.925+0000 7fca60a00700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca440a2e40 0x7fca440a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:25.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.926+0000 7fca60a00700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca440a2e40 0x7fca440a5300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fca5c109c30 tx=0x7fca48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:25.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:25.928+0000 7fca5a000700 1 -- 172.21.15.79:0/1518965968 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fca50013070 con 0x7fca5c1045c0 2024-06-03T02:36:26.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:25 smithi079 ceph-mon[31819]: pgmap v1007: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 14 op/s 2024-06-03T02:36:26.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.156+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca5c19bfe0 con 0x7fca5c1045c0 2024-06-03T02:36:26.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.158+0000 7fca5a000700 1 -- 172.21.15.79:0/1518965968 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+24805 (secure 0 0 0) 0x7fca5009c4c0 con 0x7fca5c1045c0 2024-06-03T02:36:26.160 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-06-03T02:36:26.160 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":101,"fsid":"7a1e9ad6-214d-11ef-bc9b-c7b262605968","created":"2024-06-03T02:04:43.162039+0000","modified":"2024-06-03T02:15:54.257434+0000","last_up_change":"2024-06-03T02:12:53.554535+0000","last_in_change":"2024-06-03T02:12:33.563914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T02:08:38.699457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T02:14:03.801578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T02:14:08.648682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T02:14:21.759252+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7ff0fc4b-c2d4-4408-b6cf-827bba928e4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6803","nonce":1006585875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6805","nonce":1006585875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6809","nonce":1006585875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1006585875},{"type":"v1","addr":"172.21.15.79:6807","nonce":1006585875}]},"public_addr":"172.21.15.79:6803/1006585875","cluster_addr":"172.21.15.79:6805/1006585875","heartbeat_back_addr":"172.21.15.79:6809/1006585875","heartbeat_front_addr":"172.21.15.79:6807/1006585875","state":["exists","up"]},{"osd":1,"uuid":"dc15c092-e416-48ac-8568-c7f12ae5a56d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6801","nonce":1025664412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6803","nonce":1025664412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6807","nonce":1025664412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1025664412},{"type":"v1","addr":"172.21.15.123:6805","nonce":1025664412}]},"public_addr":"172.21.15.123:6801/1025664412","cluster_addr":"172.21.15.123:6803/1025664412","heartbeat_back_addr":"172.21.15.123:6807/1025664412","heartbeat_front_addr":"172.21.15.123:6805/1025664412","state":["exists","up"]},{"osd":2,"uuid":"bf1db324-1336-4e1b-950d-3b07e158b5eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6801","nonce":3876209625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6803","nonce":3876209625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6807","nonce":3876209625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3876209625},{"type":"v1","addr":"172.21.15.135:6805","nonce":3876209625}]},"public_addr":"172.21.15.135:6801/3876209625","cluster_addr":"172.21.15.135:6803/3876209625","heartbeat_back_addr":"172.21.15.135:6807/3876209625","heartbeat_front_addr":"172.21.15.135:6805/3876209625","state":["exists","up"]},{"osd":3,"uuid":"f1394880-b33f-4511-a8df-b80c7d0273fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6811","nonce":1694165026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6813","nonce":1694165026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6817","nonce":1694165026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":1694165026},{"type":"v1","addr":"172.21.15.79:6815","nonce":1694165026}]},"public_addr":"172.21.15.79:6811/1694165026","cluster_addr":"172.21.15.79:6813/1694165026","heartbeat_back_addr":"172.21.15.79:6817/1694165026","heartbeat_front_addr":"172.21.15.79:6815/1694165026","state":["exists","up"]},{"osd":4,"uuid":"c407e697-6fbf-4447-b7ae-6a2079134052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6809","nonce":819552045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6811","nonce":819552045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6815","nonce":819552045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":819552045},{"type":"v1","addr":"172.21.15.123:6813","nonce":819552045}]},"public_addr":"172.21.15.123:6809/819552045","cluster_addr":"172.21.15.123:6811/819552045","heartbeat_back_addr":"172.21.15.123:6815/819552045","heartbeat_front_addr":"172.21.15.123:6813/819552045","state":["exists","up"]},{"osd":5,"uuid":"d3e5d988-4303-49db-9740-c3e87c443c76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6809","nonce":3110965280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6811","nonce":3110965280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6815","nonce":3110965280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3110965280},{"type":"v1","addr":"172.21.15.135:6813","nonce":3110965280}]},"public_addr":"172.21.15.135:6809/3110965280","cluster_addr":"172.21.15.135:6811/3110965280","heartbeat_back_addr":"172.21.15.135:6815/3110965280","heartbeat_front_addr":"172.21.15.135:6813/3110965280","state":["exists","up"]},{"osd":6,"uuid":"eabb74df-81e5-4e78-81b7-0d925a8317e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6819","nonce":2064944325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6821","nonce":2064944325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6825","nonce":2064944325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2064944325},{"type":"v1","addr":"172.21.15.79:6823","nonce":2064944325}]},"public_addr":"172.21.15.79:6819/2064944325","cluster_addr":"172.21.15.79:6821/2064944325","heartbeat_back_addr":"172.21.15.79:6825/2064944325","heartbeat_front_addr":"172.21.15.79:6823/2064944325","state":["exists","up"]},{"osd":7,"uuid":"eba1f465-3fc4-4eb4-88b2-01f5dd8fe7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6817","nonce":283771362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6819","nonce":283771362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6823","nonce":283771362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":283771362},{"type":"v1","addr":"172.21.15.123:6821","nonce":283771362}]},"public_addr":"172.21.15.123:6817/283771362","cluster_addr":"172.21.15.123:6819/283771362","heartbeat_back_addr":"172.21.15.123:6823/283771362","heartbeat_front_addr":"172.21.15.123:6821/283771362","state":["exists","up"]},{"osd":8,"uuid":"fd056a07-1f3b-450a-8da4-9a8c8edf22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6817","nonce":2566672579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6819","nonce":2566672579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6823","nonce":2566672579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2566672579},{"type":"v1","addr":"172.21.15.135:6821","nonce":2566672579}]},"public_addr":"172.21.15.135:6817/2566672579","cluster_addr":"172.21.15.135:6819/2566672579","heartbeat_back_addr":"172.21.15.135:6823/2566672579","heartbeat_front_addr":"172.21.15.135:6821/2566672579","state":["exists","up"]},{"osd":9,"uuid":"cbad302f-a304-4c5b-96c0-3e619ba42f9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6827","nonce":1463813812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6829","nonce":1463813812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6833","nonce":1463813812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1463813812},{"type":"v1","addr":"172.21.15.79:6831","nonce":1463813812}]},"public_addr":"172.21.15.79:6827/1463813812","cluster_addr":"172.21.15.79:6829/1463813812","heartbeat_back_addr":"172.21.15.79:6833/1463813812","heartbeat_front_addr":"172.21.15.79:6831/1463813812","state":["exists","up"]},{"osd":10,"uuid":"66fbdf38-f92c-4cb6-8724-4b1df4d472f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6825","nonce":1137082874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6827","nonce":1137082874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6831","nonce":1137082874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1137082874},{"type":"v1","addr":"172.21.15.123:6829","nonce":1137082874}]},"public_addr":"172.21.15.123:6825/1137082874","cluster_addr":"172.21.15.123:6827/1137082874","heartbeat_back_addr":"172.21.15.123:6831/1137082874","heartbeat_front_addr":"172.21.15.123:6829/1137082874","state":["exists","up"]},{"osd":11,"uuid":"8fd600f8-5366-44e9-8827-e7f9fc41e2ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6825","nonce":2083189453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6827","nonce":2083189453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6831","nonce":2083189453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":2083189453},{"type":"v1","addr":"172.21.15.135:6829","nonce":2083189453}]},"public_addr":"172.21.15.135:6825/2083189453","cluster_addr":"172.21.15.135:6827/2083189453","heartbeat_back_addr":"172.21.15.135:6831/2083189453","heartbeat_front_addr":"172.21.15.135:6829/2083189453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:07:45.554874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:10.599286+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:08:34.366332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:07.396893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:33.997208+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:09:58.858135+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:33.195580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:10:59.579422+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:25.251476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:11:58.669644+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:25.975430+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T02:12:51.581362+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6801/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:6800/54480245":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3960540784":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3085681251":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6800/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/37254677":"2024-06-04T02:05:16.591415+0000","172.21.15.79:6801/1845575623":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/603700731":"2024-06-04T02:05:16.591415+0000","172.21.15.79:0/161718363":"2024-06-04T02:05:52.008734+0000","172.21.15.79:0/3870624303":"2024-06-04T02:05:52.008734+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T02:36:26.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca440a2e40 msgr2=0x7fca440a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:26.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca440a2e40 0x7fca440a5300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fca5c109c30 tx=0x7fca48009260 comp rx=0 tx=0).stop 2024-06-03T02:36:26.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca5c1045c0 msgr2=0x7fca5c1a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:26.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca5c1045c0 0x7fca5c1a0930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca50005b40 tx=0x7fca50003a90 comp rx=0 tx=0).stop 2024-06-03T02:36:26.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 shutdown_connections 2024-06-03T02:36:26.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fca440a2e40 0x7fca440a5300 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:26.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca5c10a670 0x7fca5c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:26.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca5c1045c0 0x7fca5c1a0930 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:26.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 --2- 172.21.15.79:0/1518965968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca5c104f80 0x7fca5c1a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:26.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.161+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 >> 172.21.15.79:0/1518965968 conn(0x7fca5c100070 msgr2=0x7fca5c101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:26.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.162+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 shutdown_connections 2024-06-03T02:36:26.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:26.162+0000 7fca62800700 1 -- 172.21.15.79:0/1518965968 wait complete. 2024-06-03T02:36:26.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:25 smithi135 ceph-mon[36653]: pgmap v1007: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 14 op/s 2024-06-03T02:36:26.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:25 smithi123 ceph-mon[36640]: pgmap v1007: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 14 op/s 2024-06-03T02:36:27.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-03T02:36:27.224 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:26 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1518965968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:36:27.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:26 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1518965968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:36:27.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:26 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1518965968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T02:36:27.496 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:36:28.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:28 smithi079 ceph-mon[31819]: pgmap v1008: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-03T02:36:28.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:28 smithi135 ceph-mon[36653]: pgmap v1008: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-03T02:36:28.374 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:28 smithi123 ceph-mon[36640]: pgmap v1008: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-03T02:36:29.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.795+0000 7f0277a00700 1 -- 172.21.15.79:0/356394490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f027010a6b0 msgr2=0x7f027010aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:29.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.795+0000 7f0277a00700 1 --2- 172.21.15.79:0/356394490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f027010a6b0 0x7f027010aa90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0260009990 tx=0x7f0260009ca0 comp rx=0 tx=0).stop 2024-06-03T02:36:29.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.796+0000 7f0277a00700 1 -- 172.21.15.79:0/356394490 shutdown_connections 2024-06-03T02:36:29.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.796+0000 7f0277a00700 1 --2- 172.21.15.79:0/356394490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0270105070 0x7f0270111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:29.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.796+0000 7f0277a00700 1 --2- 172.21.15.79:0/356394490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f027010a6b0 0x7f027010aa90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:29.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.796+0000 7f0277a00700 1 --2- 172.21.15.79:0/356394490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02701046b0 0x7f0270104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:29.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.796+0000 7f0277a00700 1 -- 172.21.15.79:0/356394490 >> 172.21.15.79:0/356394490 conn(0x7f02701001a0 msgr2=0x7f02701025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:29.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.796+0000 7f0277a00700 1 -- 172.21.15.79:0/356394490 shutdown_connections 2024-06-03T02:36:29.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.796+0000 7f0277a00700 1 -- 172.21.15.79:0/356394490 wait complete. 2024-06-03T02:36:29.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.797+0000 7f0277a00700 1 Processor -- start 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.797+0000 7f0277a00700 1 -- start start 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.797+0000 7f0277a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02701046b0 0x7f027019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.797+0000 7f0277a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270105070 0x7f027019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.797+0000 7f0277a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027010a6b0 0x7f027019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.797+0000 7f0277a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f02701149e0 con 0x7f0270105070 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.797+0000 7f0277a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0270114860 con 0x7f027010a6b0 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.797+0000 7f0277a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0270114b60 con 0x7f02701046b0 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0267e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270105070 0x7f027019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0275000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02701046b0 0x7f027019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0275000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02701046b0 0x7f027019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.79:41506/0 (socket says 172.21.15.79:41506) 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0275000700 1 -- 172.21.15.79:0/1957114842 learned_addr learned my addr 172.21.15.79:0/1957114842 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:36:29.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0275a00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027010a6b0 0x7f027019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:29.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0267e00700 1 -- 172.21.15.79:0/1957114842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02701046b0 msgr2=0x7f027019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:29.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0267e00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02701046b0 0x7f027019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:29.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0267e00700 1 -- 172.21.15.79:0/1957114842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027010a6b0 msgr2=0x7f027019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:29.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0267e00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027010a6b0 0x7f027019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:29.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.798+0000 7f0267e00700 1 -- 172.21.15.79:0/1957114842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027019fe40 con 0x7f0270105070 2024-06-03T02:36:29.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.799+0000 7f0267e00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270105070 0x7f027019b370 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f025800e9f0 tx=0x7f025800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:29.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.799+0000 7f026e000700 1 -- 172.21.15.79:0/1957114842 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025800c9c0 con 0x7f0270105070 2024-06-03T02:36:29.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.799+0000 7f026e000700 1 -- 172.21.15.79:0/1957114842 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f025800cb60 con 0x7f0270105070 2024-06-03T02:36:29.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.799+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02701a0130 con 0x7f0270105070 2024-06-03T02:36:29.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.799+0000 7f026e000700 1 -- 172.21.15.79:0/1957114842 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258018680 con 0x7f0270105070 2024-06-03T02:36:29.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.799+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02701a79b0 con 0x7f0270105070 2024-06-03T02:36:29.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.800+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f027010cc40 con 0x7f0270105070 2024-06-03T02:36:29.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.802+0000 7f026e000700 1 -- 172.21.15.79:0/1957114842 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0258010430 con 0x7f0270105070 2024-06-03T02:36:29.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.804+0000 7f026e000700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f02540a2c20 0x7f02540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:29.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.804+0000 7f026e000700 1 -- 172.21.15.79:0/1957114842 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9179+0+0 (secure 0 0 0) 0x7f02580cf700 con 0x7f0270105070 2024-06-03T02:36:29.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.804+0000 7f0275000700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f02540a2c20 0x7f02540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:29.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.805+0000 7f0275000700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f02540a2c20 0x7f02540a50e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f026000b2e0 tx=0x7f0260009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:29.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:29.807+0000 7f026e000700 1 -- 172.21.15.79:0/1957114842 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f025809c250 con 0x7f0270105070 2024-06-03T02:36:30.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.080+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f027010c620 con 0x7f0270105070 2024-06-03T02:36:30.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.084+0000 7f026e000700 1 -- 172.21.15.79:0/1957114842 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f025809bb50 con 0x7f0270105070 2024-06-03T02:36:30.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.087+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f02540a2c20 msgr2=0x7f02540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:30.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.087+0000 7f0277a00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f02540a2c20 0x7f02540a50e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f026000b2e0 tx=0x7f0260009590 comp rx=0 tx=0).stop 2024-06-03T02:36:30.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.087+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270105070 msgr2=0x7f027019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:30.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.087+0000 7f0277a00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270105070 0x7f027019b370 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f025800e9f0 tx=0x7f025800ed00 comp rx=0 tx=0).stop 2024-06-03T02:36:30.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.088+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 shutdown_connections 2024-06-03T02:36:30.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.088+0000 7f0277a00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027010a6b0 0x7f027019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:30.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.088+0000 7f0277a00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02701046b0 0x7f027019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:30.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.088+0000 7f0277a00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f02540a2c20 0x7f02540a50e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:30.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.088+0000 7f0277a00700 1 --2- 172.21.15.79:0/1957114842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0270105070 0x7f027019b370 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:30.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.088+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 >> 172.21.15.79:0/1957114842 conn(0x7f02701001a0 msgr2=0x7f0270100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:30.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.088+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 shutdown_connections 2024-06-03T02:36:30.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:30.089+0000 7f0277a00700 1 -- 172.21.15.79:0/1957114842 wait complete. 2024-06-03T02:36:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-03T02:36:30.255 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-mon[31819]: pgmap v1009: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-03T02:36:30.255 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a[31815]: 2024-06-03T02:36:30.081+0000 7ff3f9a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T02:36:30.256 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g[73997]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.256 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d[73234]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.256 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d[73234]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.256 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-mds[73240]: -- 172.21.15.79:0/3745387819 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e869efa40 con 0x563e869c3000 2024-06-03T02:36:30.256 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-j[72824]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.256 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-j[72824]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.256 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-mds[72828]: -- 172.21.15.79:0/2380202976 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b7cd75a40 con 0x560b7cd48800 2024-06-03T02:36:30.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-mon[36653]: pgmap v1009: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-03T02:36:30.346 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c[67206]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.346 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c[67206]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.346 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-mds[67210]: -- 172.21.15.135:0/2779652607 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e89bc3a40 con 0x563e89b96800 2024-06-03T02:36:30.346 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c[67206]: starting mds.c at 2024-06-03T02:36:30.346 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f[66894]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.347 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f[66894]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.347 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-mds[66898]: -- 172.21.15.135:0/2708043370 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559ab91fda40 con 0x559ab91d0800 2024-06-03T02:36:30.347 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f[66894]: starting mds.f at 2024-06-03T02:36:30.347 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i[67509]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.347 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i[67509]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.347 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-mds[67513]: -- 172.21.15.135:0/527620636 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635e5aa1a40 con 0x5635e5a74c00 2024-06-03T02:36:30.347 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:36:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i[67509]: starting mds.i at 2024-06-03T02:36:30.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-mon[36640]: pgmap v1009: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-03T02:36:30.474 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e[66889]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.474 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e[66889]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.474 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-mds[66893]: -- 172.21.15.123:0/2168866816 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f84eafa40 con 0x564f84e83000 2024-06-03T02:36:30.474 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e[66889]: starting mds.e at 2024-06-03T02:36:30.474 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h[67193]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.474 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h[67193]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.475 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-mds[67197]: -- 172.21.15.123:0/2849509700 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba0410da40 con 0x55ba040e0800 2024-06-03T02:36:30.475 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h[67193]: starting mds.h at 2024-06-03T02:36:30.475 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b[67499]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.475 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b[67499]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.475 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-mds[67503]: -- 172.21.15.123:0/128918777 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558fcc1f7a40 con 0x558fcc1ca800 2024-06-03T02:36:30.475 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b[67499]: starting mds.b at 2024-06-03T02:36:30.475 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k[66578]: ignoring --setuser ceph since I am not root 2024-06-03T02:36:30.475 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k[66578]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.476 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-mds[66582]: -- 172.21.15.123:0/988584903 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f47fd87a40 con 0x55f47fd5b000 2024-06-03T02:36:30.476 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:36:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k[66578]: starting mds.k at 2024-06-03T02:36:30.551 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g[73997]: ignoring --setgroup ceph since I am not root 2024-06-03T02:36:30.551 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-mds[74007]: -- 172.21.15.79:0/1902246655 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f44cee5a40 con 0x55f44ceb8800 2024-06-03T02:36:30.551 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g[73997]: starting mds.g at 2024-06-03T02:36:30.551 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-j[72824]: starting mds.j at 2024-06-03T02:36:30.552 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:36:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d[73234]: starting mds.d at 2024-06-03T02:36:31.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:31 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1957114842' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T02:36:31.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:31 smithi079 ceph-mon[31819]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T02:36:31.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:31 smithi079 ceph-mon[31819]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T02:36:31.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:31 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1957114842' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T02:36:31.300 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:31 smithi079 ceph-mon[31819]: osdmap e102: 12 total, 12 up, 12 in 2024-06-03T02:36:31.300 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:31 smithi079 ceph-mon[31819]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-03T02:36:31.303 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-03T02:36:31.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:31 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1957114842' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T02:36:31.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:31 smithi135 ceph-mon[36653]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T02:36:31.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:31 smithi135 ceph-mon[36653]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T02:36:31.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:31 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1957114842' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T02:36:31.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:31 smithi135 ceph-mon[36653]: osdmap e102: 12 total, 12 up, 12 in 2024-06-03T02:36:31.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:31 smithi135 ceph-mon[36653]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-03T02:36:31.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:31 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1957114842' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T02:36:31.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:31 smithi123 ceph-mon[36640]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T02:36:31.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:31 smithi123 ceph-mon[36640]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T02:36:31.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:31 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1957114842' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T02:36:31.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:31 smithi123 ceph-mon[36640]: osdmap e102: 12 total, 12 up, 12 in 2024-06-03T02:36:31.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:31 smithi123 ceph-mon[36640]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-03T02:36:31.758 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:36:32.168 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: pgmap v1011: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 921 B/s wr, 10 op/s 2024-06-03T02:36:32.168 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6832/2987181779,v1:172.21.15.123:6833/2987181779] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6834/695577041,v1:172.21.15.123:6835/695577041] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6836/894440287,v1:172.21.15.123:6837/894440287] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.123:6838/2596748941,v1:172.21.15.123:6839/2596748941] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.135:6834/799489042,v1:172.21.15.135:6835/799489042] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.135:6836/839323648,v1:172.21.15.135:6838/839323648] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.135:6837/3903559030,v1:172.21.15.135:6839/3903559030] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.79:6835/78222119,v1:172.21.15.79:6837/78222119] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.79:6834/830577879,v1:172.21.15.79:6836/830577879] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: mds.? [v2:172.21.15.79:6838/3914497683,v1:172.21.15.79:6839/3914497683] up:boot 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T02:36:32.169 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T02:36:32.170 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T02:36:32.170 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T02:36:32.170 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T02:36:32.170 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T02:36:32.170 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T02:36:32.170 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:32 smithi079 ceph-mon[31819]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T02:36:32.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: pgmap v1011: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 921 B/s wr, 10 op/s 2024-06-03T02:36:32.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6832/2987181779,v1:172.21.15.123:6833/2987181779] up:boot 2024-06-03T02:36:32.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6834/695577041,v1:172.21.15.123:6835/695577041] up:boot 2024-06-03T02:36:32.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6836/894440287,v1:172.21.15.123:6837/894440287] up:boot 2024-06-03T02:36:32.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.123:6838/2596748941,v1:172.21.15.123:6839/2596748941] up:boot 2024-06-03T02:36:32.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.135:6834/799489042,v1:172.21.15.135:6835/799489042] up:boot 2024-06-03T02:36:32.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.135:6836/839323648,v1:172.21.15.135:6838/839323648] up:boot 2024-06-03T02:36:32.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.135:6837/3903559030,v1:172.21.15.135:6839/3903559030] up:boot 2024-06-03T02:36:32.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.79:6835/78222119,v1:172.21.15.79:6837/78222119] up:boot 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.79:6834/830577879,v1:172.21.15.79:6836/830577879] up:boot 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: mds.? [v2:172.21.15.79:6838/3914497683,v1:172.21.15.79:6839/3914497683] up:boot 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T02:36:32.347 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:32 smithi135 ceph-mon[36653]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: pgmap v1011: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 921 B/s wr, 10 op/s 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6832/2987181779,v1:172.21.15.123:6833/2987181779] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6834/695577041,v1:172.21.15.123:6835/695577041] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6836/894440287,v1:172.21.15.123:6837/894440287] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.123:6838/2596748941,v1:172.21.15.123:6839/2596748941] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.135:6834/799489042,v1:172.21.15.135:6835/799489042] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.135:6836/839323648,v1:172.21.15.135:6838/839323648] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.135:6837/3903559030,v1:172.21.15.135:6839/3903559030] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.79:6835/78222119,v1:172.21.15.79:6837/78222119] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.79:6834/830577879,v1:172.21.15.79:6836/830577879] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: mds.? [v2:172.21.15.79:6838/3914497683,v1:172.21.15.79:6839/3914497683] up:boot 2024-06-03T02:36:32.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T02:36:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:32 smithi123 ceph-mon[36640]: from='mgr.14152 172.21.15.79:0/1436321268' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T02:36:34.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.083+0000 7f04b6c00700 1 -- 172.21.15.79:0/1447101317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b0113370 msgr2=0x7f04b0115760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:34.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.083+0000 7f04b6c00700 1 --2- 172.21.15.79:0/1447101317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b0113370 0x7f04b0115760 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f04a800b0a0 tx=0x7f04a800b3b0 comp rx=0 tx=0).stop 2024-06-03T02:36:34.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.083+0000 7f04b6c00700 1 -- 172.21.15.79:0/1447101317 shutdown_connections 2024-06-03T02:36:34.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.083+0000 7f04b6c00700 1 --2- 172.21.15.79:0/1447101317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b0113370 0x7f04b0115760 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:34.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.083+0000 7f04b6c00700 1 --2- 172.21.15.79:0/1447101317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b00769c0 0x7f04b0076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:34.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.083+0000 7f04b6c00700 1 --2- 172.21.15.79:0/1447101317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04b00772e0 0x7f04b0112e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:34.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.083+0000 7f04b6c00700 1 -- 172.21.15.79:0/1447101317 >> 172.21.15.79:0/1447101317 conn(0x7f04b01001c0 msgr2=0x7f04b01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:34.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.083+0000 7f04b6c00700 1 -- 172.21.15.79:0/1447101317 shutdown_connections 2024-06-03T02:36:34.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.083+0000 7f04b6c00700 1 -- 172.21.15.79:0/1447101317 wait complete. 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.084+0000 7f04b6c00700 1 Processor -- start 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.085+0000 7f04b6c00700 1 -- start start 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.085+0000 7f04b6c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b00769c0 0x7f04b019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.085+0000 7f04b6c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04b00772e0 0x7f04b019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.085+0000 7f04b6c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b0113370 0x7f04b01a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.085+0000 7f04b6c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0118e50 con 0x7f04b00769c0 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.085+0000 7f04b6c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0118cd0 con 0x7f04b00772e0 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.085+0000 7f04b6c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0118fd0 con 0x7f04b0113370 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.085+0000 7f04afe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b00769c0 0x7f04b019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.085+0000 7f04af400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04b00772e0 0x7f04b019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04af400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04b00772e0 0x7f04b019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:51528/0 (socket says 172.21.15.79:51528) 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04afe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b00769c0 0x7f04b019f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58656/0 (socket says 172.21.15.79:58656) 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04af400700 1 -- 172.21.15.79:0/2940831264 learned_addr learned my addr 172.21.15.79:0/2940831264 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04b4c00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b0113370 0x7f04b01a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04af400700 1 -- 172.21.15.79:0/2940831264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b0113370 msgr2=0x7f04b01a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04af400700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b0113370 0x7f04b01a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04af400700 1 -- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b00769c0 msgr2=0x7f04b019f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04af400700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b00769c0 0x7f04b019f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04af400700 1 -- 172.21.15.79:0/2940831264 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b01a4250 con 0x7f04b00772e0 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04b4c00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b0113370 0x7f04b01a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04afe00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b00769c0 0x7f04b019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:36:34.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.086+0000 7f04af400700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04b00772e0 0x7f04b019f780 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e9a0 tx=0x7f049c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:34.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.087+0000 7f04acc00700 1 -- 172.21.15.79:0/2940831264 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c00ca10 con 0x7f04b00772e0 2024-06-03T02:36:34.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.087+0000 7f04acc00700 1 -- 172.21.15.79:0/2940831264 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f049c00cbb0 con 0x7f04b00772e0 2024-06-03T02:36:34.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.087+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b01a4540 con 0x7f04b00772e0 2024-06-03T02:36:34.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.087+0000 7f04acc00700 1 -- 172.21.15.79:0/2940831264 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c018680 con 0x7f04b00772e0 2024-06-03T02:36:34.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.087+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b01abdc0 con 0x7f04b00772e0 2024-06-03T02:36:34.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.088+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b0069180 con 0x7f04b00772e0 2024-06-03T02:36:34.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.091+0000 7f04acc00700 1 -- 172.21.15.79:0/2940831264 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f049c010430 con 0x7f04b00772e0 2024-06-03T02:36:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.092+0000 7f04acc00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f04940a2ca0 0x7f04940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.092+0000 7f04acc00700 1 -- 172.21.15.79:0/2940831264 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9609+0+0 (secure 0 0 0) 0x7f049c0cfac0 con 0x7f04b00772e0 2024-06-03T02:36:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.092+0000 7f04afe00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f04940a2ca0 0x7f04940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:34.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.093+0000 7f04afe00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f04940a2ca0 0x7f04940a5160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04a0011490 tx=0x7f04a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:34.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.094+0000 7f04acc00700 1 -- 172.21.15.79:0/2940831264 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f049c09c460 con 0x7f04b00772e0 2024-06-03T02:36:34.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:34 smithi079 ceph-mon[31819]: pgmap v1012: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 921 B/s wr, 10 op/s 2024-06-03T02:36:34.371 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:34.370+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f04b002dca0 con 0x7f04b00772e0 2024-06-03T02:36:34.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:34 smithi123 ceph-mon[36640]: pgmap v1012: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 921 B/s wr, 10 op/s 2024-06-03T02:36:34.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:34 smithi135 ceph-mon[36653]: pgmap v1012: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 921 B/s wr, 10 op/s 2024-06-03T02:36:35.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.112+0000 7f04acc00700 1 -- 172.21.15.79:0/2940831264 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7f049c09bd60 con 0x7f04b00772e0 2024-06-03T02:36:35.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f04940a2ca0 msgr2=0x7f04940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:35.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f04940a2ca0 0x7f04940a5160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04a0011490 tx=0x7f04a0009260 comp rx=0 tx=0).stop 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04b00772e0 msgr2=0x7f04b019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04b00772e0 0x7f04b019f780 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e9a0 tx=0x7f049c00ecb0 comp rx=0 tx=0).stop 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 shutdown_connections 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04b0113370 0x7f04b01a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f04940a2ca0 0x7f04940a5160 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04b00769c0 0x7f04b019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 --2- 172.21.15.79:0/2940831264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04b00772e0 0x7f04b019f780 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.116+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 >> 172.21.15.79:0/2940831264 conn(0x7f04b01001c0 msgr2=0x7f04b01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.117+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 shutdown_connections 2024-06-03T02:36:35.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:35.117+0000 7f04b6c00700 1 -- 172.21.15.79:0/2940831264 wait complete. 2024-06-03T02:36:35.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:35 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/2940831264' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:36:35.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:35 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:36:35.559 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:35 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/2940831264' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:36:35.559 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:35 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:36:35.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:35 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/2940831264' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:36:35.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:35 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T02:36:36.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-03T02:36:36.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:36 smithi079 ceph-mon[31819]: pgmap v1013: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-06-03T02:36:36.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:36 smithi079 ceph-mon[31819]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T02:36:36.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:36 smithi079 ceph-mon[31819]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T02:36:36.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:36 smithi079 ceph-mon[31819]: Cluster is now healthy 2024-06-03T02:36:36.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:36 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-03T02:36:36.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:36 smithi079 ceph-mon[31819]: fsmap 12 up:standby 2024-06-03T02:36:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:36 smithi123 ceph-mon[36640]: pgmap v1013: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-06-03T02:36:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:36 smithi123 ceph-mon[36640]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T02:36:36.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:36 smithi123 ceph-mon[36640]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T02:36:36.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:36 smithi123 ceph-mon[36640]: Cluster is now healthy 2024-06-03T02:36:36.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:36 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-03T02:36:36.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:36 smithi123 ceph-mon[36640]: fsmap 12 up:standby 2024-06-03T02:36:36.561 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:36:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:36 smithi135 ceph-mon[36653]: pgmap v1013: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-06-03T02:36:36.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:36 smithi135 ceph-mon[36653]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T02:36:36.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:36 smithi135 ceph-mon[36653]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T02:36:36.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:36 smithi135 ceph-mon[36653]: Cluster is now healthy 2024-06-03T02:36:36.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:36 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-03T02:36:36.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:36 smithi135 ceph-mon[36653]: fsmap 12 up:standby 2024-06-03T02:36:38.263 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:38 smithi079 ceph-mon[31819]: pgmap v1014: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-06-03T02:36:38.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:38 smithi123 ceph-mon[36640]: pgmap v1014: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-06-03T02:36:38.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:38 smithi135 ceph-mon[36653]: pgmap v1014: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-06-03T02:36:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.815+0000 7fac34200700 1 -- 172.21.15.79:0/830428670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 msgr2=0x7fac2c115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.815+0000 7fac34200700 1 --2- 172.21.15.79:0/830428670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 0x7fac2c115990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fac2400b0a0 tx=0x7fac2400b3b0 comp rx=0 tx=0).stop 2024-06-03T02:36:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.815+0000 7fac34200700 1 -- 172.21.15.79:0/830428670 shutdown_connections 2024-06-03T02:36:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.815+0000 7fac34200700 1 --2- 172.21.15.79:0/830428670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 0x7fac2c115990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.815+0000 7fac34200700 1 --2- 172.21.15.79:0/830428670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac2c10eac0 0x7fac2c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:38.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.815+0000 7fac34200700 1 --2- 172.21.15.79:0/830428670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2c108ac0 0x7fac2c108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:38.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.815+0000 7fac34200700 1 -- 172.21.15.79:0/830428670 >> 172.21.15.79:0/830428670 conn(0x7fac2c075c60 msgr2=0x7fac2c1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:38.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.815+0000 7fac34200700 1 -- 172.21.15.79:0/830428670 shutdown_connections 2024-06-03T02:36:38.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.815+0000 7fac34200700 1 -- 172.21.15.79:0/830428670 wait complete. 2024-06-03T02:36:38.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.816+0000 7fac34200700 1 Processor -- start 2024-06-03T02:36:38.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.816+0000 7fac34200700 1 -- start start 2024-06-03T02:36:38.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac34200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac2c108ac0 0x7fac2c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:38.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac34200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 0x7fac2c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:38.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac34200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2c10eac0 0x7fac2c1a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:38.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac34200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c118e50 con 0x7fac2c109480 2024-06-03T02:36:38.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac34200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c118cd0 con 0x7fac2c10eac0 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac34200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c118fd0 con 0x7fac2c108ac0 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac30e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 0x7fac2c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac31800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac2c108ac0 0x7fac2c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac30e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 0x7fac2c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57704/0 (socket says 172.21.15.79:57704) 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac30e00700 1 -- 172.21.15.79:0/524270083 learned_addr learned my addr 172.21.15.79:0/524270083 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac32200700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2c10eac0 0x7fac2c1a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac31800700 1 -- 172.21.15.79:0/524270083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2c10eac0 msgr2=0x7fac2c1a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac31800700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2c10eac0 0x7fac2c1a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac31800700 1 -- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 msgr2=0x7fac2c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac31800700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 0x7fac2c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac31800700 1 -- 172.21.15.79:0/524270083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c1a41d0 con 0x7fac2c108ac0 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac30e00700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 0x7fac2c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:36:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.817+0000 7fac32200700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2c10eac0 0x7fac2c1a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:36:38.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.818+0000 7fac31800700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac2c108ac0 0x7fac2c19f1c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fac2000e9f0 tx=0x7fac2000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:38.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.818+0000 7fac2a000700 1 -- 172.21.15.79:0/524270083 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20004220 con 0x7fac2c108ac0 2024-06-03T02:36:38.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.818+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c1a44c0 con 0x7fac2c108ac0 2024-06-03T02:36:38.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.818+0000 7fac2a000700 1 -- 172.21.15.79:0/524270083 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac20004d10 con 0x7fac2c108ac0 2024-06-03T02:36:38.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.818+0000 7fac2a000700 1 -- 172.21.15.79:0/524270083 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20005020 con 0x7fac2c108ac0 2024-06-03T02:36:38.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.818+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c1abd40 con 0x7fac2c108ac0 2024-06-03T02:36:38.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.819+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c069180 con 0x7fac2c108ac0 2024-06-03T02:36:38.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.822+0000 7fac2a000700 1 -- 172.21.15.79:0/524270083 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fac20042ae0 con 0x7fac2c108ac0 2024-06-03T02:36:38.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.823+0000 7fac2a000700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac140a2c20 0x7fac140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:38.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.823+0000 7fac2a000700 1 -- 172.21.15.79:0/524270083 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9609+0+0 (secure 0 0 0) 0x7fac200cf400 con 0x7fac2c108ac0 2024-06-03T02:36:38.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.823+0000 7fac30e00700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac140a2c20 0x7fac140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:38.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.824+0000 7fac30e00700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac140a2c20 0x7fac140a50e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac2c077b30 tx=0x7fac18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:38.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:38.826+0000 7fac2a000700 1 -- 172.21.15.79:0/524270083 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fac2001f3a0 con 0x7fac2c108ac0 2024-06-03T02:36:39.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.048+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fac2c1a0520 con 0x7fac2c108ac0 2024-06-03T02:36:39.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.125+0000 7fac2a000700 1 -- 172.21.15.79:0/524270083 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7fac2009bda0 con 0x7fac2c108ac0 2024-06-03T02:36:39.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.131+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac140a2c20 msgr2=0x7fac140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:39.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.131+0000 7fac34200700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac140a2c20 0x7fac140a50e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac2c077b30 tx=0x7fac18009260 comp rx=0 tx=0).stop 2024-06-03T02:36:39.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.131+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac2c108ac0 msgr2=0x7fac2c19f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:39.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.131+0000 7fac34200700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac2c108ac0 0x7fac2c19f1c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fac2000e9f0 tx=0x7fac2000ed00 comp rx=0 tx=0).stop 2024-06-03T02:36:39.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.134+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 shutdown_connections 2024-06-03T02:36:39.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.134+0000 7fac34200700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2c10eac0 0x7fac2c1a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:39.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.134+0000 7fac34200700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac2c108ac0 0x7fac2c19f1c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:39.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.134+0000 7fac34200700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fac140a2c20 0x7fac140a50e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:39.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.134+0000 7fac34200700 1 --2- 172.21.15.79:0/524270083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac2c109480 0x7fac2c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:39.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.134+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 >> 172.21.15.79:0/524270083 conn(0x7fac2c075c60 msgr2=0x7fac2c0761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:39.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.135+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 shutdown_connections 2024-06-03T02:36:39.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:39.135+0000 7fac34200700 1 -- 172.21.15.79:0/524270083 wait complete. 2024-06-03T02:36:39.139 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' removed 2024-06-03T02:36:39.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:39 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/524270083' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:39.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:39 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:39.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:39 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/524270083' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:39.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:39 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:39.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:39 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/524270083' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:39.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:39 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:40.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-03T02:36:40.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:40 smithi079 ceph-mon[31819]: pgmap v1015: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 4 op/s 2024-06-03T02:36:40.338 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:40 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T02:36:40.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:40 smithi079 ceph-mon[31819]: osdmap e103: 12 total, 12 up, 12 in 2024-06-03T02:36:40.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:40 smithi123 ceph-mon[36640]: pgmap v1015: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 4 op/s 2024-06-03T02:36:40.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:40 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T02:36:40.474 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:40 smithi123 ceph-mon[36640]: osdmap e103: 12 total, 12 up, 12 in 2024-06-03T02:36:40.553 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:36:40.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:40 smithi135 ceph-mon[36653]: pgmap v1015: 129 pgs: 129 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 4 op/s 2024-06-03T02:36:40.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:40 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T02:36:40.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:40 smithi135 ceph-mon[36653]: osdmap e103: 12 total, 12 up, 12 in 2024-06-03T02:36:42.299 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:42 smithi079 ceph-mon[31819]: pgmap v1017: 97 pgs: 97 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:42.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:42 smithi123 ceph-mon[36640]: pgmap v1017: 97 pgs: 97 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:42.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:42 smithi135 ceph-mon[36653]: pgmap v1017: 97 pgs: 97 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:42.880 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.879+0000 7f5fa3c00700 1 -- 172.21.15.79:0/3528423646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f9c1046b0 msgr2=0x7f5f9c104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:42.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.879+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/3528423646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f9c1046b0 0x7f5f9c104b30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5f880098f0 tx=0x7f5f88009c00 comp rx=0 tx=0).stop 2024-06-03T02:36:42.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.879+0000 7f5fa3c00700 1 -- 172.21.15.79:0/3528423646 shutdown_connections 2024-06-03T02:36:42.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.879+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/3528423646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f9c105070 0x7f5f9c111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:42.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.879+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/3528423646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c10a6b0 0x7f5f9c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:42.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.879+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/3528423646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f9c1046b0 0x7f5f9c104b30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:42.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.879+0000 7f5fa3c00700 1 -- 172.21.15.79:0/3528423646 >> 172.21.15.79:0/3528423646 conn(0x7f5f9c1001a0 msgr2=0x7f5f9c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:42.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.879+0000 7f5fa3c00700 1 -- 172.21.15.79:0/3528423646 shutdown_connections 2024-06-03T02:36:42.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.879+0000 7f5fa3c00700 1 -- 172.21.15.79:0/3528423646 wait complete. 2024-06-03T02:36:42.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.880+0000 7f5fa3c00700 1 Processor -- start 2024-06-03T02:36:42.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5fa3c00700 1 -- start start 2024-06-03T02:36:42.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5fa3c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c1046b0 0x7f5f9c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:42.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5fa3c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f9c105070 0x7f5f9c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:42.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5fa3c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f9c10a6b0 0x7f5f9c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:42.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5fa3c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c114a00 con 0x7f5f9c1046b0 2024-06-03T02:36:42.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5fa3c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c114880 con 0x7f5f9c105070 2024-06-03T02:36:42.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5fa3c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c114b80 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5f9be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f9c105070 0x7f5f9c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:42.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5fa1200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c1046b0 0x7f5f9c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5fa1c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f9c10a6b0 0x7f5f9c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5f9be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f9c105070 0x7f5f9c19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.79:49108/0 (socket says 172.21.15.79:49108) 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.881+0000 7f5f9be00700 1 -- 172.21.15.79:0/1664123587 learned_addr learned my addr 172.21.15.79:0/1664123587 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.882+0000 7f5fa1c00700 1 -- 172.21.15.79:0/1664123587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f9c105070 msgr2=0x7f5f9c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.882+0000 7f5fa1c00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f9c105070 0x7f5f9c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.882+0000 7f5fa1c00700 1 -- 172.21.15.79:0/1664123587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c1046b0 msgr2=0x7f5f9c19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.882+0000 7f5fa1c00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c1046b0 0x7f5f9c19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.882+0000 7f5fa1c00700 1 -- 172.21.15.79:0/1664123587 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c19fee0 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.882+0000 7f5f9be00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f9c105070 0x7f5f9c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.882+0000 7f5fa1c00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f9c10a6b0 0x7f5f9c19f7a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400efc0 tx=0x7f5f9400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.883+0000 7f5f99600700 1 -- 172.21.15.79:0/1664123587 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f940092e0 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.883+0000 7f5f99600700 1 -- 172.21.15.79:0/1664123587 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f94004bb0 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.883+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9c1a0170 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.883+0000 7f5f99600700 1 -- 172.21.15.79:0/1664123587 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94004280 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.883+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9c1a7a50 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.884+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9c069180 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.886+0000 7f5f99600700 1 -- 172.21.15.79:0/1664123587 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5f9401b030 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.887+0000 7f5f99600700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f840a2c20 0x7f5f840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:42.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.888+0000 7f5f99600700 1 -- 172.21.15.79:0/1664123587 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9055+0+0 (secure 0 0 0) 0x7f5f940cf8f0 con 0x7f5f9c10a6b0 2024-06-03T02:36:42.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.888+0000 7f5fa1200700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f840a2c20 0x7f5f840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:42.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.888+0000 7f5fa1200700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f840a2c20 0x7f5f840a50e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005fd0 tx=0x7f5f90009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:42.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:42.890+0000 7f5f99600700 1 -- 172.21.15.79:0/1664123587 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5f94022020 con 0x7f5f9c10a6b0 2024-06-03T02:36:43.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.116+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5f9c02dca0 con 0x7f5f9c10a6b0 2024-06-03T02:36:43.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.138+0000 7f5f99600700 1 -- 172.21.15.79:0/1664123587 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f5f94010040 con 0x7f5f9c10a6b0 2024-06-03T02:36:43.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f840a2c20 msgr2=0x7f5f840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:43.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f840a2c20 0x7f5f840a50e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005fd0 tx=0x7f5f90009310 comp rx=0 tx=0).stop 2024-06-03T02:36:43.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f9c10a6b0 msgr2=0x7f5f9c19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:43.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f9c10a6b0 0x7f5f9c19f7a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400efc0 tx=0x7f5f9400c3a0 comp rx=0 tx=0).stop 2024-06-03T02:36:43.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 shutdown_connections 2024-06-03T02:36:43.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f9c10a6b0 0x7f5f9c19f7a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:43.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7f5f840a2c20 0x7f5f840a50e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:43.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c1046b0 0x7f5f9c19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:43.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 --2- 172.21.15.79:0/1664123587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f9c105070 0x7f5f9c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:43.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.142+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 >> 172.21.15.79:0/1664123587 conn(0x7f5f9c1001a0 msgr2=0x7f5f9c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:43.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.144+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 shutdown_connections 2024-06-03T02:36:43.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:43.144+0000 7f5fa3c00700 1 -- 172.21.15.79:0/1664123587 wait complete. 2024-06-03T02:36:43.148 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' removed 2024-06-03T02:36:43.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:43 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/1664123587' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:43.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:43 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:43.530 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:43 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/1664123587' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:43.530 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:43 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:43.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:43 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/1664123587' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:43.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:43 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:44.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-03T02:36:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:44 smithi123 ceph-mon[36640]: pgmap v1018: 97 pgs: 97 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:44 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T02:36:44.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:44 smithi123 ceph-mon[36640]: osdmap e104: 12 total, 12 up, 12 in 2024-06-03T02:36:44.485 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T02:36:44.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:44 smithi135 ceph-mon[36653]: pgmap v1018: 97 pgs: 97 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:44.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:44 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T02:36:44.497 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:44 smithi135 ceph-mon[36653]: osdmap e104: 12 total, 12 up, 12 in 2024-06-03T02:36:44.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:44 smithi079 ceph-mon[31819]: pgmap v1018: 97 pgs: 97 active+clean; 1.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:44.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:44 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T02:36:44.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:44 smithi079 ceph-mon[31819]: osdmap e104: 12 total, 12 up, 12 in 2024-06-03T02:36:46.693 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:46 smithi079 ceph-mon[31819]: pgmap v1020: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:46.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:46 smithi135 ceph-mon[36653]: pgmap v1020: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:46.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:46 smithi123 ceph-mon[36640]: pgmap v1020: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:46.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.993+0000 7fea5d000700 1 -- 172.21.15.79:0/1741335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea5810ee90 msgr2=0x7fea58111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:46.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.993+0000 7fea5d000700 1 --2- 172.21.15.79:0/1741335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea5810ee90 0x7fea58111280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fea50009f90 tx=0x7fea5000b330 comp rx=0 tx=0).stop 2024-06-03T02:36:46.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.994+0000 7fea5d000700 1 -- 172.21.15.79:0/1741335 shutdown_connections 2024-06-03T02:36:46.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.994+0000 7fea5d000700 1 --2- 172.21.15.79:0/1741335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea5810ee90 0x7fea58111280 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:46.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.994+0000 7fea5d000700 1 --2- 172.21.15.79:0/1741335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea580746c0 0x7fea58074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:46.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.994+0000 7fea5d000700 1 --2- 172.21.15.79:0/1741335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea58074fe0 0x7fea5810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:46.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.994+0000 7fea5d000700 1 -- 172.21.15.79:0/1741335 >> 172.21.15.79:0/1741335 conn(0x7fea580795f0 msgr2=0x7fea58079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:46.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.994+0000 7fea5d000700 1 -- 172.21.15.79:0/1741335 shutdown_connections 2024-06-03T02:36:46.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.994+0000 7fea5d000700 1 -- 172.21.15.79:0/1741335 wait complete. 2024-06-03T02:36:46.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.995+0000 7fea5d000700 1 Processor -- start 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.995+0000 7fea5d000700 1 -- start start 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea5d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea580746c0 0x7fea5819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea5d000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58074fe0 0x7fea5819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea5d000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5810ee90 0x7fea5819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea5d000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fea581149d0 con 0x7fea58074fe0 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea5d000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea58114850 con 0x7fea5810ee90 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea5d000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea58114b50 con 0x7fea580746c0 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea4d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58074fe0 0x7fea5819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea4d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58074fe0 0x7fea5819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46730/0 (socket says 172.21.15.79:46730) 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea4d400700 1 -- 172.21.15.79:0/3087375537 learned_addr learned my addr 172.21.15.79:0/3087375537 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-06-03T02:36:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea55e00700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea580746c0 0x7fea5819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:46.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.996+0000 7fea56800700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5810ee90 0x7fea5819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:46.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea55e00700 1 -- 172.21.15.79:0/3087375537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5810ee90 msgr2=0x7fea5819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:46.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea55e00700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5810ee90 0x7fea5819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea55e00700 1 -- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58074fe0 msgr2=0x7fea5819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea55e00700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58074fe0 0x7fea5819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea55e00700 1 -- 172.21.15.79:0/3087375537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5819fdb0 con 0x7fea580746c0 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea4d400700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58074fe0 0x7fea5819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea55e00700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea580746c0 0x7fea5819ada0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fea4800e9c0 tx=0x7fea4800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea4f400700 1 -- 172.21.15.79:0/3087375537 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4800c9c0 con 0x7fea580746c0 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea581a00a0 con 0x7fea580746c0 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea4f400700 1 -- 172.21.15.79:0/3087375537 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4800cb60 con 0x7fea580746c0 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea4f400700 1 -- 172.21.15.79:0/3087375537 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48018680 con 0x7fea580746c0 2024-06-03T02:36:46.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.997+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581a7920 con 0x7fea580746c0 2024-06-03T02:36:47.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:46.999+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea58069180 con 0x7fea580746c0 2024-06-03T02:36:47.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.001+0000 7fea4f400700 1 -- 172.21.15.79:0/3087375537 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fea4800cd10 con 0x7fea580746c0 2024-06-03T02:36:47.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.002+0000 7fea4f400700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea3c0a2c20 0x7fea3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T02:36:47.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.002+0000 7fea4f400700 1 -- 172.21.15.79:0/3087375537 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8496+0+0 (secure 0 0 0) 0x7fea480cf5f0 con 0x7fea580746c0 2024-06-03T02:36:47.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.002+0000 7fea4d400700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea3c0a2c20 0x7fea3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T02:36:47.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.003+0000 7fea4d400700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea3c0a2c20 0x7fea3c0a50e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fea58110a50 tx=0x7fea40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T02:36:47.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.004+0000 7fea4f400700 1 -- 172.21.15.79:0/3087375537 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea4809c410 con 0x7fea580746c0 2024-06-03T02:36:47.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.231+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fea581a7b50 con 0x7fea580746c0 2024-06-03T02:36:47.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.516+0000 7fea4f400700 1 -- 172.21.15.79:0/3087375537 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v105) ==== 178+0+0 (secure 0 0 0) 0x7fea4809bd10 con 0x7fea580746c0 2024-06-03T02:36:47.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.525+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea3c0a2c20 msgr2=0x7fea3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:47.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.525+0000 7fea5d000700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea3c0a2c20 0x7fea3c0a50e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fea58110a50 tx=0x7fea40009260 comp rx=0 tx=0).stop 2024-06-03T02:36:47.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.525+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea580746c0 msgr2=0x7fea5819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T02:36:47.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.525+0000 7fea5d000700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea580746c0 0x7fea5819ada0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fea4800e9c0 tx=0x7fea4800ecd0 comp rx=0 tx=0).stop 2024-06-03T02:36:47.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.526+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 shutdown_connections 2024-06-03T02:36:47.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.526+0000 7fea5d000700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea5810ee90 0x7fea5819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:47.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.526+0000 7fea5d000700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea580746c0 0x7fea5819ada0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:47.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.526+0000 7fea5d000700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:6800/2278164,v1:172.21.15.79:6801/2278164] conn(0x7fea3c0a2c20 0x7fea3c0a50e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:47.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.526+0000 7fea5d000700 1 --2- 172.21.15.79:0/3087375537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea58074fe0 0x7fea5819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T02:36:47.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.526+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 >> 172.21.15.79:0/3087375537 conn(0x7fea580795f0 msgr2=0x7fea580ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T02:36:47.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.528+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 shutdown_connections 2024-06-03T02:36:47.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-06-03T02:36:47.528+0000 7fea5d000700 1 -- 172.21.15.79:0/3087375537 wait complete. 2024-06-03T02:36:47.535 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data_ec' removed 2024-06-03T02:36:47.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:47 smithi079 ceph-mon[31819]: from='client.? 172.21.15.79:0/3087375537' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:47.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:47 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:47.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:47 smithi135 ceph-mon[36653]: from='client.? 172.21.15.79:0/3087375537' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:47.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:47 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:47 smithi123 ceph-mon[36640]: from='client.? 172.21.15.79:0/3087375537' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:47.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:47 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T02:36:48.384 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T02:36:48.495 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T02:36:48.538 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T02:36:48.567 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-03T02:36:48.567 DEBUG:teuthology.orchestra.run.smithi079:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T02:36:48.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:48 smithi079 ceph-mon[31819]: pgmap v1021: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:48.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:48 smithi079 ceph-mon[31819]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T02:36:48.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:48 smithi079 ceph-mon[31819]: osdmap e105: 12 total, 12 up, 12 in 2024-06-03T02:36:48.624 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T02:36:48.639 DEBUG:teuthology.orchestra.run.smithi135:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T02:36:48.654 INFO:tasks.cephadm:Stopping all daemons... 2024-06-03T02:36:48.655 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-03T02:36:48.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.a 2024-06-03T02:36:48.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:48 smithi135 ceph-mon[36653]: pgmap v1021: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:48.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:48 smithi135 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T02:36:48.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:48 smithi135 ceph-mon[36653]: osdmap e105: 12 total, 12 up, 12 in 2024-06-03T02:36:48.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:48 smithi123 ceph-mon[36640]: pgmap v1021: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T02:36:48.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:48 smithi123 ceph-mon[36640]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T02:36:48.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:48 smithi123 ceph-mon[36640]: osdmap e105: 12 total, 12 up, 12 in 2024-06-03T02:36:49.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:48 smithi079 systemd[1]: Stopping Ceph mon.a for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:36:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a[31815]: 2024-06-03T02:36:49.201+0000 7ff400800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:36:49.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a[31815]: 2024-06-03T02:36:49.201+0000 7ff400800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-03T02:36:49.589 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:49 smithi079 podman[120664]: 2024-06-03 02:36:49.438496023 +0000 UTC m=+0.391605679 container died 5de779843158d430dc7548c8771faf16ee775b9749afa27296b350a1a23e87fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-03T02:36:50.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:50 smithi079 podman[120664]: 2024-06-03 02:36:50.073829517 +0000 UTC m=+1.026939162 container cleanup 5de779843158d430dc7548c8771faf16ee775b9749afa27296b350a1a23e87fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:36:50.342 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:50 smithi079 bash[120664]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a 2024-06-03T02:36:50.342 INFO:journalctl@ceph.mon.a.smithi079.stdout:Jun 03 02:36:50 smithi079 podman[120736]: 2024-06-03 02:36:50.317914745 +0000 UTC m=+0.874303973 container remove 5de779843158d430dc7548c8771faf16ee775b9749afa27296b350a1a23e87fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-a, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:36:50.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-03T02:36:50.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:36:50.982 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-03T02:36:50.983 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-03T02:36:50.983 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.b 2024-06-03T02:36:51.305 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:51 smithi123 systemd[1]: Stopping Ceph mon.b for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:36:51.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:51 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-b[36636]: 2024-06-03T02:36:51.303+0000 7f0cb34bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:36:51.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:51 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-b[36636]: 2024-06-03T02:36:51.303+0000 7f0cb34bf700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-03T02:36:51.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:51 smithi123 podman[73322]: 2024-06-03 02:36:51.426893214 +0000 UTC m=+0.235337932 container died 0144ff2843368ba55bc53d431826131ac926607504d9ecfe1cd7f84f2ce1423c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True) 2024-06-03T02:36:51.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:51 smithi123 podman[73322]: 2024-06-03 02:36:51.804591612 +0000 UTC m=+0.613036432 container cleanup 0144ff2843368ba55bc53d431826131ac926607504d9ecfe1cd7f84f2ce1423c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-b, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-03T02:36:51.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 03 02:36:51 smithi123 bash[73322]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-b 2024-06-03T02:36:52.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-03T02:36:52.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:36:52.563 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-03T02:36:52.563 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-03T02:36:52.563 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.c 2024-06-03T02:36:52.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:52 smithi135 systemd[1]: Stopping Ceph mon.c for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:36:53.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:53 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-c[36649]: 2024-06-03T02:36:53.269+0000 7f00f9578700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:36:53.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:53 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-c[36649]: 2024-06-03T02:36:53.269+0000 7f00f9578700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-03T02:36:53.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:53 smithi135 podman[73630]: 2024-06-03 02:36:53.33101496 +0000 UTC m=+0.514215627 container died abe230c9a7e3aba64925ecf47c45931d0c2767466f84fea2fdb8d5523c8f5c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-03T02:36:54.014 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:53 smithi135 podman[73630]: 2024-06-03 02:36:53.713449797 +0000 UTC m=+0.896650464 container cleanup abe230c9a7e3aba64925ecf47c45931d0c2767466f84fea2fdb8d5523c8f5c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-03T02:36:54.015 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 03 02:36:53 smithi135 bash[73630]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mon-c 2024-06-03T02:36:54.147 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-03T02:36:54.197 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:36:54.197 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-03T02:36:54.197 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-03T02:36:54.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.x 2024-06-03T02:36:54.588 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:36:54 smithi079 systemd[1]: Stopping Ceph mgr.x for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:36:55.089 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:36:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x[32042]: 2024-06-03T02:36:54.827+0000 7f1c2aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:36:55.089 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:36:54 smithi079 podman[120861]: 2024-06-03 02:36:54.963279156 +0000 UTC m=+0.354967090 container died 35cbaa7fc85adb21c958d5413fac02c628da0a74594113f28434a943586778f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-03T02:36:56.384 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:36:56 smithi079 podman[120861]: 2024-06-03 02:36:56.114401446 +0000 UTC m=+1.506089357 container cleanup 35cbaa7fc85adb21c958d5413fac02c628da0a74594113f28434a943586778f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-03T02:36:56.384 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:36:56 smithi079 bash[120861]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x 2024-06-03T02:36:56.838 INFO:journalctl@ceph.mgr.x.smithi079.stdout:Jun 03 02:36:56 smithi079 podman[120874]: 2024-06-03 02:36:56.382944427 +0000 UTC m=+1.383811809 container remove 35cbaa7fc85adb21c958d5413fac02c628da0a74594113f28434a943586778f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-03T02:36:56.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-03T02:36:57.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:36:57.020 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-03T02:36:57.020 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-03T02:36:57.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.y 2024-06-03T02:36:57.371 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:36:57 smithi123 systemd[1]: Stopping Ceph mgr.y for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:36:57.693 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:36:57 smithi123 podman[73445]: 2024-06-03 02:36:57.371815187 +0000 UTC m=+0.150009641 container died a4a1e043d3c42b73136cafb43f39b692c602fdbb0f185a22092c6806a29b3efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-03T02:36:57.973 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:36:57 smithi123 podman[73445]: 2024-06-03 02:36:57.793288305 +0000 UTC m=+0.571482758 container cleanup a4a1e043d3c42b73136cafb43f39b692c602fdbb0f185a22092c6806a29b3efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-03T02:36:57.974 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 03 02:36:57 smithi123 bash[73445]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-y 2024-06-03T02:36:58.218 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-03T02:36:58.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:36:58.253 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-03T02:36:58.253 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-03T02:36:58.253 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.z 2024-06-03T02:36:58.596 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:36:58 smithi135 systemd[1]: Stopping Ceph mgr.z for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:36:58.903 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:36:58 smithi135 podman[73755]: 2024-06-03 02:36:58.613981753 +0000 UTC m=+0.168459923 container died 5f6977c5c1792e5d4a1a48dcef3ae391ce89ccbd68b8c2262db7c1e48d1945ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:36:59.328 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:36:59 smithi135 podman[73755]: 2024-06-03 02:36:59.019159642 +0000 UTC m=+0.573637812 container cleanup 5f6977c5c1792e5d4a1a48dcef3ae391ce89ccbd68b8c2262db7c1e48d1945ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-03T02:36:59.329 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 03 02:36:59 smithi135 bash[73755]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mgr-z 2024-06-03T02:36:59.453 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-03T02:36:59.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:36:59.486 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-03T02:36:59.486 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-03T02:36:59.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.0 2024-06-03T02:36:59.839 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:36:59 smithi079 systemd[1]: Stopping Ceph osd.0 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:37:01.089 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0[46633]: 2024-06-03T02:37:00.811+0000 7fc978a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:37:01.089 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0[46633]: 2024-06-03T02:37:00.811+0000 7fc978a00700 -1 osd.0 105 *** Got signal Terminated *** 2024-06-03T02:37:01.089 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0[46633]: 2024-06-03T02:37:00.811+0000 7fc978a00700 -1 osd.0 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:37:06.970 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:06 smithi079 podman[120985]: 2024-06-03 02:37:06.591087439 +0000 UTC m=+6.670909714 container died 865659a5536725149594614520085e017bc10b91c5ec5d8749e5e9951ee3b70e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-03T02:37:07.223 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:07 smithi079 podman[120985]: 2024-06-03 02:37:07.194701435 +0000 UTC m=+7.274523687 container cleanup 865659a5536725149594614520085e017bc10b91c5ec5d8749e5e9951ee3b70e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-03T02:37:07.223 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:07 smithi079 bash[120985]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0 2024-06-03T02:37:07.838 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:07 smithi079 podman[121002]: 2024-06-03 02:37:07.463532503 +0000 UTC m=+0.844911813 container remove 865659a5536725149594614520085e017bc10b91c5ec5d8749e5e9951ee3b70e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-03T02:37:08.501 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:08 smithi079 podman[121089]: 2024-06-03 02:37:08.155595659 +0000 UTC m=+0.038896440 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:37:08.501 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:08 smithi079 podman[121089]: 2024-06-03 02:37:08.318025651 +0000 UTC m=+0.201326451 container create ab75ecfd8529c0c1f58b3a1fccf30920942c37dba6a0ec6e1dc7885e6d01740b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-03T02:37:09.089 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:08 smithi079 podman[121089]: 2024-06-03 02:37:08.610600431 +0000 UTC m=+0.493901234 container init ab75ecfd8529c0c1f58b3a1fccf30920942c37dba6a0ec6e1dc7885e6d01740b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:37:09.089 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:08 smithi079 podman[121089]: 2024-06-03 02:37:08.6230751 +0000 UTC m=+0.506375878 container start ab75ecfd8529c0c1f58b3a1fccf30920942c37dba6a0ec6e1dc7885e6d01740b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:37:09.089 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:08 smithi079 podman[121089]: 2024-06-03 02:37:08.742458351 +0000 UTC m=+0.625759111 container attach ab75ecfd8529c0c1f58b3a1fccf30920942c37dba6a0ec6e1dc7885e6d01740b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-03T02:37:09.839 INFO:journalctl@ceph.osd.0.smithi079.stdout:Jun 03 02:37:09 smithi079 podman[121089]: 2024-06-03 02:37:09.418275196 +0000 UTC m=+1.301575983 container died ab75ecfd8529c0c1f58b3a1fccf30920942c37dba6a0ec6e1dc7885e6d01740b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:37:10.540 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-03T02:37:10.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:37:10.649 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-03T02:37:10.649 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-03T02:37:10.649 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.1 2024-06-03T02:37:10.973 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:10 smithi123 systemd[1]: Stopping Ceph osd.1 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:37:11.474 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:10 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1[41221]: 2024-06-03T02:37:10.980+0000 7fb3f4013700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:37:11.474 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:10 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1[41221]: 2024-06-03T02:37:10.980+0000 7fb3f4013700 -1 osd.1 105 *** Got signal Terminated *** 2024-06-03T02:37:11.474 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:10 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1[41221]: 2024-06-03T02:37:10.980+0000 7fb3f4013700 -1 osd.1 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:37:16.806 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:16 smithi123 podman[73581]: 2024-06-03 02:37:16.500348292 +0000 UTC m=+5.646259884 container died e18da9247846901f2c3fc27b757583709762057992c8c7dbb2680bcfdb23d975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-06-03T02:37:17.165 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:16 smithi123 podman[73581]: 2024-06-03 02:37:16.914421153 +0000 UTC m=+6.060332737 container cleanup e18da9247846901f2c3fc27b757583709762057992c8c7dbb2680bcfdb23d975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-03T02:37:17.165 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:16 smithi123 bash[73581]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1 2024-06-03T02:37:17.473 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:17 smithi123 podman[73599]: 2024-06-03 02:37:17.165688142 +0000 UTC m=+0.666978393 container remove e18da9247846901f2c3fc27b757583709762057992c8c7dbb2680bcfdb23d975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-03T02:37:17.902 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:17 smithi123 podman[73689]: 2024-06-03 02:37:17.494507611 +0000 UTC m=+0.017277508 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:37:17.902 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:17 smithi123 podman[73689]: 2024-06-03 02:37:17.617025705 +0000 UTC m=+0.139795588 container create 5b4125610fac3a493036afb1aa54939cacb97af3e8283d637196dc0e0959df41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-03T02:37:18.224 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:17 smithi123 podman[73689]: 2024-06-03 02:37:17.902632166 +0000 UTC m=+0.425402049 container init 5b4125610fac3a493036afb1aa54939cacb97af3e8283d637196dc0e0959df41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:37:18.224 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:17 smithi123 podman[73689]: 2024-06-03 02:37:17.90580516 +0000 UTC m=+0.428575051 container start 5b4125610fac3a493036afb1aa54939cacb97af3e8283d637196dc0e0959df41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-03T02:37:18.224 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:18 smithi123 podman[73689]: 2024-06-03 02:37:18.002881873 +0000 UTC m=+0.525651770 container attach 5b4125610fac3a493036afb1aa54939cacb97af3e8283d637196dc0e0959df41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1-deactivate, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:37:18.528 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 03 02:37:18 smithi123 podman[73689]: 2024-06-03 02:37:18.250297168 +0000 UTC m=+0.773067054 container died 5b4125610fac3a493036afb1aa54939cacb97af3e8283d637196dc0e0959df41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-1-deactivate, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-03T02:37:18.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-03T02:37:18.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:37:18.921 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-03T02:37:18.921 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-03T02:37:18.921 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.2 2024-06-03T02:37:19.231 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:18 smithi135 systemd[1]: Stopping Ceph osd.2 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:37:19.536 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2[41736]: 2024-06-03T02:37:19.230+0000 7f0d752a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:37:19.536 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2[41736]: 2024-06-03T02:37:19.230+0000 7f0d752a3700 -1 osd.2 105 *** Got signal Terminated *** 2024-06-03T02:37:19.537 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2[41736]: 2024-06-03T02:37:19.230+0000 7f0d752a3700 -1 osd.2 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:37:25.047 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:24 smithi135 podman[73885]: 2024-06-03 02:37:24.713110757 +0000 UTC m=+5.596573133 container died 4dfd9dee0a85d0c0b7a70843af84220dd35ef8f755199e370aefe5210283a4ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-03T02:37:25.345 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:25 smithi135 podman[73885]: 2024-06-03 02:37:25.047513805 +0000 UTC m=+5.930976186 container cleanup 4dfd9dee0a85d0c0b7a70843af84220dd35ef8f755199e370aefe5210283a4ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-03T02:37:25.346 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:25 smithi135 bash[73885]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2 2024-06-03T02:37:25.346 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:25 smithi135 podman[73912]: 2024-06-03 02:37:25.266061737 +0000 UTC m=+0.548318844 container remove 4dfd9dee0a85d0c0b7a70843af84220dd35ef8f755199e370aefe5210283a4ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-06-03T02:37:26.028 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:25 smithi135 podman[74001]: 2024-06-03 02:37:25.61821784 +0000 UTC m=+0.017243953 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:37:26.028 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:25 smithi135 podman[74001]: 2024-06-03 02:37:25.758723477 +0000 UTC m=+0.157749578 container create 4a66be272c8b56b1a51c1f2894f101c080c661893a6c114b7e82b1aa0aa970e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-03T02:37:26.345 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:26 smithi135 podman[74001]: 2024-06-03 02:37:26.028676889 +0000 UTC m=+0.427702995 container init 4a66be272c8b56b1a51c1f2894f101c080c661893a6c114b7e82b1aa0aa970e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:37:26.346 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:26 smithi135 podman[74001]: 2024-06-03 02:37:26.03216291 +0000 UTC m=+0.431189020 container start 4a66be272c8b56b1a51c1f2894f101c080c661893a6c114b7e82b1aa0aa970e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-03T02:37:26.346 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:26 smithi135 podman[74001]: 2024-06-03 02:37:26.136515046 +0000 UTC m=+0.535541162 container attach 4a66be272c8b56b1a51c1f2894f101c080c661893a6c114b7e82b1aa0aa970e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:37:26.845 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 03 02:37:26 smithi135 podman[74001]: 2024-06-03 02:37:26.582231572 +0000 UTC m=+0.981257676 container died 4a66be272c8b56b1a51c1f2894f101c080c661893a6c114b7e82b1aa0aa970e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-06-03T02:37:27.257 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-03T02:37:27.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:37:27.296 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-03T02:37:27.296 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-03T02:37:27.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.3 2024-06-03T02:37:27.839 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:27 smithi079 systemd[1]: Stopping Ceph osd.3 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:37:28.205 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:27 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3[53533]: 2024-06-03T02:37:27.871+0000 7efeec000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:37:28.206 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:27 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3[53533]: 2024-06-03T02:37:27.871+0000 7efeec000700 -1 osd.3 105 *** Got signal Terminated *** 2024-06-03T02:37:28.206 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:27 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3[53533]: 2024-06-03T02:37:27.871+0000 7efeec000700 -1 osd.3 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:37:28.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:28 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:28.483+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:29.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:28 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:28.683+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:29.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:29 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:29.520+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:29.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:29 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:29.416+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:30.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:29 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:29.679+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:30.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:30.372+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:30.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:30 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:30.504+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:30.973 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:30.493+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:31.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:30.694+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:31.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:31 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:31.457+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:31.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:31 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:31.386+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:31.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:31 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:31.463+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:32.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:31.673+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:32.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:31.682+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:32.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:32 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:32.422+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:32.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:32 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:32.479+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:32.973 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:32 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:32.495+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:33.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:32 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:32.648+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:33.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:32 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:32.696+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:33.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:33 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:33.406+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:33.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:33 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:33.493+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:33.839 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:33 smithi079 podman[121194]: 2024-06-03 02:37:33.608958549 +0000 UTC m=+5.888343897 container died 20cc42a16e0a0abd4515a14caae8df8997323ffd4a1a3b9dada544b84b274de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:37:33.973 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:33 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:33.480+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:34.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:33 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:33.614+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:34.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:33 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:33.723+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:34.537 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:34 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:34.402+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:34.538 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:34 smithi079 podman[121194]: 2024-06-03 02:37:34.282268832 +0000 UTC m=+6.561654182 container cleanup 20cc42a16e0a0abd4515a14caae8df8997323ffd4a1a3b9dada544b84b274de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-06-03T02:37:34.538 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:34 smithi079 bash[121194]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3 2024-06-03T02:37:34.838 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:34 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:34.536+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:34.839 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:34 smithi079 podman[121225]: 2024-06-03 02:37:34.625148442 +0000 UTC m=+1.007209137 container remove 20cc42a16e0a0abd4515a14caae8df8997323ffd4a1a3b9dada544b84b274de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-03T02:37:34.973 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:34.494+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:35.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:34 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:34.647+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:35.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:34 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:34.679+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:35.524 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:35 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:35.414+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:35.524 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:35 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:35.522+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:35.524 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:35 smithi079 podman[121313]: 2024-06-03 02:37:35.153936601 +0000 UTC m=+0.038439692 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:37:35.525 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:35 smithi079 podman[121313]: 2024-06-03 02:37:35.292967834 +0000 UTC m=+0.177470928 container create 0bedc171f1de0057bff436e5870054440486847652463e3d669c025dac73615e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:37:35.839 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:35 smithi079 podman[121313]: 2024-06-03 02:37:35.588381435 +0000 UTC m=+0.472884573 container init 0bedc171f1de0057bff436e5870054440486847652463e3d669c025dac73615e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-03T02:37:35.839 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:35 smithi079 podman[121313]: 2024-06-03 02:37:35.600572087 +0000 UTC m=+0.485075188 container start 0bedc171f1de0057bff436e5870054440486847652463e3d669c025dac73615e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-03T02:37:35.839 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:35 smithi079 podman[121313]: 2024-06-03 02:37:35.703882295 +0000 UTC m=+0.588385436 container attach 0bedc171f1de0057bff436e5870054440486847652463e3d669c025dac73615e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-06-03T02:37:35.973 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:35 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:35.503+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:36.082 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:35.697+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:36.082 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:35.664+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:36.339 INFO:journalctl@ceph.osd.3.smithi079.stdout:Jun 03 02:37:36 smithi079 podman[121313]: 2024-06-03 02:37:36.046718926 +0000 UTC m=+0.931222067 container died 0bedc171f1de0057bff436e5870054440486847652463e3d669c025dac73615e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-03T02:37:36.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:36 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:36.421+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:36.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:36 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:36.565+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:36.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-03T02:37:36.973 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:36 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:36.542+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:37.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:37:37.069 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-03T02:37:37.069 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-03T02:37:37.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.4 2024-06-03T02:37:37.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:36 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:36.707+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:37.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:36 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:36.702+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:37.393 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:37 smithi123 systemd[1]: Stopping Ceph osd.4 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:37:37.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:37 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:37.556+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:37.723 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:37 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:37:37.392+0000 7ff11d05e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:37:37.724 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:37 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:37:37.392+0000 7ff11d05e700 -1 osd.4 105 *** Got signal Terminated *** 2024-06-03T02:37:37.724 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:37 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:37:37.392+0000 7ff11d05e700 -1 osd.4 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:37:37.724 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:37 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:37:37.480+0000 7ff119652700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.969292+0000 front 2024-06-03T02:37:13.969267+0000 (oldest deadline 2024-06-03T02:37:36.869495+0000) 2024-06-03T02:37:37.838 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:37 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:37.592+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:37.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:37 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:37.429+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:38.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:37.728+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:38.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:37.692+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:38.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:37.692+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:38.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:38.286+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:38.724 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:38 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:37:38.458+0000 7ff119652700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.969292+0000 front 2024-06-03T02:37:13.969267+0000 (oldest deadline 2024-06-03T02:37:36.869495+0000) 2024-06-03T02:37:38.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:38 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:38.356+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:38.725 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:38 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:38.561+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:38.725 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:38 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:38.561+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:38.838 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:38 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:38.563+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:38.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:38 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:38.479+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:39.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:38.694+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:39.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:38.680+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:39.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:38.680+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:39.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:39.267+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:39.723 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:39 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:37:39.506+0000 7ff119652700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.969292+0000 front 2024-06-03T02:37:13.969267+0000 (oldest deadline 2024-06-03T02:37:36.869495+0000) 2024-06-03T02:37:39.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:39 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:39.340+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:39.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:39 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:39.529+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:39.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:39 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:39.529+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:39.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:39 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:39.430+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:39.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:39 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:39.430+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:39.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:39 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:39.581+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:39.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:39 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:39.581+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:40.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:39.696+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:40.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:39.696+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:40.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:39.699+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:40.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:39.699+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:40.508 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:40 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:40.387+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:40.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:40.282+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:40.783 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:40 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:37:40.508+0000 7ff119652700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.969292+0000 front 2024-06-03T02:37:13.969267+0000 (oldest deadline 2024-06-03T02:37:36.869495+0000) 2024-06-03T02:37:40.784 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:40 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:40.531+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:40.784 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:40 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:40.531+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:40.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:40 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:40.443+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:40.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:40 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:40.443+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:40.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:40 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:40.592+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:40.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:40 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:40.592+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:41.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:40.689+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:41.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:40.689+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:41.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:40.707+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:41.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:40.707+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:41.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:41.320+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:41.723 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:41 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4[48019]: 2024-06-03T02:37:41.462+0000 7ff119652700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.969292+0000 front 2024-06-03T02:37:13.969267+0000 (oldest deadline 2024-06-03T02:37:36.869495+0000) 2024-06-03T02:37:41.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:41 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:41.360+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:41.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:41 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:41.505+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:41.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:41 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:41.505+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:41.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:41 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:41.408+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:41.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:41 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:41.408+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:41.840 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:41 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:41.580+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:41.840 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:41 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:41.580+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:42.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:41.686+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:42.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:41.686+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:42.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:41.698+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:42.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:41.698+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:42.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:42 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:42.276+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:42.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:42 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:42.404+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:42.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:42 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:42.516+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:42.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:42 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:42.516+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:42.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:42 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:42.389+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:42.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:42 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:42.389+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:42.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:42 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:42.601+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:42.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:42 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:42.601+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:43.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:42 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:42.694+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:43.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:42 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:42.694+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:43.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:42 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:42.734+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:43.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:42 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:42.734+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:43.410 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:43 smithi123 podman[73819]: 2024-06-03 02:37:43.107254648 +0000 UTC m=+5.843561251 container died fb8d3392ad36834d2e9ef5f00429f9ac34a39e24efa0a9c03861f5ed6e33747a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True) 2024-06-03T02:37:43.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:43 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:43.234+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:43.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:43 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:43.514+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:43.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:43 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:43.514+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:43.724 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:43 smithi123 podman[73819]: 2024-06-03 02:37:43.509718769 +0000 UTC m=+6.246025369 container cleanup fb8d3392ad36834d2e9ef5f00429f9ac34a39e24efa0a9c03861f5ed6e33747a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-03T02:37:43.724 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:43 smithi123 bash[73819]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4 2024-06-03T02:37:43.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:43 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:43.417+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:43.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:43.397+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:43.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:43.397+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:43.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:43.555+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:43.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:43 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:43.555+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:44.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:43 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:43.680+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:44.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:43 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:43.680+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:44.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:43 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:43.733+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:44.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:43 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:43.733+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:44.127 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:43 smithi123 podman[73835]: 2024-06-03 02:37:43.786123399 +0000 UTC m=+0.680652708 container remove fb8d3392ad36834d2e9ef5f00429f9ac34a39e24efa0a9c03861f5ed6e33747a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-03T02:37:44.423 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:44 smithi123 podman[73928]: 2024-06-03 02:37:44.029722449 +0000 UTC m=+0.017146136 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:37:44.423 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:44 smithi123 podman[73928]: 2024-06-03 02:37:44.14526547 +0000 UTC m=+0.132689150 container create 9d67c7c4be788ca5cdfa8b65ceb1038e4dbce70c9414d1ef640807f3ad529b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-03T02:37:44.423 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:44 smithi123 podman[73928]: 2024-06-03 02:37:44.37306846 +0000 UTC m=+0.360492132 container init 9d67c7c4be788ca5cdfa8b65ceb1038e4dbce70c9414d1ef640807f3ad529b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-03T02:37:44.423 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:44 smithi123 podman[73928]: 2024-06-03 02:37:44.376349022 +0000 UTC m=+0.363772707 container start 9d67c7c4be788ca5cdfa8b65ceb1038e4dbce70c9414d1ef640807f3ad529b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-03T02:37:44.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:44 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:44.277+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:44.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:44 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:44.422+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:44.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:44 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:44.468+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:44.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:44 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:44.468+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:44.724 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:44 smithi123 podman[73928]: 2024-06-03 02:37:44.473153808 +0000 UTC m=+0.460577497 container attach 9d67c7c4be788ca5cdfa8b65ceb1038e4dbce70c9414d1ef640807f3ad529b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-03T02:37:44.724 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 03 02:37:44 smithi123 podman[73928]: 2024-06-03 02:37:44.663765023 +0000 UTC m=+0.651188712 container died 9d67c7c4be788ca5cdfa8b65ceb1038e4dbce70c9414d1ef640807f3ad529b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-03T02:37:44.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:44 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:44.413+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:44.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:44 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:44.413+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:44.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:44 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:44.552+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:44.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:44 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:44.552+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:45.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:44 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:44.675+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:45.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:44 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:44.675+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:45.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:44 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:44.713+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:45.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:44 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:44.713+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:45.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-03T02:37:45.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:37:45.306 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-03T02:37:45.306 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-03T02:37:45.306 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.5 2024-06-03T02:37:45.595 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:45 smithi135 systemd[1]: Stopping Ceph osd.5 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:37:45.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:45 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:45.237+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:45.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:45 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:45.425+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:45.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:45 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:45.435+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:45.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:45 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:45.435+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:45.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:45 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:45.415+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:45.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:45 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:45.415+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:45.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:45 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:45.508+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:45.840 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:45 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:45.508+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:46.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:45 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:45.693+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:46.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:45 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:45.693+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:46.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:45 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:45.626+0000 7fac6804f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:37:46.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:45 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:45.626+0000 7fac6804f700 -1 osd.5 105 *** Got signal Terminated *** 2024-06-03T02:37:46.097 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:45 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:45.626+0000 7fac6804f700 -1 osd.5 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:37:46.097 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:45 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:45.665+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:46.097 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:45 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:45.665+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:46.097 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:45 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:45.665+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:19.634918+0000 front 2024-06-03T02:37:19.634866+0000 (oldest deadline 2024-06-03T02:37:44.935056+0000) 2024-06-03T02:37:46.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:46 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:46.226+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:46.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:46 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:46.417+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:46.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:46 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:46.388+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:46.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:46 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:46.388+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:46.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:46.383+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:46.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:46.383+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:46.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:46.529+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:46.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:46.529+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:47.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:46 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:46.683+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:47.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:46 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:46.683+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:47.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:46 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:46.655+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:47.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:46 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:46.655+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:47.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:46 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:46.655+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:19.634918+0000 front 2024-06-03T02:37:19.634866+0000 (oldest deadline 2024-06-03T02:37:44.935056+0000) 2024-06-03T02:37:47.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:47 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:47.183+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:47.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:47 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:47.443+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:47.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:47 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:47.443+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:47.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:47 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:47.397+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:47.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:47 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:47.397+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:47.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:47 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:47.397+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:47.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:47 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:47.348+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:47.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:47 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:47.348+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:47.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:47 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:47.503+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:47.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:47 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:47.503+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:48.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:47 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:47.675+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:48.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:47 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:47.675+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:48.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:47 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:47.675+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:48.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:47 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:47.624+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:48.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:47 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:47.624+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:48.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:47 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:47.624+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:19.634918+0000 front 2024-06-03T02:37:19.634866+0000 (oldest deadline 2024-06-03T02:37:44.935056+0000) 2024-06-03T02:37:48.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:48 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:48.224+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:48.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:48 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:48.452+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:48.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:48 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:48.452+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:48.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:48 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:48.388+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:48.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:48 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:48.388+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:48.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:48 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:48.388+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:48.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:48 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:48.393+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:48.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:48 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:48.393+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:48.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:48 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:48.544+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:48.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:48 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:48.544+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:49.095 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:48 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:48.613+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:49.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:48 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:48.613+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:49.096 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:48 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:48.613+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:19.634918+0000 front 2024-06-03T02:37:19.634866+0000 (oldest deadline 2024-06-03T02:37:44.935056+0000) 2024-06-03T02:37:49.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:48 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:48.684+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:49.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:48 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:48.684+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:49.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:48 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:48.684+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:49.570 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:49 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:49.187+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:49.570 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:49 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:49.187+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:49.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:49 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:49.455+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:49.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:49 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:49.455+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:49.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:49 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:49.407+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:49.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:49 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:49.407+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:49.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:49 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:49.407+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:49.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:49.426+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:49.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:49.426+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:49.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:49.559+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:49.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:49.559+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:49.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:49 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:49.559+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:49.845 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:49 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:49.570+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:49.845 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:49 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:49.570+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:49.845 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:49 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:49.570+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:19.634918+0000 front 2024-06-03T02:37:19.634866+0000 (oldest deadline 2024-06-03T02:37:44.935056+0000) 2024-06-03T02:37:49.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:49 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:49.680+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:49.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:49 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:49.680+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:49.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:49 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:49.680+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:50.580 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:50 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:50.205+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:50.580 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:50 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:50.205+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:50.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:50 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:50.471+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:50.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:50 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:50.471+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:50.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:50 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:50.363+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:50.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:50 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:50.363+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:50.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:50 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:50.363+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:50.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:50.377+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:50.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:50.377+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:50.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:50.575+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:50.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:50.575+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:50.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:50 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:50.575+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:50.846 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:50 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:50.580+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.732138+0000 front 2024-06-03T02:37:02.732003+0000 (oldest deadline 2024-06-03T02:37:28.632115+0000) 2024-06-03T02:37:50.846 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:50 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:50.580+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:13.233799+0000 front 2024-06-03T02:37:13.233866+0000 (oldest deadline 2024-06-03T02:37:39.133842+0000) 2024-06-03T02:37:50.846 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:50 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5[48663]: 2024-06-03T02:37:50.580+0000 7fac6063b700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:19.634918+0000 front 2024-06-03T02:37:19.634866+0000 (oldest deadline 2024-06-03T02:37:44.935056+0000) 2024-06-03T02:37:50.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:50 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:50.698+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:50.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:50 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:50.698+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:50.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:50 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:50.698+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:51.535 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:51 smithi135 podman[74106]: 2024-06-03 02:37:51.230674086 +0000 UTC m=+5.730947088 container died 5d8758198dd26c3962658859944cec62572c377b2ff1c4423e45ad18d88adaeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-03T02:37:51.535 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:51 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:51.176+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:51.535 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:51 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:51.176+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:51.588 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:51 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:51.333+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:51.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:51 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:51.333+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:51.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:51 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:51.333+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.006327+0000 front 2024-06-03T02:37:24.006431+0000 (oldest deadline 2024-06-03T02:37:50.408211+0000) 2024-06-03T02:37:51.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:51 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:51.474+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:51.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:51 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:51.474+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:51.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:51 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:51.362+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:51.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:51 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:51.362+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:51.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:51 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:51.362+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:51.845 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:51 smithi135 podman[74106]: 2024-06-03 02:37:51.635652779 +0000 UTC m=+6.135925781 container cleanup 5d8758198dd26c3962658859944cec62572c377b2ff1c4423e45ad18d88adaeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-03T02:37:51.846 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:51 smithi135 bash[74106]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5 2024-06-03T02:37:51.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:51 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:51.714+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:51.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:51 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:51.714+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:51.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:51 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:51.714+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:52.088 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:51 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:51.607+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:52.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:51 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:51.607+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:52.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:51 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:51.607+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:52.303 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:51 smithi135 podman[74122]: 2024-06-03 02:37:51.995584995 +0000 UTC m=+0.765124750 container remove 5d8758198dd26c3962658859944cec62572c377b2ff1c4423e45ad18d88adaeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-03T02:37:52.303 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:52 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:52.217+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:52.303 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:52 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:52.217+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:52.595 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:52 smithi135 podman[74212]: 2024-06-03 02:37:52.206241771 +0000 UTC m=+0.017583342 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:37:52.596 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:52 smithi135 podman[74212]: 2024-06-03 02:37:52.341838752 +0000 UTC m=+0.153180363 container create 6098828bb146f587f78f662716f795be0b56cb2b1739d0a4ccd9d9f8222e958e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:37:52.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:52 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:52.481+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:52.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:52 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:52.481+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:52.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:52 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:52.408+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:52.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:52 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:52.408+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:52.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:52 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:52.408+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:52.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:52.352+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:52.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:52.352+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:52.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:52.352+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.006327+0000 front 2024-06-03T02:37:24.006431+0000 (oldest deadline 2024-06-03T02:37:50.408211+0000) 2024-06-03T02:37:52.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:52.563+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:52.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:52.563+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:52.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:52.563+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:52.905 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:52 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:52.737+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:52.905 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:52 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:52.737+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:52.906 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:52 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:52.737+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:52.906 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:52 smithi135 podman[74212]: 2024-06-03 02:37:52.607624457 +0000 UTC m=+0.418966006 container init 6098828bb146f587f78f662716f795be0b56cb2b1739d0a4ccd9d9f8222e958e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-03T02:37:52.906 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:52 smithi135 podman[74212]: 2024-06-03 02:37:52.610745446 +0000 UTC m=+0.422087007 container start 6098828bb146f587f78f662716f795be0b56cb2b1739d0a4ccd9d9f8222e958e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-03T02:37:52.906 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:52 smithi135 podman[74212]: 2024-06-03 02:37:52.707571511 +0000 UTC m=+0.518913078 container attach 6098828bb146f587f78f662716f795be0b56cb2b1739d0a4ccd9d9f8222e958e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-03T02:37:53.193 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 03 02:37:52 smithi135 podman[74212]: 2024-06-03 02:37:52.906250291 +0000 UTC m=+0.717591862 container died 6098828bb146f587f78f662716f795be0b56cb2b1739d0a4ccd9d9f8222e958e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-03T02:37:53.470 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:53 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:53.258+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:53.470 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:53 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:53.258+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:53.495 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-03T02:37:53.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:37:53.534 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-03T02:37:53.535 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-03T02:37:53.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.6 2024-06-03T02:37:53.582 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:53.306+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:53.582 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:53.306+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:53.582 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:53.306+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.006327+0000 front 2024-06-03T02:37:24.006431+0000 (oldest deadline 2024-06-03T02:37:50.408211+0000) 2024-06-03T02:37:53.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:53 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:53.503+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:53.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:53 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:53.503+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:53.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:53 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:53.389+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:53.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:53 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:53.389+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:53.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:53 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:53.389+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:53.839 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:53 smithi079 systemd[1]: Stopping Ceph osd.6 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:37:53.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:53.581+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:53.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:53.581+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:53.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:53 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:53.581+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:54.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:53 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:53.786+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:54.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:53 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:53.786+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:54.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:53 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:53.786+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:54.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:54.139+0000 7f86f5400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:37:54.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:54.139+0000 7f86f5400700 -1 osd.6 105 *** Got signal Terminated *** 2024-06-03T02:37:54.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:54.139+0000 7f86f5400700 -1 osd.6 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:37:54.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:54.297+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:54.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:54.297+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:54.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:54.297+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.006327+0000 front 2024-06-03T02:37:24.006431+0000 (oldest deadline 2024-06-03T02:37:50.408211+0000) 2024-06-03T02:37:54.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:54 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:54.246+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:54.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:54 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:54.246+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:54.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:54 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:54.468+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:54.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:54 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:54.468+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:54.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:54 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:54.380+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:54.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:54 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:54.380+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:54.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:54 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:54.380+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:54.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:54 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:54.380+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:37:55.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:54.611+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:55.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:54.611+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:55.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:54 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:54.611+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:55.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:54 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:54.836+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:55.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:54 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:54.836+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:55.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:54 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:54.836+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:55.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:55.267+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:55.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:55.267+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:55.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:55.267+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.006327+0000 front 2024-06-03T02:37:24.006431+0000 (oldest deadline 2024-06-03T02:37:50.408211+0000) 2024-06-03T02:37:55.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:55.267+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.409206+0000 front 2024-06-03T02:37:30.409750+0000 (oldest deadline 2024-06-03T02:37:54.509470+0000) 2024-06-03T02:37:55.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:55 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:55.261+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:55.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:55 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:55.261+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:55.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:55 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:55.261+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:37:55.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:55 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:55.513+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:55.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:55 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:55.513+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:55.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:55 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:55.409+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:55.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:55 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:55.409+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:55.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:55 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:55.409+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:55.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:55 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:55.409+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:37:56.088 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:55.629+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:56.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:55.629+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:56.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:55 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:55.629+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:56.228 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:55 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:55.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:56.228 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:55 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:55.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:56.228 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:55 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:55.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:56.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:56.280+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:56.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:56.280+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:56.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:56.280+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.006327+0000 front 2024-06-03T02:37:24.006431+0000 (oldest deadline 2024-06-03T02:37:50.408211+0000) 2024-06-03T02:37:56.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:56.280+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.409206+0000 front 2024-06-03T02:37:30.409750+0000 (oldest deadline 2024-06-03T02:37:54.509470+0000) 2024-06-03T02:37:56.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:56 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:56.228+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:56.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:56 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:56.228+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:56.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:56 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:56.228+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:37:56.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:56 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:56.504+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:56.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:56 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:56.504+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:56.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:56 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:56.504+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:37:56.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:56 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:56.390+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:56.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:56 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:56.390+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:56.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:56 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:56.390+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:56.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:56 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:56.390+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:37:57.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:56.678+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:57.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:56.678+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:57.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:56.678+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:57.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:56 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:56.825+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:57.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:56 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:56.825+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:57.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:56 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:56.825+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:57.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:57 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:57.286+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:57.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:57 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:57.286+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:57.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:57 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:57.286+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.006327+0000 front 2024-06-03T02:37:24.006431+0000 (oldest deadline 2024-06-03T02:37:50.408211+0000) 2024-06-03T02:37:57.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:57 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:57.286+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.409206+0000 front 2024-06-03T02:37:30.409750+0000 (oldest deadline 2024-06-03T02:37:54.509470+0000) 2024-06-03T02:37:57.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:57 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:57.272+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:57.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:57 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:57.272+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:57.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:57 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:57.272+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:37:57.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:57 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:57.495+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:57.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:57 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:57.495+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:57.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:57 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:57.495+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:37:57.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:57 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:57.403+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:57.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:57 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:57.403+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:57.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:57 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:57.403+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:57.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:57 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:57.403+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:37:58.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:57 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:57.649+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:58.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:57 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:57.649+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:58.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:57 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:57.649+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:58.298 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:57 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:57.845+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:58.298 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:57 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:57.845+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:58.298 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:57 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:57.845+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:58.588 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:58.295+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.000649+0000 front 2024-06-03T02:37:04.000708+0000 (oldest deadline 2024-06-03T02:37:29.300942+0000) 2024-06-03T02:37:58.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:58.295+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.603378+0000 front 2024-06-03T02:37:14.603317+0000 (oldest deadline 2024-06-03T02:37:38.703578+0000) 2024-06-03T02:37:58.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:58.295+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.006327+0000 front 2024-06-03T02:37:24.006431+0000 (oldest deadline 2024-06-03T02:37:50.408211+0000) 2024-06-03T02:37:58.589 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:37:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6[60476]: 2024-06-03T02:37:58.295+0000 7f86f0c00700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.409206+0000 front 2024-06-03T02:37:30.409750+0000 (oldest deadline 2024-06-03T02:37:54.509470+0000) 2024-06-03T02:37:58.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:58 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:58.298+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:58.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:58 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:58.298+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:58.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:58 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:58.298+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:37:58.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:58 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:58.544+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:58.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:58 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:58.544+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:58.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:58 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:58.544+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:37:58.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:58 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:58.411+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:58.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:58 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:58.411+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:58.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:58 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:58.411+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:58.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:58 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:58.411+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:37:59.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:58.601+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:59.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:58.601+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:59.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:58.601+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:59.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:58 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:58.601+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:37:59.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:58 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:58.812+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:37:59.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:58 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:58.812+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:37:59.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:58 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:58.812+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:37:59.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:59 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:59.326+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:37:59.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:59 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:59.326+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:37:59.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:37:59 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:37:59.326+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:37:59.625 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:59 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:59.534+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:37:59.625 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:59 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:59.534+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:37:59.625 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:37:59 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:37:59.534+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:37:59.625 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:59 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:59.372+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:37:59.625 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:59 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:59.372+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:37:59.625 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:59 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:59.372+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:37:59.625 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:37:59 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:37:59.372+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:37:59.838 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:59 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:59.582+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:37:59.838 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:59 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:59.582+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:37:59.838 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:59 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:59.582+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:37:59.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:37:59 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:37:59.582+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:00.095 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:59 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:59.825+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:00.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:59 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:59.825+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:00.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:37:59 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:37:59.825+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:00.338 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:00 smithi079 podman[121419]: 2024-06-03 02:38:00.02613184 +0000 UTC m=+6.041532486 container died aeb61145c801e446f3de8cf6e686348a3319ed12fb95738554a90484fd4a1bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-06-03T02:38:00.643 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:00.550+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:00.643 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:00.550+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:00.643 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:00.550+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:00.643 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:00.550+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:00.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:00 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:00.520+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:38:00.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:00 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:00.520+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:38:00.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:00 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:00.520+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:38:00.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:00 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:00.394+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:00.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:00 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:00.394+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:00.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:00 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:00.394+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:00.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:00 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:00.394+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:00.827 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:00 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:00.362+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:00.827 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:00 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:00.362+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:00.827 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:00 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:00.362+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:00.927 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:00 smithi079 podman[121419]: 2024-06-03 02:38:00.641851804 +0000 UTC m=+6.657252443 container cleanup aeb61145c801e446f3de8cf6e686348a3319ed12fb95738554a90484fd4a1bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-03T02:38:00.928 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:00 smithi079 bash[121419]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6 2024-06-03T02:38:01.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:00 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:00.826+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:01.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:00 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:00.826+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:01.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:00 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:00.826+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:01.338 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:00 smithi079 podman[121437]: 2024-06-03 02:38:00.926812373 +0000 UTC m=+0.890544673 container remove aeb61145c801e446f3de8cf6e686348a3319ed12fb95738554a90484fd4a1bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-03T02:38:01.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:01 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:01.327+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:01.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:01 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:01.327+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:01.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:01 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:01.327+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:01.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:01 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:01.512+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:38:01.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:01 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:01.512+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:38:01.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:01 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:01.512+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:38:01.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:01 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:01.418+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:01.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:01 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:01.418+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:01.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:01 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:01.418+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:01.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:01 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:01.418+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:01.811 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:01 smithi079 podman[121523]: 2024-06-03 02:38:01.694419341 +0000 UTC m=+0.032822584 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:38:01.811 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:01 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:01.551+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:01.811 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:01 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:01.552+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:01.811 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:01 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:01.552+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:01.811 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:01 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:01.552+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:02.089 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:01 smithi079 podman[121523]: 2024-06-03 02:38:01.811034815 +0000 UTC m=+0.149438098 container create d35d8a4eabb305005f48fa397e78b2f41c14f778b554bdee53774af341972d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-03T02:38:02.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:01 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:01.864+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:02.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:01 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:01.864+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:02.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:01 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:01.864+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:02.579 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:02 smithi079 podman[121523]: 2024-06-03 02:38:02.106681187 +0000 UTC m=+0.445084510 container init d35d8a4eabb305005f48fa397e78b2f41c14f778b554bdee53774af341972d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-03T02:38:02.580 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:02 smithi079 podman[121523]: 2024-06-03 02:38:02.117529056 +0000 UTC m=+0.455932346 container start d35d8a4eabb305005f48fa397e78b2f41c14f778b554bdee53774af341972d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-06-03T02:38:02.580 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:02 smithi079 podman[121523]: 2024-06-03 02:38:02.26395673 +0000 UTC m=+0.602360057 container attach d35d8a4eabb305005f48fa397e78b2f41c14f778b554bdee53774af341972d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-03T02:38:02.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:02 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:02.488+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:38:02.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:02 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:02.488+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:38:02.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:02 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:02.488+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:38:02.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:02 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:02.369+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:02.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:02 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:02.369+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:02.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:02 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:02.369+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:02.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:02 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:02.369+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:02.838 INFO:journalctl@ceph.osd.6.smithi079.stdout:Jun 03 02:38:02 smithi079 podman[121523]: 2024-06-03 02:38:02.581610856 +0000 UTC m=+0.920014159 container died d35d8a4eabb305005f48fa397e78b2f41c14f778b554bdee53774af341972d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-03T02:38:02.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:02 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:02.577+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:02.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:02 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:02.577+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:02.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:02 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:02.577+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:02.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:02 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:02.577+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:02.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:02 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:02.358+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:02.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:02 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:02.358+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:02.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:02 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:02.358+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:03.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:02 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:02.909+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:03.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:02 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:02.909+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:03.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:02 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:02.909+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:03.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-03T02:38:03.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:38:03.708 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-03T02:38:03.708 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-03T02:38:03.708 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.7 2024-06-03T02:38:03.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:03 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:03.446+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:38:03.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:03 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:03.446+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:38:03.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:03 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:03.446+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:38:03.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:03 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:03.345+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:03.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:03 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:03.345+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:03.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:03 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:03.345+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:03.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:03 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:03.345+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:03.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:03 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:03.584+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:03.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:03 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:03.584+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:03.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:03 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:03.584+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:03.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:03 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:03.584+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:03.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:03 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:03.356+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:03.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:03 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:03.356+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:03.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:03 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:03.356+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:04.178 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:03 smithi123 systemd[1]: Stopping Ceph osd.7 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:38:04.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:03 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:03.943+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:04.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:03 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:03.943+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:04.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:03 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:03.943+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:04.473 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:04.374+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:04.473 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:04.374+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:04.473 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:04.374+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:04.473 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:04.374+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:04.474 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:04.177+0000 7fa1d4c26700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:38:04.474 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:04.177+0000 7fa1d4c26700 -1 osd.7 105 *** Got signal Terminated *** 2024-06-03T02:38:04.474 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:04.177+0000 7fa1d4c26700 -1 osd.7 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:38:04.474 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:04.416+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:38:04.474 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:04.416+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:38:04.474 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:04 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:04.416+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:38:04.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:04 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:04.358+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:04.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:04 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:04.358+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:04.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:04 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:04.358+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:05.088 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:04 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:04.610+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:05.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:04 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:04.610+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:05.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:04 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:04.610+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:05.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:04 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:04.610+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:05.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:04 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:04.969+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:05.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:04 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:04.969+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:05.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:04 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:04.969+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:05.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:05 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:05.409+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:38:05.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:05 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:05.409+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:38:05.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:05 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:05.409+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:38:05.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:05 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:05.361+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:05.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:05 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:05.361+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:05.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:05 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:05.361+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:05.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:05 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:05.361+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:05.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:05 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:05.402+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:05.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:05 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:05.402+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:05.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:05 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:05.402+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:06.088 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:05 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:05.634+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:06.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:05 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:05.634+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:06.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:05 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:05.634+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:06.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:05 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:05.634+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:06.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:05 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:05.988+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:06.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:05 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:05.988+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:06.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:05 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:05.988+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:06.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:06 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:06.440+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:38:06.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:06 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:06.440+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:38:06.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:06 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:06.440+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:38:06.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:06 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:06.440+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:41.932695+0000 front 2024-06-03T02:37:41.932664+0000 (oldest deadline 2024-06-03T02:38:06.032828+0000) 2024-06-03T02:38:06.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:06 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:06.350+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:06.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:06 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:06.350+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:06.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:06 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:06.350+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:06.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:06 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:06.350+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:06.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:06 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:06.378+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:06.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:06 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:06.378+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:06.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:06 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:06.378+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:06.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:06 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:06.378+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:07.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:06 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:06.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:07.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:06 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:06.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:07.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:06 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:06.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:07.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:06 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:06.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:07.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:06 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:06.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:07.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:06 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:06.997+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:07.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:06 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:06.997+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:07.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:06 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:06.997+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:07.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:06 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:06.997+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:07.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:07.483+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:38:07.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:07.483+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:38:07.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:07.483+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:38:07.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:07.483+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:41.932695+0000 front 2024-06-03T02:37:41.932664+0000 (oldest deadline 2024-06-03T02:38:06.032828+0000) 2024-06-03T02:38:07.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:07.389+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:07.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:07.389+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:07.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:07.389+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:07.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:07.389+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:07.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:07 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:07.353+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:07.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:07 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:07.353+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:07.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:07 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:07.353+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:07.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:07 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:07.353+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:08.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:07 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:07.697+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:08.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:07 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:07.697+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:08.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:07 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:07.697+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:08.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:07 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:07.697+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:08.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:07 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:07.697+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:08.324 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:07 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:07.959+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:08.324 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:07 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:07.959+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:08.324 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:07 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:07.959+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:08.324 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:07 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:07.959+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:08.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:08 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:08.323+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:08.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:08 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:08.323+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:08.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:08 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:08.323+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:08.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:08 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:08.323+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:08.723 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:08 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:08.477+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.826733+0000 front 2024-06-03T02:37:14.826689+0000 (oldest deadline 2024-06-03T02:37:37.726829+0000) 2024-06-03T02:38:08.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:08 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:08.477+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:24.029060+0000 front 2024-06-03T02:37:24.029152+0000 (oldest deadline 2024-06-03T02:37:47.429635+0000) 2024-06-03T02:38:08.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:08 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:08.477+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:30.330724+0000 front 2024-06-03T02:37:30.330672+0000 (oldest deadline 2024-06-03T02:37:56.230527+0000) 2024-06-03T02:38:08.724 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:08 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7[54995]: 2024-06-03T02:38:08.477+0000 7fa1d121a700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:41.932695+0000 front 2024-06-03T02:37:41.932664+0000 (oldest deadline 2024-06-03T02:38:06.032828+0000) 2024-06-03T02:38:08.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:08 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:08.354+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:08.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:08 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:08.354+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:08.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:08 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:08.354+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:08.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:08 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:08.354+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:08.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:08 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:08.354+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:09.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:08.649+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:09.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:08.649+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:09.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:08.649+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:09.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:08.649+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:09.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:08 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:08.649+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:09.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:08 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:08.996+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:09.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:08 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:08.996+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:09.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:08 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:08.996+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:09.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:08 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:08.996+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:09.705 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:09 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:09.399+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:09.705 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:09 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:09.399+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:09.705 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:09 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:09.399+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:09.705 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:09 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:09.399+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:09.705 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:09 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:09.399+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:09.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:09 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:09.371+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:09.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:09 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:09.371+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:09.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:09 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:09.371+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:09.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:09 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:09.371+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:09.973 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:09 smithi123 podman[74047]: 2024-06-03 02:38:09.705214608 +0000 UTC m=+5.797585057 container died 15fbbe69ba48cd3fcc44e635b5d0479211d9e38a7c15857c2d6c61d0ef040d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-03T02:38:10.088 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:09 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:09.656+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:10.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:09 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:09.656+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:10.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:09 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:09.656+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:10.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:09 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:09.656+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:10.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:09 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:09.656+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:10.331 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:10 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:10.035+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:10.331 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:10 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:10.035+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:10.331 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:10 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:10.035+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:10.331 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:10 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:10.035+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:10.347 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:10 smithi123 podman[74047]: 2024-06-03 02:38:10.071217422 +0000 UTC m=+6.163587863 container cleanup 15fbbe69ba48cd3fcc44e635b5d0479211d9e38a7c15857c2d6c61d0ef040d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-03T02:38:10.347 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:10 smithi123 bash[74047]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7 2024-06-03T02:38:10.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:10 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:10.330+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:10.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:10 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:10.330+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:10.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:10 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:10.330+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:10.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:10 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:10.330+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:10.687 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:10 smithi123 podman[74063]: 2024-06-03 02:38:10.347710374 +0000 UTC m=+0.642670125 container remove 15fbbe69ba48cd3fcc44e635b5d0479211d9e38a7c15857c2d6c61d0ef040d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:38:10.687 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:10 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:10.352+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:10.687 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:10 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:10.352+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:10.688 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:10 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:10.352+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:10.688 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:10 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:10.352+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:10.688 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:10 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:10.352+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:10.967 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:10 smithi123 podman[74151]: 2024-06-03 02:38:10.590277657 +0000 UTC m=+0.016992072 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:38:10.967 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:10 smithi123 podman[74151]: 2024-06-03 02:38:10.715265377 +0000 UTC m=+0.141979801 container create 93d508ce3184a2db33ed3998b8eb3d65485495c49eb6f6006edad647978b0f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-03T02:38:10.967 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:10 smithi123 podman[74151]: 2024-06-03 02:38:10.967487009 +0000 UTC m=+0.394201400 container init 93d508ce3184a2db33ed3998b8eb3d65485495c49eb6f6006edad647978b0f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-03T02:38:11.088 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:10 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:10.672+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:11.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:10 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:10.672+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:11.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:10 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:10.672+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:11.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:10 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:10.672+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:11.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:10 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:10.672+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:11.223 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:10 smithi123 podman[74151]: 2024-06-03 02:38:10.97060901 +0000 UTC m=+0.397323413 container start 93d508ce3184a2db33ed3998b8eb3d65485495c49eb6f6006edad647978b0f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-06-03T02:38:11.224 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:11 smithi123 podman[74151]: 2024-06-03 02:38:11.059163994 +0000 UTC m=+0.485878421 container attach 93d508ce3184a2db33ed3998b8eb3d65485495c49eb6f6006edad647978b0f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528) 2024-06-03T02:38:11.338 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:11.020+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:11.339 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:11.020+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:11.339 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:11.020+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:11.339 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:11.020+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:11.537 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 03 02:38:11 smithi123 podman[74151]: 2024-06-03 02:38:11.266732197 +0000 UTC m=+0.693446604 container died 93d508ce3184a2db33ed3998b8eb3d65485495c49eb6f6006edad647978b0f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:38:11.537 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:11.381+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:11.537 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:11.381+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:11.537 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:11.381+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:11.537 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:11.381+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:11.537 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:11.381+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:11.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:11.337+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:11.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:11.337+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:11.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:11.337+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:11.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:11.337+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:11.839 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-03T02:38:11.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:38:11.918 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-03T02:38:11.918 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-03T02:38:11.918 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.8 2024-06-03T02:38:12.088 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:11 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:11.625+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:12.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:11 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:11.625+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:12.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:11 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:11.625+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:12.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:11 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:11.625+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:12.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:11 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:11.625+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:12.249 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:11 smithi135 systemd[1]: Stopping Ceph osd.8 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:38:12.249 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:11.996+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:12.249 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:11.996+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:12.249 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:11.996+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:12.250 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:11.996+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:12.250 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:11 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:11.996+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:12.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:12 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:12.247+0000 7eff9c77a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:38:12.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:12 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:12.247+0000 7eff9c77a700 -1 osd.8 105 *** Got signal Terminated *** 2024-06-03T02:38:12.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:12 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:12.247+0000 7eff9c77a700 -1 osd.8 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:38:12.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:12 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:12.362+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:12.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:12 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:12.362+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:12.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:12 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:12.362+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:12.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:12 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:12.362+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:12.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:12 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:12.380+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:12.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:12 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:12.380+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:12.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:12 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:12.380+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:12.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:12 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:12.380+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:12.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:12 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:12.380+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:13.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:12 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:12.614+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:13.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:12 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:12.614+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:13.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:12 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:12.614+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:13.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:12 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:12.614+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:13.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:12 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:12.614+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:13.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:12 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:12.614+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:13.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.008+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:13.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.008+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:13.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.008+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:13.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.008+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:13.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.008+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:13.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:13 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:13.343+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:13.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:13 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:13.343+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:13.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:13 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:13.343+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:13.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:13 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:13.343+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:13.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:13 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:13.343+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:13.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:13 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:13.343+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:13.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:13.363+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:13.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:13.363+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:13.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:13.363+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:13.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:13.363+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:14.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:13 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:13.653+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:14.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:13 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:13.653+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:14.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:13 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:13.653+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:14.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:13 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:13.653+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:14.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:13 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:13.653+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:14.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:13 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:13.653+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:14.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.993+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:14.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.993+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:14.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.993+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:14.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.993+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:14.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:13 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:13.993+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:14.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:14 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:14.318+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:14.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:14 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:14.318+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:14.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:14 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:14.318+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:14.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:14 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:14.318+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:14.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:14 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:14.318+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:14.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:14 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:14.318+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:14.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:14.357+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:14.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:14.357+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:14.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:14.357+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:14.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:14.357+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:15.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:14.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:15.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:14.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:15.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:14.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:15.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:14.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:15.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:14.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:15.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:14 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:14.673+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:15.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:14.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:15.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:14.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:15.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:14.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:15.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:14.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:15.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:14 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:14.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:15.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:15 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:15.347+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:15.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:15 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:15.347+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:15.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:15 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:15.347+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:15.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:15 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:15.347+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:15.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:15 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:15.347+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:15.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:15 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:15.347+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:15.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:15.361+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:15.845 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:15.361+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:15.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:15.361+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:15.846 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:15.361+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:16.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:15 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:15.680+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:16.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:15 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:15.680+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:16.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:15 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:15.680+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:16.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:15 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:15.680+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:16.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:15 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:15.680+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:16.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:15 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:15.680+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:16.338 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:15.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:16.338 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:15.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:16.338 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:15.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:16.338 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:15.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:16.338 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:15 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:15.972+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:16.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:16.336+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.865622+0000 front 2024-06-03T02:37:15.865672+0000 (oldest deadline 2024-06-03T02:37:37.565650+0000) 2024-06-03T02:38:16.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:16.336+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.367433+0000 front 2024-06-03T02:37:23.367471+0000 (oldest deadline 2024-06-03T02:37:49.168271+0000) 2024-06-03T02:38:16.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:16.336+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.469534+0000 front 2024-06-03T02:37:31.469507+0000 (oldest deadline 2024-06-03T02:37:54.869929+0000) 2024-06-03T02:38:16.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:16.336+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:40.171455+0000 front 2024-06-03T02:37:40.171341+0000 (oldest deadline 2024-06-03T02:38:06.071426+0000) 2024-06-03T02:38:16.596 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8[55647]: 2024-06-03T02:38:16.336+0000 7eff98d6e700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:50.072813+0000 front 2024-06-03T02:37:50.072833+0000 (oldest deadline 2024-06-03T02:38:15.972970+0000) 2024-06-03T02:38:16.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:16 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:16.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:16.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:16 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:16.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:16.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:16 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:16.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:16.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:16 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:16.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:16.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:16 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:16.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:16.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:16 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:16.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:17.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:16 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:16.691+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:17.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:16 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:16.691+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:17.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:16 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:16.691+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:17.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:16 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:16.691+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:17.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:16 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:16.691+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:17.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:16 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:16.691+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:17.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:16.956+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:17.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:16.956+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:17.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:16.956+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:17.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:16.956+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:17.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:16 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:16.956+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:17.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:17 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:17.280+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:17.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:17 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:17.280+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:17.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:17 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:17.280+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:17.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:17 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:17.280+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:17.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:17 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:17.280+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:17.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:17 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:17.280+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:18.058 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:17 smithi135 podman[74320]: 2024-06-03 02:38:17.794417144 +0000 UTC m=+5.681334056 container died d3d103cd354d57c93594c6a5301e9bd4d314f94c4e729ce63037689537f89c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:38:18.058 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:17 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:17.988+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:18.058 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:17 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:17.988+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:18.058 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:17 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:17.988+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:18.058 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:17 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:17.988+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:18.059 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:17 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:17.988+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:18.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:17 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:17.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:18.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:17 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:17.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:18.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:17 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:17.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:18.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:17 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:17.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:18.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:17 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:17.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:18.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:17 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:17.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:18.345 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:18 smithi135 podman[74320]: 2024-06-03 02:38:18.148938339 +0000 UTC m=+6.035855243 container cleanup d3d103cd354d57c93594c6a5301e9bd4d314f94c4e729ce63037689537f89c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-03T02:38:18.345 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:18 smithi135 bash[74320]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8 2024-06-03T02:38:18.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:18 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:18.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:18.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:18 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:18.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:18.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:18 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:18.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:18.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:18 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:18.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:18.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:18 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:18.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:18.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:18 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:18.306+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:18.771 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:18 smithi135 podman[74338]: 2024-06-03 02:38:18.434199977 +0000 UTC m=+0.641349991 container remove d3d103cd354d57c93594c6a5301e9bd4d314f94c4e729ce63037689537f89c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:38:19.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:18 smithi135 podman[74426]: 2024-06-03 02:38:18.674641991 +0000 UTC m=+0.017244820 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:38:19.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:18 smithi135 podman[74426]: 2024-06-03 02:38:18.793163982 +0000 UTC m=+0.135766795 container create 94c1f34835081a9ffc29f1d66752ee352c70211f76a5689d880101c93f57b7ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-03T02:38:19.045 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.004+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:19.045 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.004+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:19.046 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.004+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:19.046 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.004+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:19.046 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.004+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:19.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:18 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:18.637+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:19.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:18 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:18.637+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:19.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:18 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:18.637+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:19.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:18 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:18.637+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:19.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:18 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:18.637+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:19.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:18 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:18.637+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:19.316 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:19 smithi135 podman[74426]: 2024-06-03 02:38:19.045525599 +0000 UTC m=+0.388128418 container init 94c1f34835081a9ffc29f1d66752ee352c70211f76a5689d880101c93f57b7ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-03T02:38:19.316 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:19 smithi135 podman[74426]: 2024-06-03 02:38:19.049150549 +0000 UTC m=+0.391753375 container start 94c1f34835081a9ffc29f1d66752ee352c70211f76a5689d880101c93f57b7ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-03T02:38:19.316 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:19 smithi135 podman[74426]: 2024-06-03 02:38:19.177743098 +0000 UTC m=+0.520345920 container attach 94c1f34835081a9ffc29f1d66752ee352c70211f76a5689d880101c93f57b7ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True) 2024-06-03T02:38:19.595 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 03 02:38:19 smithi135 podman[74426]: 2024-06-03 02:38:19.352510956 +0000 UTC m=+0.695113783 container died 94c1f34835081a9ffc29f1d66752ee352c70211f76a5689d880101c93f57b7ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-03T02:38:19.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:19.289+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:19.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:19.289+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:19.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:19.289+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:19.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:19.289+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:19.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:19.289+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:19.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:19 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:19.289+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:19.978 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-03T02:38:20.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:38:20.015 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-03T02:38:20.015 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-03T02:38:20.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.9 2024-06-03T02:38:20.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:19 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:19.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:20.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:19 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:19.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:20.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:19 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:19.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:20.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:19 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:19.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:20.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:19 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:19.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:20.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:19 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:19.679+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:20.257 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.982+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:20.257 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.982+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:20.257 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.982+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:20.258 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.982+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:20.258 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:19.982+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:20.588 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 systemd[1]: Stopping Ceph osd.9 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:38:20.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:20.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:20.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:20.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:20.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:20.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:20.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:20.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:20.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:20.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:20.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:20 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:20.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:21.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:20.625+0000 7fdf7ec00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:38:21.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:20.625+0000 7fdf7ec00700 -1 osd.9 105 *** Got signal Terminated *** 2024-06-03T02:38:21.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:20.625+0000 7fdf7ec00700 -1 osd.9 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:38:21.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:20.712+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:21.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:20.712+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:21.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:20.712+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:21.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:20.712+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:21.090 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:20.712+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:21.090 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:20 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:20.712+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:21.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:20 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:20.945+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:21.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:20 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:20.945+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:21.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:20 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:20.945+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:21.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:20 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:20.945+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:21.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:20 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:20.945+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:21.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:21.208+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:21.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:21.208+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:21.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:21.208+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:21.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:21.208+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:21.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:21.208+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:21.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:21 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:21.208+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:22.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:21 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:21.713+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:22.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:21 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:21.713+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:22.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:21 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:21.713+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:22.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:21 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:21.713+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:22.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:21 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:21.713+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:22.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:21 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:21.713+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:22.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:21 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:21.930+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:22.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:21 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:21.930+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:22.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:21 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:21.930+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:22.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:21 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:21.930+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:22.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:21 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:21.930+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:22.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:21 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:21.930+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:22.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:22 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:22.226+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:22.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:22 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:22.226+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:22.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:22 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:22.226+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:22.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:22 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:22.226+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:22.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:22 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:22.226+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:22.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:22 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:22.226+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:23.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:22.666+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:23.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:22.666+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:23.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:22.666+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:23.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:22.666+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:23.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:22.666+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:23.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:22 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:22.666+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:23.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:22 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:22.889+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:23.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:22 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:22.889+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:23.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:22 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:22.889+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:23.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:22 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:22.889+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:23.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:22 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:22.889+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:23.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:22 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:22.889+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:23.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:23 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:23.219+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:23.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:23 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:23.219+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:23.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:23 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:23.219+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:23.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:23 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:23.219+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:23.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:23 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:23.219+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:23.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:23 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:23.219+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:24.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:23 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:23.716+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:24.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:23 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:23.716+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:24.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:23 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:23.716+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:24.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:23 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:23.716+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:24.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:23 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:23.716+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:24.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:23 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:23.716+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:24.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:23 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:23.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:24.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:23 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:23.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:24.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:23 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:23.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:24.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:23 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:23.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:24.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:23 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:23.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:24.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:23 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:23.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:24.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:24 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:24.249+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:24.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:24 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:24.249+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:24.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:24 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:24.249+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:24.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:24 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:24.249+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:24.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:24 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:24.249+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:24.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:24 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:24.249+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:24.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:24 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:24.249+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:25.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:24 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:24.723+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:04.878225+0000 front 2024-06-03T02:37:04.878550+0000 (oldest deadline 2024-06-03T02:37:27.778417+0000) 2024-06-03T02:38:25.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:24 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:24.723+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:14.181842+0000 front 2024-06-03T02:37:14.181896+0000 (oldest deadline 2024-06-03T02:37:39.482123+0000) 2024-06-03T02:38:25.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:24 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:24.723+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:23.584144+0000 front 2024-06-03T02:37:23.584208+0000 (oldest deadline 2024-06-03T02:37:49.484386+0000) 2024-06-03T02:38:25.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:24 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:24.723+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:31.786448+0000 front 2024-06-03T02:37:31.786734+0000 (oldest deadline 2024-06-03T02:37:57.686605+0000) 2024-06-03T02:38:25.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:24 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:24.723+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:37.687895+0000 front 2024-06-03T02:37:37.687836+0000 (oldest deadline 2024-06-03T02:38:05.889578+0000) 2024-06-03T02:38:25.089 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:24 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9[67472]: 2024-06-03T02:38:24.723+0000 7fdf7a400700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.992137+0000 front 2024-06-03T02:37:49.992536+0000 (oldest deadline 2024-06-03T02:38:11.692350+0000) 2024-06-03T02:38:25.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:24 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:24.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:25.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:24 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:24.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:25.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:24 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:24.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:25.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:24 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:24.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:25.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:24 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:24.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:25.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:24 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:24.896+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:25.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:25 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:25.202+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:25.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:25 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:25.203+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:25.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:25 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:25.203+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:25.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:25 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:25.203+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:25.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:25 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:25.203+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:25.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:25 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:25.203+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:25.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:25 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:25.203+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:26.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:25 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:25.859+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:26.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:25 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:25.859+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:26.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:25 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:25.859+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:26.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:25 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:25.859+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:26.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:25 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:25.859+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:26.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:25 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:25.859+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:26.473 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:26 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:26.218+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:26.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:26 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:26.218+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:26.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:26 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:26.218+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:26.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:26 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:26.218+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:26.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:26 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:26.218+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:26.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:26 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:26.218+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:26.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:26 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:26.218+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:26.994 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:26 smithi079 podman[121628]: 2024-06-03 02:38:26.688271984 +0000 UTC m=+6.211264972 container died 6951ae1c31e24a8328866aa3a831734b59ce1503be44ea16b2efaf5ba109b830 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:38:27.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:26 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:26.817+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:27.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:26 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:26.817+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:27.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:26 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:26.817+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:27.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:26 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:26.817+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:27.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:26 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:26.817+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:27.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:26 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:26.817+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:27.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:27 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:27.176+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:27.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:27 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:27.176+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:27.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:27 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:27.176+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:27.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:27 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:27.176+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:27.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:27 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:27.176+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:27.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:27 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:27.176+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:27.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:27 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:27.176+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:27.570 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:27 smithi079 podman[121628]: 2024-06-03 02:38:27.268038991 +0000 UTC m=+6.791031926 container cleanup 6951ae1c31e24a8328866aa3a831734b59ce1503be44ea16b2efaf5ba109b830 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:38:27.571 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:27 smithi079 bash[121628]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9 2024-06-03T02:38:27.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:27 smithi079 podman[121647]: 2024-06-03 02:38:27.569460374 +0000 UTC m=+0.864748773 container remove 6951ae1c31e24a8328866aa3a831734b59ce1503be44ea16b2efaf5ba109b830 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-03T02:38:28.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:27 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:27.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:28.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:27 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:27.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:28.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:27 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:27.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:28.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:27 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:27.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:28.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:27 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:27.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:28.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:27 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:27.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:28.473 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:28 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:28.189+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:28.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:28 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:28.189+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:28.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:28 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:28.189+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:28.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:28 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:28.189+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:28.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:28 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:28.189+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:28.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:28 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:28.189+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:28.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:28 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:28.189+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:28.492 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:28 smithi079 podman[121735]: 2024-06-03 02:38:28.103053811 +0000 UTC m=+0.036214568 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:38:28.493 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:28 smithi079 podman[121735]: 2024-06-03 02:38:28.22850524 +0000 UTC m=+0.161666002 container create e4ae7f9511757424b5ed7d08564833bc9fb72074b53dd1742542e79a1d4fdd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:38:28.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:28 smithi079 podman[121735]: 2024-06-03 02:38:28.492413801 +0000 UTC m=+0.425574572 container init e4ae7f9511757424b5ed7d08564833bc9fb72074b53dd1742542e79a1d4fdd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-03T02:38:28.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:28 smithi079 podman[121735]: 2024-06-03 02:38:28.503251496 +0000 UTC m=+0.436412237 container start e4ae7f9511757424b5ed7d08564833bc9fb72074b53dd1742542e79a1d4fdd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-03T02:38:28.839 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:28 smithi079 podman[121735]: 2024-06-03 02:38:28.620396929 +0000 UTC m=+0.553557661 container attach e4ae7f9511757424b5ed7d08564833bc9fb72074b53dd1742542e79a1d4fdd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True) 2024-06-03T02:38:29.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:28 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:28.844+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:29.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:28 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:28.844+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:29.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:28 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:28.844+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:29.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:28 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:28.844+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:29.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:28 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:28.844+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:29.097 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:28 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:28.844+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:29.332 INFO:journalctl@ceph.osd.9.smithi079.stdout:Jun 03 02:38:29 smithi079 podman[121771]: 2024-06-03 02:38:29.015734882 +0000 UTC m=+0.031498559 container died e4ae7f9511757424b5ed7d08564833bc9fb72074b53dd1742542e79a1d4fdd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-03T02:38:29.646 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:29 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:29.231+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:29.646 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:29 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:29.231+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:29.646 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:29 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:29.231+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:29.646 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:29 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:29.231+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:29.646 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:29 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:29.231+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:29.647 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:29 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:29.231+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:29.647 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:29 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:29.231+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:29.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-03T02:38:29.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:38:29.910 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-03T02:38:29.910 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-03T02:38:29.910 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.10 2024-06-03T02:38:30.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:29 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:29.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:30.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:29 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:29.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:30.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:29 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:29.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:30.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:29 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:29.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:30.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:29 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:29.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:30.096 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:29 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:29.853+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:30.223 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:29 smithi123 systemd[1]: Stopping Ceph osd.10 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:38:30.473 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.222+0000 7f8573414700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:38:30.473 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.222+0000 7f8573414700 -1 osd.10 105 *** Got signal Terminated *** 2024-06-03T02:38:30.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.222+0000 7f8573414700 -1 osd.10 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:38:30.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.238+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:30.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.239+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:30.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.239+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:30.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.239+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:30.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.239+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:30.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.239+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:30.474 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:30 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:30.239+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:31.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:30.903+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:31.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:30.903+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:31.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:30.903+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:31.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:30.903+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:31.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:30.903+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:31.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:30 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:30.903+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:31.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:31 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:31.246+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:31.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:31 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:31.246+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:31.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:31 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:31.246+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:31.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:31 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:31.246+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:31.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:31 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:31.247+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:31.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:31 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:31.247+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:31.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:31 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:31.247+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:32.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:31.952+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:32.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:31.952+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:32.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:31.952+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:32.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:31.952+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:32.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:31.952+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:32.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:31.952+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:32.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:32 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:32.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:32.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:32 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:32.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:32.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:32 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:32.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:32.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:32 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:32.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:32.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:32 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:32.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:32.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:32 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:32.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:32.725 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:32 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:32.256+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:33.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:32 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:32.989+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:33.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:32 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:32.989+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:33.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:32 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:32.989+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:33.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:32 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:32.989+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:33.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:32 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:32.989+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:33.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:32 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:32.989+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:33.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:32 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:32.989+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:33.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:33 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:33.288+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:33.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:33 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:33.288+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:33.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:33 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:33.288+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:33.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:33 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:33.288+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:33.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:33 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:33.288+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:33.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:33 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:33.288+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:33.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:33 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:33.288+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:34.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:34 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:34.018+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:34.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:34 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:34.018+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:34.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:34 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:34.018+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:34.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:34 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:34.018+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:34.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:34 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:34.018+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:34.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:34 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:34.018+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:34.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:34 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:34.018+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:34.723 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:34.262+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:02.257139+0000 front 2024-06-03T02:37:02.257409+0000 (oldest deadline 2024-06-03T02:37:29.857742+0000) 2024-06-03T02:38:34.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:34.262+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:15.560037+0000 front 2024-06-03T02:37:15.560064+0000 (oldest deadline 2024-06-03T02:37:37.860123+0000) 2024-06-03T02:38:34.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:34.262+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:20.562347+0000 front 2024-06-03T02:37:20.562385+0000 (oldest deadline 2024-06-03T02:37:46.462456+0000) 2024-06-03T02:38:34.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:34.262+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6814 osd.3 since back 2024-06-03T02:37:28.764115+0000 front 2024-06-03T02:37:28.764064+0000 (oldest deadline 2024-06-03T02:37:54.063458+0000) 2024-06-03T02:38:34.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:34.262+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:42.264875+0000 front 2024-06-03T02:37:42.264907+0000 (oldest deadline 2024-06-03T02:38:08.065529+0000) 2024-06-03T02:38:34.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:34.262+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:49.766728+0000 front 2024-06-03T02:37:49.766607+0000 (oldest deadline 2024-06-03T02:38:13.266442+0000) 2024-06-03T02:38:34.724 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:34 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10[62008]: 2024-06-03T02:38:34.262+0000 7f856ba00700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.567750+0000 front 2024-06-03T02:37:58.567803+0000 (oldest deadline 2024-06-03T02:38:23.867581+0000) 2024-06-03T02:38:35.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:35.040+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:35.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:35.040+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:35.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:35.040+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:35.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:35.040+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:35.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:35.040+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:35.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:35.040+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:35.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:35 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:35.040+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:35.971 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:35 smithi123 podman[74267]: 2024-06-03 02:38:35.662246268 +0000 UTC m=+5.566330667 container died 59b5ae5c94f03eadf8f44cd997968dc46dcff1de2ac62ef16e0973f762b1c38e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:38:35.971 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:35 smithi123 podman[74267]: 2024-06-03 02:38:35.972050883 +0000 UTC m=+5.876135280 container cleanup 59b5ae5c94f03eadf8f44cd997968dc46dcff1de2ac62ef16e0973f762b1c38e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-03T02:38:36.223 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:35 smithi123 bash[74267]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10 2024-06-03T02:38:36.225 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:36 smithi123 podman[74285]: 2024-06-03 02:38:36.181618468 +0000 UTC m=+0.516279685 container remove 59b5ae5c94f03eadf8f44cd997968dc46dcff1de2ac62ef16e0973f762b1c38e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-03T02:38:36.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:36 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:36.066+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:36.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:36 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:36.066+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:36.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:36 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:36.066+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:36.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:36 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:36.066+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:36.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:36 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:36.066+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:36.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:36 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:36.066+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:36.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:36 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:36.066+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:36.768 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:36 smithi123 podman[74380]: 2024-06-03 02:38:36.397897589 +0000 UTC m=+0.017103374 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:38:36.768 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:36 smithi123 podman[74380]: 2024-06-03 02:38:36.499197221 +0000 UTC m=+0.118403028 container create e654396bc4f41ce020645bb0ae513b1976d85cac93b7f885d149e9e67531e851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-03T02:38:36.768 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:36 smithi123 podman[74380]: 2024-06-03 02:38:36.701856227 +0000 UTC m=+0.321062012 container init e654396bc4f41ce020645bb0ae513b1976d85cac93b7f885d149e9e67531e851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-03T02:38:36.768 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:36 smithi123 podman[74380]: 2024-06-03 02:38:36.704845291 +0000 UTC m=+0.324051079 container start e654396bc4f41ce020645bb0ae513b1976d85cac93b7f885d149e9e67531e851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-03T02:38:36.768 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:36 smithi123 podman[74380]: 2024-06-03 02:38:36.768484604 +0000 UTC m=+0.387690409 container attach e654396bc4f41ce020645bb0ae513b1976d85cac93b7f885d149e9e67531e851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-06-03T02:38:37.223 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 03 02:38:36 smithi123 podman[74380]: 2024-06-03 02:38:36.998726357 +0000 UTC m=+0.617932151 container died e654396bc4f41ce020645bb0ae513b1976d85cac93b7f885d149e9e67531e851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-06-03T02:38:37.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.085+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:37.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.085+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:37.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.085+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:37.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.085+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:37.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.085+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:37.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.085+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:37.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.085+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:37.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-03T02:38:37.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:38:37.659 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-03T02:38:37.659 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-03T02:38:37.659 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.11 2024-06-03T02:38:37.958 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 systemd[1]: Stopping Ceph osd.11 for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:38:37.958 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.958+0000 7f46296e4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:38:37.958 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.958+0000 7f46296e4700 -1 osd.11 105 *** Got signal Terminated *** 2024-06-03T02:38:37.959 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:37 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:37.958+0000 7f46296e4700 -1 osd.11 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T02:38:38.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:38.072+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:38.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:38.072+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:38.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:38.072+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:38.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:38.072+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:38.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:38.072+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:38.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:38.072+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:38.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:38 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:38.072+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:39.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:39.070+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:39.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:39.070+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:39.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:39.070+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:39.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:39.070+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:39.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:39.070+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:39.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:39.070+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:39.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:39.070+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:39.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:39 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:39.070+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-06-03T02:38:16.163280+0000 front 2024-06-03T02:38:16.163256+0000 (oldest deadline 2024-06-03T02:38:38.463308+0000) 2024-06-03T02:38:40.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.047+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:40.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.047+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:40.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.047+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:40.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.047+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:40.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.047+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:40.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.047+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:40.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.047+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:40.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:40 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.047+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-06-03T02:38:16.163280+0000 front 2024-06-03T02:38:16.163256+0000 (oldest deadline 2024-06-03T02:38:38.463308+0000) 2024-06-03T02:38:41.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.999+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:41.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.999+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:41.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.999+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:41.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.999+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:41.347 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.999+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:41.347 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.999+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:41.347 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.999+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:41.347 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:40.999+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-06-03T02:38:16.163280+0000 front 2024-06-03T02:38:16.163256+0000 (oldest deadline 2024-06-03T02:38:38.463308+0000) 2024-06-03T02:38:42.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:41.973+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-06-03T02:37:05.750412+0000 front 2024-06-03T02:37:05.750393+0000 (oldest deadline 2024-06-03T02:37:31.551047+0000) 2024-06-03T02:38:42.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:41.973+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-03T02:37:12.052459+0000 front 2024-06-03T02:37:12.052471+0000 (oldest deadline 2024-06-03T02:37:37.253308+0000) 2024-06-03T02:38:42.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:41.973+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-03T02:37:21.255324+0000 front 2024-06-03T02:37:21.255174+0000 (oldest deadline 2024-06-03T02:37:47.155381+0000) 2024-06-03T02:38:42.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:41.973+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-03T02:37:38.357087+0000 front 2024-06-03T02:37:38.356995+0000 (oldest deadline 2024-06-03T02:38:06.557542+0000) 2024-06-03T02:38:42.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:41.973+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-03T02:37:46.558099+0000 front 2024-06-03T02:37:46.557971+0000 (oldest deadline 2024-06-03T02:38:11.758524+0000) 2024-06-03T02:38:42.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:41.973+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.79:6822 osd.6 since back 2024-06-03T02:37:58.160399+0000 front 2024-06-03T02:37:58.160353+0000 (oldest deadline 2024-06-03T02:38:21.660139+0000) 2024-06-03T02:38:42.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:41.973+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-03T02:38:08.061862+0000 front 2024-06-03T02:38:08.061843+0000 (oldest deadline 2024-06-03T02:38:32.161852+0000) 2024-06-03T02:38:42.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:41 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11[62223]: 2024-06-03T02:38:41.973+0000 7f4621cd0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-06-03T02:38:16.163280+0000 front 2024-06-03T02:38:16.163256+0000 (oldest deadline 2024-06-03T02:38:38.463308+0000) 2024-06-03T02:38:43.850 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:43 smithi135 podman[74546]: 2024-06-03 02:38:43.59798839 +0000 UTC m=+5.748968051 container died f3518e6805d038b54e819f6ffd704dab005823bf6acc09a799fe2b61015a8001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-03T02:38:44.267 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:43 smithi135 podman[74546]: 2024-06-03 02:38:43.949487978 +0000 UTC m=+6.100467645 container cleanup f3518e6805d038b54e819f6ffd704dab005823bf6acc09a799fe2b61015a8001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-03T02:38:44.267 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:43 smithi135 bash[74546]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11 2024-06-03T02:38:44.578 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:44 smithi135 podman[74562]: 2024-06-03 02:38:44.267511847 +0000 UTC m=+0.665167507 container remove f3518e6805d038b54e819f6ffd704dab005823bf6acc09a799fe2b61015a8001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-06-03T02:38:44.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:44 smithi135 podman[74650]: 2024-06-03 02:38:44.481595496 +0000 UTC m=+0.017674931 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T02:38:44.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:44 smithi135 podman[74650]: 2024-06-03 02:38:44.584888153 +0000 UTC m=+0.120967565 container create 9e4ffaf5544298416b09a9d7cf0d2b8782863ec00021a8cf8323460639984356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:38:44.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:44 smithi135 podman[74650]: 2024-06-03 02:38:44.795573154 +0000 UTC m=+0.331652581 container init 9e4ffaf5544298416b09a9d7cf0d2b8782863ec00021a8cf8323460639984356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-03T02:38:44.846 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:44 smithi135 podman[74650]: 2024-06-03 02:38:44.798732512 +0000 UTC m=+0.334811929 container start 9e4ffaf5544298416b09a9d7cf0d2b8782863ec00021a8cf8323460639984356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-03T02:38:45.345 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:44 smithi135 podman[74650]: 2024-06-03 02:38:44.889900749 +0000 UTC m=+0.425980181 container attach 9e4ffaf5544298416b09a9d7cf0d2b8782863ec00021a8cf8323460639984356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-03T02:38:45.346 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:45 smithi135 podman[74650]: 2024-06-03 02:38:45.087961193 +0000 UTC m=+0.624040611 container died 9e4ffaf5544298416b09a9d7cf0d2b8782863ec00021a8cf8323460639984356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:38:45.633 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 03 02:38:45 smithi135 podman[74677]: 2024-06-03 02:38:45.615456276 +0000 UTC m=+0.517932756 container remove 9e4ffaf5544298416b09a9d7cf0d2b8782863ec00021a8cf8323460639984356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-osd-11-deactivate, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-03T02:38:45.637 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-03T02:38:45.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:38:45.674 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-03T02:38:45.674 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-03T02:38:45.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.a 2024-06-03T02:38:46.088 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:38:45 smithi079 systemd[1]: Stopping Ceph mds.a for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:38:46.589 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:38:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a[74407]: 2024-06-03T02:38:46.265+0000 7fa863a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:38:46.589 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:38:46 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a[74407]: 2024-06-03T02:38:46.265+0000 7fa863a00700 -1 mds.a *** got signal Terminated *** 2024-06-03T02:38:50.588 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:38:50 smithi079 podman[121851]: 2024-06-03 02:38:50.316423266 +0000 UTC m=+4.179251762 container died f1472a61b4ec51edac531e6906d0dd9af04979339d27185fc147834af747509e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-06-03T02:38:51.338 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:38:50 smithi079 podman[121851]: 2024-06-03 02:38:50.876144923 +0000 UTC m=+4.738973416 container cleanup f1472a61b4ec51edac531e6906d0dd9af04979339d27185fc147834af747509e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:38:51.339 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:38:50 smithi079 bash[121851]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a 2024-06-03T02:38:51.339 INFO:journalctl@ceph.mds.a.smithi079.stdout:Jun 03 02:38:51 smithi079 podman[121862]: 2024-06-03 02:38:51.086260189 +0000 UTC m=+0.750877369 container remove f1472a61b4ec51edac531e6906d0dd9af04979339d27185fc147834af747509e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-a, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:38:51.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-03T02:38:51.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:38:51.739 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-03T02:38:51.739 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-03T02:38:51.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.d 2024-06-03T02:38:52.338 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:38:51 smithi079 systemd[1]: Stopping Ceph mds.d for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:38:52.838 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:38:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d[73234]: 2024-06-03T02:38:52.369+0000 7f1611600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:38:52.839 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:38:52 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d[73234]: 2024-06-03T02:38:52.369+0000 7f1611600700 -1 mds.d *** got signal Terminated *** 2024-06-03T02:38:54.579 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:38:54 smithi079 podman[121968]: 2024-06-03 02:38:54.298158387 +0000 UTC m=+2.060155528 container died 4f6fb36aef5945b21fd19cc2ab5b74bfd07d7dfe8d84e61e683c91dfa1483a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-03T02:38:55.338 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:38:54 smithi079 podman[121968]: 2024-06-03 02:38:54.838639716 +0000 UTC m=+2.600636821 container cleanup 4f6fb36aef5945b21fd19cc2ab5b74bfd07d7dfe8d84e61e683c91dfa1483a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:38:55.338 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:38:54 smithi079 bash[121968]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d 2024-06-03T02:38:55.339 INFO:journalctl@ceph.mds.d.smithi079.stdout:Jun 03 02:38:55 smithi079 podman[121981]: 2024-06-03 02:38:55.031677538 +0000 UTC m=+0.713131770 container remove 4f6fb36aef5945b21fd19cc2ab5b74bfd07d7dfe8d84e61e683c91dfa1483a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:38:55.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-03T02:38:55.669 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:38:55.669 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-03T02:38:55.669 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-03T02:38:55.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.g 2024-06-03T02:38:56.088 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:38:55 smithi079 systemd[1]: Stopping Ceph mds.g for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:38:56.588 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:38:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g[73997]: 2024-06-03T02:38:56.255+0000 7ff34ca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:38:56.588 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:38:56 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g[73997]: 2024-06-03T02:38:56.255+0000 7ff34ca00700 -1 mds.g *** got signal Terminated *** 2024-06-03T02:38:58.573 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:38:58 smithi079 podman[122087]: 2024-06-03 02:38:58.296324662 +0000 UTC m=+2.160207784 container died 64f029655276317e6590527380b646d0cf543e83845851695461b79e1bd9cdcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-03T02:38:59.145 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:38:58 smithi079 podman[122087]: 2024-06-03 02:38:58.894851984 +0000 UTC m=+2.758735119 container cleanup 64f029655276317e6590527380b646d0cf543e83845851695461b79e1bd9cdcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:38:59.145 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:38:58 smithi079 bash[122087]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g 2024-06-03T02:38:59.510 INFO:journalctl@ceph.mds.g.smithi079.stdout:Jun 03 02:38:59 smithi079 podman[122098]: 2024-06-03 02:38:59.143892 +0000 UTC m=+0.830870484 container remove 64f029655276317e6590527380b646d0cf543e83845851695461b79e1bd9cdcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-g, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T02:38:59.531 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-03T02:38:59.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:38:59.616 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-03T02:38:59.616 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-03T02:38:59.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.j 2024-06-03T02:38:59.838 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:38:59 smithi079 systemd[1]: Stopping Ceph mds.j for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:39:00.589 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:39:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-j[72824]: 2024-06-03T02:39:00.119+0000 7f4878200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:39:00.589 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:39:00 smithi079 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-j[72824]: 2024-06-03T02:39:00.119+0000 7f4878200700 -1 mds.j *** got signal Terminated *** 2024-06-03T02:39:02.588 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:39:02 smithi079 podman[122214]: 2024-06-03 02:39:02.3096858 +0000 UTC m=+2.316883477 container died 305331830f1e718598d6b195bd6f113df4aafbf37bc31347c5d4437b11accf82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-j, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-03T02:39:02.895 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:39:02 smithi079 podman[122214]: 2024-06-03 02:39:02.871288905 +0000 UTC m=+2.878486546 container cleanup 305331830f1e718598d6b195bd6f113df4aafbf37bc31347c5d4437b11accf82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-03T02:39:02.895 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:39:02 smithi079 bash[122214]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-j 2024-06-03T02:39:03.338 INFO:journalctl@ceph.mds.j.smithi079.stdout:Jun 03 02:39:03 smithi079 podman[122226]: 2024-06-03 02:39:03.072302397 +0000 UTC m=+0.742580708 container remove 305331830f1e718598d6b195bd6f113df4aafbf37bc31347c5d4437b11accf82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:39:03.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-03T02:39:03.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:39:03.646 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-03T02:39:03.646 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-03T02:39:03.646 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.b 2024-06-03T02:39:03.946 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:03 smithi123 systemd[1]: Stopping Ceph mds.b for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:39:04.217 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:03 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b[67499]: 2024-06-03T02:39:03.945+0000 7f70286d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:39:04.217 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:03 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b[67499]: 2024-06-03T02:39:03.945+0000 7f70286d4700 -1 mds.b *** got signal Terminated *** 2024-06-03T02:39:06.473 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:06 smithi123 podman[74485]: 2024-06-03 02:39:06.168213078 +0000 UTC m=+2.329978814 container died a6f06cdcd13f55684dd6ba3a17cfb3e4c3e3ebc6cdf5728d87991308fbae1039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-03T02:39:06.805 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:06 smithi123 podman[74485]: 2024-06-03 02:39:06.587947269 +0000 UTC m=+2.749713004 container cleanup a6f06cdcd13f55684dd6ba3a17cfb3e4c3e3ebc6cdf5728d87991308fbae1039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-06-03T02:39:06.806 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:06 smithi123 bash[74485]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b 2024-06-03T02:39:07.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-03T02:39:07.056 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:06 smithi123 podman[74509]: 2024-06-03 02:39:06.805789737 +0000 UTC m=+0.634626603 container remove a6f06cdcd13f55684dd6ba3a17cfb3e4c3e3ebc6cdf5728d87991308fbae1039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-b, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:39:07.056 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:07 smithi123 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.b.service: Deactivated successfully. 2024-06-03T02:39:07.056 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:07 smithi123 systemd[1]: Stopped Ceph mds.b for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:39:07.056 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 03 02:39:07 smithi123 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.b.service: Consumed 23.972s CPU time. 2024-06-03T02:39:07.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:39:07.070 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-03T02:39:07.070 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-03T02:39:07.070 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.e 2024-06-03T02:39:07.383 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:07 smithi123 systemd[1]: Stopping Ceph mds.e for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:39:07.723 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e[66889]: 2024-06-03T02:39:07.382+0000 7fdf6c968700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:39:07.723 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:07 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e[66889]: 2024-06-03T02:39:07.382+0000 7fdf6c968700 -1 mds.e *** got signal Terminated *** 2024-06-03T02:39:10.467 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:10 smithi123 podman[74618]: 2024-06-03 02:39:10.162395962 +0000 UTC m=+2.875031404 container died 6b4da3e35fdd33b28cf4df8d0cc35a2b758126be51ba56acb2ba3eb55459a0c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True) 2024-06-03T02:39:10.723 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:10 smithi123 podman[74618]: 2024-06-03 02:39:10.541686587 +0000 UTC m=+3.254322048 container cleanup 6b4da3e35fdd33b28cf4df8d0cc35a2b758126be51ba56acb2ba3eb55459a0c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-03T02:39:10.723 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:10 smithi123 bash[74618]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e 2024-06-03T02:39:10.979 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-03T02:39:11.000 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:10 smithi123 podman[74631]: 2024-06-03 02:39:10.74273772 +0000 UTC m=+0.572928508 container remove 6b4da3e35fdd33b28cf4df8d0cc35a2b758126be51ba56acb2ba3eb55459a0c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-03T02:39:11.000 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:10 smithi123 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.e.service: Deactivated successfully. 2024-06-03T02:39:11.000 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:10 smithi123 systemd[1]: Stopped Ceph mds.e for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:39:11.000 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 03 02:39:10 smithi123 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.e.service: Consumed 2.461s CPU time. 2024-06-03T02:39:11.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:39:11.012 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-03T02:39:11.012 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-03T02:39:11.012 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.h 2024-06-03T02:39:11.311 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:39:11 smithi123 systemd[1]: Stopping Ceph mds.h for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:39:11.723 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:39:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h[67193]: 2024-06-03T02:39:11.311+0000 7fc94c839700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:39:11.723 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:39:11 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h[67193]: 2024-06-03T02:39:11.311+0000 7fc94c839700 -1 mds.h *** got signal Terminated *** 2024-06-03T02:39:14.437 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:39:14 smithi123 podman[74738]: 2024-06-03 02:39:14.160971317 +0000 UTC m=+2.934046953 container died 47d4e72909ab5b2a6bcedd75fde733dbcb3bad032517a93f435b3ffa7d379e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-03T02:39:14.723 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:39:14 smithi123 podman[74738]: 2024-06-03 02:39:14.511782722 +0000 UTC m=+3.284858341 container cleanup 47d4e72909ab5b2a6bcedd75fde733dbcb3bad032517a93f435b3ffa7d379e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:39:14.724 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 03 02:39:14 smithi123 bash[74738]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-h 2024-06-03T02:39:14.959 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-03T02:39:14.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:39:14.990 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-03T02:39:14.990 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-03T02:39:14.990 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.k 2024-06-03T02:39:15.315 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:39:15 smithi123 systemd[1]: Stopping Ceph mds.k for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:39:15.723 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:39:15 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k[66578]: 2024-06-03T02:39:15.314+0000 7f1da8759700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:39:15.723 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:39:15 smithi123 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k[66578]: 2024-06-03T02:39:15.314+0000 7f1da8759700 -1 mds.k *** got signal Terminated *** 2024-06-03T02:39:18.473 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:39:18 smithi123 podman[74860]: 2024-06-03 02:39:18.153742264 +0000 UTC m=+2.943310477 container died cd3843bbaf1ddfc1bdc1ed045c53acad17613b0f6d899cd8fed8f5279ed173ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-03T02:39:18.473 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:39:18 smithi123 podman[74860]: 2024-06-03 02:39:18.473543609 +0000 UTC m=+3.263111867 container cleanup cd3843bbaf1ddfc1bdc1ed045c53acad17613b0f6d899cd8fed8f5279ed173ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T02:39:18.723 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:39:18 smithi123 bash[74860]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k 2024-06-03T02:39:18.723 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 03 02:39:18 smithi123 podman[74874]: 2024-06-03 02:39:18.682797018 +0000 UTC m=+0.526871121 container remove cd3843bbaf1ddfc1bdc1ed045c53acad17613b0f6d899cd8fed8f5279ed173ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T02:39:18.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-03T02:39:18.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:39:18.921 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-03T02:39:18.922 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-03T02:39:18.922 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.c 2024-06-03T02:39:19.215 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:18 smithi135 systemd[1]: Stopping Ceph mds.c for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:39:19.478 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c[67206]: 2024-06-03T02:39:19.214+0000 7fc383095700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:39:19.478 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:19 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c[67206]: 2024-06-03T02:39:19.214+0000 7fc383095700 -1 mds.c *** got signal Terminated *** 2024-06-03T02:39:22.533 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:22 smithi135 podman[74755]: 2024-06-03 02:39:22.196643051 +0000 UTC m=+3.075604547 container died e93d986aac4d05e726aaff8c484d266e1bea69b2b21a3a7ba884ea19587dad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-03T02:39:22.806 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:22 smithi135 podman[74755]: 2024-06-03 02:39:22.63058193 +0000 UTC m=+3.509543427 container cleanup e93d986aac4d05e726aaff8c484d266e1bea69b2b21a3a7ba884ea19587dad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-03T02:39:22.806 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:22 smithi135 bash[74755]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c 2024-06-03T02:39:23.053 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-03T02:39:23.074 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:22 smithi135 podman[74780]: 2024-06-03 02:39:22.806659187 +0000 UTC m=+0.606309301 container remove e93d986aac4d05e726aaff8c484d266e1bea69b2b21a3a7ba884ea19587dad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-03T02:39:23.074 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:23 smithi135 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.c.service: Deactivated successfully. 2024-06-03T02:39:23.074 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:23 smithi135 systemd[1]: Stopped Ceph mds.c for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:39:23.074 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 03 02:39:23 smithi135 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.c.service: Consumed 4.205s CPU time. 2024-06-03T02:39:23.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:39:23.088 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-03T02:39:23.088 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-03T02:39:23.088 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.f 2024-06-03T02:39:23.345 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:23 smithi135 systemd[1]: Stopping Ceph mds.f for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:39:23.845 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:23 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f[66894]: 2024-06-03T02:39:23.425+0000 7f34a9d07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:39:23.845 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:23 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f[66894]: 2024-06-03T02:39:23.425+0000 7f34a9d07700 -1 mds.f *** got signal Terminated *** 2024-06-03T02:39:26.452 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:26 smithi135 podman[74889]: 2024-06-03 02:39:26.191884473 +0000 UTC m=+2.884897905 container died db117653c628e33ee691f7b2817c6614cee5ae93133ed184185cac3c70b1ef28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-03T02:39:26.735 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:26 smithi135 podman[74889]: 2024-06-03 02:39:26.54305993 +0000 UTC m=+3.236073364 container cleanup db117653c628e33ee691f7b2817c6614cee5ae93133ed184185cac3c70b1ef28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-03T02:39:26.735 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:26 smithi135 bash[74889]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f 2024-06-03T02:39:26.968 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-03T02:39:26.989 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:26 smithi135 podman[74903]: 2024-06-03 02:39:26.735717688 +0000 UTC m=+0.534451364 container remove db117653c628e33ee691f7b2817c6614cee5ae93133ed184185cac3c70b1ef28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-03T02:39:26.989 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:26 smithi135 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.f.service: Deactivated successfully. 2024-06-03T02:39:26.989 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:26 smithi135 systemd[1]: Stopped Ceph mds.f for 7a1e9ad6-214d-11ef-bc9b-c7b262605968. 2024-06-03T02:39:26.989 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 03 02:39:26 smithi135 systemd[1]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.f.service: Consumed 2.751s CPU time. 2024-06-03T02:39:27.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:39:27.001 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-03T02:39:27.001 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-03T02:39:27.002 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.i 2024-06-03T02:39:27.330 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:39:27 smithi135 systemd[1]: Stopping Ceph mds.i for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:39:27.595 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:39:27 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i[67509]: 2024-06-03T02:39:27.330+0000 7f6e0bd02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:39:27.595 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:39:27 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i[67509]: 2024-06-03T02:39:27.330+0000 7f6e0bd02700 -1 mds.i *** got signal Terminated *** 2024-06-03T02:39:30.451 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:39:30 smithi135 podman[75011]: 2024-06-03 02:39:30.188464845 +0000 UTC m=+2.966481590 container died 2232e148aa25dbadcff13804c3a8c2023dfdd8f201c1014ce061ba43eb6b3793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-03T02:39:30.806 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:39:30 smithi135 podman[75011]: 2024-06-03 02:39:30.555095637 +0000 UTC m=+3.333112381 container cleanup 2232e148aa25dbadcff13804c3a8c2023dfdd8f201c1014ce061ba43eb6b3793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-03T02:39:30.806 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 03 02:39:30 smithi135 bash[75011]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-i 2024-06-03T02:39:30.990 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-03T02:39:31.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:39:31.022 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-03T02:39:31.022 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-03T02:39:31.022 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.l 2024-06-03T02:39:31.345 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:39:31 smithi135 systemd[1]: Stopping Ceph mds.l for 7a1e9ad6-214d-11ef-bc9b-c7b262605968... 2024-06-03T02:39:31.845 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:39:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l[66592]: 2024-06-03T02:39:31.349+0000 7f6ffaec8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T02:39:31.845 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:39:31 smithi135 ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l[66592]: 2024-06-03T02:39:31.349+0000 7f6ffaec8700 -1 mds.l *** got signal Terminated *** 2024-06-03T02:39:33.095 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:39:32 smithi135 podman[75130]: 2024-06-03 02:39:32.80263519 +0000 UTC m=+1.569097014 container died 037a71e766e0435a82133c46b769771d8b01fe1517edd7b0825865ff540e53e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-03T02:39:33.529 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:39:33 smithi135 podman[75130]: 2024-06-03 02:39:33.117496794 +0000 UTC m=+1.883958618 container cleanup 037a71e766e0435a82133c46b769771d8b01fe1517edd7b0825865ff540e53e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-06-03T02:39:33.530 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:39:33 smithi135 bash[75130]: ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l 2024-06-03T02:39:33.530 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 03 02:39:33 smithi135 podman[75143]: 2024-06-03 02:39:33.3434496 +0000 UTC m=+0.538944360 container remove 037a71e766e0435a82133c46b769771d8b01fe1517edd7b0825865ff540e53e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-03T02:39:33.551 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a1e9ad6-214d-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-03T02:39:33.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T02:39:33.581 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-03T02:39:33.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T02:39:33.899 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:39:38.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T02:39:39.139 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:39:41.658 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T02:39:41.846 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:39:44.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T02:39:44.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T02:39:44.446 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T02:39:44.472 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-03T02:39:44.474 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/remote/smithi079/crash 2024-06-03T02:39:44.475 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T02:39:44.576 INFO:teuthology.orchestra.run.smithi079.stderr:tar: /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T02:39:44.576 INFO:teuthology.orchestra.run.smithi079.stderr:tar: Error is not recoverable: exiting now 2024-06-03T02:39:44.580 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/remote/smithi123/crash 2024-06-03T02:39:44.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T02:39:44.621 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T02:39:44.621 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-06-03T02:39:44.623 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/remote/smithi135/crash 2024-06-03T02:39:44.624 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T02:39:44.670 INFO:teuthology.orchestra.run.smithi135.stderr:tar: /var/lib/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T02:39:44.670 INFO:teuthology.orchestra.run.smithi135.stderr:tar: Error is not recoverable: exiting now 2024-06-03T02:39:44.671 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-03T02:39:44.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-03T02:39:44.825 INFO:tasks.cephadm:Compressing logs... 2024-06-03T02:39:44.825 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T02:39:44.830 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T02:39:44.832 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T02:39:44.869 INFO:teuthology.orchestra.run.smithi135.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-03T02:39:44.869 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T02:39:44.869 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T02:39:44.869 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-03T02:39:44.869 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-03T02:39:44.869 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T02:39:44.869 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-03T02:39:44.869 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-03T02:39:44.869 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-03T02:39:44.870 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-03T02:39:44.870 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-03T02:39:44.870 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mon.c.log: /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T02:39:44.870 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-03T02:39:44.870 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-03T02:39:44.870 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T02:39:44.870 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.11.log: /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-03T02:39:44.876 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-03T02:39:44.876 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T02:39:44.877 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T02:39:44.877 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T02:39:44.877 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-03T02:39:44.880 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-03T02:39:44.880 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-03T02:39:44.880 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-03T02:39:44.880 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-03T02:39:44.880 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-03T02:39:44.880 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-03T02:39:44.880 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T02:39:44.880 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-03T02:39:44.881 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T02:39:44.881 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-03T02:39:44.882 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-03T02:39:44.898 INFO:teuthology.orchestra.run.smithi079.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-03T02:39:44.898 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T02:39:44.899 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T02:39:44.900 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T02:39:44.902 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-03T02:39:44.902 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-03T02:39:44.903 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-03T02:39:44.904 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-03T02:39:44.905 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-03T02:39:44.906 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-03T02:39:44.906 INFO:teuthology.orchestra.run.smithi079.stderr: 87.2% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T02:39:44.907 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-03T02:39:44.908 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T02:39:44.909 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-03T02:39:44.910 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T02:39:44.911 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-03T02:39:44.911 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-03T02:39:44.912 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-03T02:39:44.915 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.9.log: /var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-03T02:39:44.916 INFO:teuthology.orchestra.run.smithi079.stderr: 94.3% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-03T02:39:44.940 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mon.b.log: /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.k.log: 94.7% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-03T02:39:44.955 INFO:teuthology.orchestra.run.smithi079.stderr: 83.4% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T02:39:44.965 INFO:teuthology.orchestra.run.smithi123.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T02:39:44.965 INFO:teuthology.orchestra.run.smithi123.stderr: 93.1% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-03T02:39:44.969 INFO:teuthology.orchestra.run.smithi123.stderr: 93.1% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-03T02:39:44.971 INFO:teuthology.orchestra.run.smithi079.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T02:39:44.987 INFO:teuthology.orchestra.run.smithi123.stderr: 83.3% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T02:39:44.989 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-volume.log: /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.l.log: 93.4% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-03T02:39:44.989 INFO:teuthology.orchestra.run.smithi123.stderr: 90.4% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T02:39:44.998 INFO:teuthology.orchestra.run.smithi123.stderr: 87.2% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T02:39:45.021 INFO:teuthology.orchestra.run.smithi123.stderr: 93.4% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-03T02:39:45.034 INFO:teuthology.orchestra.run.smithi135.stderr: 83.2% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T02:39:45.037 INFO:teuthology.orchestra.run.smithi135.stderr: 87.2% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T02:39:45.053 INFO:teuthology.orchestra.run.smithi135.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T02:39:45.074 INFO:teuthology.orchestra.run.smithi079.stderr: 90.9% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T02:39:45.080 INFO:teuthology.orchestra.run.smithi079.stderr: 90.3% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T02:39:45.127 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T02:39:45.173 INFO:teuthology.orchestra.run.smithi135.stderr: 94.4% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-03T02:39:45.187 INFO:teuthology.orchestra.run.smithi135.stderr: 90.5% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T02:39:45.209 INFO:teuthology.orchestra.run.smithi135.stderr: 91.0% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T02:39:45.300 INFO:teuthology.orchestra.run.smithi079.stderr: 94.6% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-03T02:39:45.423 INFO:teuthology.orchestra.run.smithi079.stderr: 94.6% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-03T02:39:45.558 INFO:teuthology.orchestra.run.smithi079.stderr: 93.0% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-03T02:39:45.805 INFO:teuthology.orchestra.run.smithi135.stderr: 93.1% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-03T02:39:46.265 INFO:teuthology.orchestra.run.smithi135.stderr: 94.6% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-03T02:39:46.311 INFO:teuthology.orchestra.run.smithi123.stderr: 92.0% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-03T02:39:46.340 INFO:teuthology.orchestra.run.smithi079.stderr: 90.3% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-03T02:39:47.321 INFO:teuthology.orchestra.run.smithi135.stderr: 94.6% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-03T02:39:48.832 INFO:teuthology.orchestra.run.smithi135.stderr: 91.7% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-03T02:39:52.410 INFO:teuthology.orchestra.run.smithi079.stderr: 90.3% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-03T02:40:03.786 INFO:teuthology.orchestra.run.smithi123.stderr: 94.2% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-03T02:41:02.623 INFO:teuthology.orchestra.run.smithi135.stderr: 90.6% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-03T02:42:59.254 INFO:teuthology.orchestra.run.smithi123.stderr: 90.5% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-03T02:43:39.605 INFO:teuthology.orchestra.run.smithi123.stderr: 90.5% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-03T02:43:41.249 INFO:teuthology.orchestra.run.smithi123.stderr: 90.5% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-03T02:44:23.281 INFO:teuthology.orchestra.run.smithi135.stderr: 90.9% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-03T02:44:44.551 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-03T02:44:44.553 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-06-03T02:44:44.554 INFO:teuthology.orchestra.run.smithi123.stderr:real 4m59.706s 2024-06-03T02:44:44.554 INFO:teuthology.orchestra.run.smithi123.stderr:user 5m8.076s 2024-06-03T02:44:44.554 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m10.623s 2024-06-03T02:46:12.898 INFO:teuthology.orchestra.run.smithi135.stderr: 90.6% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-03T02:46:48.357 INFO:teuthology.orchestra.run.smithi135.stderr: 90.6% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-03T02:46:48.359 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-06-03T02:46:48.359 INFO:teuthology.orchestra.run.smithi135.stderr:real 7m3.513s 2024-06-03T02:46:48.359 INFO:teuthology.orchestra.run.smithi135.stderr:user 6m15.447s 2024-06-03T02:46:48.359 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m13.324s 2024-06-03T02:51:01.005 INFO:teuthology.orchestra.run.smithi079.stderr: 90.7% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-03T02:51:41.493 INFO:teuthology.orchestra.run.smithi079.stderr: 90.6% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-03T02:51:53.624 INFO:teuthology.orchestra.run.smithi079.stderr: 90.7% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-03T02:52:46.079 INFO:teuthology.orchestra.run.smithi079.stderr: 90.7% -- replaced with /var/log/ceph/7a1e9ad6-214d-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-03T02:52:46.084 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-06-03T02:52:46.084 INFO:teuthology.orchestra.run.smithi079.stderr:real 13m1.218s 2024-06-03T02:52:46.084 INFO:teuthology.orchestra.run.smithi079.stderr:user 15m42.244s 2024-06-03T02:52:46.084 INFO:teuthology.orchestra.run.smithi079.stderr:sys 1m50.341s 2024-06-03T02:52:46.085 INFO:tasks.cephadm:Archiving logs... 2024-06-03T02:52:46.086 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/remote/smithi079/log 2024-06-03T02:52:46.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T02:53:55.673 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/remote/smithi123/log 2024-06-03T02:53:55.675 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T02:54:51.446 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/remote/smithi135/log 2024-06-03T02:54:51.448 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T02:55:58.585 INFO:tasks.cephadm:Removing cluster... 2024-06-03T02:55:58.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 --force 2024-06-03T02:55:59.095 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:56:01.067 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 --force 2024-06-03T02:56:01.292 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:56:01.967 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a1e9ad6-214d-11ef-bc9b-c7b262605968 --force 2024-06-03T02:56:02.238 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 7a1e9ad6-214d-11ef-bc9b-c7b262605968 2024-06-03T02:56:02.977 INFO:tasks.cephadm:Removing cephadm ... 2024-06-03T02:56:02.977 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T02:56:03.058 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T02:56:03.286 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T02:56:03.308 INFO:tasks.cephadm:Teardown complete 2024-06-03T02:56:03.310 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-03T02:56:03.334 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-03T02:56:03.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T02:56:03.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T02:56:03.345 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T02:56:03.506 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-03T02:56:03.516 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-03T02:56:03.517 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T02:56:03.525 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T02:56:03.528 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T02:56:03.541 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-06-03T02:56:03.542 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-06-03T02:56:03.553 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T02:56:03.553 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-06-03T02:56:03.553 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +773us[ +802us] +/- 40ms 2024-06-03T02:56:03.553 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -781us[ -754us] +/- 57ms 2024-06-03T02:56:03.553 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -318us[ -290us] +/- 62ms 2024-06-03T02:56:03.553 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:56:03.572 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-06-03T02:56:03.605 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T02:56:03.605 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-06-03T02:56:03.605 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +773us[ +803us] +/- 40ms 2024-06-03T02:56:03.605 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 -785us[ -785us] +/- 57ms 2024-06-03T02:56:03.605 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -324us[ -294us] +/- 62ms 2024-06-03T02:56:03.605 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:56:03.693 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T02:56:03.693 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-06-03T02:56:03.693 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +787us[ +768us] +/- 40ms 2024-06-03T02:56:03.694 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 -785us[ -804us] +/- 57ms 2024-06-03T02:56:03.694 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -318us[ -318us] +/- 62ms 2024-06-03T02:56:03.694 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T02:56:03.696 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-03T02:56:03.706 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-03T02:56:03.707 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-03T02:56:03.723 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T02:56:04.125 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T02:56:04.277 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T02:56:04.379 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T02:56:04.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T02:56:04.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T02:56:04.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T02:56:04.504 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T02:56:04.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T02:56:04.533 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-03T02:56:04.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-03T02:56:04.567 INFO:teuthology.task.internal:Duration was 3652.631774 seconds 2024-06-03T02:56:04.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-03T02:56:04.583 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-03T02:56:04.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T02:56:04.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T02:56:04.594 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T02:56:04.649 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T02:56:04.654 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T02:56:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T02:56:05.205 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-03T02:56:05.205 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-06-03T02:56:05.205 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T02:56:05.269 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-06-03T02:56:05.270 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T02:56:05.303 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2024-06-03T02:56:05.303 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T02:56:05.334 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-03T02:56:05.334 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T02:56:05.342 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T02:56:05.346 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T02:56:05.424 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-03T02:56:05.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T02:56:05.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T02:56:05.432 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T02:56:06.789 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-03T02:56:06.800 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-03T02:56:06.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T02:56:06.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T02:56:06.928 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T02:56:06.958 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-03T02:56:06.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T02:56:06.979 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T02:56:06.981 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T02:56:07.015 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-06-03T02:56:07.024 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2024-06-03T02:56:07.087 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-06-03T02:56:07.139 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T02:56:07.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T02:56:07.186 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T02:56:07.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T02:56:07.202 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T02:56:07.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T02:56:07.219 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-03T02:56:07.230 INFO:teuthology.task.internal:Transferring archived files... 2024-06-03T02:56:07.231 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/remote/smithi079 2024-06-03T02:56:07.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T02:56:07.372 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/remote/smithi123 2024-06-03T02:56:07.372 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T02:56:07.414 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738807/remote/smithi135 2024-06-03T02:56:07.414 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T02:56:07.453 INFO:teuthology.task.internal:Removing archive directory... 2024-06-03T02:56:07.454 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T02:56:07.462 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T02:56:07.464 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T02:56:07.512 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-03T02:56:07.526 INFO:teuthology.task.internal:Not uploading archives. 2024-06-03T02:56:07.526 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-03T02:56:07.535 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-03T02:56:07.536 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T02:56:07.541 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T02:56:07.543 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T02:56:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 02:56 /home/ubuntu/cephtest 2024-06-03T02:56:07.560 INFO:teuthology.orchestra.run.smithi135.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 02:56 /home/ubuntu/cephtest 2024-06-03T02:56:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 02:56 /home/ubuntu/cephtest 2024-06-03T02:56:07.592 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-03T02:56:07.604 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-03T02:56:07.630 INFO:teuthology.run:Summary data: client.0-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/postgres}} duration: 3652.631774187088 failure_reason: 'The following counters failed to be set on mds daemons: {''mds.imported'', ''mds.exported''}' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-06-03T02:56:07.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-03T02:56:07.723 INFO:teuthology.run:FAIL